Serverless Workflow is a standard from the Cloud Native Computing Foundation (CNCF). Kogito implements the Serverless Workflow specifications to define workflows for event-driven, serverless applications using a DSL-based model.
Serverless Framework is an open source framework that builds, compiles, and packages code for serverless deployment. The framework provides implementations for different cloud providers, including Knative.
This article walks you through the steps to integrate Kogito with Serverless Framework to build a working example on the Red Hat OpenShift Platform. The article is based on code you can find in my GitHub repository.
Prerequisites
To run the demo in this article, you need the following tools on your local system:
- Maven (at least 3.8.6)
- Java SDK 11+
- Docker
- Bash terminal
- The serverless command-line interface (CLI) from the Serverless Framework
You also need accounts on the following systems:
- OpenShift 4.8+ (logged in as an account with the
cluster-adminrole) - A Docker Hub credential
- A Quay.io account
Introducing the Kogito Newsletter Subscription Showcase example
The Kogito Newsletter Subscription Showcase is a demo based on the Kogito implementation of the Serverless Workflow specification. This example consists of two applications:
subscription-flow: The workflow orchestrator, defined using the Serverless Workflow specificationsubscription-service: A Quarkus application implementing the orchestrated services.
Figure 1 shows the system architecture of the example application.
The documentation in the repository describes how to deploy and run the application on Knative, using the YAML configurations generated by the Maven build of the project, through the knative build profile, running on minikube.
This article shows how to implement the same deployment in the cloud using the Serverless Framework. Our target is a Knative environment installed on OpenShift, but the principles extend to other cloud settings as well.
Images for the Kogito Newsletter Subscription Showcase
The first step is to build and publish the images of the two applications that make up the Newsletter Subscription Showcase example: subscription-flow and subscription-service.
You can download prebuilt images or build new ones from source. The prebuilt images for our example are available in the quay.io/dmartino repository; if you download them, you can skip ahead to the next section, entitled "Installing Knative on OpenShift."
If you prefer to build the images yourself, you'll need to clone the Kogito Examples repository, build the applications using the knative profile, and finally push the Docker images to your Quay repository, using the following commands. Replace QUAY_USER_ID with your actual ID.
git clone https://github.com/kiegroup/kogito-examples.git
cd kogito-examples
git checkout stable
cd serverless-workflow-examples/serverless-workflow-newsletter-subscription
docker login quay.io
mvn clean install -DskipTests -Pknative \
-Dquarkus.container-image.registry=quay.io \
-Dquarkus.container-image.group=QUAY_USER_ID \
-Dquarkus.container-image.push=true
Verify that two images have been generated with the expected tag (as of today, it is 1.25.0.Final) on your Quay.io account, and modify the visibility of the images to make them publicly accessible.
Installing Knative on OpenShift
Knative can be easily installed on OpenShift using the OpenShift Serverless Operator, and this is the recommended approach we are going to follow.
Install the Red Hat Serverless Operator from the administrator console (Figure 2). The sequence of menu items to choose is OperatorHub→Red Hat OpenShift Serverless→Install. Accept the default settings.
One instance of the KnativeServing custom resource is required to manage Knative serverless applications (Figure 3). Create the instance in the knative-serving namespace by selecting the knative-serving project from the administrator console. Then select Installed Operators→Red Hat OpenShift Serverless→Knative Serving→Create KnativeServing. Accept the default settings.
Additionally, one instance of the KnativeEventing Custom Resource is required to manage the events around the Knative serverless applications (Figure 4). Create the instance in the knative-eventing namespace by selecting the knative-eventing project from the administrator console. Then select Installed Operators→Red Hat OpenShift Serverless→Knative Eventing→Create KnativeEventing. Accept the default settings.
Installing the example application
To run the application, you need to install a PostgreSQL database. The application also requires some configuration changes to bring it up to date.
Installing the newsletter-postgres service
The newsletter-postgres service is a regular OpenShift deployment of PostgreSQL in a namespace called newsletter-subscription-db. Execute the following instructions to install the newsletter-postgres service:
git clone https://github.com/dmartinol/kogito-serverless-workflow-with-serverless-framework.git
cd kogito-serverless-workflow-with-serverless-framework
oc create namespace newsletter-subscription-db
oc adm policy add-scc-to-user anyuid -z default -n newsletter-subscription-db
oc apply -f newsletter-postgres/newsletter-postgres.yaml
Preparing the Serverless Framework
To successfully run the example on OpenShift, we had to apply a few changes to the original implementation of the Knative Cloud Provider in the Serverless Framework. Such updates are needed to align the original Knative version to the one installed with the OpenShift Serverless Operator, and to introduce some extensions that support new settings and fix a few issues. Details are available in a GitHub repository.
Because the example would not run using the default implementation of the Knative Cloud Provider, the package.json descriptor includes the following dependency:
"devDependencies": {
"serverless-knative": "https://github.com/dmartinol/serverless-knative.git"
}
The necessary changes are available in the following GitHub repositories:
- Serverless Knative Plugin: The Knative Cloud Provider implementation for the Serverless Framework
- knative-serving: A Node.js module to manage Knative Serving instances
- knative-eventing: A Node.js module to manage Knative Eventing instances
Unwrapping the Serverless Framework descriptor
The heart of the Serverless Framework deployment is the serverless.yml file that sits at the local root of the kogito-serverless-workflow-with-serverless-framework repository:
service: newsletter
frameworkVersion: '3'
provider:
name: knative
# optional Docker Hub credentials you need if you're using local Dockerfiles as function handlers
docker:
username: ${env:DOCKER_HUB_USERNAME}
password: ${env:DOCKER_HUB_PASSWORD}
functions:
event-display:
handler: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display@sha256:a214514d6ba674d7393ec8448dd272472b2956207acb3f83152d3071f0ab1911
# autoscaler field is managed by knative provider
# Just add any autoscaling related annotation and it will be propagated to the deployed Service and Revision
# The plugin automatically adds the 'autoscaling.knative.dev/' prefix to the annotation name
autoscaler:
min-scale: 1
max-scale: 2
events:
- custom:
name: new.subscription.2.event-display
filter:
attributes:
type: new.subscription
- custom:
name: confirm.subscription.2.event-display
filter:
attributes:
type: confirm.subscription
subscription-service:
handler: Dockerfile.jvm
context: ./subscription-service
subscription-flow:
handler: Dockerfile.jvm
context: ./subscription-flow
events:
- custom:
filter:
attributes:
type: confirm.subscription
- sinkBinding: {}
plugins:
- serverless-knative
To replicate the architecture of the original example, this deployment includes the functions listed in the following sections (the equivalents of the Knative Service resources).
event-display
This is an event logger application implemented with a prebuilt image from the Google Cloud Container Registry. The image is configured with a minimum of one instance to simplify logging activity, and has two custom events that are mapped onto two Knative Trigger instances.
subscription-service
This is the service running the original subscription-service application. The original source code was copied from the Kogito Examples repository under the subscription-service folder, to show the option to locally build an application and deploy the serverless service using the Serverless Framework CLI.
The Dockerfile.jvm file defined by the handler property builds the Quarkus application and injects the binding properties to connect to the newsletter-postgres database:
ENV POSTGRES_PASSWORD=cGFzcwo=
ENV POSTGRES_HOST=newsletter-postgres.newsletter-subscription-db
subscription-flow
This function runs the subscription-flow image that you previously built, but with overridden properties to locate the newsletter-postgres service:
FROM quay.io/dmartino/serverless-workflow-newsletter-subscription-flow:1.25.0.Final
ENV SUBSCRIPTION_API_URL=http://newsletter-subscription-service.sls-newsletter-dev.svc.cluster.local
ENV POSTGRES_HOST=newsletter-postgres.newsletter-subscription-db
This serverless function is configured with one custom event, mapped to a Knative Trigger instance, and one sinkBinding event that generates the Knative SinkBinding to connect the Knative Service with the default Knative Broker. The default Knative Broker is automatically created by the eventing.knative.dev/injection annotation attached to the Knative Trigger instances.
Deploying the application with the Serverless Framework
The first step is to build the subscription-service application:
$ cd subscription-service
$ mvn clean package
The following instructions assume that you have already installed the serverless CLI, and set the environment variables DOCKER_HUB_USERNAME and DOCKER_HUB_PASSWORD to define the access credentials to the Docker Hub repository. Now deployment is just a matter of running the serverless deploy command:
$ serverless deploy
The info command returns the deployment status of your application:
$ serverless info
...
Service Information
service: newsletter
namespace: sls-newsletter-dev
Deployed functions
event-display:
- url: https://newsletter-event-display-sls-newsletter-dev.DOMAIN
- custom
- custom
subscription-service:
- url: https://newsletter-subscription-service-sls-newsletter-dev.DOMAIN
subscription-flow:
- url: https://newsletter-subscription-flow-sls-newsletter-dev.DOMAIN
- custom
- sinkBinding
Validating the applications
Run the applications using the default browser by executing the following:
$ open -t $(oc get ksvc newsletter-subscription-flow -n sls-newsletter-dev -ojsonpath='{.status.url}{"\n"}')
$ open -t $(oc get ksvc newsletter-subscription-service -n sls-newsletter-dev -ojsonpath='{.status.url}{"\n"}')
Now you can monitor the CloudEvents events through the logs of the event-display pod:
$ oc logs -l serving.knative.dev/service=newsletter-event-display -f -n sls-newsletter-dev -c user-container
Serverless Framework supports cloud deployments
By using the Serverless Framework software, we successfully deployed a serverless application on Red Hat OpenShift, using Knative as the serverless framework.
The default implementation of the Knative Cloud Provider is missing some features and is not compatible with the Red Hat OpenShift Serverless Operator, so a patched implementation was used for the purposes of this article.
The application is defined using the Kogito implementation of the CNCF Serverless Workflow specification, a DSL-based model that targets the serverless technology domain.
Serverless Framework claims to be a cloud-agnostic tool, so nothing prevents us from extending the exercise in the future and adapting this deployment model to run on another cloud platform such as AWS or Azure.
For more information, please read the blog posting Orchestrating Events with Knative and Kogito.