Build a Kogito Serverless Workflow using Serverless Framework

September 28, 2022
Daniele Martinoli
Related topics:
Serverless
Related products:
Red Hat OpenShift Container Platform

Serverless Workflow is a standard from the Cloud Native Computing Foundation (CNCF). Kogito implements the Serverless Workflow specifications to define workflows for event-driven, serverless applications using a DSL-based model.

Serverless Framework is an open source framework that builds, compiles, and packages code for serverless deployment. The framework provides implementations for different cloud providers, including Knative.

This article walks you through the steps to integrate Kogito with Serverless Framework to build a working example on the Red Hat OpenShift Platform. The article is based on code you can find in my GitHub repository.

Prerequisites

To run the demo in this article, you need the following tools on your local system:

You also need accounts on the following systems:

  • OpenShift 4.8+ (logged in as an account with the cluster-admin role)
  • A Docker Hub credential
  • A Quay.io account

Introducing the Kogito Newsletter Subscription Showcase example

The Kogito Newsletter Subscription Showcase is a demo based on the Kogito implementation of the Serverless Workflow specification. This example consists of two applications:

  • subscription-flow: The workflow orchestrator, defined using the Serverless Workflow specification
  • subscription-service: A Quarkus application implementing the orchestrated services.

Figure 1 shows the system architecture of the example application.

The Newsletter Subscription Showcase is made of two applications, the Subscription Flow and the Subscription Service, an event Broker and a Persistent Storage
Figure 1: The architecture of the Newsletter Subscription Showcase example.
Created by KIE group, License under Apache 2.0. Thanks to KIE https://www.kie.org/,

The documentation in the repository describes how to deploy and run the application on Knative, using the YAML configurations generated by the Maven build of the project, through the knative build profile, running on minikube.

This article shows how to implement the same deployment in the cloud using the Serverless Framework. Our target is a Knative environment installed on OpenShift, but the principles extend to other cloud settings as well.

Images for the Kogito Newsletter Subscription Showcase

The first step is to build and publish the images of the two applications that make up the Newsletter Subscription Showcase example: subscription-flow and subscription-service.

You can download prebuilt images or build new ones from source. The prebuilt images for our example are available in the quay.io/dmartino repository; if you download them, you can skip ahead to the next section, entitled "Installing Knative on OpenShift."

If you prefer to build the images yourself, you'll need to clone the Kogito Examples repository, build the applications using the knative profile, and finally push the Docker images to your Quay repository, using the following commands. Replace QUAY_USER_ID with your actual ID.

git clone https://github.com/kiegroup/kogito-examples.git
cd kogito-examples
git checkout stable
cd serverless-workflow-examples/serverless-workflow-newsletter-subscription
docker login quay.io
mvn clean install -DskipTests -Pknative \
  -Dquarkus.container-image.registry=quay.io \
  -Dquarkus.container-image.group=QUAY_USER_ID \
  -Dquarkus.container-image.push=true

Verify that two images have been generated with the expected tag (as of today, it is 1.25.0.Final) on your Quay.io account, and modify the visibility of the images to make them publicly accessible.

Installing Knative on OpenShift

Knative can be easily installed on OpenShift using the OpenShift Serverless Operator, and this is the recommended approach we are going to follow.

Install the Red Hat Serverless Operator from the administrator console (Figure 2). The sequence of menu items to choose is OperatorHub→Red Hat OpenShift Serverless→Install. Accept the default settings.

Administrator console -> OperatorHub -> Red Hat OpenShift Serverless -> Install using the default settings.
Figure 2: Select the Red Hat OpenShift Serverless operator from the OperatorHub page and install it using the default settings.

One instance of the KnativeServing custom resource is required to manage Knative serverless applications (Figure 3). Create the instance in the knative-serving namespace by selecting the knative-serving project from the administrator console. Then select Installed Operators→Red Hat OpenShift Serverless→Knative Serving→Create KnativeServing. Accept the default settings.

Administrator console -> Project: knative-serving -> Installed Operators -> Red Hat OpenShift Serverless -> Knative Serving -> Create KnativeServing using the default settings.
Figure 3: Install the KnativeServing custom resource from the knative-serving project.

Additionally, one instance of the KnativeEventing Custom Resource is required to manage the events around the Knative serverless applications (Figure 4). Create the instance in the knative-eventing namespace by selecting the knative-eventing project from the administrator console. Then select Installed Operators→Red Hat OpenShift Serverless→Knative Eventing→Create KnativeEventing. Accept the default settings.

Administrator console -> Project: knative-eventing -> Installed Operators -> Red Hat OpenShift Serverless -> Knative Eventing -> Create KnativeEventing using the default settings.
Figure 4: Install the KnativeEventing custom resource from the knative-eventing project.

Installing the example application

To run the application, you need to install a PostgreSQL database. The application also requires some configuration changes to bring it up to date.

Installing the newsletter-postgres service

The newsletter-postgres service is a regular OpenShift deployment of PostgreSQL in a namespace called newsletter-subscription-db. Execute the following instructions to install the newsletter-postgres service:

git clone https://github.com/dmartinol/kogito-serverless-workflow-with-serverless-framework.git
cd kogito-serverless-workflow-with-serverless-framework
oc create namespace newsletter-subscription-db
oc adm policy add-scc-to-user anyuid -z default -n newsletter-subscription-db
oc apply -f newsletter-postgres/newsletter-postgres.yaml

Preparing the Serverless Framework

To successfully run the example on OpenShift, we had to apply a few changes to the original implementation of the Knative Cloud Provider in the Serverless Framework. Such updates are needed to align the original Knative version to the one installed with the OpenShift Serverless Operator, and to introduce some extensions that support new settings and fix a few issues. Details are available in a GitHub repository.

Because the example would not run using the default implementation of the Knative Cloud Provider, the package.json descriptor includes the following dependency:

  "devDependencies": {
    "serverless-knative": "https://github.com/dmartinol/serverless-knative.git"
  }

The necessary changes are available in the following GitHub repositories:

Unwrapping the Serverless Framework descriptor

The heart of the Serverless Framework deployment is the serverless.yml file that sits at the local root of the kogito-serverless-workflow-with-serverless-framework repository:

service: newsletter

frameworkVersion: '3'

provider:
  name: knative
  # optional Docker Hub credentials you need if you're using local Dockerfiles as function handlers
  docker:
    username: ${env:DOCKER_HUB_USERNAME}
    password: ${env:DOCKER_HUB_PASSWORD}
functions:
  event-display:
    handler: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display@sha256:a214514d6ba674d7393ec8448dd272472b2956207acb3f83152d3071f0ab1911
    # autoscaler field is managed by knative provider
    # Just add any autoscaling related annotation and it will be propagated to the deployed Service and Revision
    # The plugin automatically adds the 'autoscaling.knative.dev/' prefix to the annotation name
    autoscaler:
      min-scale: 1
      max-scale: 2
    events:
      - custom:
          name: new.subscription.2.event-display
          filter:
            attributes:
              type: new.subscription
      - custom:
          name: confirm.subscription.2.event-display
          filter:
            attributes:
              type: confirm.subscription
  subscription-service:
    handler: Dockerfile.jvm
    context: ./subscription-service
  subscription-flow:
    handler: Dockerfile.jvm
    context: ./subscription-flow
    events:
      - custom:
          filter:
            attributes:
              type: confirm.subscription
      - sinkBinding: {}
plugins:
  - serverless-knative

To replicate the architecture of the original example, this deployment includes the functions listed in the following sections (the equivalents of the Knative Service resources).

event-display

This is an event logger application implemented with a prebuilt image from the Google Cloud Container Registry. The image is configured with a minimum of one instance to simplify logging activity, and has two custom events that are mapped onto two Knative Trigger instances.

subscription-service

This is the service running the original subscription-service application. The original source code was copied from the Kogito Examples repository under the subscription-service folder, to show the option to locally build an application and deploy the serverless service using the Serverless Framework CLI.

The Dockerfile.jvm file defined by the handler property builds the Quarkus application and injects the binding properties to connect to the newsletter-postgres database:

ENV POSTGRES_PASSWORD=cGFzcwo=
ENV POSTGRES_HOST=newsletter-postgres.newsletter-subscription-db

subscription-flow

This function runs the subscription-flow image that you previously built, but with overridden properties to locate the newsletter-postgres service:

FROM quay.io/dmartino/serverless-workflow-newsletter-subscription-flow:1.25.0.Final

ENV SUBSCRIPTION_API_URL=http://newsletter-subscription-service.sls-newsletter-dev.svc.cluster.local
ENV POSTGRES_HOST=newsletter-postgres.newsletter-subscription-db

This serverless function is configured with one custom event, mapped to a Knative Trigger instance, and one sinkBinding event that generates the Knative SinkBinding to connect the Knative Service with the default Knative Broker. The default Knative Broker is automatically created by the eventing.knative.dev/injection annotation attached to the Knative Trigger instances.

Deploying the application with the Serverless Framework

The first step is to build the subscription-service application:

$ cd subscription-service
$ mvn clean package

The following instructions assume that you have already installed the serverless CLI, and set the environment variables DOCKER_HUB_USERNAME and DOCKER_HUB_PASSWORD to define the access credentials to the Docker Hub repository. Now deployment is just a matter of running the serverless deploy command:

$ serverless deploy

The info command returns the deployment status of your application:

$ serverless info
...
Service Information
service: newsletter
namespace: sls-newsletter-dev

Deployed functions
event-display:
  - url: https://newsletter-event-display-sls-newsletter-dev.DOMAIN
  - custom
  - custom
subscription-service:
  - url: https://newsletter-subscription-service-sls-newsletter-dev.DOMAIN
subscription-flow:
  - url: https://newsletter-subscription-flow-sls-newsletter-dev.DOMAIN
  - custom
  - sinkBinding

Validating the applications

Run the applications using the default browser by executing the following:

$ open -t $(oc get ksvc newsletter-subscription-flow -n sls-newsletter-dev -ojsonpath='{.status.url}{"\n"}')
$ open -t $(oc get ksvc newsletter-subscription-service -n sls-newsletter-dev -ojsonpath='{.status.url}{"\n"}')

Now you can monitor the CloudEvents events through the logs of the event-display pod:

$ oc logs -l serving.knative.dev/service=newsletter-event-display -f -n sls-newsletter-dev -c user-container

Serverless Framework supports cloud deployments

By using the Serverless Framework software, we successfully deployed a serverless application on Red Hat OpenShift, using Knative as the serverless framework.

The default implementation of the Knative Cloud Provider is missing some features and is not compatible with the Red Hat OpenShift Serverless Operator, so a patched implementation was used for the purposes of this article.

The application is defined using the Kogito implementation of the CNCF Serverless Workflow specification, a DSL-based model that targets the serverless technology domain.

Serverless Framework claims to be a cloud-agnostic tool, so nothing prevents us from extending the exercise in the future and adapting this deployment model to run on another cloud platform such as AWS or Azure.

For more information, please read the blog posting Orchestrating Events with Knative and Kogito.

Last updated: March 18, 2024