Orchestrate event-driven, distributed services with Serverless Workflow and Kubernetes

Orchestrate event-driven, distributed services with Serverless Workflow and Kubernetes

Serverless workflows have gained renewed interest and usefulness with the rise of serverless architectures. Once seen as centralized and monolithic, they now play a key role in cloud-based event and service orchestration. Until recently, there was no vendor-neutral way to describe service orchestration, so developers were dependent on vendors and vendor implementations. We realized that we needed a common, standards-based language for describing serverless workflows.

In this article, we introduce the Serverless Workflow specification, now in its 0.5 version release. Our goal with this project is to empower anyone to develop serverless workflow libraries, tooling, and infrastructure for modeling workflows across different cloud platforms.

About the Serverless Workflow specification

The Serverless Workflow specification defines a common, declarative, domain-specific language for workflows. Developers can use it to describe both stateless and stateful orchestrations. The specification is a Cloud Native Computing Foundation (CNCF) sandbox project hosted by the CNCF Serverless Working Group.

The Serverless Workflow project’s main focus is hosting a vendor-neutral, portable, community-driven workflow language based on standards rather than proprietary definitions. We’ve targeted the orchestration of event-driven, distributed services. You can use both YAML and JSON formats to describe a workflow using the Serverless Workflow specification.

Everything you need to grow your career.

With your free Red Hat Developer program membership, unlock our library of cheat sheets and ebooks on next-generation application development.

SIGN UP

Structure of the Serverless Workflow language

Figure 1 shows the structure of the Serverless Workflow language. The structure includes three main parts. Each part builds on existing standards.

A diagram of the Serverless Workflow language structure, which includes defining the workflow control-flow logic, events, and services and their operations.

Figure 1: Serverless Workflow’s language structure consists of three main parts.

In the next sections, we’ll describe the three main parts of the Serverless Workflow language structure.

Part 1: Defining events

Serverless Workflow utilizes the CloudEvents specification for defining events that can be consumed or produced during workflow execution. The specification defines a clear one-to-one mapping between how events are defined within the CloudEvents format and how you define them in your workflow language definitions. You can use CloudEvents context attributes to define one or many event-correlation rules. Event definitions in Serverless Workflow are reusable, meaning that you can reuse them between multiple workflows. Figure 2 shows an event definition in the CloudEvent format compared to the same event definition in the Serverless Workflow format.

One screen shows an example of a Serverless Workflow event definition, the other shows a similar event definition in the CloudEvents format.

Figure 2: Compare the event definitions in the Serverless Workflow and CloudEvents formats.

Part 2: Defining services or functions

Serverless Workflow uses the OpenAPI specification to define the services and associated operations that will be invoked during orchestration. To define a service operation, you only need to specify the path to the OpenAPI service definition(s) and select the unique operation ID for the service operation that you want to invoke. Figure 3 shows Serverless Workflow service definitions in relation to their OpenAPI definitions.

One screen shown sample Serverless Workflow service definitions and the other shows the OpenAPI definitions. Lines connect the two screens and highlight the relationship between the two definitions.

Figure 3: Serverless Workflow service definitions in relation to their OpenAPI definition.

As shown in Figure 4, Serverless Workflow lets you describe the invocation of both RESTful and event-triggered services or functions.

One screen shows the definition of a RESTful service invocation and the other shows an event-triggered invocation.

Figure 4: Describing the invocation of RESTful and event-triggered services.

Part 3: Defining the control-flow logic

Serverless Workflow defines pattern-based control-flow logic constructs for defining what will happen during workflow execution. You can define workflow states or steps and their associated transitions, error handling, retries, data management, and more. Figure 5 shows control-flow logic constructs in Serverless Workflow. You can use these to define simple sequences and more complex constructs such as parallel executions, looping, retries, and user interactions or decision steps for controlling the overall orchestration.

An illustration of the many control-flow logic constructs that you can define using Serverless Workflow. Options include data-based looping, sequences, human decision sequences, and more.

Figure 5: Control-flow logic constructs in Serverless Workflow.

Additional components of Serverless Workflow

In addition to the workflow language definition described in the JSON Schema definitions, specification, Serverless Workflow provides a set of language extensions. The extensions enhance workflow definitions to improve the overall performance, cost, and effectiveness of an orchestration. Serverless Workflow also provides software development kits (SDKs) for Java, and Go, with more planned in the future. Additionally, it includes a Visual Studio Code extension and an online editor, providing useful features such as code completions and diagram generation for both JSON and YAML workflow formats. Figure 6 shows components of the Serverless Workflow project.

The components of a Serverless Workflow project shown in concentric circles. The Serverless Workflow language is at the center of the circles.

Figure 6: Components of the Serverless Workflow project.

Use cases for Serverless Workflow

You can use the Serverless Workflow specification language for various use cases, including payment processing, data analysis, continuous integration deployments, and many others. The specification’s use cases document describes several use cases in detail. Figure 7 shows a flow diagram for a sample use case.

A sample Serverless Workflow use case in a flow diagram.

Figure 7: A sample Serverless Workflow use case.

Video demo: A Kubernetes use case

Serverless Workflow is well-suited for container environments. You can directly translate its event and service definitions into Kubernetes-specific constructs such as brokers, sinks, and so on. See our KubeCon + CloudNativeCon North America 2020 talk for a demonstration of how to deploy Serverless Workflow to manage and orchestrate services running on a Kubernetes cluster.

Get involved with Serverless Workflow

The success of the Serverless Workflow project depends on community involvement. We invite anyone to join and contribute to our weekly community meetings. The community meeting is a good place to bring up questions about the project. You can also use the Serverless Workflow GitHub repository to raise issues or reach out to us on the Serverless Workflow community Slack channel.

Conclusion

Serverless Workflow is a portable, community-driven workflow language that is based on existing standards. Being part of the CNCF cloud-native landscape ensures its continuous growth, development, and usability. Serverless Workflow is the leading vendor-neutral workflow language in the serverless space. Our goal with this project is to empower anyone to make a difference in the evolution of container orchestration.

You can use the following resources to learn more about the Serverless Workflow specification and how to use it in your projects:

Share