Low-code microservices orchestration with Syndesis

Low-code microservices orchestration with Syndesis

Recently I wrote about decoupling infrastructure code from microservices. I found that Apache Camel and Debezium provided the middleware I needed for that project, with minimal coding on my end. After my successful experiment, I wondered if it would be possible to orchestrate two or more similarly decoupled microservices into a new service–and could I do it without writing any code at all? I decided to find out.

This article is a quick dive into orchestrating microservices without writing any code. We will use Syndesis (an open source integration platform) as our orchestration platform. Note that the examples assume that you are familiar with Debezium and Kafka.

About Syndesis

Syndesis is an open source integration platform that can connect to any service you use. It targets citizen integrators, who are usually business-domain experts who want to quickly combine existing data sources into new microservices without writing any code. These users have valuable business expertise, and a microservices architecture provides the ideal platform for transforming their knowledge into business services. What’s needed is an integration platform to connect their business expertise to the technology back end.

Note that Syndesis is the open source project for Red Hat Fuse Online. Fuse Online runs on Red Hat OpenShift Online, Red Hat OpenShift Dedicated, and Red Hat OpenShift Container Platform.

Everything you need to grow your career.

With your free Red Hat Developer program membership, unlock our library of cheat sheets and ebooks on next-generation application development.

SIGN UP

Microservices orchestration with Syndesis

Our goal is to integrate two or more microservices to create a new notification service for a game-store platform. Using Syndesis as our integrated platform-as-a-service (iPaaS) lets us orchestrate existing microservices without writing any code. We assume a microservices architecture where Debezium and Kafka have already been implemented to decouple infrastructure code from the microservices, following an approach similar to the one demonstrated in my previous article.

Our game store has an established set of microservices running different parts of the business. For this project, we’re interested in the GamePrice and UserPreference services. GamePrice provides the price of each game in the store. UserPreference is an AI engine that matches users with games they will like. Each microservice exposes an API that we can view in its OpenAPI documentation.

The game store occasionally uses the GamePrice service for product promotions and giveaways. Now we want to integrate GamePrice with UserPreference to create a new microservice for email alerts. The service will alert subscribers when a selected game is offered for free. Our integration platform, Syndesis, integrated with a Debezium connector, makes it easy to capture changes in the GamePrice domain. It will also automatically generate the integration logic connecting our existing microservices, and it will send a personalized email alert to interested users’ (UserPreference domain). Kafka will be the messaging infrastructure where Debezium produces change data events. We won’t need to write any code to get our new notification system up and running.

Step 1: Configure microservices endpoints

To start, we need to configure Syndesis to use the GamePrice and UserPreference APIs. We do that by providing their OpenAPI endpoints, as shown in Figure 1. (Note that this feature is provided by Apicurio.)

A screenshot of the OpenAPI endpoint screen in Syndesis.

Figure 1. Configure the OpenAPI endpoints.

Step 2: Capture data changes in the database

Next, we need to capture changes to price data in the database. We can create event streams to capture such changes. We’ll use a Syndesis Debezium connector pointing to a Kafka broker, which we assume has been previously configured. Kafka is the infrastructure used by Debezium to produce change data events. By selecting the broker we will tell Syndesis which events we need to consume. Figure 2 shows this configuration.

A screenshot of the database-configuration page in Syndesis.

Figure 2. Capture data changes.

Step 3: Configure the email service

Finally, we configure the email service. For this example, I chose to use an SMTP service (as shown in Figure 3), but you could use virtually any notification type or service.

A screenshot of the email service configuration screen in Syndesis.

Figure 3. Configure the email service.

Building the integration pipeline

We’ve configured all the microservices we need for our orchestrated email alerts. Our next step will be to ingrate the GamePrice, UserPreference, GamePrice CDC, and Email services into a new pipeline. These services are shown in Figure 4.

A screenshot of available microservices in the Syndesis integration dashboard.

Figure 4. Microservices displayed in the Syndesis integration .

Our completed integration pipeline will look like the diagram in Figure 5.

A diagram of the integration pipeline.

Figure 5. The completed integration pipeline.

Let’s get started!

Step 1: Create the integration pipeline

We’ll start by selecting the Debezium connector as a source. We select the Kafka topic we need to listen to and a special Kafka topic as shown in Figure 6. Syndesis will use the events produced in the first topic in order to pick the GamePricedrop. It will use the second topic to analyze the event data structure and allow the integrator to map that data to each step in the integration.

A screenshot of the new-integration screen in Syndesis.

Figure 6. Create the integration pipeline.

Step 2: Add steps and connections

Next, we begin filling in our pipeline as shown in Figure 7. As a destination, we use a conditional flow that filters for price updates. Notice that you can add other conditions based on the complexity of your integration. We’ll also add a logging action to trace the results of each execution.

A screenshot of the Syndesis screen for adding new steps and connections to the integration pipeline.

Figure 7. Add steps and connections to the integration pipeline.

Step 3: Retrieve user preferences

Next, we call the UserPreference API to retrieve the list of users potentially interested in a particular offer as shown in Figure 8. We’ll use this list to send emails informing them of promotions and giveaways.

A screenshot showing the mapping between game_id and gameLiked in the UserPreference API.

Figure 8. Retrieve user preferences.

Before we continue to the final steps, let’s look at the overall flow of the pipeline, as shown in Figure 9.

A screenshot showing each phase and action in the pipeline.

Figure 9. The integration pipeline from end to end.

In the first step (1 in figure 9), a change in price starts the flow. The next step is the basic filter (2 in figure 9), which we’ve configured to notify us of events where a price has dropped to zero (price=0). Next, we go to the UserPreference API (4 in figure 9), where we select the endpoint and retrieve a list of users likely to be interested in the specified event. Notice that in the UserPreference API, we also had to do some data mapping (3 in figure 9), which we did visually thanks to a feature provided by AtlasMap.

Step 4: Split the data

Next, we continue to (5 in figure 9), where we split the list of users obtained in (4 in figure 9). We need to split data in order to treat each user separately as we want to personalize email message.

Step 5: Customize the message

Now, we are almost ready to send our selected users a promotional invitation. But before we do, let’s use the template feature (7 in figure 9) to personalize our message, as shown in Figure 10.

A screenshot of the template screen in Syndesis.

Figure 10. Templates in the Syndesis pipeline.

After we’ve uploaded our custom template, we return to the Data Mapper (8 in figure 9) to define a new mapping that will generate a personalized message for each user. (This configuration is similar to what we did in (3 in figure 9).)

Step 6: Send the email

Finally, we are ready to send the email, which combines our personalized message with the email addresses we pulled from the UserPreference service. Figure 11 shows the result of a live execution.

Screenshot of an email for 'Game giveaway!'

Figure 11. A personalized email sent from the new microservice.

The code and steps to run this example are available in my GitHub repository.

Conclusion

You’ve seen in this article how to use Syndesis to integrate two existing microservices into a new, personalized service for added business value–and you’ve seen that it’s possible to do it without writing a single line of code. Using Syndesis, a business expert without programming experience could discover microservices in a services dashboard, compose a pipeline based on one or more of those existing services, correlate data from the services, and build the integration.

All that’s left to do is to start dreaming up integrations that will bring added value to your business.

Share