Setting Up a Failover Scenario Using Apache Camel ZooKeeper

In this article, we will discuss using the Apache Camel ZooKeeper component, and demonstrate how easily we can set up a fail-over scenario for Apache Camel Routes. While working in a clustered environment, situations arise where a user wants to have a backup or slave route which will become active only when the master (or the currently active) route stops working. There are different ways to achieve this: one can use Quartz to configure a master-slave setup; JGroups can also be used. In a Fabric8 environment, there is a master component which can easily be set up as a failover scenario.

There is one more way to discuss here in detail using ZooKeeperRoutePolicy. This policy uses a common znode (zookeeper registry) path across all instances. Each instance will be registered with a unique-id in the znode path; thus there will be a list of unique-ids. Each unique-id represents one instance of a route deployed with ZooKeeperRoutePolicy route policy.

Whichever node is at the top of the list will be considered as the master route. Other remaining nodes will be dormant and not active. Once the master route stops, one of the slaves would be elected as the master route. Then it will continue to serve requests.

Setting Up The Project and Configuring Camel Zookeeper as the MasterSlave Route

Now let's discuss the coding portion. I have tested this example in Red Hat JBoss Fuse 6.3.0. You can find the code base at my personal github link.

1. Create a Maven project with following structure:

[cpandey@cpandey CamelZookeeperMasterSlave]$ tree
.
├── pom.xml
├── ReadMe.txt
└── src
 └── main
  ├── java
   └── com
    └── test
     └── pckg
      └── MyRouteBuilder.java
  └── resources
   ├── log4j.properties
   └── OSGI-INF
    └── blueprint
     └── blueprint.xml

2. The pom.xml should have following dependencies:

 <dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-core</artifactId>
 </dependency>
 <dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-blueprint</artifactId>
 </dependency>
 <dependency>
 <groupId>org.apache.camel</groupId>
 <artifactId>camel-zookeeper</artifactId>
 </dependency>

3. A version of these artifacts are picked from the BOM version mentioned in pom.xml:

 <dependencyManagement>
 <dependencies>
 <dependency>
 <groupId>org.jboss.fuse.bom</groupId>
 <artifactId>jboss-fuse-parent</artifactId>
 <version>6.3.0.redhat-310</version>
 <type>pom</type>
 <scope>import</scope>
 </dependency>
 </dependencies>
 </dependencyManagement>

4. Here, I have used JAVA DSL for the Camel route. Hence in blueprint.xml, I have defined bean, which refer class com.test.pckg.MyRouteBuilder where camel route is defined.

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v1.0.0 https://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">

 <camelContext id="_context1" xmlns="http://camel.apache.org/schema/blueprint">
  <routeBuilder ref="myBuilder"/>
 </camelContext>
 <bean class="com.test.pckg.MyRouteBuilder" id="myBuilder"/>
</blueprint>

5. Following the camel route defined in MyRouteBuilder.java</code>:

@Override
public void configure() {
System.out.println("......Inside MyRouteBuilder.....");
ZooKeeperRoutePolicy policy = new ZooKeeperRoutePolicy("zookeeper:localhost:" + 2181 + "/someapp/somepolicy", 1);
from("file:/path/to/someFolder").routePolicy(policy).id("Master").log("Body... ${body} "+new Date());
}

• Define a ZooKeeperRoutePolicy(String uri, int enabledCount) which creates znode at path /someapp/somepolicy. ZooKeeper Server is listening at localhost:2181.
• We want to have one instance active at a time. For a Master-slave scenario, the route is configured with 1 route instance and only the first entry in the listing will start its route. However, if there is a scenario when we need two or more route instances to be up at the same time, we can set enabledCount to 2 or more.
• We then have a file based consumer polling at path/to/someFolder where routePolicy is set to ZooKeeperRoutePolicy.  Finally, we log the body which is set with the content of the file polled or consumed.

How to Deploy in Red Hat JBoss Fuse with a Fabric8 Environment

1. Fabric8 uses Apache ZooKeeper (which is a highly reliable distributed coordination service) as its registry to store the cluster configuration and node registration. Hence, we don't need to setup a different ZooKeeper server. We can register znode in an existing ZooKeeper registry and use it to set up MasterSlave route.

2. Follow the steps below to deploy this application from Karaf terminal of Red Hat JBoss Fuse:

profile-edit --feature fabric-zookeeper-commands default
container-create-child root abc
container-create-child root pqr 
profile-create testZK
profile-edit --feature camel-zookeeper --feature camel-blueprint testZK
profile-edit --bundle mvn:com.mycompany/camel-zookeeper-example/1.0.0-SNAPSHOT testZK
container-add-profile abc testZK
container-add-profile pqr testZK

3. As per our route, at location /path/to/someFolder, create a simple text file and add some text data to it and save it.

4. In logs for one of the nodes we can check following; which will ensure that  znode is created and that the route is active:

2018-03-25 17:30:30,075 | WARN | masterSlaveRoute | ZookeeperProducer | 191 - org.apache.camel.camel-core - 2.17.0.redhat-630329 | Node '/someapp/somepolicy/localhost.localdomain-cd65c741-4252-420a-88a4-f084a933fcaf' did not exist, creating it.
2018-03-25 17:30:30,080 | INFO | masterSlaveRoute | ZooKeeperElection | 195 - org.apache.camel.camel-zookeeper - 2.17.0.redhat-630329 | Candidate node '/someapp/somepolicy/localhost.localdomain-cd65c741-4252-420a-88a4-f084a933fcaf' has been created
2018-03-25 17:30:30,107 | INFO | masterSlaveRoute | BlueprintCamelContext | 191 - org.apache.camel.camel-core - 2.17.0.redhat-630329 | Route: election-route-localhost.localdomain-cd65c741-4252-420a-88a4-f084a933fcaf started and consuming from: Endpoint[zookeeper://localhost:2181/someapp/somepolicy]
2018-03-25 17:30:30,112 | INFO | masterSlaveRoute | Master | 191 - org.apache.camel.camel-core - 2.17.0.redhat-630329 | Body... Sun Mar 25 17:27:32 IST 2018

4. Using the following command from Karaf terminal, we can easily identify which nodes are registered. This unique number we can be identified in logs also:

zk:list -r /someapp/somepolicy
#Output we get is like
/someapp/somepolicy/localhost.localdomain-cd65c741-4252-420a-88a4-f084a933fcaf0000000013

5. If the route or application is stopped, this znode entry will be deleted.

 How to Deploy in Red Hat JBoss Fuse in a Standalone Environment

1. Download ZooKeepers latest release from here.  I downloaded zookeeper-3.4.11.

2. Extract zookeeper-3.4.11.tar.gz. Now go to zookeeper-3.4.11/conf folder:

$ vi conf/zoo.cfg

tickTime = 2000
initLimit = 10
syncLimit = 5
dataDir = /path/to/zookeeper/data
clientPort = 2181

3.  Finally, start the ZooKeeper server:

$ bin/zkServer.sh start

4. Now on Red Hat JBoss Fuse 6.3.0 Karaf terminal, run the following commands:

features:install camel-zookeeper
osgi:install -s mvn:com.mycompany/camel-zookeeper-example/1.0.0-SNAPSHOT

5.  As per our route, at location /path/to/someFolder, create a simple text file and add some text data to it and save it. We will get similar logs (as mentioned above) for deployment in the Fabric8 environment. Here we can have multiple Red Hat JBoss Fuse standalone environments running at different nodes having the same application deployment.

6. If deployment and testing are done, you can stop the ZooKeeper server with the following command:

bin/zkServer.sh stop

That's it. I hope it helps you to to set up a master-slave scenario with camel-routes. This might help you to achieve a successful failover scenario.