Scaling AMQ 7 Brokers with AMQ Interconnect

Red Hat JBoss AMQ Interconnect provides flexible routing of messages between AMQP-enabled endpoints, including clients, brokers, and standalone services. With a single connection to a network of AMQ Interconnect routers, a client can exchange messages with any other endpoint connected to the network.

AMQ Interconnect can create various topologies to manage a high volume of traffic or define an elastic network in front of AMQ 7 brokers. This article shows a sample AMQ Interconnect topology for scaling AMQ 7 brokers easily.

AMQ Interconnect does not use master-slave clusters for high availability. It is typically deployed in topologies of multiple routers with redundant network paths, which it uses to provide reliable connectivity. AMQ Interconnect can distribute messaging workloads across the network and achieve new levels of scale with very low latency.

The router accepts AMQP protocol–based connections from clients and creates AMQP connections to brokers or AMQP services. The router classifies incoming AMQP messages and routes the messages between message producers and message consumers.

A messaging client can make a single AMQP connection into a messaging bus built with routers, and over that connection it can exchange messages with one or more message brokers connected to any router in the network. At the same time, the client can exchange messages directly with other endpoints without involving a broker at all.

Topologies

AMQ Interconnect will help create different topologies to manage a high volume of traffic or define an elastic network in front of AMQ 7 brokers. Some of these topologies could be:

Connection aggregation: Clients will manage a lot of connections; however, the routers will manage fewer connections to the brokers deployed in the back end. Routers will manage a huge amount of connections instead of the final broker instances.
Elastic network: This topology will help us to modify the AMQ 7 broker instances topology without any impact to the clients. We could add or remove more brokers and the clients will manage the same connections.

In general, these topology use cases could provide the following features:

Ability to handle large volume and high throughput
Ability to scale up the broker vertically through interconnect routers
Ability to automatically load balance traffic among shards of brokers
Freedom to add or remove shards without clients noticing

Proposed Topology

This article will describe a topology sample to show some of the AMQ Interconnect features. This diagram shows the proposed topology:

AMQ Interconnect topology

The routers deployed will be:

Aggregator router: Router to manage any connection from or to AMQ 7 broker instances. This broker will aggregate any connection from the other routers to the AMQ 7 broker instances.
Producer router: Router to manage connections from the producer applications.
Consumer router: Router to manage connections from the consumer applications.

After finalizing the configuration, we can see from an AMQ 7 management web console the topology deployed:

AMQ Interconnect routers deployed

Prerequisites

To deploy and test this topology, you should have at least one AMQ 7 broker deployed and running on some host. If you have more instances, all of them should work in the same cluster definition.

Samples will work with a set of queues that you should define in your AMQ 7 brokers. To do that, please add to address section of the $AMQ_BROKER/etc/broker.xml file the following definitions:

<address name="SampleQueue">
  <anycast>
    <queue name="SampleQueue" />
  </anycast>
</address>

Also, we will need three hosts to deploy each kind of AMQ Interconnect router.

Installation

AMQ Interconnect is distributed as a set of RPM packages, which are available through your Red Hat subscription. Enable the following repositories:

$ sudo subscription-manager repos --enable=amq-interconnect-1-for-rhel-7-server-rpms --enable=a-mq-clients-1-for-rhel-7-server-rpms

Run this command to install the main packages:

$ sudo yum install qpid-dispatch-router qpid-dispatch-tools

Run the router as a service in Red Hat Enterprise Linux 7:

$ sudo systemctl start qdrouterd.service

The AMQ Interconnect router configuration file is located at /etc/qpid-dispatch/qdrouterd.conf. Any configuration will be defined in this file.

We will repeat these steps in each host used to deploy the AMQ Interconnect router.

Aggregator Router

This router will manage the incoming and outgoing messages from other routers to the AMQ 7 HA cluster topology behind it.

Here's the general router definition:

router {
    mode: interior
    id: Router.Aggregator
}

Listeners: Three listeners will be defined to manage the different client connections:

General listener for clients
Inter-router listener for other routers
General listener to attend connections from AMQ 7 management web console

# Listener for clients
listener {
  host: 0.0.0.0
  port: amqp
  authenticatePeer: no
  saslMechanisms: ANONYMOUS
}
 
# Listener for inter-router connections
listener {
  host: 0.0.0.0
  port: 5772
  authenticatePeer: no
  role: inter-router
}
 
# Listener for AMQ console
listener {
  name: amq-console
  role: normal
  host: 0.0.0.0
  port: 5773
  http: yes
}

Connectors: This section will define the connectors for each AMQ 7 broker defined in the back end.

connector {
   name: amq7.master01
   host: amq7master01host
   port: 5672
   role: route-container
   saslMechanisms: ANONYMOUS
}
 
# Definitions for other AMQ 7 brokers
...

Addresses: This sample will define an address for SampleQueue to the different brokers defined in the connector section.

address {
  prefix: SampleQueue
  # Routing messages through a broker queue
  waypoint: yes
}

A waypoint address identifies a queue on a broker to route messages.

Now we define the auto-links between the address and the different back ends. This case represents a sharded queue because there are multiple brokers. Using queue sharding, it is possible to distribute a single queue over multiple brokers.

Auto-links will connect the router to the broker. Auto-links will handle the client traffic on the router, not the broker. Clients attach their links to the router and then the router uses internal auto-links to connect to the queue on the broker. This means that the queue will always have a single producer and consumer regardless of how many clients are attached to the router.

These blocks will define the links to send messages to the brokers from the router:

autoLink { 
  addr: SampleQueue 
  connection: amq7.master01 
  dir: out 
} 

# Definitions for other AMQ 7 brokers 
...

These blocks will define the links to receive messages from the brokers into the router:

autoLink {
  addr: SampleQueue
  connection: amq7.master01
  dir: in
}
 
# Definitions for other AMQ 7 brokers
...

Producer Router

This router will manage the incoming messages from producers to the aggregator router.

Here is the general router definition:

router {
    mode: interior
    id: Router.Producer
}

Listeners: Two listeners will be defined to manage the different client connections:

General listener for clients
Inter-router listener for other routers

# Listener for clients
listener {
  host: 0.0.0.0
  port: amqp
  authenticatePeer: no
  saslMechanisms: ANONYMOUS
}
 
# Listener for inter-router connections
listener {
  host: 0.0.0.0
  port: 5772
  authenticatePeer: no
  role: inter-router
}

Connectors: This section will define the connectors to the aggregator router. Note that we will connect to the inter-router port, not to the AMQP service port.

connector {
  name: Router.Aggregator
  host: amqinterconnectaggregatorhost
  port: 5772
  role: inter-router
}

Addresses: This sample will define an address for SampleQueue to distribute the messages between the different routers in the network.

address {
  prefix: SampleQueue
  distribution: closest
}

This address defines a route pattern based on the closest method, which determines the shortest path based on the topology cost to reach each of the consumers.

Consumer Router

This router will manage the outgoing messages from the aggregator router to the consumer router.

Here is the general router definition:

router {
    mode: interior
    id: Router.Consumer
}

Listeners: Two listeners will be defined to manage the different client connections:

General listener for clients
Inter-router listener for other routers

# Listener for clients
listener {
  host: 0.0.0.0
  port: amqp
  authenticatePeer: no
  saslMechanisms: ANONYMOUS
}
 
# Listener for inter-router connections
listener {
  host: 0.0.0.0
  port: 5772
  authenticatePeer: no
  role: inter-router
}

Connectors: This section will define the connectors to the aggregator router. Note that we will connect to the inter-router port, not to the AMQP service port.

connector {
  name: Router.Aggregator
  host: amqinterconnectaggregatorhost
  port: 5772
  role: inter-router
}

Addresses: This sample will define an address for SampleQueue to distribute the messages between the different routers in the network.

address {
  prefix: SampleQueue
  waypoint: yes
}

Testing

The following GitHub repository includes a set of main Java applications using Apache Qpid JMS dependencies to send and receive messages using the AMQP protocol. After building and packaging the applications, you can execute the samples and test how the messages are routed from the AMQ Interconnect routers to and from the AMQ 7 brokers.

The sender application from the publish/subscribe sample will send 10 messages to the producer router, which will route the messages to the aggregator route, and the messages will be stored at the AMQ 7 broker.

The sender application will output the following:

$ java -DUSER=application -DPASSWORD=application -cp "target/classes/:target/dependency/*" org.apache.qpid.jms.example.pubsub.Sender
Sending up to 10 messages.
Specify a message count as the program argument if you wish to send a different amount.
Sender connected to amqp://producerrouterhost:5672
Sent message 10
Sent 10 messages in 314ms

The receiver application from the same sample will consume 10 messages from the consumer router. The aggregator router will get the messages from the AMQ 7 broker, and it will route them to the consumer router.

The receiver application will output this:

$ java -DUSER=application -DPASSWORD=application -cp "target/classes/:target/dependency/rg.apache.qpid.jms.example.pubsub.Receiver
Consuming up to 10 messages.
Specify a message count as the program argument if you wish to consume a different amount.
Receiver connected to amqp://consumerroutehost:5672
Got message 1. Body: Text message.
Got message 2. Body: Text message.
Got message 3. Body: Text message.
Got message 4. Body: Text message.
Got message 5. Body: Text message.
Got message 6. Body: Text message.
Got message 7. Body: Text message.
Got message 8. Body: Text message.
Got message 9. Body: Text message.
Got message 10. Body: Text message.
Received 10 messages in 339ms

During the execution of these applications, you will see the following in the AMQ 7 management console:

Conclusion

This article shows you a sample AMQ Interconnect topology for scaling your AMQ 7 brokers easily, but it is not the only topology. AMQ Interconnect allows you to design and deploy other router topologies to scale up your AMQ 7 brokers. These topologies could cover other use cases such as:

Complex internal infrastructures: AMQ Interconnect routers to forward messages to assigned brokers
Secured brokers, firewall restrictions: AMQ Interconnect routers as a reverse proxy in a DMZ
Instance data feed, information integration: AMQ Interconnect routers send messages directly to a receiving application service
Distributed data centers, cross-timezone branches: AMQ Interconnect routers to communicate with brokers and applications across the World Wide Web

With all this flexibility, your use case can easily be accommodated by using the messaging routing capabilities from the AMQ Interconnect router.

The following links provide more details about this router:

Last updated: May 22, 2018

Scaling AMQ 7 Brokers with AMQ Interconnect

Topologies

Proposed Topology

Prerequisites

Installation

Aggregator Router

Producer Router

Consumer Router

Testing

Conclusion

How we added support for the C++23 assume feature in GCC

3 improvements to the OpenShift 4.12 developer experience

4 steps to run an application under OpenShift Service Mesh

Introducing the new Red Hat Enterprise Linux download experience

How to generate code using Fabric8 Kubernetes Client

Comments