In this post, I wanted to address how to configure mKahaDB persistence storage on ActiveMQ for better management and reducing disk usage.
Default configured KahaDB persistence adapter works well when all the destinations (queues/topics) being managed by the broker have similar performance. However, an enterprise solution where several third parties are involved is never the case.
There are multiple queues or topics and different consumers or listeners listening to these queues/topics. Some consumers might be slower than other consumers. This will grow the message store's disk usage rapidly. Due to this situation and being single KahaDB all store destinations might perform slow.
The multi KahaDB (mKahaDB) persistence adapter allows us to have a broker's queues/topics across multiple KahaDB message stores. So, we can assign the different KahaDB store to different queues thus reducing the impact of slow performing destination (because of slow consumers) on normal or fast processing destinations. We can assign a separate KahaDB store to slow the performing destination with help of mKahaDB.
Let's do a practice session.
1. We are using Red Hat JBoss Fuse/AMQ 6.3.0 for this practice session. The same should apply to all Red Hat JBoss/AMQ 6.x version or ActiveMq 5.x versions.
2. The broker's configuration file activemq.xml exists in the etc folder of Red Hat JBoss Fuse/AMQ 6.3.0, we will be adding or modifying configurations to this file.
3. Let's set memoryLimit of queues to a higher value i.e. 100 MB. So that we can send more messages of 1mb size to the broker.
<broker xmlns=.... > --- <destinationPolicy> <policyMap> <policyEntries> <policyEntry topic=">" producerFlowControl="true"> <pendingMessageLimitStrategy> <constantPendingMessageLimitStrategy limit="1000"/> </pendingMessageLimitStrategy> </policyEntry> <policyEntry queue=">" producerFlowControl="true" memoryLimit="100mb"> </policyEntry> </policyEntries> </policyMap> </destinationPolicy> --- </broker>
Note: We have only configured memoryLimit="100mb".
4. Now we will comment default KahaDB store configuration and replace with mKahaDB configurations.
<broker xmlns=.... >
---
<persistenceAdapter>
<!-- <kahaDB directory="${data}/kahadb"/> -->
<mKahaDB directory="${data}/kahadb">
<filteredPersistenceAdapters>
<filteredKahaDB queue="A.*.C">
<persistenceAdapter>
<kahaDB/>
</persistenceAdapter>
</filteredKahaDB>
<filteredKahaDB queue="Q.>">
<persistenceAdapter>
<kahaDB/>
</persistenceAdapter>
</filteredKahaDB>
<filteredKahaDB queue="Test.*">
<persistenceAdapter>
<kahaDB/>
</persistenceAdapter>
</filteredKahaDB>
<filteredKahaDB queue="X.Y.Z">
<persistenceAdapter>
<kahaDB/>
</persistenceAdapter>
</filteredKahaDB>
<!-- match all destinations -->
<filteredKahaDB>
<persistenceAdapter>
<kahaDB/>
</persistenceAdapter>
</filteredKahaDB>
</filteredPersistenceAdapters>
</mKahaDB>
</persistenceAdapter>
---
</broker>
5. Start the broker.
6. Now if we check the location jboss-fuse-6.3.0.redhat-310/data/amq/kahadb, we will get the following directories.
[cpandey@cpandey kahadb]$ ls -ltr
total 28
drwxrwxr-x. 2 cpandey cpandey 4096 Nov 18 07:05 #210
drwxrwxr-x. 2 cpandey cpandey 4096 Nov 18 07:05 txStore
-rw-rw-r--. 1 cpandey cpandey 8 Nov 18 08:20 lock
drwxrwxr-x. 2 cpandey cpandey 4096 Nov 18 08:20 queue#3a#2f#2fA.#2a.C
drwxrwxr-x. 2 cpandey cpandey 4096 Nov 18 08:20 queue#3a#2f#2fQ.#3e
drwxrwxr-x. 2 cpandey cpandey 4096 Nov 18 08:20 queue#3a#2f#2fTest.#2a
drwxrwxr-x. 2 cpandey cpandey 4096 Nov 18 08:20 queue#3a#2f#2fX.Y.ZPoints to note here:
- Queue.* matches one word after prefix Queue for destination name.
- Queue.> all queues prefixed with Queue.
- txStore: This is an implementation detail of mKahaDB. It contains transactions that involve different KahaDB instances.
- #210: This KahaDB store is for all expect the queues with specific/filtered KahaDB store.
- queue#3a#2f#2fA.#2a.C: This KahaDB store is for queues with queue_name as expression A.*.C
- queue#3a#2f#2fQ.#3e: This KahaDB store is for all queues starting with prefix 'Q.>'. Where '>' stands for any string.
- queue#3a#2f#2fTest.#2a: This queue is specific for all queues with expression Test.*.
- queue#3a#2f#2fX.Y.Z: This queue is specific for X.Y.Z queue.
7. mKahaDB is an extension of KahaDB, hence KahaDB tuning parameters applies for mKahaDB too. Just make sure for each filter, we would have to apply tuning parameters as each behaves as a separate KahaDB store. More details at activemq documentation.
8. We can use java code to send a message. But here I am using Red Hat JBoss Fuse/AMQ Karaf terminal to send a message.
Scenario 1: KahaDB store for one specific queue. Now let use send data to X.Y.Z queue.
# If we check the file created in KahaDB store "queue#3a#2f#2fX.Y.Z", we see journal files.
# At present no messages.
JBossFuse:karaf@root dstat queues
Name Queue Size Producer # Consumer # Enqueue # Dequeue # Forward # Memory %
A.B.C 0 0 0 0 0 0 0
A.N.C 0 0 0 0 0 0 0
BAR 0 0 0 0 0 0 0
FOO 0 0 0 0 0 0 0
Q.1 0 0 0 0 0 0 0
Q.1.2 0 0 0 0 0 0 0
Q.T 0 0 0 0 0 0 0
Test.1 0 0 0 0 0 0 0
Test.2 0 0 0 0 0 0 0
Test.2.3 0 0 0 0 0 0 0
JBossFuse:karaf@root activemq:producer --user admin --password admin --destination X.Y.Z --messageCount 35 --messageSize 1000000
JBossFuse:karaf@root dstat queues
Name Queue Size Producer # Consumer # Enqueue # Dequeue # Forward # Memory %
A.B.C 0 0 0 0 0 0 0
A.N.C 0 0 0 0 0 0 0
BAR 0 0 0 0 0 0 0
FOO 0 0 0 0 0 0 0
Q.1 0 0 0 0 0 0 0
Q.1.2 0 0 0 0 0 0 0
Q.T 0 0 0 0 0 0 0
Test.1 0 0 0 0 0 0 0
Test.2 0 0 0 0 0 0 0
Test.2.3 0 0 0 0 0 0 0
X.Y.Z 35 0 0 35 0 0 33
JBossFuse:karaf@root
[cpandey@cpandey queue#3a#2f#2fX.Y.Z]$ ls -ltrh
total 34M
-rw-rw-r--. 1 cpandey cpandey 33K Nov 18 09:53 db.redo
-rw-rw-r--. 1 cpandey cpandey 32K Nov 18 09:53 db.data
-rw-rw-r--. 1 cpandey cpandey 32M Nov 18 09:53 db-1.log
-rw-rw-r--. 1 cpandey cpandey 32M Nov 18 09:53 db-2.log
# Now we will purge queue X.Y.Z so that Memory is released, and we can continue test other scenarios.
JBossFuse:karaf@root activemq:purge X.Y.Z
INFO: Purging all messages in queue: X.Y.ZScenario 2: KahaDB store for queues with some common expression. In this case, A.*.C is the expression for queue names.
# Now let's send data to queue A.B.C.
JBossFuse:karaf@root activemq:producer --user admin --password admin --destination A.B.C --messageCount 35 --messageSize 1000000
JBossFuse:karaf@root dstat queues
Name Queue Size Producer # Consumer # Enqueue # Dequeue # Forward # Memory %
A.B.C 35 0 0 35 0 0 33
A.N.C 0 0 0 0 0 0 0
BAR 0 0 0 0 0 0 0
FOO 0 0 0 0 0 0 0
Q.1 0 0 0 0 0 0 0
Q.1.2 0 0 0 0 0 0 0
Q.T 0 0 0 0 0 0 0
Test.1 0 0 0 0 0 0 0
Test.2 0 0 0 0 0 0 0
Test.2.3 0 0 0 0 0 0 0
X.Y.Z 0 0 0 35 35 0 0
JBossFuse:karaf@root
# Let us check KahaDB store "queue#3a#2f#2fA.#2a.C".
[cpandey@cpandey kahadb]$ cd "queue#3a#2f#2fA.#2a.C"
[cpandey@cpandey queue#3a#2f#2fA.#2a.C]$ ls -ltr
total 34376
-rw-rw-r--. 1 cpandey cpandey 33554432 Nov 18 10:01 db-1.log
-rw-rw-r--. 1 cpandey cpandey 33554432 Nov 18 10:04 db-2.log
-rw-rw-r--. 1 cpandey cpandey 32824 Nov 18 10:04 db.redo
-rw-rw-r--. 1 cpandey cpandey 32768 Nov 18 10:04 db.data
[cpandey@cpandey queue#3a#2f#2fA.#2a.C]$
#Now we will send data to queue A.N.C
JBossFuse:karaf@root activemq:producer --user admin --password admin --destination A.N.C --messageCount 35 --messageSize 1000000
JBossFuse:karaf@root dstat queues
Name Queue Size Producer # Consumer # Enqueue # Dequeue # Forward # Memory %
A.B.C 35 0 0 35 0 0 33
A.N.C 35 0 0 35 0 0 33
BAR 0 0 0 0 0 0 0
FOO 0 0 0 0 0 0 0
Q.1 0 0 0 0 0 0 0
Q.1.2 0 0 0 0 0 0 0
Q.T 0 0 0 0 0 0 0
Test.1 0 0 0 0 0 0 0
Test.2 0 0 0 0 0 0 0
Test.2.3 0 0 0 0 0 0 0
X.Y.Z 0 0 0 35 35 0 0
JBossFuse:karaf@root
# If we again observe datastore "queue#3a#2f#2fA.#2a.C", we find that a new journal file is created db-3.log hence content of queue A.N.C is also persisted to this datastore.
[cpandey@cpandey queue#3a#2f#2fA.#2a.C]$ ls -ltr
total 68608
-rw-rw-r--. 1 cpandey cpandey 33554432 Nov 18 10:01 db-1.log
-rw-rw-r--. 1 cpandey cpandey 33554432 Nov 18 10:08 db-2.log
-rw-rw-r--. 1 cpandey cpandey 33554432 Nov 18 10:09 db-3.log
-rw-rw-r--. 1 cpandey cpandey 32824 Nov 18 10:09 db.redo
-rw-rw-r--. 1 cpandey cpandey 53248 Nov 18 10:09 db.data
[cpandey@cpandey queue#3a#2f#2fA.#2a.C]$
# Now we will purge queue A.B.C and A.N.C so that Memory is released, and we can continue test other scenarios.
JBossFuse:karaf@root activemq:purge A.B.C A.N.C
INFO: Purging all messages in queue: A.N.C
INFO: Purging all messages in queue: A.B.CScenario 3: KahaDB store for all queues with the same prefix. Here we have prefix 'Q.>'.
# Sending message to queue Q.1.
JBossFuse:karaf@root activemq:producer --user admin --password admin --destination Q.1 --messageCount 35 --messageSize 1000000
JBossFuse:karaf@root dstat queues
Name Queue Size Producer # Consumer # Enqueue # Dequeue # Forward # Memory %
A.B.C 0 0 0 0 0 0 0
A.N.C 0 0 0 0 0 0 0
BAR 0 0 0 0 0 0 0
FOO 0 0 0 0 0 0 0
Q.1 35 0 0 35 0 0 33
Q.1.2 0 0 0 0 0 0 0
Q.T 0 0 0 0 0 0 0
Test.1 0 0 0 0 0 0 0
Test.2 0 0 0 0 0 0 0
Test.2.3 0 0 0 0 0 0 0
X.Y.Z 0 0 0 0 0 0 0
JBossFuse:karaf@root
# We see that in folder 'queue#3a#2f#2fQ.#3e' which is journal store for queues with prefix 'Q.' is having db-1.log and db-2.log journals.
[cpandey@cpandey queue#3a#2f#2fQ.#3e]$ ls -ltrh
total 34M
-rw-rw-r--. 1 cpandey cpandey 32M Nov 18 20:37 db-1.log
-rw-rw-r--. 1 cpandey cpandey 32M Nov 18 20:39 db-2.log
-rw-rw-r--. 1 cpandey cpandey 73K Nov 18 20:39 db.redo
-rw-rw-r--. 1 cpandey cpandey 72K Nov 18 20:39 db.data
[cpandey@cpandey queue#3a#2f#2fQ.#3e]$
# Now let us send data to queue Q.1.T and Q.T
JBossFuse:karaf@root activemq:producer --user admin --password admin --destination Q.1.T --messageCount 35 --messageSize 1000000
JBossFuse:karaf@root dstat queues
Name Queue Size Producer # Consumer # Enqueue # Dequeue # Forward # Memory %
A.B.C 0 0 0 0 0 0 0
A.N.C 0 0 0 0 0 0 0
BAR 0 0 0 0 0 0 0
FOO 0 0 0 0 0 0 0
Q.1 35 0 0 35 0 0 33
Q.1.T 35 0 0 35 0 0 33
Q.T 0 0 0 0 0 0 0
Test.1 0 0 0 0 0 0 0
Test.2 0 0 0 0 0 0 0
Test.2.3 0 0 0 0 0 0 0
X.Y.Z 0 0 0 0 0 0 0
JBossFuse:karaf@root
[cpandey@cpandey queue#3a#2f#2fQ.#3e]$ ls -ltrh
total 68M
-rw-rw-r--. 1 cpandey cpandey 32M Nov 18 20:37 db-1.log
-rw-rw-r--. 1 cpandey cpandey 32M Nov 18 20:41 db-2.log
-rw-rw-r--. 1 cpandey cpandey 32M Nov 18 20:43 db-3.log
-rw-rw-r--. 1 cpandey cpandey 73K Nov 18 20:43 db.redo
-rw-rw-r--. 1 cpandey cpandey 92K Nov 18 20:43 db.data
JBossFuse:karaf@root activemq:producer --user admin --password admin --destination Q.T --messageCount 35 --messageSize 1000000
[cpandey@cpandey queue#3a#2f#2fQ.#3e]$ ls -ltrh
total 101M
-rw-rw-r--. 1 cpandey cpandey 32M Nov 18 20:37 db-1.log
-rw-rw-r--. 1 cpandey cpandey 32M Nov 18 20:41 db-2.log
-rw-rw-r--. 1 cpandey cpandey 73K Nov 18 20:43 db.redo
-rw-rw-r--. 1 cpandey cpandey 92K Nov 18 20:43 db.data
-rw-rw-r--. 1 cpandey cpandey 32M Nov 18 20:43 db-3.log
-rw-rw-r--. 1 cpandey cpandey 32M Nov 18 20:43 db-4.log
[cpandey@cpandey queue#3a#2f#2fQ.#3e]$
# Above we observe that db journal files has been created in same KahaDB store. Just all queue with prefix 'Q.'
JBossFuse:karaf@root activemq:purge Q.1 Q.1.T Q.T
INFO: Purging all messages in queue: Q.1
INFO: Purging all messages in queue: Q.1.T
INFO: Purging all messages in queue: Q.TScenario 4: When messages are sent to any other queues not mentioned in the filters. We will observe here that all the non-filtered messages will go to one different KahaDB store (i.e all within #210).
JBossFuse:karaf@root activemq:producer --user admin --password admin --messageCount 35 --messageSize 1000000 --destination FOO
JBossFuse:karaf@root activemq:dstat queues
Name Queue Size Producer # Consumer # Enqueue # Dequeue # Forward # Memory %
A.B.C 0 0 0 0 0 0 0
A.N.C 0 0 0 0 0 0 0
BAR 0 0 0 0 0 0 0
FOO 35 0 0 35 0 0 33
Q.1 0 0 0 0 0 0 0
Q.1.2 0 0 0 0 0 0 0
Q.1.T 0 0 0 0 0 0 0
Q.T 0 0 0 0 0 0 0
Test.1 0 0 0 0 0 0 0
Test.2 0 0 0 0 0 0 0
Test.2.3 0 0 0 0 0 0 0
X.Y.Z 0 0 0 0 0 0 0
JBossFuse:karaf@root
[cpandey@cpandey #210]$ ls -ltr
total 34836
-rw-rw-r--. 1 cpandey cpandey 33554432 Nov 18 23:09 db-1.log
-rw-rw-r--. 1 cpandey cpandey 33554432 Nov 18 23:12 db-2.log
-rw-rw-r--. 1 cpandey cpandey 32824 Nov 18 23:12 db.redo
-rw-rw-r--. 1 cpandey cpandey 217088 Nov 18 23:12 db.data
[cpandey@cpandey #210]$
JBossFuse:karaf@root activemq:producer --user admin --password admin --messageCount 35 --messageSize 1000000 --destination BAR
JBossFuse:karaf@root activemq:dstat queues
Name Queue Size Producer # Consumer # Enqueue # Dequeue # Forward # Memory %
A.B.C 0 0 0 0 0 0 0
A.N.C 0 0 0 0 0 0 0
BAR 35 0 0 35 0 0 33
FOO 35 0 0 35 0 0 33
Q.1 0 0 0 0 0 0 0
Q.1.2 0 0 0 0 0 0 0
Q.1.T 0 0 0 0 0 0 0
Q.T 0 0 0 0 0 0 0
Test.1 0 0 0 0 0 0 0
Test.2 0 0 0 0 0 0 0
Test.2.3 0 0 0 0 0 0 0
X.Y.Z 0 0 0 0 0 0 0
JBossFuse:karaf@root
[cpandey@cpandey #210]$ ls -ltr
total 69044
-rw-rw-r--. 1 cpandey cpandey 33554432 Nov 18 23:09 db-1.log
-rw-rw-r--. 1 cpandey cpandey 33554432 Nov 18 23:14 db-2.log
-rw-rw-r--. 1 cpandey cpandey 33554432 Nov 18 23:14 db-3.log
-rw-rw-r--. 1 cpandey cpandey 32824 Nov 18 23:14 db.redo
-rw-rw-r--. 1 cpandey cpandey 217088 Nov 18 23:14 db.data
[cpandey@cpandey #210]$I hope this article will provide a better understanding of mKahaDB and its implementation and benefits.
Join the Red Hat Developer Program (it’s free) and get access to related cheat sheets, books, and product downloads.
Last updated: November 27, 2017