Persistence vs. Durability in Messaging. Do you know the difference?

Messaging is a critical aspect of integrating systems, and while there are many different messaging platforms and infrastructures, a common request is for “zero loss of messages.”  From there, the terms “Persistence” and “Durability” often get thrown around, but what do those two things really mean?

Persistence

At a basic level, persistence means that when failure occurs during message processing, the message will still be there (where you found it the first time) to process again once the failure is resolved.

Take JBoss Active MQ for example. In AMQ we have brokers that do the communicating of the messages. For simplicity’s sake, let’s assume we only have a single broker doing the communication to and from a queue. Should this broker be shut down while a message is in the queue, ready to be processed, once the broker comes back up the message will be processed normally.

So how does this work?  In order for messages to “persist” they must be stored somewhere other than just broker memory.  Depending on the platform this could be a temporary folder, a database, a log file, etc.

Now, why would anyone not use persistent messaging?  Well, for one thing it tends slows things down. Otherwise, maybe some messages are okay to lose in the event of a broker shutting down, and it’s not worth the complexity. 

If you think of messaging in the context of status checking, the system may want to periodically ensure that a device is up and running. The device send a status message every few minutes. In the event of a broker restart without using persistent messaging we may lose 1 status message, but that might not be a problem since another message is probably on its way — in this case some data-loss may be acceptable.

Durability

Queues and Topics are important parts of messaging (particularly JMS). A queue by itself is great for point-to-point messaging, often one producer to one consumer. Topics on the other hand are most often used when you have a single producer (or multiple for the same purpose) and many consumers.

A common pattern is to have the producer send the message to the topic and then have the queues subscribe to the topic. This allows each queue to receive its own copy of the message. But what happens to the message if it is sent to the topic, but no queues are online (remember our queues subscribe to the topic)?

This is where durability comes into play. When a durable subscription is set up between a queue and a topic, the queue can be offline when the message hits the topic. Once the queue comes back online, the message can be received.

If the subscription is non-durable, then any messages received to the topic while the topic subscriber is offline will not be received by the subscriber (in this case the queue).

Preventing Message Loss

So what do we need to prevent message loss?  If you are using both queues and topics, then using both persistent messaging and durable subscriptions is your best bet. This will ensure you have a back up of the message in case of broker failure and that your subscriptions will always receive the proper messages. Just remember that certain messaging systems, such as Amazon’s SQS and SNS, may not support durable subscriptions.


Join the Red Hat Developer Program (it’s free) and get access to related cheat sheets, books, and product downloads.

Share
  • Why would there be a choice for a non durable subscription?

    When one sets up a subscription, isn’t the expectation that it will be durable (except for possible loss/corruption)? “Durable subscription” seems redundant.

    • Mary Cochran

      Not necessarily. Even having the possibility of durable subscriptions is dependent on the messaging technology being used. For example if I want to use Amazon AWS’s SNS and SQS as mentioned above I do not have the option for a durable subscription.

      Think about the situation where you have a Topic T1 and a Queue Q1 subscribed to T1. Now, someone comes in who doesn’t know your system and accidentally unsubscribes Q1 from T1. In a durable subscription Q1 will still be able to receive the messages from the time it wasn’t subscribed to T1 once it gets back attached. In a non-durable subscription such as those with AWS any messages sent to T1 during this disconnected time will never get to Q1 and if there were no other subscriptions at the time T1 received the message than the message is lost forever.

      If you are confident your system settings will never get touched, a region won’t go down, etc then I supposed your original subscriptions set ups “in theory” would always do their jobs.