ActiveMQ: Topics vs. Queues

by jmorris 9. February 2011 13:47

The JMS specification defines two different ways of defining message destinations: Queues and Topics. Understanding the behavior of each is vital to implementing messaging within a system and fulfilling any expectations of message availability.

First a quick aside, the purpose of this post is mostly to keep myself from committing the mistakes I made today based upon invalid assumptions regarding the availability messages sent to a queue within an environment with multiple listeners. In a nutshell, I was writing a test to queue up a single message and I have a listener within the test receive the message.  For some reason the message was never being received…baffling given the simplicity of the test and the maturity of the code.

Here is the test (ignore that the test is not well written, I just want to quickly test that the code works while I am developing):


I edited this code a bit for simplicity, but in essence a message should be pumped to the console and displayed (real test validates that message was received after x seconds delay). The only problem was I that I was getting nada…note that I am not testing the queuing API itself, I am testing that the strategy will push a message onto the queue without throwing an exception. It’s basically a throwaway test to prove that it works as expected in an integrated environment.

The problem was that the message was never being received by the listener (QueueReceiver is the listener), however other similar tests that are messaging API specific were working as expected. Running through the usual checks turned up nothing. The code simple didn’t work…but only for this test! Swapping out for another queue worked perfectly :(

Fool me once, shame on you. Fool me twice…

So, what was causing the problem? Well it all became very clear once I went back and reflected on the basics of JMS messaging and the differences in how messages are delivered to queues and topics.

The JMS specification supports two models for sending messages: point-to-point for queues and publish and subscribe for topics. In a nutshell, only one consumer of a queue will receive a messaging regardless of the number of consumers; round-robin message dispersal. Whereas a topic broadcasts a copy of the message to every registered consumer.

It turns out out that I had a windows service running that was a consumer of the same queue and It was the first listener, so it always got the first message. The unit test never received a copy because only one message was sent and the consumer on the windows service always pulled it off. This was proved by simply calling publish twice:


This consumer (_receiver) got the second message, because the first one was consumed by the windows service and the next round-robin listener was this unit test. ARGGGHHHH, what a waste of time and brain power!

Lessons learnt:

  1. Don’t write unit tests with external dependencies and assume they will behave as expected
  2. Don’t use shared queues for testing
  3. Don’t forget the difference between queues and topics



Apache.NMS, JMS | rant

ActiveMQ via C# using Apache.NMS Part 2 - Queues

by jmorris 29. July 2010 13:35

ActiveMQ via C# Part 2
In my part 1 of this series, I discussed the basics of JMS and messaging schematics: publish/subscribe and sender/receiver. I introduced Apache.NMS and the ActiveMQ client for NMS, Apache.NMS.ActiveMQ. I created client classes for publishing messages to ActiveMQ via topics and subscribing to a topic to receive messages. Finally, I tested the classes by writing a unit test that contained a publisher that sent messages to a topic and a subscriber that wrote the output to the console. In this example, I will expand on C# client by writing a simple, reusable configuration based API for sending and receiving messages using ActiveMQ and Apache.NMS.

In situations where you have multiple projects in need of a common messaging infrastructure, relatively low messaging domain knowledge across teams and an integration requirement between .NET and other Java based systems, a simple wrapper over a more complex JMS/NMS API can quicken client development. Additionally, the JMS/NMS programming model lends itself to a configuration based API that utilizes factory methods to create configured objects for sending and receiving objects.

The benefit of going with this model is that users of the API can get up and running quickly and need very little understanding of the nuances of JMS/NMS. Also, APIs like this make it easier to manage deployments amongst the different software environments in most organizations: dev, staging, prod, etc.

More Messaging Schematics
JMS provides two major schemes for sending and receiving messages: point-to-point (PTP) and publish/subscribe (Pub/Sub). PTP messaging is always queue based, while the Pub/Sub model uses topics.

In JMS speak topics and queues are simple message destinations or channels that producers send messages to and consumers listen to via notification semantics that messages are available. For topics, each consumer that subscribes to a topic will receive a copy of a message if the consumer is an active subscriber at the time the broker receives the message. If a producer sends a message to a destination and there are ten consumers actively subscribing to the destination, all ten consumers will receive a copy of the message. Pub/Sub is useful in which you want every consumer or client to receive a message sent to a destination by a producer:

On the other hand, queues are like load balancers: if a producer sends a message to the destination, then the message will be kept by the broker until a consumer is available and then the message will be delivered. Note that if there is one producer and ten consumers of the queue, exactly one consumer will receive a copy of the message. The messages are load-balanced in that if another message is sent, the next consumer will receive a copy of the message and then the next, and so on and so forth as messages are sent to the destination. A scenario where this is useful is where you have say a pool of processes and you want each message sent to the pool to be received by a single process and each subsequent message to be processed by exactly one other, different process within the pool.

In part 1 I created a consumer and producer of messages using pub/Sub schematics, in this example I will augment this API with classes for sending message using PTP messaging schematics. These classes will allow clients to send and receive messages to queues with consumers of these messages receiving at most one copy of the message that was sent from a producer. Additionally, I’ll provide a simplified API for using ActiveMQ across multiple projects.

Utilizing Queues – PTP Messaging
As I stated previously, queue based messaging is always PTP; a producer sends a message to a queue and exactly one consumer will receive a copy of the message. For our API we will define a producer and a consumer of messages. First the producer:

This is simple class takes a session and a string identifying the destination or queue that we will be sending messages to as constructor arguments. Usage is even simpler, just call the SendMessage method and pass in a string to send. Note that this could be augmented to send more complex message types, but for simplicity I have omitted anything other than string messages.

The consumer class is likewise rather simple and has a constructor that takes the same two arguments, a session and a string identifier of the queue that messages will be received from:

The QueueReceiver requires that you register with a delegate that is fired when a message is received by the consumer. The client starts listening to the by calling the Start method and passing in a string that uniquely identifies the client. Like subscribers (and all other message consumers), the QueueReceiver should be a long-lived object, with a lifetime of the entire application.

The entire API, looks like the following:

Usage is very simple, provide a long-lived consumer object with a queue as its destination and then create shorter lived producer objects that send messages to the broker. The following unit test shows an example of the round-robin way that messages are received by the registered consumers of a queue:

Tags: , , , ,


ActiveMQ via C# using Apache.NMS Part 1

by jmorris 12. July 2010 18:17

ActiveMQ via C# using Apache.NMS Part 1

Java Message Service (JMS) is the de facto standard for asynchronous messaging between loosely coupled, distributed applications. Per the specification, it provides a common way for Java application to create, send, receive and read messages. This is great for enterprises or organizations whose architecture depends upon a single platform (Java), but the reality is that most organizations have hi-bred architectures consisting of Java and .NET (and others). Oftentimes these systems need to communicate using common messaging schematics: ActiveMQ and Apache.NMS satisfy this integration requirement.

JMS Primer
The JMS specification outlines the requirements for system communication between Java Messaging Middleware and the clients that use them. Products that implement the JMS specification do so by developing a provider that supports the set of JMS interfaces and messaging semantics. Examples of JMS providers include open source offerings such as ActiveMQ, HornetQ and GlassFish and proprietary offerings such as SonicMQ and WebSphere MQ. The specification simply makes it easier for third parties to develop providers.

All messaging in JMS is peer-2-peer; clients are either JMS or non JMS applications that send and receive messages via a provider. JMS applications are pure Java based applications whereas non JMS use JMS styled APIs such as ActiveMQ.NMS which uses OpenWire, a cross language wire protocol that allows native access to the ActiveMQ provider.

JMS messaging schematics are defined into two separate domains: queue based and topic based applications. Queue based or more formally, point-to-point (PTP) clients rely on “senders” sending messages to specific queues and “receivers” registering as listeners to the queue. In scenarios where more a queue has more than one listener, the messages are delivered in a round-robin fashion between each listener; only one copy of the message is delivered. Think of this as something like a phone call between you and another person.

Topic based application follow the publish/subscribe metaphor in which (in most cases) a single publisher client publishes a message to a topic and all subscribers to that topic receive a copy. This type of messaging metaphor is often referred to as broadcast messaging because a single client sends messages to all client subscribers. This is some analogous to a TV station broadcasting a television show to you and any other people who wish to “subscribe” to a specific channel.

JMS API Basics
The JMS Standard defines a series of interfaces that client applications and providers use to send messages and receive messages. From a client perspective, this makes learning the various JMS implementations relatively easy, since once you learn one you can apply what you learned to another implementation relatively easily and NMS is no exception. The core components of JMS are as follows: ConnectionFactory, Connection, Destination, Session, MessageProducer, and MessageConsumer. The following diagram illustrates communication and creational aspects of each object:

NMS supplies similar interfaces to the .NET world which allows for clients to send messages to and from the ActiveMQ JMS via OpenWire. A quick rundown of the NMS interfaces are as follows:

Note that the Apache.NMS namespace contains several more interfaces and classes, but these are the essential interfaces that map to the JMS specification. The following diagram illustrates the signature that each interface provides:

The interfaces above are all part of the Apache.NMS 1.30 API available for download here. In order to use NMS in your .NET code you also need to down load the Apache.NMS.ActiveMQ client as well and to test your code, you will need to download and install the ActiveMQ broker, which is written in Java so it requires the JRE to be installed as well. The following table provides links to each download:

For my examples I will be using the latest release of Apache.NMS and Apache.NMS.ActiveMQ as of this writing time. You should simple pick the latest version that is stable. The same applies for ActiveMQ and the JDK/JRE…note that you only need the Java Runtime Environment (JRE) to host install ActiveMQ. Install the JDK if you want to take advantage of some the tools that it offers for working with JMS providers.

To start ActiveMQ, install the JRE (if you do not already have it installed – most people do already) and unzip the ActiveMQ release into a directory…in directory will do. Open a command prompt and navigate to the folder with the ActiveMQ release and locate the “bin” folder, then type ‘activemq”. You should see something like the following:

Download and install the Apache.NMS and Apache.NMS.ActiveMQ libraries from the links defined in the table above. Unzip them into a directory on your hard drive, so that you can reference them from Visual Studio.

Open Visual Studio 2008/2010 and create a new Windows project of type “Class Library”:

And once the project is created, using the “Add Reference” dialog, browse to the directory where you unzipped the Apache.NMS files defined above and a reference to the Apache.NMS.dll. Do the same for the Apache.NMS.ActiveMQ download. Note that each download contains builds for several different .NET versions; I chose the “net-3.5” version of each dll since I am using VS 2008 and targeting the 3.5 version of .NET.

For my examples you will also need to install the latest and greatest version NUnit from After you have installed NUnit, add a reference to the nunit.framework.dll. Note that any unit testing framework should work.

Add three classes to the project:

  1. A test harness class (ApacheNMSActiveMQTests.cs)
  2. A publisher class (TopicPublisher.cs) 
  3. A subscriber class (TopicSubscriber.cs).

Your solution explorer should look something like the following:

The test harness will be used to demonstrate the use of the two other classes. The TopicPublisher class represents a container for a message producer and the TopicSubcriber represents a container for a message consumer.

The Publisher
The publisher, TopicPublisher is a simple container/wrapper class that allows a client to easily send messages to a topic. Remember from my previous discussion about topics that topics allow for broadcast messaging scenarios: a single publisher sends a message to one or more subscribers and that all subscribers will receive a copy of the message.

Message producers typically have a lifetime equal to the amount of time it takes to send a message, however for performance reasons you can extend the life out to the length of the application’s lifetime.

The Subscriber
Like the TopicPublisher above, the TopicSubscriber class is container/wrapper class that allows clients to “listen in” or “subscribe” to a topic.

The TopicSubscriber class is typically has a lifetime that is the equal to the lifetime of the application. The reason is pretty obvious: a publisher always knows when it will publish, but a subscriber never knows when the publisher will send the message. What the subscriber does is create a permanent “listener” to the topic, when a publisher sends a message to the topic, the subscriber will receive and process the message.

Usage Example
The following unit test shows the classes above used in conjunction with the Apache.NMS and Apache.NMS.ActiveMQ API’s to send and receive messages to ActiveMQ which is Java based, from the .NET world!

Here is quick rundown of the ApachNMSActiveMQTests class:

  1. Declare variables for the required NMS objects and the TopicSubscriber
  2. Declare variables for  the broker URI, the topic to subscribe/publish to, and the client and consumer ids
  3. Create a ConnectionFactory object, create and start a Connection, and then create a Session to work with.
  4. Create and start the TopicSubscriber which will be a listener/subscriber to the “TestTopic” topic. Also, to receive messages you must register an event handler or lambda expression with the MessageReceivedDelegate delegate. In this example I in-lined a lambda expression for simplicity.
  5. On the test the method, create a temporary publisher and send a message to the topic.
  6. Tear down and dispose of the subscriber and Session.
  7. Tear down and dispose of the Connection.

After you run the unit test you should see something like the following message:

Note that ActiveMQ must be up and running for the example to work.


kick it on

Tags: , , , ,

Apache.NMS | JMS

Weird Exception Using Spring.NET NMS and ActiveMQ

by jmorris 28. April 2010 15:09

I am doing some testing with Spring.NET NMS and ActiveMQ (via Apache.NMS) and I came across this gem:

Note that the path in exception message does not exist on my local machine?!

Tags: , , ,

Jeff Morris

Tag cloud

Month List

Page List