ActiveMQ: Topics vs. Queues

by jmorris 9. February 2011 13:47

The JMS specification defines two different ways of defining message destinations: Queues and Topics. Understanding the behavior of each is vital to implementing messaging within a system and fulfilling any expectations of message availability.

First a quick aside, the purpose of this post is mostly to keep myself from committing the mistakes I made today based upon invalid assumptions regarding the availability messages sent to a queue within an environment with multiple listeners. In a nutshell, I was writing a test to queue up a single message and I have a listener within the test receive the message.  For some reason the message was never being received…baffling given the simplicity of the test and the maturity of the code.

Here is the test (ignore that the test is not well written, I just want to quickly test that the code works while I am developing):

image

I edited this code a bit for simplicity, but in essence a message should be pumped to the console and displayed (real test validates that message was received after x seconds delay). The only problem was I that I was getting nada…note that I am not testing the queuing API itself, I am testing that the strategy will push a message onto the queue without throwing an exception. It’s basically a throwaway test to prove that it works as expected in an integrated environment.

The problem was that the message was never being received by the listener (QueueReceiver is the listener), however other similar tests that are messaging API specific were working as expected. Running through the usual checks turned up nothing. The code simple didn’t work…but only for this test! Swapping out for another queue worked perfectly :(

Fool me once, shame on you. Fool me twice…

So, what was causing the problem? Well it all became very clear once I went back and reflected on the basics of JMS messaging and the differences in how messages are delivered to queues and topics.

The JMS specification supports two models for sending messages: point-to-point for queues and publish and subscribe for topics. In a nutshell, only one consumer of a queue will receive a messaging regardless of the number of consumers; round-robin message dispersal. Whereas a topic broadcasts a copy of the message to every registered consumer.

It turns out out that I had a windows service running that was a consumer of the same queue and It was the first listener, so it always got the first message. The unit test never received a copy because only one message was sent and the consumer on the windows service always pulled it off. This was proved by simply calling publish twice:

image

This consumer (_receiver) got the second message, because the first one was consumed by the windows service and the next round-robin listener was this unit test. ARGGGHHHH, what a waste of time and brain power!

Lessons learnt:

  1. Don’t write unit tests with external dependencies and assume they will behave as expected
  2. Don’t use shared queues for testing
  3. Don’t forget the difference between queues and topics

References

Tags:

Apache.NMS, JMS | rant

ActiveMQ via C# Part 3: Simplifying the API using a Provider Pattern

by jmorris 24. September 2010 16:05

Background

In my previous posts I developed simple sender and receiver classes for topics and queues using Apache.NMS and ActiveMQ. In this post I will use the provider pattern to simplify creation and management of NMS connections, senders and receivers. Additionally, I’ll improve the syntax and ease of using NMS by adding fluent-interface behavior to the sender and receiver classes previously developed in part 1 and part 2, as well as a little refactoring to better manage the lifetime of the objects.

  1. http://rantdriven.com/post/ActiveMQ-via-C-sharp-and-dotnet-using-ApacheNMS-Part-1.aspx
  2. http://rantdriven.com/post/ActiveMQ-via-C-using-ApacheNMS-Part-2-Queues.aspx

Provider Pattern Background

The provider pattern is a software pattern developed at Microsoft during the development of ASP.NET 2.0 and used pretty extensively in that release. The most noticeable implementation was the Membership API, where the data storage of your security was abstracted away from the implementation. The aim is the pattern is loose coupling similar to what is provided by an IoC and DI, such as Ninject or Windsor container via settings in a configuration file as injected parameters.

There has been a bit of debate over whether or not it is a true pattern with most people agreeing that it’s a really several patterns or simply another pattern with a different name; most notable the Factory and Strategy patterns in some circles and the Singleton or Bridge patterns in others. The Bridge pattern definition most closely matches the goals of the pattern:

"decouple an abstraction from its implementation so that the two can vary independently"

- Gamma, Helm, Johnson, and Vlissides (gof)

As far as I am concerned it is a sort of hybrid pattern; a composition of patterns if you will.

Provider Pattern Implementation

The provider pattern itself is deeply ingrained within .NET itself; there is even a namespace with base classes supporting the pattern – System.Configuration.Provider. This namespace contains the following base classes, BaseProvider and ProviderCollection which are the building blocks of configuration based providers. System.Configuration adds another abstract class that makes working with .NET configuration files easier. Note that you could in theory create your own provider implementation without using these classes, but they provide an easy to follow framework for making providers using .NET.

The following diagram illustrates the entire provider implementation:

provider-api

Here is a rundown of the various classes required:

  • NmsProvider – derives from BaseProvider and provides initialization for the provider plus factory methods for creating NMS senders and receievers. Only one instance of a configuration should exist for an AppDomain or process and all operations should be thread safe.
  • NmsProviderCollection – derives from ProviderCollection and maintains a list of configured providers. For example you could have providers configured for multiple ActiveMQ servers i.e. receive from one server and send to another.
  • NmsProviderConfigurationSection – provides a defined section in your configuration file for each provider implementation.
  •  NmsProviderService – a singleton ‘service’ for maintaining the collection of providers while your application is running. Note that providers are long lived objects, hence the need for a singleton to maintain the list.

The NmsProvider class is really a façade encapsulating the interaction between the JMS broker and the client’s it services. From a client perspective it manages the connection between the client and the JMS broker, the sessions that are generated per conversation, as well as the creation of objects for sending JMS messages to receiving messages from destinations (queues and topics). An important point to remember is that JMS Connections are heavyweight, long lived objects and are typically created 1:1 with clients, which “fits” well with the lifecycle of providers.

All providers use configuration files to store the type of provider to initialize and any additionally information that must be injected into the newly created instance. Note that this is a form of Dependency Injection (DI). The configuration file may contain configurations for multiple instances of each provider configured differently. For example, each provider may be using a different broker to send or receive messages. Here is a listing of a configuration that lists several provider instances using the same broker:

 

When the NmsProviderService is created, it will read the contents of NmsProviderConfigurationSection and construct providers for each entry in the “providers” section; in this case two providers are listed: NmsProvider0 and NmsProvider1. The System.Configuration namespace contains a special class to help in creating the providers called ProvidersHelper. After the providers are created they are added to the NmsProviderCollection, which is a thread safe collection of providers.

When the provider is created by the ProvidersHelper.InstantiateProviders method in the NmsProviderService class, the overriden ProviderBase.Initialize method will be called in the NmsProvider class. The Initialize method will assign the values stored in the configuration file to their respective properties on the NmsProvider class and then create an instance of the Apache.NMS.ConnectionFactory class.

image

As previously discussed, the NmsProvider is façade encapsulating the interaction between the JMS server and the clients that it services. It is also a factory in that it offers methods for creating objects for sending and receiving messages. Notable it hold references to Apache.NMS.IConnection and Apache.NMS.IConnectionFactory objects. The IConnectionFactory class is a factory for creating IConnection instances based on the configuration supplied.

Clients use providers by name via and indexer on the NmsProviderService after calling the Start() method on the provider to create the connection between the client and the JMS server:

 image

 

The Start() method creates a new IConnection if the provider does not have one already initialized. Then it checks to see if the IConnection is started, if not then the IConnection’s Start() method is called which establishes the connection with the JMS server.  Additional methods on the provider allow the IConnection to be temporarily stopped or permanently closed:

image

Once the provider has been initialized, the usage is quite simple:

image

**Make sure you have ActiveMQ running and your configuration file is configured with the appropriate uri. See http://rantdriven.com/post/ActiveMQ-via-C-sharp-and-dotnet-using-ApacheNMS-Part-1.aspx for details on how to get ActiveMq up and running on a windows box.

References

 

 

Tags: , ,

Apache.NMS, JMS

Jeff Morris

Tag cloud

Month List

Page List