I recently had a customer that wanted us to do some load testing of Red Hat’s JBoss A-MQ for them. In particular, this customer wanted the tests performed using the AMQP protocol instead of ActiveMQ’s native OpenWire. From previous engagements, I knew that there would be a performance difference. But after a quick look I didn’t see any blogs or posts on the subject. More specifically, I didn’t see any posts that detailed how to run the tests yourself so that you could get real numbers in your own environment. So I figured I’d write up some steps and post my results for future reference.
Please note that the purpose of this blog post is not to give you a number of msg/s to expect or tell you the absolute best way to tune your broker. The purpose of this post is to show you how to run the tests yourself and give you some very rough idea of the performance difference between the two protocols. All of my testing was done on my laptop using the default configurations. You will probably get wildly different performance in your environment and will likely need to tune the broker specific to your use case to get the best performance possible.
Both AMQP (Advanced Message Queuing Protocol) and OpenWire are what we refer to as “wire-level protocols”. They define how a client and broker will negotiate a connection, how they’ll format messages, and how they’ll communicate in general. ActiveMQ can speak many different wire-level protocols, but OpenWire and AMQP are supposed to be the fastest as they are both binary in nature. In both cases, you have multiple options as to what language your clients can be written in (ie, Java, .NET, C++, …). It really doesn’t matter as long as they are communicating via the same wire-level protocol as the broker. In this case, I’ll be using Java simply because of the availability of test tools.
There are a ton of available test tools that you can use to drive traffic to your brokers. For this blog, I used the ActiveMQ Perf Maven Plugin. This is a great tool that comes with the ActiveMQ source code and allows you to run as both producers and consumers. It is easily configurable and gives you a ton of options to test various scenarios (ie, persistent/non-persistent, transacted/non-transacted, 1-n producer/consumer threads, …). The only caveat to the tool is that it will load a connection factory that uses the OpenWire protocol. Luckily, it was written in such a way that we can extend it to use AMQP as well. To do that, we only need to create one class that knows how to load an AMQP connection factory. The source for the class is as follows:
Then we just need to make sure that our custom class and the Qpid client libraries are on the Maven classpath.
The tricky part is that we need them on the classpath of the Maven plugin itself and not necessarily the classpath of the project. Take a look at the pom.xml file to see what I mean. The full code for this test can be found at https://github.com/joshdreagan/amqp-perf-test. Feel free to just clone it down and use it directly.
If you take a look at the plugin’s documentation, you’ll see several properties that can be set for the test as a whole, the consumer, the producer, and the connection factory. You can configure the properties using “-Dkey=value” arguments on the command line, or via a .properties file (or some combination of the two). If you plan to run the test multiple times, it’s probably worth creating a .properties file. The only detail that I’ll give here is that the connection factory properties are set via reflection and will be specific to the connection factory used. I mention this because we swap out the connection factory when we run the AMQP tests. To give a concrete example, when using the
org.apache.activemq.tool.spi.ActiveMQReflectionSPI you will set the URI for the broker using
-Dfactory.brokerURL=tcp://localhost:61616, but when using the
org.jboss.examples.amqp.spi.AMQPReflectionSPIConnectionFactory you will set the URI for the broker using
-Dfactory.remoteURI=amqp://localhost:5672. This is because the “setters” are named differently for the different connection factory implementations. The available options for the AMQP connection factory can be found in the Qpid docs. All other options (ie, producer, consumers, & test) should be the same and are found on the plugin docs.
Here are some sample runs that I did. Please note the disclaimer at the beginning of this blog post regarding performance results.
That’s it! You can see that OpenWire is quite a bit faster in this case. But please don’t take my word for it… clone the project, tweak some configurations, try scaling the producers/consumers/brokers, and run the test yourself to see what results you’ll get.