Chapter 1 Message Queuing Overview
Message queuing gives you a mechanism to allow an application to asynchronously send a message to a receiver. This means that the sender and receiver do not need to interact with the message at the same time. A message is sent to a queue where it is stored until the receiver retrieves the message.
Message queues can be inter-process where the queue resides in memory on a single server or for integrating systems across multiple servers. This can be done by using in-memory queues but it is also common to use durable queues in which the messages are persisted to disk, meaning that messages are not lost should any system or server go offline for any period of time.
Message queuing systems come in many forms-both as commercial proprietry products and as open-source products. An example of a commercial solution is IBM MQ. Examples of open-source message queuing systems include RabbitMQ, JBoss Messaging, and Apache ActiveMQ.
The open-source message queuing systems are free to download and use but it is also common for companies to provide commercial support agreements (which you can pay for that allow your organization to reach out for technical support should you run into any production issues). RabbitMQ is no different in this respect and Pivotal Software, Inc., the company that owns RabbitMQ, provides a purchasable support agreement. Having a support agreement is normally a prerequisite for large organizations adopting an open-source platform.
Common Message Queuing Traits
A messaging broker is installed onto a server or a set of servers, and a series of name queues and exchanges are defined where they are registered with the message broker. An application can then send a message to the queue and any number of receiving systems can receive the message and act on its contents.
Queuing systems such as RabbitMQ will typically have the following features available to them:
- Durability: Messages can either be kept in memory or persisted to disk. If they are persisted to disk, then the message will be preserved should the server containing the queues crash. If this happens, once the server comes back up, the receiving applications can pick up where they left off.
- Security: A message broker should be able to let you set up different security permissions for different users. Some applications may have permission to write but not read from the queue, or another application could just have permissions to read from the queue.
- Time to Live: A message may have a time to live property set on it. This will mean a message can be set to expire after a certain timespan.
- Filtering and Routing: A messaging system might have the ability to filter the messages to decide to which queue a message should be delivered.
- Batching: Batching allows messages to be saved up and then delivered all at the same time as part of a larger batch; this is typically used when a destination server is not always online.
- Receipt Acknowledgement: A message publisher may require an acknowledgement from the message broker that the message has been successfully delivered.
Message Queuing Protocols
Many commercial message queuing systems are based on proprietary protocols. These protocols are kept closed so that the vendor can restrict what operating systems (OSes) and development platforms can interact with the message broker. This closed nature means that, if any customer of the messaging broker provider wanted it to work with another platform, they might have to pay a high consultancy fee to get the support they need.
With the advent of open-source messaging systems, a number of open queuing protocol standards have been developed. The two most predominant standards are:
- Advanced Message Queuing Protocol (AMQP): This is a feature-rich message queuing protocol and it is the protocol used by RabbitMQ.
- Streaming Text-Oriented Messaging Protocol (STOMP): Stomp is a simple text-based messaging protocol.
Later on in this book, we will look in more detail at the AMQP.
