RabbitMQ Cluster using Docker Compose

RabbitMQ

RabbitMQ is the most widely used open-source message broker service developed by Pivotal Tracker.

RabbitMQ is developed using Erlang and can be easily integrated into your microservices architecture.

RabbitMq supports below protocols

• AMQP
• HTTP
• STOMP
• MQTT

How RabbitMQ Works?

RabbitMQ has an “Exchange” component which is responsible for routing packages to queues. Each consumer gets its own Queue based on a logic that you use, there are 4 types of logic that you can use in Exchange.

- Direct Exchange: Will be direct to queue based on a message routing key

- Fanout Exchange: Will publish all queues that have the same routing key

- Topic Exchange: Will publish to all queues that have the same routing key and routing pattern specified in the binding.

• Headers Exchange: Headers mean headers in sending a file HTTP, like when you send Image the header is “image/*”.

RabbitMQ Components

RabbitMQ has four basic components :

• Producer : Responsible for publishing message to queue
• Exchange: Responsible for routing message to it’s destination queue.
• Queue: Message Storage which is bounded to Exchange using binding_key.
• Consumer: Linked to one of the queues to consume messages.

Why use RabbitMQ?

To overcome the disadvantages that are faced in monolithic architecture like single-point failure and scalability. Microservices architecture deals with decoupling the core components of your application

into small manageable applications. This system design also improves the Single Responsibility Principle.

As the application is decoupled it is also flexible to develop to the next phase. It also provides developers the flexibility to connect two different applications developed using different languages.

Advantages of RabbitMQ:

• HA
• Multi-protocol support
• Many Clients
• Clustering
• Management UI
• Huge Community
• Plugin system

RabbitMQ vs Apache Kafka

The most basic difference between both technologies is:

RabbitMQ has a smart broker & dumb consumer.

Kafka has dumb broker and smart consumer.

Setup RabbitMQ with Docker Compose

I have created a docker image that you can use to deploy a RabbitMQ cluster on your machine.

Github: https://github.com/kailashyogeshwar85/docker-rabbitmq-cluster

DockerHub: https://hub.docker.com/r/lucifer8591/rabbitmq-server

Let’s discuss the components that were used in creating RabbitMQ Cluster and build using docker-compose.

Architecture:

So we will have three containers each running RabbitMQ instance and other slave brokers are connected to its master component.

docker-compose.yml

version: "3"
services:
rabbit1:
image: lucifer8591/rabbitmq-server:3.7.17
hostname: rabbit1
ports:
- "5672:5672"
- "15672:15672"
environment:
  - RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER:-admin}
  - RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS:-admin}
rabbit2:
  image: lucifer8591/rabbitmq-server:3.7.17
  hostname: rabbit2
  links:
    - rabbit1
  environment:
    - CLUSTERED=true
    - CLUSTER_WITH=rabbit1
    - RAM_NODE=true
  ports:
    - "5673:5672"
    - "15673:15672"
rabbit3:
   image: lucifer8591/rabbitmq-server:3.7.17
   hostname: rabbit3
   links:
    - rabbit1
    - rabbit2
   environment:
    - CLUSTERED=true
    - CLUSTER_WITH=rabbit1
   ports:
    - "5674:5672"

Here we have defined three services rabbit1, rabbit2, rabbit3.

rabbit1 — Master

rabbit2 — Slave (links to master node rabbit1)

rabbit3 — Slave (links to master node rabbit1, rabbit2)

Here we have provided the administrator user’s username and password for accessing Management UI.

Docker Images

The final image is created using a base erlang image which is used to build RabbitMQ:3.7.17, RabbitMQ Server image which is used to create a cluster of servers.

If you want to build on your local system you just need a docker-compose.yml file which you will find in a repo and execute docker-compose up inside directory containing your docker-compose.yml

RabbitMQ Server Image

RabbitMQ server image consists of a server configuration file and Rabbitmq cookie which is needed by the server to allow administrator login.

It also has a start script to upstart your RabbitMQ instances.

Screenshots

Startup RabbitMQ

RabbitMQ Management UI

Management UI

RabbitMQ Containers

There are finally 3 containers that are created on your system:

• Container1: cluster_rabbit1_1
• Container2: cluster_rabbit2_2
• Container3: cluster_rabbit3_3

You can specify your own name for containers I have left it to docker-compose to name the container with its default pattern to use parent folder name as its prefix and service name as it’s a suffix.

Troubleshoot

To troubleshoot any configuration issue or Management UI not accessible kindly check Server logs in the container.

docker-compose logs -f