RabbitMQ with HTTP backend plugin

Introduction

In this tutorial we'll look at how to extend the AMQP messaging system RabbitMQ with the HTTP backend plugin - the rabbitmq_auth_backend_http plugin located here.

First we must mention that at the time of this writing there are several authentication mechanisms available in RabbitMQ. Available authentication mechanisms are:

• SASL PLAIN: Username and password are send in plantext, so they can be easily intercepted. If we're going to use the PLAIN mechanism, we should at least send the username and password over SSL.
• SASL AMQPLAIN: Basically the same as SASL PLAIN.
• SASL EXTERNAL: Available in RabbitMQ 2.3.1 and latter versions. This mechanism determines if username is allowed to login via some external mechanism, outside the scope of the protocol. This mechanism can be used to implement a number of additional authentication mechanisms, some of which are:
• SSL (plugin rabbitmq-auth-mechanism-ssl): In this case the external mechanism is SSL, which can authenticate users with client certificate rather than username and password. The username is Common Name (CN) of the presented certificate, and the password is the certificate itself.
• LDAP (plugin rabbitmq-auth-backend-ldap): This external mechanism can be used to authenticate users against backend LDAP database. Configuring this plugin for authentication is easy once we have the LDAP database in place.
• HTTP (plugin rabbitmq-auth-backend-http): External mechanism here is HTTP, which can be used to send/receive queries to/from some web server, which knows about user credentials and can allow or disallow certain user to access a resource.

But what about authorization that specifies the scope where the users have access:

• Internal: The RabbitMQ has internal database, which can handle authorization without a problem.

• LDAP: Specifying authorization in LDAP is particularly hard, since the LDAP does not have any ideas how permissions should work in AMQP. RabbitMQ adds a simple query mechanism that can use used to determine whether a particular user is allowed to login to certain virtual host.

SASL is a Simple Authentication and Security Layer that is used for authentication. It allows any application protocol to use it as a blackbox - the application protocol doesn't need to know anything about it, except when a specific user has been successfully authenticated or when login failed. SASL mechanism basically consists of sending a username and password over the wire to authenticate against the appropriate backend database, whatever it is. We can read more about SASL mechanisms and SASL aware application protocols here: Wikipedia.


In this article we'll look at the EXTERNAL HTTP backend mechanism used to authenticate users.

Installing and configuring the RabbitMQ environment

I followed the RabbitMQ Plugin Development guide and had some problems installing and configuring the RabbitMQ environment, that's why we'll configure the environment specific to the HTTP plugin here.

First we have to follow some steps that are directly copied from the linked guide:

# hg clone http://hg.rabbitmq.com/rabbitmq-public-umbrella
# cd rabbitmq-public-umbrella
# make co
# cd ..


Now we have the basic environment that can be used for developing a plugin. After that we should clone the rabbitmq-auth-backend-http plugin form the github, since that plugin is not available in the default RabbitMQ environment, because it's at an early stage of development.

# git clone https://github.com/simonmacmullen/rabbitmq-auth-backend-http.git
# cd rabbitmq-auth-backend-http
# make
# cd ..

Afterwards we should enable the HTTP plugin we just downloaded:


# mkdir -p rabbitmq-server/plugins
# cd rabbitmq-server/plugins
# ln -s ../../rabbitmq-erlang-client
# ln -s ../../rabbitmq-auth-backend-http
# cd ..
# scripts/rabbitmq-plugins enable rabbitmq_auth_backend_http

The plugins directory should contain the following two links:
# ls -l plugins/
lrwxrwxrwx 1 eleanor eleanor 32 Jul 2 22:27 rabbitmq-auth-backend-http -> ../../rabbitmq-auth-backend-http
lrwxrwxrwx 1 eleanor eleanor 28 Jul 2 22:27 rabbitmq-erlang-client -> ../../rabbitmq-erlang-client

To start the RabbitMQ now with our HTTP plugin, all we need to do is run:
# make run

But as we can see, it doesn't work quite as easily. We need to tweak the configuration a little bit.

Creating RabbitMQ configuration files

First we need to create the etc/ directory under rabbitmq-server and add two files specified here (note the dot at the end of the first file, it's important, don't forget it):
# vim etc/enabled_plugins
[rabbitmq_auth_backend_http].

# vim etc/rabbitmq.config
[
{rabbit, [{auth_backends, [rabbit_auth_backend_http]}]},
{rabbitmq_auth_backend_http,
[{user_path, "http://127.0.0.1:8000/auth/user"},
{vhost_path, "http://127.0.0.1:8000/auth/vhost"},
{resource_path, "http://127.0.0.1:8000/auth/resource"}]}
].

Afterwards we need to tell RabbitMQ where those files are located, because by default it will try to access system RabbitMQ files, which just won't work, since we want to create a custom environment. To do that we need to edit rabbitmq-server/scripts/rabbitmq-defaults and change the following two variables:
CONFIG_FILE=/home/user/rabbitmq/rabbitmq-server/etc/rabbitmq.config
ENABLED_PLUGINS_FILE=/home/user/rabbitmq/rabbitmq-server/etc/enabled_plugins

Then we need to run the following in bash in order for RabbitMQ to detect the config file.
# cd rabbitmq-server
# RABBITMQ_CONFIG_FILE=/home/user/rabbitmq/rabbitmq-server/etc/rabbitmq make run

If we would like to specify the path to the config file in the rabbitmq-env.config we can check RabbitMQ Configuration.


To check whether the config file is picked up by the RabbitMQ, we can run the following command, which should output the "config file(s)" parameter that is either none if config files isn't present or specifies the actual path to the config file:
# make run
config file(s) : /home/user/rabbitmq/rabbitmq-server/etc/rabbitmq.config


We can see that the config file was detected.

Using the HTTP backend authentication

Ok, now the make run detects the config file, which in turn specifies the configuration file, which configures the rabbitmq_auth_backend_http plugin that is used for authenticating against RabbitMQ.

This is all fine and dandy, but how to use that in reality. First we need to run the django application that's located in the rabbitmq-auth-backend-http/examples/rabbitmq_auth_backend_django/ directory. We can do that with the following commands:
# cd rabbitmq-auth-backend-http/examples/rabbitmq_auth_backend_django/
# python manage.py syncdb
# python manage.py runserver

This will first create a sqlite database in the /tmp directory. Then it will start the server on the localhost on port 8000 as is shown here:

# python manage.py runserver
0 errors found
Django version 1.3.1, using settings 'rabbitmq_auth_backend_django.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

We need to connect to the http://127.0.0.1:8000/admin/ with the configured username:password and add another user. Let's add the username test with the password test. Now we can input the following address into default web browser to check whether django application is working and successfully authenticating users:

GET http://127.0.0.1:8000/auth/user?username=test&password=test HTTP/1.0

It should return allow management.

Ok, it works. But how can we connect to the configured RabbitMQ server with a client script and do some work. Here's the python client code that connects to the configured RabbitMQ server:

#!/usr/bin/env python
import pika

credentials = pika.PlainCredentials('test', 'test')
params = pika.ConnectionParameters(credentials=credentials, host="localhost", virtual_host='/')
connection = pika.BlockingConnection(params)
channel = connection.channel()
channel.queue_declare(queue='')
channel.basic_publish(exchange='logs', routing_key='', body='Hello World!')
connection.close()

The python script basically connect to the RabbitMQ server with credentials test:test as we specified in the django application, then sends the message "Hello World!" to the null queue. Each client that is also associated with that queue should receive that message, but that's not part of the HTTP plugin - it's basic RabbitMQ design.

Note, that if we don't specify the PlainCredentials in the pika module in python, it will by default use guest:guest, which is the default RabbitMQ server username and password. This is quite a useful piece of information, because if the username and password isn't specified, how can the python script have access to post the message to the queue?

To read more about the python RabbitMQ code read Pika and java RabbitMQ code Java.

Final thoughts

That's it for now. We've looked how to configure the local RabbitMQ environment, add and build the rabbitmq_auth_backend_http module, start the http web server (django application), and use the python client RabbitMQ client code to authenticate against it.


I hope you enjoyed the tutorial and please let me know if anything isn't clear or if you want to have anything added.