Which issues are fixed by using message queues, such as RabbitMQ, and why this is interesting to me.

  • Enes Aktay

First of all, thanks for all the feedback I got for my last post. 🤟
For everyone who didn’t read it: This post is an addition to my previous post.
❗: This post might be not very interesting to people that already know about message queues. 😬

This blogpost will briefly describe what message queues are, how and where this technology will be used and as I promised in my previous blogpost, and here it is. 🙌

Let’s tackle this topic with:

What are message queues?

What are message queues in the first place? 📤 ✉️ 📥

Message queues are used to decouple applications by adding a common communication layer, that makes applications communicate over this layer rather than peer to peer. The message could possibly be anything that can be saved as simple text. An application or a software sends a message to the message queue where it's being stored until an application or software takes the message off again. There are several technologies that gets the job done but one of the most famous ones, nowadays, is RabbitMQ.
This is a very brief explanation, so bear with me if it’s simplified too much because I don't want to deepdive in the basics too much.

An image showing the Queuesystem

To put the advantages of message queuing in a nutshell:

  • Fault tolerance 😶 because, if one system breaks for whatever reason another system can still send messages to the queue
  • Improve scalability 😮 because, if there is a unexpected load on an application you can create multiple instances of this application to balance the load and it will easily use message queues for communication
  • Decrease latency 😲 because, since messages are being sent asynchronously, systems don't have to wait for each other before finishing a cross-system task

How am I going to use message queues?

There is a framework called Sylius, which I will not be explaining. If you are interested in it, but don't want to spend hours reading their doc, have a look at one of Lukas' software evualuation post about Sylius. Basically, this framework is an eCommerce framework which is based on Symfony. People that know Symfony, know symfonies bundle-system.
For the others: You can extend Symfony applications easily by adding bundles. Bundles can be seen like plugins to a Symfony application. There is an open-source bundle called “SyliusImportExportPlugin”. As the name implies, you can import and export data of the eCommerce application with this bundle. The main topic of my bachelor’s thesis is, to add a mq system into this import/export plugin and see how the mq system affects the latency, scalability and fault tolerance by doing benchmarks.
The following (beautiful) drawing illustrates the changes I'll be doing to the bundle:

The drawing shows the bundle before and after I changed my topic outlined above

The regular file export and the red part will be benchmarked and implifications will be discussed in my scientific paper.
I will make a follow up post about the benchmarks, findings, as well as the comparison before and after my project.

Why did I choose this subject?

I liked the subject because it gives me the opportunity, to learn something about a solution, to the problem of having no fault tolerance, not being able scale and systems having the need of wait for each other. Two different systems talking to each other is not an easy task. Before having message queues: An admin had to export data from the system to a file and then (bestcase) import the file without changing anything to another system. By queuing messages, this task can be automated.

"Great things in business are never done by one person. They're done by a team of people."

  • Steve Jobs

Thanks to everyone that is helping me with working on the bundle and in addition for helping me on my way to graduation.
Thanks to Michael Weibel and Fabian Ryf for helping me with this blogpost.
You guys are all 🔥

Tell us what you think