Shulou(Shulou.com)06/01 Report--

How to analyze the ultra-fast comparison between Kafka and message queue? aiming at this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Kafka was originally a technology developed by the Linkedin community. In short, it's a bit like a message queuing system, but unlike a message queuing system, it supports pub/sub, can be extended on many servers, and replays messages.

Usually you may not pay much attention to these issues, but these are the things you need to pay attention to when you want to use a responsive programming style rather than an imperative programming style.

The difference between imperative programming and responsive programming

Imperative programming is the type of programming that we used from the beginning. When something happens, in other words, the event happens, and your code is told that the event happened. For example, if the user clicks a button, where you handle the event in the code determines which actions you want the system to trigger next. You can save records to a database, invoke another service, send an e-mail, or combine these actions. The most important point here is that events are directly coupled to these specific actions.

Responsive programming enables users to respond to events that occur, usually in the form of a stream. Multiple concerns can subscribe to the same event and let the event have an impact in its domain, regardless of what happens in other domains. In other words, it supports loosely coupled code and can be easily extended to more functions. It is possible that a variety of large dirty systems encoded in different stacks can be affected by events, or even a bunch of serverless functions executed somewhere in the cloud.

From message queuing to Kafka

To understand what Kafka brings to your architecture, let's talk about message queuing first. We started with message queuing because we will discuss its limitations and then see how Kafka solves these problems.

Message queuing allows a group of subscribers to extract one or more messages from the end of the queue. Queues usually allow certain levels of transactions to be performed before the message is removed to ensure that the required actions are performed before the message is deleted.

Not all queuing systems have the same functionality, but once the message is processed, it is deleted from the queue. If you think about it, it's actually very similar to imperative programming, where something has to happen first, and then the system starts to decide what to do in the downstream system.

Although multiple consumers can be extended in the queue, they all contain the same functionality, and this is just to handle the load and process messages in parallel; in other words, it does not allow you to start multiple independent operations based on the same event. All processors of queue messages will perform the same type of logic in the same domain. This means that the message in the queue is actually a command, which is suitable for imperative programming, not an event suitable for responsive programming.

For queues, the same logic is usually performed for each message in the queue in the same domain

On the other hand, with Kafka, you can publish messages / events to topics, and they are persisted. When consumers receive these messages, they will not be removed. This allows you to replay the message, but more importantly, it allows a large number of consumers to handle different logic based on the same message / event.

You can still do parallel processing in the same domain, but more importantly, you can also add different types of consumers who perform different logic based on the same event. In other words, for Kafka, users can adopt a passive pub/sub architecture.

Different logic can be executed by different systems based on the same event.

In the case of Kafka, this is possible because the information is retained, and so is the concept of the consumer community. Kafka's consumer groups position themselves as Kafka when they ask Kafka for information about a topic. Kafka will record which messages (offsets) are delivered to which consumer group so that it will no longer serve it. In fact, it's a little more complicated than that because you have many configuration options available to control this, but we don't need to explore these options comprehensively, just to understand Kafka at a high level.

This is the answer to the question on how to analyze the ultra-fast comparison between Kafka and message queue. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.