Shulou(Shulou.com)06/02 Report--

This article mainly explains "what are the differences between common message queues". The content of the explanation is simple and clear, and it is easy to learn and understand. let's study and learn "what are the differences between common message queues"?

First, the origin of message queue

You may have heard a lot of concepts, such as JMS,AMQP,ActiveMQ,RabbitMQ,RocketMQ,Kafka, before you know anything about message queuing.

A message middleware, how to come up with so many concepts?

Don't panic, let's sort out the relationship between these MQ and agreements from a historical point of view!

As a matter of fact, message middleware was born very early. In the era when Internet applications were still barren, an Indian brother Vivek Ranadive in the United States imagined a general software bus, using the publish and subscribe mode, similar to the bus on the computer motherboard. If new devices or programs want to communicate with other device software on the computer, they only need to dock the bus according to the protocol to complete the access and communication!

In 1983, the 26-year-old Indian brother Vivek Ranadive founded a company, Teknekron, and implemented the world's first messaging middleware The Information Bus (TIB).

Soon TIB software was welcomed by enterprises and was initially used by Goldman Sachs to solve financial transactions, and the speed of business development of Teknekron even attracted the attention of IBM, the most powerful IT company at that time.

So IBM began to form a team to develop its own message queuing software, which became known as wesphere mq, and Microsoft soon joined the regiment.

Due to commercial barriers, each software manufacturer realizes software communication in accordance with its own standards, resulting in enterprise customers can not casually change the MQ platform.

In order to break down this barrier and to enable messages to communicate among message queuing platforms, JMS (Java Message Service) arises at the historic moment.

JMS attempts to hide the implementation interface provided by individual MQ product vendors by providing public Java API, so as to overcome the barriers and solve the problem of interoperability.

Technically speaking, Java applications only need to be programmed for JMS API and choose the appropriate MQ driver, and JMS will take care of other parts, just like JDBC. For developers, they only need to write sql, whether to use oracle or mysql or sqlserver, and the specific vendor can provide the driver package files. Developers do not need to care about the specific database vendor. As a result, the development efficiency is greatly improved and the development difficulty is reduced.

ActiveMQ is a concrete implementation of JMS.

JMS-point-to-point model

JMS-point-to-point model

JMS-publish and subscribe model

JMS-publish and subscribe model

Although the use of standardized interfaces can effectively integrate many different MQ products, it also exposes many problems. For example, some MQ products provide very advanced functions, but due to the limitations of standardized interfaces, users can not use them, so there is an urgent need for a new message communication standardization scheme.

In June 2006, the public standard of AMQP was jointly formulated by Cisco, Redhat, iMatix and others, from which AMQP stepped onto the stage of history.

AMQP is an open standard of application layer protocol to solve the requirements and topology problems of many message middleware. It is designed for message-oriented middleware. The client and message middleware based on this protocol can transmit messages without being restricted by product, development language and other conditions.

JMS vs AMQP

RabbitMQ is a concrete implementation of AMQP.

AMQP-model

With the passage of time, although AMQP specification can be applied to many business scenarios, LinkedIn (LinkedIn) feels that AMQP specification is not suitable for it when implementing message queues, so when designing Kafka, it does not support all the features of AMQP.

At the same time, Alibaba's RocketMQ also draws lessons from the idea of Kakfa and does not support AMQP protocol, and you will find that there are similar concepts of Topic and Consumer Group in both Kafka and RocketMQ, but these concepts do not exist in AMQP protocol.

Second, why use message queuing

Although message middleware has been developed for many years, not every project has access to message queues. For students who first come into contact with MQ, there will inevitably be some questions!

What is a message queue? Why use message queuing? What are the disadvantages of using message queues?

For traditional applications, if you need to send information to another application, you just need to send a request to it!

This approach is simple and straightforward, but if Application 2 suddenly dies, Application 1 may not be able to continue to provide services because of service exceptions!

Imagine inserting a message service between application 1 and application 2, mainly for receiving and sending messages, so that the dependencies between application 1 and application 2 are decoupled. At the same time, it will not be because either party cannot continue to provide the service when the service is not available!

The message service that is inserted is called message queue!

Thus, the advantages of introducing message queuing are obvious:

Program decoupling: when application 1 and application 2 interact, the service will not stop because of the interruption of one party's service.

Asynchronous processing: after the program is decoupled, the biggest benefit is that it can be processed asynchronously. Application 1 only sends the message to the message middleware, and application 2 only needs to receive the message from the message middleware and process it.

At the same time, based on the asynchronous processing feature, in some business scenarios, such as commodity flash sale activity, after the introduction of message queue, when the client request volume is very large, traffic peaking can be effectively carried out!

If there is no middle layer to do buffer, when the commodity second kill, a sudden influx of requests, is likely to cause direct paralysis of the system, or even downtime!

In the large website system, how to quickly locate the abnormal code of the system in real time through the log is very important!

Message queue Kafka developed by LinkedIn can be said to be the king of log collection. In medium-and large-scale system development, using message queue Kafka in log processing can effectively solve the problem of a large number of log transmission.

Of course, the introduction of message queuing also brings obvious disadvantages:

Reduced system availability: before introducing message queuing, you don't have to think about message loss or message queuing service hanging, but you need to consider these issues after introducing message queuing!

Increased system complexity: after joining the message queue, you need to make sure that the message is not repeatedly consumed, that the message is not handled correctly, and so on!

Although the introduction of message queue will bring some problems, as the saying goes, troops will block, water will cover the earth, this sentence also applies to IT developers, there is a hole to fill the hole!

For the reduction of system availability, the commonly used solution is to build a message service cluster. The specific technical implementation can be a master-slave architecture or a distributed architecture, even if a message queuing service machine dies. It will not affect that message queues cannot provide services!

For the improvement of system complexity, there are also many common solutions. For example, after the recipient receives the message, it can be written to the database first, even if it is not processed correctly, or it can be processed manually, or the message consumption fails. Re-queue the message to wait for the next consumption, and so on.

Third, the common message queue comparison

At present, the more mainstream MQ products, there are ActiveMQ,RabbitMQ,RocketMQ,Kafka, and they are open source, they also have their own characteristics.

The summary is as follows

The 1.ActiveMQ community is relatively mature, but compared to the current, ActiveMQ performance is relatively poor, and version iteration is very slow, so it is not recommended.

Although 2.RabbitMQ is slightly inferior to Kafka and RocketMQ in terms of throughput, because it is based on erlang development, it has strong concurrency, excellent performance, and low latency, reaching microsecond levels. But also because RabbitMQ is based on erlang development, few domestic companies have the strength to do erlang source-level research and customization. If the concurrency requirement of the business scenario is not too high (100,000, million), then RabbitMQ is preferred among the four message queues. If it is real-time computing, log collection and other scenarios in big data's field, using Kafka is the industry standard, absolutely no problem. The community is very active and will never be yellow, not to mention it is almost a factual norm in this field all over the world.

3.RocketMQ Ali produced, Java is an open source project, the source code we can read directly, and then we can customize our own company's MQ, and RocketMQ has Alibaba's actual business scenarios of the actual combat test. The activity of the RocketMQ community is relatively mediocre, but it is OK, and the documentation is relatively simple. There is also the technology introduced by Ali, you have to deal with the risk of community pornography in case this technology is abandoned. If your company has technical strength, I think it is good to use RocketMQ.

The characteristics of 4.Kafka are obvious, that is, it provides only a few core functions, but provides ultra-high throughput, ms-level latency, high availability and reliability, and the distribution can be extended at will. At the same time, Kafka is best to support a small number of topic to ensure its ultra-high throughput. The only disadvantage of Kafka is the possibility of repeated consumption of messages, which will have a very slight impact on data accuracy, which can be ignored in big data's field and log collection. Kafka is naturally suitable for big data's real-time computing and log collection.

Thank you for your reading. the above is the content of "what are the differences between common message queues". After the study of this article, I believe you have a deeper understanding of the differences between common message queues. Specific use also needs to be verified by practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.