Shulou(Shulou.com)06/03 Report--

In the Internet scenario of high concurrency and high message throughput such as IM, MQ message middleware is a very important infrastructure, which plays the roles of message transfer, message peaking and message exchange asynchronization in the server architecture of IM system.

Of course, the role of MQ message middleware is much more than that, its value not only exists in technology, but also changes the previous thinking of synchronous processing of messages.

For example, when storing the history of IM messages, the traditional information system may synchronously store a message in the database as soon as it is received, which can work well in the case of small concurrency, but it is a disaster in the large concurrency environment of the Internet.

MQ message middleware can be understood as a pool, the end of the pool is the message producer, and the other end of the pool is the message consumer. Producers and messengers do not need to connect directly, which will bring a lot of benefits: business decoupling, architecture distribution, etc., producers and consumers are completely transparent to each other.

However, there are many MQ message middleware products in the market. As an indispensable part of the IM system, how should we choose them?

What is message queuing middleware

Message queuing middleware (referred to as message middleware) refers to the use of efficient and reliable message delivery mechanism for platform-independent data exchange, and based on data communication for distributed system integration.

By providing message passing and message queuing model, it can provide the functions of application decoupling, elastic scaling, redundant storage, traffic peaking, asynchronous communication, data synchronization and so on in distributed environment. As an important component of distributed system architecture, it plays an important role.

At present, open source message middleware is full of eyes, and there are many that can be familiar to everyone, such as ActiveMQ, RabbitMQ, Kafka, RocketMQ, ZeroMQ and so on. No matter which one you choose, it will be useful. After all, it is not tailor-made for you.

Some large manufacturers may have accumulated some experience in the long-term use process, coupled with the relatively stable and stable usage scenarios of their message queues, or the message middleware currently on the market can not meet their own needs. at the same time, it also has enough energy and manpower to choose to tailor a message middleware for itself.

However, most companies still do not choose to repeat the wheel, so it is particularly important to choose a message middleware that suits them.

Even the former will go through such a selection process before developing stable and reliable related products.

The introduction of message middleware into the overall architecture is bound to consider many factors, such as costs and benefits, how to achieve the best performance-to-price ratio?

Although there are many kinds of message middleware, each has its own focus, and it is undoubtedly the best way to choose suitable for yourself and enhance your strengths and circumvent your weaknesses. If you are at a loss about this, you may be able to refer to one or two in this article.

Brief introduction of all kinds of message queues

ActiveMQ

Message-oriented middleware produced by Apache and written in Java language based on JMS1.1 specification provides efficient, scalable, stable and secure enterprise-level message communication for applications.

However, due to the heavy burden due to historical reasons, the current market share is not as large as the latter three kinds of message middleware, and its latest architecture is named Apollo, known as the next generation ActiveMQ, which can be understood by interested students.

RabbitMQ

The message middleware of AMQP protocol implemented in Erlang language originally originated from the financial system and was used to store and forward messages in the distributed system.

Today, RabbitMQ has been recognized by more and more people, which is inseparable from its excellent performance in reliability, usability, scalability, rich functions and so on.

Kafka

A distributed, multi-partition, multi-copy and distributed messaging system based on ZooKeeper coordination, which was originally developed by LinkedIn in Scala language, has been donated to the Apache Foundation.

It is a high-throughput distributed publish and subscribe messaging system, which is widely used because of its horizontal scalability and high throughput. At present, more and more open source distributed processing systems such as Cloudera, Apache Storm, Spark, Flink and so on support integration with Kafka.

RocketMQ

Is Ali open source message middleware, has been donated to the Apache Foundation, it is developed by the Java language, with high throughput, high availability, suitable for large-scale distributed system applications and other characteristics, experienced the baptism of double 11, the strength can not be underestimated.

ZeroMQ

Known as the fastest message queue in history, it is developed based on the C language. ZeroMQ is a message processing queue library that scales between multiple threads, multiple kernels, and hosts.

Although most of the time we are used to classifying it into the message queue family, it is essentially different from the previous ones. ZeroMQ itself is not a message queue server, but more like a set of underlying network communication libraries, adding a layer of encapsulation to the original Socket API.

At present, there are many message middleware in the market, such as PhxQueue, CMQ, CKafka of Tencent, and NSQ based on Go language. Sometimes people also regard products like Redis as a kind of message middleware.

Of course, they are all excellent, but the space limit of this article is not exhaustive. Below, we will select two typical message middleware, RabbitMQ and Kafka, to analyze them, and strive to explain the main points in the selection of message middleware from a fair and just standpoint.

Key points of type selection of message middleware

Whether a message middleware meets the requirements needs to be examined from multiple dimensions.

The first thing is the functional dimension, which directly determines whether you can maximize out-of-the-box use, and then shorten the project cycle, reduce costs and so on.

If the function of a message middleware can not reach the desired function, then secondary development is needed, which will increase the technical difficulty and complexity of the project and increase the project cycle.

Specific selection index of message middleware

Functional dimension

Functional dimensions can be divided into multiple sub-dimensions, which can be roughly divided into the following.

Priority queue

Priority queues are different from first-in-first-out queues, high-priority messages have the privilege of being consumed first, which can provide different message levels of guarantee for downstream.

However, this priority also requires a premise: if the consumer consumes faster than the producer, and there is no message accumulation in the message middleware server (commonly referred to as Broker).

Then there is no real meaning to set the priority of the message sent, because the producer is consumed by the consumer as soon as the producer sends a message, then there is at most one message in the Broker, and priority is meaningless for a single message.

Delay queue

Whether you will encounter such a prompt when you are shopping online: "if you do not pay within 30 minutes, the order will be cancelled automatically", which is a typical application scenario of delayed queue.

The delay queue stores the corresponding delay message. The so-called "delay message" means that when the message is sent, the consumer does not want to get the message immediately, but waits for a specific time before the consumer can get the message for consumption.

There are generally two types of delay queues:

Message-based delay means that a different delay time is set for each message, so that whenever a new message enters the queue, it will be reordered according to the delay time, which will also have a great impact on performance.

Queue-based delay is mostly used in practical applications. Queues with different delay levels are set, such as 5s, 10s, 30s, 1min, 5mins, 10mins and so on. The delay time of messages in each queue is the same, which avoids the performance suffering of delay sorting, and timeout messages can be delivered through a certain scanning strategy (such as timing).

Dead letter queue

Messages cannot be delivered correctly for some reasons. in order to ensure that messages will not be discarded for no reason, they are generally placed in a queue with a special role, which is called a dead letter queue.

Correspondingly, there is a concept of "fallback queue". If a consumer has an exception during consumption, then the consumption will not be Ack, and the message will always be placed at the top of the queue after the rollback operation occurs, and then it will be processed and rolled back constantly, causing the queue to fall into an endless loop.

To solve this problem, you can set up a fallback queue for each queue, which, together with the dead letter queue, provides a mechanism for exception handling. In practice, the role of fallback queues can be played by dead letter queues and retry queues.

Retry queue

In fact, it can be regarded as a fallback queue, specifically, when the consumer fails to consume a message, it rolls back the message to the Broker in order to prevent the message from being lost for no reason.

Unlike the fallback queue, the retry queue is generally divided into multiple retry levels, and each retry level generally sets the redelivery delay. The more retries, the greater the delivery delay.

For example, when a message fails to be consumed for the first time, the redelivery delay of Q1Magi Q1 in the retry queue is 5s, and the message is redelivered after 5s.

If the message consumption fails again, the redelivery delay of Q2Magi Q2 in the retry queue is 10s, and the message is delivered again after 10s.

By analogy, the more times you retry, the longer it will take to re-deliver. For this reason, you need to set an upper limit. If you exceed the number of deliveries, you will join the dead letter queue.

The delay level needs to be set between the retry queue and the delay queue. The difference between them is that the delay queue action is triggered internally, while the retry queue action is triggered by the external consumer. The delay queue acts once, and the scope of the retry queue is passed backward.

Consumption pattern

Consumption patterns are divided into push mode and pull mode:

Push mode means that Broker actively pushes messages to consumers with good real-time performance, but it needs a certain streaming mechanism to ensure that the messages pushed from the server will not overwhelm the consumers.

Pull mode means that the consumer side actively requests to pull (usually timed or quantitative) messages from the Broker side, and the real-time performance is worse than the push mode, but the amount of messages pulled can be controlled according to its own processing capacity.

Broadcasting consumption

There are generally two modes of message delivery-point-to-point (P2P) and publish / subscribe (Pub/Sub) mode:

For peer-to-peer patterns, messages are no longer stored in the queue after they are consumed, so it is impossible for message consumers to consume messages that have already been consumed. Although queues can support multiple consumers, a message is consumed by only one consumer.

The publish and subscribe model defines how to publish and subscribe messages to a content node called a Topic, which can be thought of as an intermediary for message delivery, where the message publisher publishes the message to a topic, while the message subscriber subscribes to the message from the topic.

The topic makes the subscriber and the publisher of the message independent of each other and ensures the delivery of the message without contact. The publish / subscribe mode is used in the one-to-many broadcast of the message.

RabbitMQ is a typical peer-to-peer model, while Kafka is a typical publish-subscribe model.

However, RabbitMQ can achieve the effect of broadcast consumption by setting the switch type to achieve the publish and subscribe model, and Kafka can also consume in the form of peer-to-peer. You can completely regard the concept of Consumer Group as the concept of queue.

However, in contrast, Kafka has stronger support for broadcast consumption than RabbitMQ because of the message backtracking feature.

Message backtracking

A general message is processed after the consumption is completed, and then the message can no longer be consumed. Message backtracking, on the contrary, refers to the message that can be consumed before it is consumed after the consumption is completed.

For messages, the common problem is "message loss". As for whether it is really due to the defect loss of message middleware or due to the misuse of the user, it is generally difficult to trace.

If the message middleware itself has the function of message backtracking, we can reproduce the "lost" message by backtracking consumption and find out the source of the problem.

The function of message backtracking is much more than that, for example, there are index recovery, local cache reconstruction, and some business compensation schemes can also be implemented by backtracking.

Message accumulation + persistence

Traffic peaking is a very important function of message middleware, and this function actually benefits from its message stacking ability.

In a sense, if a message middleware does not have the ability to stack messages, it cannot be regarded as a qualified message middleware.

Message stacking integral memory stacking and disk stacking:

RabbitMQ is a typical memory stacking, but this is not absolute. After some conditions are triggered, there will be a paging action to fetch messages in memory to disk (the paging action will affect throughput), or directly use lazy queues to persist messages directly to disk.

Kafka is a typical disk stacking in which all messages are stored on disk.

Generally speaking, the capacity of a disk is much larger than that of memory, and its stacking capacity for disk stacking is the size of the entire disk.

On the other hand, message accumulation also provides redundant storage for message middleware. Citing the case of the New York Times, it uses Kafka directly as a storage system.

Message tracking

Link tracing (Trace) in distributed architecture systems is no stranger to us. For message middleware, message link tracking (hereinafter referred to as message tracking) is equally important. The most popular understanding is to know where the message comes from, where it exists, and where it is sent.

Based on this function, we can carry out link tracking service for messages sent or consumed, and then we can quickly locate and troubleshoot problems.

Message filtering

Message filtering refers to the provision of specified categories of messages to downstream users in accordance with established filtering rules.

In the case of Kafka, it is entirely possible to send different types of messages to different Topic, thus achieving message filtering in some sense, or Kafka can also classify messages in the same Topic according to partition.

However, in a more strict sense, message filtering should take a certain way to filter established messages in accordance with certain filtering rules.

Similarly, take Kafka as an example, messages can be filtered through the Consumer Interceptor interface provided by the client or the Filter function of Kafka Stream.

Multi-tenant

Can also be called multi-lease technology, is a software architecture technology, mainly used to achieve multi-user environment to share the same system or program components, and can still ensure the isolation of data among users.

RabbitMQ can support multi-tenant technology, and each tenant is represented as a VHost, which is essentially an independent small RabbitMQ server with its own independent queues, switches and binding relationships, and it has its own independent permissions.

VHost is like a virtual machine in a physical machine. They provide logical separation between instances and allow data for different programs to securely and securely. It can not only distinguish many customers in the same RabbitMQ, but also avoid naming conflicts such as queues and switches.

Multi-protocol support

Messages are the carriers of information. In order for both producers and consumers to understand the information carried (producers need to know how to construct messages and consumers need to know how to parse messages), they need to describe messages according to a unified format, which is called message protocol.

A valid message must have a format, and a message without a format is meaningless.

General message-level protocols include AMQP, MQTT, STOMP, XMPP and so on (JMS in the message field is more of a specification than a protocol). The more protocols it supports, the wider its scope of application and the stronger its versatility.

For example, RabbitMQ can support the MQTT protocol, which gives it a place in the application of the Internet of things. There are also message middleware operating based on its own proprietary protocols, such as Kafka.

Cross-language support

For many companies, there will be a variety of programming languages in their technology stack, such as Cmax Candle +, Java, Go, PHP and so on. Message middleware itself has the feature of application decoupling. If it can further support multi-client languages, then the performance of this feature can be expanded.

Cross-language support also reflects the popularity of a messaging middleware.

Flow control

In order to solve the problem of speed mismatch between sender and receiver, a speed matching service is provided to suppress the transmission rate so that the reading rate of the receiver application is adapted to it. The usual flow control methods include Stop-and-Wait, sliding window, token bucket and so on.

Message orderliness

As the name implies, it means to ensure that the message is orderly. A common application scenario for this feature is CDC (Change Data Chapture).

Take MySQL as an example, if the order of the Binlog transmitted is wrong, such as adding 1 to a piece of data and then multiplying it by 2, it will be multiplied by 2 and then 1 after sending the wrong order, resulting in data inconsistency.

Security mechanism

After Kafka version 0.9, two security mechanisms, identity authentication and permission control, have been added:

Identity authentication refers to the identity authentication between the client and the server, including the connection authentication between the client and Broker, between Broker and Broker, and between Broker and ZooKeeper. At present, SSL, SASL and other authentication mechanisms are supported.

Permission control refers to the permission control of the read and write operations of the client, including the permission control of messages or Kafka cluster operations. Access control is pluggable and supports integration with external authorization services.

For RabbitMQ, it also provides security mechanisms for identity authentication (TLS/SSL, SASL) and permission control (read and write operations).

Message idempotency

To ensure that messages are transmitted between producers and consumers, there are generally three types of transmission guarantees (Delivery Guarantee):

At most once, at most once, the message may be lost, but it will never be transmitted repeatedly.

At least once, at least once, the message is never lost, but it may be repeated.

Exactly once, exactly once, each message must be transmitted once and only once.

For most message middleware, there are only two kinds of transmission guarantee: At most once and At least once, but it is difficult to do for the third one, so it is difficult to guarantee the idempotency of messages.

Idempotency and transaction have been introduced into Kafka since version 0.11. The idempotency of Kafka refers to the idempotency of a single producer to a single partition and single session.

Transactions can be atomically written to multiple partitions, that is, messages written to multiple partitions are either successful or rolled back. Together, these two functions can make Kafka have the ability of EOS (Exactly Once Semantic).

However, if we want to consider the global idempotence, we also need to consider comprehensively from the upstream and downstream aspects, that is, the related business level, idempotent processing itself is also an important issue to be considered at the business level.

At the downstream consumer level, for example, it is possible that an exception occurs after consumers consume a message without having time to confirm the message, and then they have to re-consume the original consumed message after recovery. Then this type of message idempotence cannot be guaranteed by the message middleware level.

If the global idempotence is to be guaranteed, more external resources need to be introduced to guarantee it, such as using the order number as the unique identity and setting a deduplicated table downstream.

Transactional message

Transaction itself is a familiar word, which consists of all the operations performed between the beginning (Begin Transaction) and the end (End Transaction) of the transaction.

There are not a few message middleware that support transactions, both Kafka and RabbitMQ support, but these two transactions refer to the transaction in which the message occurs to the producer, either successfully or unsuccessfully.

Message middleware can be used as a means to implement distributed transactions, but it does not provide the function of global distributed transactions.

The following table is a summary comparison and supplementary description of the functions of Kafka and RabbitMQ:

Performance

Functional dimension is an important reference dimension in the selection of message middleware, but it is not the only dimension, sometimes performance is more important than function, and performance and function are often contrary to each other.

Kafka degrades its performance when it turns on idempotent and transactional functions, and RabbitMQ greatly affects its performance when it opens the rabbitmq_tracing plug-in.

What does performance mean?

The performance of message middleware generally refers to its throughput. Although in terms of functional dimension, the advantage of RabbitMQ is greater than that of Kafka, the throughput of Kafka is 1 or 2 orders of magnitude higher than RabbitMQ.

Generally speaking, the stand-alone QPS of RabbitMQ is within ten thousand levels, while the stand-alone QPS of Kafka can be maintained at the level of 100,000, or even reach the level of one million.

Note: the throughput of message middleware is always limited at the hardware level. Take the bandwidth of the network card as an example. If the bandwidth of the stand-alone network card is 1Gbps, if you want to achieve million-level throughput, then the message body size must not exceed (1Gb/8) / 100W, that is, about 134B.

In other words, if the body size of the message is larger than 134B, it is impossible to reach a million-level throughput. This method of calculation can also be applied to memory and disk.

What are the performance metrics?

Delay, as an important indicator of performance dimension, is often ignored in the field of message middleware, because the requirement of timeliness in general scenarios using message middleware is not very high. If timeliness is required, it can be achieved by RPC.

Message middleware has the ability to pile up messages. the greater the message accumulation, the longer the end-to-end delay. At the same time, delay queue is also a major feature of some message middleware. So why pay attention to the latency of message middleware?

Message middleware can decouple the system. For a message middleware with low latency, it can enable upstream producers to return quickly after sending messages, and it can also allow consumers to get messages more quickly. Without stacking, it can make the cascade action between the whole upstream and downstream applications more efficient.

Although it is not recommended to use message middleware in scenarios with high timeliness, if the delay of the message middleware used is excellent, then the performance of the overall system will be greatly improved.

Reliability + availability

Message loss is a pain point that we have to face when using message middleware, and the reliability of messages behind it is also a key factor to measure the quality of message middleware. Especially in the field of financial payment, the reliability of information is particularly important.

When it comes to reliability, however, when it comes to usability, note the difference between the two:

The reliability of message middleware refers to the degree of guarantee that messages will not be lost.

The availability of message middleware refers to the percentage of time it takes to run without failure, which is usually measured by several 9s.

In a narrow sense, distributed system architecture is the application implementation of consistency protocol theory, and it can also be traced back to the consistency protocol behind message middleware in terms of message reliability and availability.

For Kafka, it uses a consistency protocol similar to PacificA, ensures synchronization between multiple replicas through ISR (In-Sync-Replica), and supports strong consistency semantics (implemented through Acks).

The corresponding RabbitMQ implements multi-copy and strong consistency semantics through mirrored ring queues.

Multiple replicas can ensure that Slave can be promoted as a new Master and continue to provide services to ensure availability after an abnormal Master node outage.

Kafka is designed for log processing at the beginning, which gives people a bad impression that the requirement of data reliability is not high, but with the upgrade and optimization of the version, its reliability has been greatly enhanced. Please refer to KIP101 for details.

At present, RabbitMQ is mostly used in the field of financial payment, while Kafka is mostly used in log processing, big data and so on. With the continuous improvement of RabbitMQ performance and the further enhancement of Kafka reliability, it is believed that each other can get a piece of the pie in areas that they were not good at before.

Synchronous flushing is an effective way to enhance the reliability of a component, and message middleware is no exception. Both Kafka and RabbitMQ can support synchronous flushing.

However, the author has some doubts about synchronous flushing: in most scenarios, the reliability of a component should not be guaranteed by synchronous flushing, which is extremely wasteful, but by using a multi-copy mechanism.

Another aspect I would like to mention here is extensibility, which I narrowly summarize to the dimension of usability. The extensibility of message middleware can enhance its availability and scope. For example, RabbitMQ supports a variety of message protocols mentioned earlier, which is based on its plug-in extension implementation.

And in terms of cluster deployment, thanks to the horizontal expansion ability of Kafka, it can basically reach the level of linear capacity improvement. In the introduction of LinkedIn practice, it is mentioned that there are Kafka clusters with more than a thousand devices.

Operation and maintenance management

In the process of using message middleware, there will inevitably be a variety of anomalies, including client and server, so how to monitor and repair timely and effectively?

There are peaks and troughs in business line traffic, especially in the field of e-commerce, so how to carry out effective capacity assessment, especially during the promotion period? Kicking power supply, network cable digging and other events emerge one after another, how to effectively do more work in different places?

All these are inseparable from the derivative product of message middleware-operation and maintenance management. Operation and maintenance management can also be further subdivided, such as application, audit, monitoring, alarm, management, disaster recovery, deployment and so on.

Application and review are well understood. Managing and controlling resources at the source can not only effectively correct the usage specifications of the application side, but also do a good job in traffic statistics and evaluation with monitoring.

In general, the application and review are well integrated with the company's internal system, so it is not suitable to use open source products.

Monitoring and alarm are also easy to understand. Omni-directional monitoring of the use of message middleware can not only provide benchmark data for the system, but also cooperate with alarm when abnormal conditions are detected, so that operators and developers can intervene quickly.

In addition to general monitoring items (such as hardware, GC, etc.), message middleware also needs to focus on end-to-end delay, message audit, message accumulation, and so on:

For RabbitMQ, the most orthodox monitoring and management tool is the rabbitmq_management plug-in, but there are many excellent products in the community, such as AppDynamics, Collectd, DataDog, Ganglia, Munin, Nagios, New Relic, Prometheus, Zenoss and so on.

Kafka is not inferior in this respect, such as: Kafka Manager, Kafka Monitor, Kafka Offset Monitor, Burrow, Chaperone, Confluent Control Center and other products, especially Cruise can also provide automatic operation and maintenance functions.

Whether it is capacity expansion, downgrade, version upgrade, cluster node deployment, or fault handling are inseparable from the application of management tools, a complete set of management tools can achieve twice the result with half the effort in the event of changes.

Faults can be large or small, generally some application anomalies, or stand-alone failures such as machine power outage, network anomaly, disk damage and so on.

If it is a failure of the data center, it will involve remote disaster recovery. The key point is how to copy data effectively. Kafka can refer to MirrorMarker, uReplicator and other products, while RabbitMQ can refer to Federation and Shovel.

Community dynamics and ecological development

For popular programming languages such as Java and Python, if you encounter some anomalies during use, you can basically solve them with the help of search engines, because the more people use a product, the more holes you step on and the more corresponding solutions.

Message middleware is also applicable. If you choose an "obscure" message middleware, it may be easy to use in some ways, but the version is updated slowly and it is difficult to get support from the community when you encounter thorny problems.

On the contrary, if you choose a "popular" message middleware, its update is strong, which can not only quickly make up for the previous shortcomings, but also adapt to the rapid development of technology to change some new functions. this allows you to "stand on the shoulders of giants".

In the dimension of operation and maintenance management, we mentioned that both Kafka and RabbitMQ have a series of open source monitoring and management products, which benefit from the rapid development of their community and ecology.

Summary of misunderstandings in the selection of message Middleware

Misunderstanding of type selection

Before selecting message middleware, you can ask yourself a question: do you really need a message middleware?

After figuring out this question, you can continue to ask yourself a question: do you need to maintain your own set of messaging middleware?

In order to save costs, many startups will choose to buy cloud services related to message middleware directly. They only need to pay attention to sending and receiving messages, and the rest can be outsourced.

Many people have an impulse to develop message middleware on their own. You can simply encapsulate ArrayBlockingQueue in Java, or you can form a message middleware based on file, database, Redis and other underlying storage encapsulation.

As a basic component, message middleware is not as simple as expected, and it also needs a complete set of products to manage the whole ecology of operation and maintenance.

Self-research will also have handover problems, if the documentation is not complete and the operation is not standard, it will bring a nightmare experience to the newcomers.

Is there really a need for self-research? If it is not the oppression of KPI, you can consider the following two questions first:

Are the message middleware on the market really unable to meet the current business needs?

Does the team have enough ability, manpower, financial resources and energy to support self-research?

Many people will refer to a lot of comparative articles on the network when making the selection of message middleware, but their professionalism, rigor, and their political positions need to be examined with skepticism.

For example, it is best for some articles to directly define a certain message middleware without any qualification or scenario.

Some articles do comparative analysis of function and performance without specifying the version of message middleware and test environment, and such articles can be spurned.

Message middleware is like a pony crossing the river, choosing the right one is the most important. This needs to be in line with their own business needs, technology to serve the business, generally speaking, according to the function, performance and other six dimensions mentioned in the previous section to filter one by one. The deeper choice is whether you can master its soul.

The author disdain: RabbitMQ lies in Routing, and Kafka lies in Streaming, so it is particularly important to understand its root for oneself to be able to choose the right message middleware.

Message middleware type selection must not blindly pursue performance or function, performance can be optimized, function can be secondary development.

If you have to make a choice between functionality and performance, then performance is preferred, because in general, there is less room for performance optimization than for functional expansion. However, looking at the long-term development, ecology is more important than performance and function.

Reliability misunderstanding

In many cases, there is also a misunderstanding in terms of reliability: if you want to find a product to ensure the absolute reliability of the message, unfortunately there is nothing absolute in the world, it can only be said to be as perfect as possible.

To ensure the reliability of messages as much as possible depends not only on the message middleware itself, but also on the upstream and downstream, which needs to be guaranteed from three dimensions: the production side, the server side and the consumer side.

Another consideration for the selection of message middleware is to fit the team's own technology stack system as much as possible. although there are no bad message middleware, only bad programmers, it is much easier for a C stack team to dig deep into PhxQueue than to dig deep into Kafka written by Scala.

Message middleware road to simple: one send, one deposit, one consumption, there is no best message middleware, only the most suitable message middleware.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.