Shulou(Shulou.com)06/01 Report--

The editor in this issue will bring you the method of realizing message queue in redis, which will be analyzed and described from a professional point of view. I hope you can get something after reading this article.

Message queue, Message Queue, is often used to solve the problem of resource consistency in concurrent systems, improve peak processing capacity, ensure the sequence, recoverability and delivery of messages, decouple applications, or realize asynchronous communication.

There are many MQ applications on the market (for example, Kafka,RabbitMQ,Disque), and they can also be implemented based on Redis. Typical solutions are:

Implementation of LPUSH+BRPOP based on List

PUB/SUB, subscription / publish mode

Implementation based on Sorted-Set

Implementation based on Stream type

In message queuing usage, there are producer producter and consumer consumer. The producer is responsible for generating the message, and the consumer is responsible for using the processing message.

Production refers to putting messages into message queues.

Consumption refers to reading and processing messages. Usually, after a message is consumed, it should be deleted from the message queue.

Implementation of LPUSH+BRPOP based on List

A typical command is:

LPUSH, BRPOP the message queue, pull the message from the queue, block the mode

Is a typical solution based on FIFL queues. LPUSH is what producers do, and BRPOP is what consumers do.

This model has many advantages:

Easy to implement

Reids supports persistence of messages, which means that messages will not be lost and can be viewed repeatedly (note that it is not consumption, just look at the instructions of the LRANGE class).

The order can be guaranteed, and the LPUSH command can be used to ensure the sequence of messages.

With RPUSH, messages can be placed at the beginning of the queue to achieve the purpose of priority messages, and a simple message priority queue can be achieved.

At the same time, there are some disadvantages:

It is troublesome to make a consumption confirmation ACK, but it cannot guarantee the downtime after the consumer has not been processed after reading it. Caused the message to be lost unexpectedly. You usually need to maintain your own Pending list to ensure that messages are processed and confirmed.

Cannot do broadcast mode, such as typical Pub/Discribe mode.

Cannot repeat consumption, once consumption will be deleted

Group consumption is not supported, and you need to solve it at the business logic layer.

PUB/SUB, subscription / publish mode

SUBSCRIBE, used to subscribe to channel PUBLISH, send messages to channel UNSUBSCRIBE, and unsubscribe

Producers and consumers interact through the same channel (Channel). A channel is actually a queue. There are usually multiple consumers. Multiple consumers subscribe to the same channel, and when the producer publishes messages to the channel, the channel immediately publishes the messages to each consumer one by one. It can be seen that the channel is a divergent channel for consumers, and each consumer can get the same message. A typical many-to-many relationship.

Typical advantages are:

In a typical broadcast mode, a message can be posted to multiple consumers

Multi-channel subscription, consumers can subscribe to multiple channels at the same time to receive multiple types of messages

The message is sent immediately, and the message does not have to wait for the consumer to read, and the consumer will automatically receive the message released by the channel.

There are also some disadvantages:

Once a message is published, it cannot be received. In other words, if the client is not online at the time of release, the message will be lost and cannot be found.

There is no guarantee that the time received by every consumer is the same.

If the consumer client has a backlog of messages, to a certain extent, it will be forced to disconnect, resulting in accidental loss of messages. It usually occurs when the production of messages is much faster than the speed of consumption.

It can be seen that Pub/Sub mode is not suitable for message storage and message backlog business, but is good at dealing with broadcasting, instant messaging and instant feedback business.

Implementation of ordered set based on SortedSet

ZADD KEY score member, press into the collection ZRANGEBYSCORE, and get the members according to score

The scheme of ordered collection is more commonly used when determining the message sequence ID, using the Score of the collection members as the message ID to ensure the order and monotonous increment of the message ID. You can usually use the scheme of timestamp + sequence number. The monotonous increment of message ID is ensured, and an ordered message queue can be made by using the characteristics of SortedSet sorted according to Score.

Compared with the above scenario, the advantage is that the message ID can be customized, which is more important when the message ID is meaningful. The disadvantage is also obvious: duplicate messages are not allowed (thought to be collections), and the message ID determines that there is an error that will cause the message to be in the wrong order.

So, if you don't need a custom message ID, this solution seems to be a bit of a chicken rib.

Implementation based on Stream type

This Stream type redis is designed to implement message queues. Support for automatic generation of message ID, packet consumption, ACK, message transfer, queue monitoring and other core message queue functions

The above is the method of implementing message queuing in redis shared by Xiaobian. If you have similar doubts, it will not hinder your understanding with reference to the above analysis. If you want to know more about it, please follow the industry information.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.