Shulou(Shulou.com)06/02 Report--

This article mainly explains "how to use redis pipeline well". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to use redis pipeline well.

Editor's note: pipeline is a throughput improvement mechanism of Redis and is suitable for multi-key read and write scenarios, such as reading value of multiple key at the same time, or updating value of multiple key. In the course of work, I found that many friends know something about pipeline, but I don't know if I have a deeper understanding or which pit. Let's analyze the redis pipeline mechanism together and uncover its mystery.

Redis itself is based on the Request/Response protocol (stop mechanism). Normally, the client sends a command and waits for Redis to return the result. Redis receives the command and responds after processing. In this case, if you need to execute a large number of commands at the same time, it is to wait for the reply of the last command, which not only has more RTT (Round Time Trip), but also calls the system IO frequently to send network requests. To improve efficiency, pipeline appeared, which allows clients to send more than one command at a time without waiting for the result of the previous command, which is a bit like the network's Nagel algorithm (TCP_NODELAY option). Pipeline not only reduces RTT, but also reduces the number of IO calls (IO calls involve switching from user state to kernel state).

To support Pipeline, you need both server support and client support. For the server, what is needed is to be able to handle multiple commands sent by a client over the same TCP connection, which can be understood to mean that multiple commands are split in the same way as a single command (the clich é phenomenon of stickiness). This is how Redis handles it. On the other hand, the client needs to cache multiple commands, send them when the buffer is full or meet the sending conditions, and finally process the Redis reply.

Note: Redis's Pipeline is different from Transaction (Redis transaction). Transaction stores the client's command and executes it at last, while Pipeline processes one (batch) and responds to one. From the different processing mechanisms of the two, the execution of the command in the Redis transaction is atomic (note that some of the commands will continue to be executed if there is an error. Here the atom says that the command execution is complete. It is not interrupted by other Redis commands in the middle, and the execution of commands in pipeline is not necessarily atomic. But there is one difference here, that is, in the pipeline mechanism, the client does not call read to read the buffered data in the socket (unless all the commands in the pipeline have been issued), which results in that if the Redis reply data fills the receive buffer (SO_RECVBUF), then the client will use the ACK,WIN=0 (receive window) to control that the server can no longer send data. The data is buffered in Redis's client response buffer. So you need to pay attention to controlling the size of the Pipeline. As shown below:

We can imagine that if the client uses ACK,WIN=0 (receive window) to control that the server can no longer send data, then the data will accumulate in the server's socket send buffer. If the server's socket send buffer is also full, then the server's call to write (socket) will block or fail.

Now that we have mentioned the concept of sliding window in tcp/ip, let's briefly summarize the sliding window:

The function of sliding window in TCP is to provide the reliability and flow control characteristics of TCP. Sliding window can be divided into sending window and receiving window, which correspond to send buffer and receive buffer respectively. The size of the send window is set according to the size of the client receive buffer (the purpose of the three-way handshake is to connect to the designated port of the server, establish a TCP connection, synchronize the serial numbers and confirmation numbers of both sides of the connection, and exchange TCP window size information).

The send window contains data that has been sent but has not yet been received from the Ack and data that has not been sent but is allowed to be sent by the peer.

The TCP receiving buffer contains data applied as read, received data (ACK has been replied), and to be received, wherein the space to be received can be called a receiving window.

In the process of using pipeline, we should pay attention to controlling the total size of commands in the pipeline at one time, so that the response result can not burst the socket receiving buffer. Here's a question: is there any other way to improve the processing performance of pipeline? In theory, for example, we can use data compression mechanism to further reduce the total size of data transmission, but this requires the client and server to provide a decompression mechanism, while consuming a certain amount of server CPU.

Previous selection

Redis has been a passer-by ever since.

Core concepts of sentinel

Statistical Mechanism of sentinel sliding window

Principle of sentinel Cluster flow Control

Sentinel dubbo adaptation mechanism

Design and implementation of distributed Lock

Think the article is good, enlighten and help you. I hope it can be forwarded to more friends. If you have any questions, please follow the official account below and send me questions. Thank you. At this point, I believe you have a deeper understanding of "how to use redis pipeline". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.