Shulou(Shulou.com)05/31 Report--

This article will explain in detail the example analysis of the command execution process in Redis. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

How does Redis execute commands from a remote client?

Redis client (client)

Redis is a single-threaded application. How does it link to multiple client resumes and process commands?

Because Redis is based on Redis O multiplexing technology, in order to be able to handle requests from multiple clients, Redis creates a redisClient data structure locally for each client linked to the redisClient server, which contains the status and commands executed by each client. The Redis server uses a linked list to maintain multiple redisClient data structures.

Use a linked list to manage all redisClient on the server side.

Struct redisServer {/ /... List * clients; / * List of active clients * /.}

So I'll take a look at the data structures and important parameters that redisClient contains:

Typedef struct redisClient {/ / client status flag int flags; / * REDIS_SLAVE | REDIS_MONITOR | REDIS_MULTI. * / / socket descriptor int fd; / / currently in use database redisDb * db; / / id (number) int dictid; / / client name robj * name; / * As set by CLIENT SETNAME * / / query buffer sds querybuf / / query buffer length peak size_t querybuf_peak; / * Recent (100ms or more) peak of querybuf size * / / Parameter number int argc; / / Parameter object array robj * * argv; / / record the command struct redisCommand * cmd executed by the client, * lastcmd; / / the type of request: inline command or multiple commands int reqtype; / / number of remaining unread command contents int multibulklen / * number of multi bulk arguments left to read * / / length of command content long bulklen; / * length of bulk argument in multi bulk request * / / reply list list * reply; / / Total size of objects in the list unsigned long reply_bytes; / * Tot bytes of objects in reply list * / / sent bytes, int sentlen; / * Amount of bytes already sent in the current buffer or object being sent is used to deal with short write. * / / reply offset int bufpos; / / reply buffer char buf [redis _ REPLY_CHUNK_BYTES]; / /.}

It is important to note that redisClient does not refer to a remote client, but to a local data structure of a Redis service, and we can understand that this redisClient is a mapping or proxy of a remote client.

Flags

Flags represents the current role of the client, as well as the current state. He is special and can represent one or more states alone.

Querybuf

Querybuf is a sds dynamic string type, and the so-called buf description means that it is just a buffer for storing commands that have not been parsed.

Argc & argv

The querybuf above is an unprocessed command. When Redis parses the querybuf command, the resulting number of parameters and parameters are saved in argc and argv, respectively. Argv is an array of redisObject.

Cmd

Redis uses a dictionary to keep all the redisCommand. Key is the name of redisCommand, and the value is a redisCommand structure, which stores the statistical information such as the implementation function of the command, the flag of the command, the number of parameters that the command should be given, the number of times the command is executed and the total time spent. Cmd is a redisCommand.

When Redis parses argv and argc, it will query the corresponding redisCommand in the dictionary according to the array argv [0]. In the above example, Redis will go to the dictionary to look up the redisCommand corresponding to the command SET. Redis executes the implementation function of the command in redisCommand.

Buf & bufpos & reply

Buf is an array of length REDIS_REPLY_CHUNK_BYTES. After Redis performs the corresponding operation, the returned data to be returned will be stored in buf. Bufpos is used to record the number of bytes used in buf. When the data to be recovered is greater than REDIS_REPLY_CHUNK_BYTES, redis will use the linked list of reply to save the data.

Other parameters

Other parameters can be understood by looking at the notes, which means literally. The omitted parameters are basically related to the parameters of Redis cluster management, which will be explained in a later article.

Link and disconnect of the client

As mentioned above, redisServer uses a linked list to maintain all redisClient states. Every time a client initiates a link, a corresponding redisClient data structure is generated in Redis, which is added to the linked list of clients.

A client is likely to be disconnected for many reasons.

There are several types in general:

The client actively exits or is kill.

Timeout timed out.

In order to protect itself, Redis will break clients that develop data that exceeds the limit.

In order to protect itself, Redis will cut off clients that need to return more data than the limit.

Call summary

When the client-side and server-side nesting words become readable, the server invokes the command to request the processor to do the following:

Read the data in the nested word and write it to querybuf.

Parse the commands in querybuf and record them in argc and argv.

Find the corresponding recommand according to argv [0].

Execute the corresponding implementation function of recommand.

After execution, the result is stored in buf & bufpos & reply and returned to the caller.

Redis Server (server side)

The above is to observe the execution of commands from the perspective of redisClient. The next part of the article will observe how Redis implements the execution of commands from the code level of Redis.

Startup of redisServer

Before you understand how redisServer works, you need to understand what redisServer startup does:

You can continue to observe the main () function of Redis.

Int main (int argc, char * * argv) {/ /... / create and initialize the server data structure initServer (); / /.}

We only focus on the function initServer (), which is responsible for initializing the data structure of the server. Continue to track the code:

Void initServer () {/ /... / / create eventLoop server.el = aeCreateEventLoop (server.maxclients+REDIS_EVENTLOOP_FDSET_INCR); / * Create an event handler for accepting new connections in TCP and Unix * domain sockets. * / / call for (j = 0; j) for the TCP connection Association connection response (accept) processor / / the connect () used to accept and reply to the client

< server.ipfd_count; j++) { if (aeCreateFileEvent(server.el, server.ipfd[j], AE_READABLE, acceptTcpHandler,NULL) == AE_ERR) { redisPanic( "Unrecoverable error creating server.ipfd file event."); } } // 为本地套接字关联应答处理器 if (server.sofd >

0 & & aeCreateFileEvent (server.el,server.sofd,AE_READABLE, acceptUnixHandler,NULL) = = AE_ERR) redisPanic ("Unrecoverable error creating server.sofd file event."); / /...}

Due to the space limit, we omit a lot of code that has nothing to do with this article, and retain the core logic code.

As we explained in the previous article, "the event-driven model in Redis," redis uses different event handlers to handle different events.

In this code:

EventLoop with event handler initialized

Two event handlers, acceptTcpHandler and acceptUnixHandler, are registered with eventLoop to handle remote links and local links, respectively.

The creation of redisClient

When a remote client connects to Redis's server, the acceptTcpHandler event handler is triggered.

The acceptTcpHandler event handler, which creates a link. Then continue to call acceptCommonHandler.

The purpose of the acceptCommonHandler event handler is:

Call the createClient () method to create a redisClient

Check whether the created redisClient exceeds the upper limit of the number allowed by server

Refuse a remote connection if the limit is exceeded

Otherwise, the redisClient is created successfully

And update the count of connections, update the flags field of redisClinet

At this point, Redis creates the redisClient data structure on the server side, and the remote client creates a proxy in redisServer. The remote client establishes contact with the Redis server and can send commands to the server.

Processing command

In the number of createClient () lines:

/ / bind read event to event loop (start receiving command request) if (aeCreateFileEvent (server.el,fd,AE_READABLE,readQueryFromClient, c) = = AE_ERR)

ReadQueryFromClient is registered with eventLoop. The purpose of readQueryFromClient is to read the contents of the client's query buffer from client.

The function processInputBuffer is then called to handle the client's request. There are several core functions in processInputBuffer:

ProcessInlineBuffer and processMultibulkBuffer parse the commands in querybuf and record them in argc and argv.

ProcessCommand finds the corresponding recommen according to argv [0] and executes the corresponding execution function of recommend. The correctness of the command is also verified before execution. Store the results in buf & bufpos & reply

Return data

Once everything is ready, you need to return the data to the remote caller after executing the command. The call chain is as follows

ProcessCommand-> addReply-> prepareClientToWrite

We have seen familiar code in prepareClientToWrite:

AeCreateFileEvent (server.el, c-> fd, AE_WRITABLE,sendReplyToClient, c) = = AE_ERR) return REDIS_ERR

The sendReplyToClient event handler is bound to eventloop.

Observing the code in sendReplyToClient, it is found that if the bufpos is greater than 0, the buf will be sent to the remote client. If the length of the linked list reply is greater than 0, the linked list reply will be traversed and sent to the remote client. It should be noted here that in order to avoid excessive amount of reply data, the Redis will be slow due to excessive occupation of resources. To solve this problem, when the total number of writes is greater than REDIS_MAX_WRITE_PER_EVENT, Redis will temporarily interrupt the write, record the progress of the operation, give up processing time to other operations, and wait for the rest to continue next time. We have seen too many such tricks along the way.

Summary

After the remote client connects to the redis, the redis server creates a redisClient as a proxy for the remote client.

Redis reads the data in the nested word and writes it to querybuf.

Parse the commands in querybuf and record them in argc and argv.

Find the corresponding recommand according to argv [0].

Execute the corresponding execution function of recommend.

After execution, the result is stored in buf & bufpos & reply.

Return to the caller. When the data is returned, the amount of data written will be controlled, and it will be divided into several times if the meeting is over. Ensure the appropriate time for redis.

As a single-threaded application, Redis has always carried out the idea that there is an upper limit for the execution of each step (including the upper limit of execution time or the upper limit of file size). Once the upper limit is reached, the current execution progress will be recorded and executed next time. It ensures that the Redis can respond in time without blocking.

This is the end of the article on "sample analysis of command execution process in Redis". I hope the above content can be helpful to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.