Shulou(Shulou.com)05/31 Report--

This article mainly explains the "Redis slow log related underlying principles", the content of the article is simple and clear, easy to learn and understand, the following please follow the editor's ideas slowly in depth, together to study and learn "Redis slow log related underlying principles" bar!

01, preface

I believe that many friends know that Redis has the query function of slow log when using Redis, and they have seen it more or less. So what does the bottom layer of Redis look like if you create slow logs and what is the structure of slow logs? This article will introduce you to it. Let's first look at a screenshot of a slow log.

Using the slowlog get 2 command to view the two recent slow log messages, such as the figure above, we can see that the information contained in each log is composed of six parts, numbered 0-5 from top to bottom, which in turn means

0: the unique number of the log ID

1: the current timestamp of the command execution

2: the time taken for the execution of the command is subtle.

3: specific execution commands and parameters

4: ip and port of the client (only supported by version 4.0 or above)

5: client name (supported above version 4.0)

As shown in the figure above, the ID of the first slow log is 41, the timestamp of command execution is 1575729996, and 16129 subtleties are executed. The specific command executed is slowlog get,ip and port is 27.38.56.88. The name of the client is not set.

02. Slow log command settings

View command

We already know the format of a slow log above. Naturally, the question we can think of is how long a command is executed, we can think of it as a slow query, and how many slow logs can be saved at most.

We can view the length setting of Redis through the config get slowlog-log-slower-than command and the maximum number of slow log entries through config get slowlog-max-len. As shown in the following picture.

Setup command

Above, we use the config get command to check the duration setting and the number of messages. On the contrary, we can use config set to set the relevant parameters. As shown in the figure below, we first check the configuration, and then use the config set slowlog-log-slower-than 1000 command and config set slowlog-max-len 64 command to set the specific values:

From the above operation, we can see that the relevant configuration changes have taken effect.

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

Slowlog-log-slower-than: this parameter means that any command that takes longer than this time will be recorded as a slow log, in microseconds.

Slowlog-max-len: this parameter indicates the maximum number of slow logs recorded. When this value is set, the new log is added and the minimum ID log is deleted.

To verify the second point above, I set slowlog-log-slower-than to 10 microseconds and slowlog-max-len to 5 for the experiment. The first time I used the slowlog get command to query the five slow logs, the numbers were from 83 to 87.

Using the slowlog get command again, the result of the query is 84-88, indicating that the one with ID 83 has been deleted.

03. Storage principle of slow log

Storage structure

Struct redisServer {ID list * slowlog;// of the next slow query log of long long slowlog_entry_id;// saves the linked list of all slow query logs the value of the long long slowlog_log_slower_than;// server configuration slowlog-log-slower-than option the value of the slowlog-max-len configured by the unsigned long slowlog_max_len;// server}

The slowlog_entry_id property that stores slow logs in the server state of Redis has an initial value of 0, which increases by 1 each time a slow log is created.

All the slow logs are stored in the slowlog linked list, which is made up of slowlogEntry structures, and each slowlogEntry represents a slow log.

Slowlog_log_slower_than and slowlog_max_len are the relevant parameters of the previous server configuration.

Slowlog linked list

Typedef struct slowlogEntry {long long id;// unique identifier time_t time; / / the time the command was executed, formatted as unix timestamp, the time consumed by the long long duration;// command, in microseconds, robj * * argv;// command and command parameters int argc; / / number of command and command parameters} slowlogEntry

Image-20191211220858341

The relevant fields of the slowlogEntry entity have the following meanings:

Id: a unique ID that identifies slow logs

Time: timestamp of command execution

Duration: command execution is time-consuming, in subtlety

Agrv: array of commands and parameters

Argc: number of commands and parameters

The figure above shows that a slow log occurred during the execution of the command set number 520. the execution of the command took 10 microseconds.

04. Slow operation log

Knowing the storage structure of slow logs, we need to consider how to create slow logs according to conditions when executing commands.

First of all, we need to record the timestamp before and after the execution of the command, then subtract to calculate the execution time of the command, and then compare the slowlog-log-slower-than configured by the Redis server to decide whether to record the slow log. In addition, when recording the slow log, we need to determine whether to delete the longest log information according to the slowlog_max_len value. The pseudo code is as follows:

/ / 1. Record the timestamp long before = now () before the command is executed; / / 2. Execute command execute (argv, argc); / / 3. Record the timestamp long after = now () after the command is executed; / / 4. Call the create slow log function slowlogPushEntryIfNeed (argv, argc, after-before)

The slowlogPushEntryIfNeed function is mainly used to determine whether to insert data and whether to delete old data.

Void slowlogPushEntryIfNeed (robj * * argv, int argc, long long duration) {/ 1. Determine whether to enable slow log if (server.slowlog_log_slower_than)

Server.slowlog_log_slower_than) {/ / insert} while (listLength (server.slowlog) > server.slowlog_max_len) {/ / delete}}

We first determine whether the server is configured with a timeout parameter, and return directly if the timeout parameter is less than 0, otherwise compare whether the command execution time has timed out, and insert a slow log if the timeout occurs; finally, whether the number of slow logs reaches the upper limit, and delete if it reaches the limit.

Thank you for your reading, the above is the content of "Redis's slow log related underlying principles". After the study of this article, I believe you have a deeper understanding of Redis's slow log related underlying principles, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.