Shulou(Shulou.com)05/31 Report--

This article will explain in detail what the expiration strategy in Redis is like. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

Save expiration time

Redis can set the expiration time for each key, and each key with an expiration time set will be placed in a separate dictionary. [related recommendation: Redis video tutorial]

Typedef struct redisDb {int id; / / id is the database serial number, which is the average ttl (time to live) of database objects stored in 0-15 (default Redis has 16 databases). It is used to count the expiration time of all key-value dict * expires; / / storage key in dict * dict; / / storage database dict * blocking_keys;//blpop storage blocking key and client object dict * ready_keys / / Post-blocking push response blocking client Storage Post-blocking push key and client object dict * watched_keys;// Storage key and client object monitored by watch} redisDb

Dict is used to maintain all Key-Value key-value pairs contained in a Redis database, while expires is used to maintain keys in an Redis database with expiration time set (that is, the mapping between key and expiration time). Note that the failure time here is represented by a timestamp of milliseconds. For example, if the failure time expires at 22:45:02 on 2022-01-02, the value is 1641134702000.

When we use the expire command to set the expiration time of a key, Redis first goes to the dict dictionary table to find out whether the key to be set exists, and if so, adds the key and expiration time to the expires dictionary table.

When we use the setex command to insert data into the system, Redis first adds Key and Value to the dict dictionary table, and then adds Key and expiration time to the expires dictionary table. Note that setex can only be used for strings.

In a nutshell, the key with the failure time set and the specific failure time are all maintained in the dictionary table expires.

Set expiration time

The use of expire

The expire command is used as follows: expire key ttl (in seconds)

127.0.0.1 expire name 2 # 2 second invalidation (integer) 1 127.0.0.1 get name (nil) 127.0.0.1 get name > set name zhangfei OK 127.0.0.1 Vista 6379 > ttl name # permanent validity (integer)-1 127.0.0.1 set name zhangfei OK 6379 > expire name 30 # 30 seconds invalidation (integer) 1 127.0.1 0.1 Ranger 6379 > ttl name # and 24 seconds failure (integer) 24 127.0 .0.1 6379 > ttl name # failure (integer)-2

Redis has four different commands to set the survival time of the key (how long the key can live) or the expiration time (when the key will be deleted):

The expire command is used to set the lifetime of the key key to ttl seconds

The pexpire command sets the lifetime of the key key to ttl milliseconds

The expireat command sets the expiration time of the key key to the number of seconds timestamp specified by timestamp

The pexpireat command sets the expiration time of the key key to the millisecond timestamp specified by timestamp.

Note that the final implementations of expire, pexpire, and expireat are all implemented through pexpireat, which means that no matter which command the client executes, Redis is converted to pexpireat command execution. So the time stored in the expires dictionary is the expiration time of the key in millisecond timestamps.

Expiration policy

If a key expires, when will it be deleted?

There are three expiration policies

Timing deletion: while setting the expiration time of the key, create a timer that allows the timer to delete the key immediately when the expiration time of the key is approaching. (create timer to delete)

Lazy deletion: let the expiration of the key be ignored, but every time you get the key from the key space, check whether the obtained key expires, delete the key if it expires, and return the key if it does not expire. (delete when in use)

Delete regularly: every once in a while, the program checks the database and deletes the expired keys. It is up to the algorithm to decide how many expired keys to delete and how many databases to check. (regularly scanned and deleted)

Scheduled deletion

Advantages

1. Most memory-friendly: by using timers, you can ensure that expired keys will be deleted as soon as possible, freeing up occupied memory.

Shortcoming

1. Most unfriendly to cpu: in the case of a large number of expired keys, deleting expired keys may take up a considerable part of cpu time and affect the response time and throughput of the server.

Lazy deletion

Advantages

1. Most friendly to cpu: expired keys are checked only when keys are taken out, that is, there is no need for regular cpu scans or the creation of a large number of timers.

Shortcoming

1. Least memory-friendly: if a key has expired but will not be accessed later, it will remain in the database all the time. If there are too many such keys, it will undoubtedly take up a lot of memory.

Delete periodically

Periodic deletion is a compromise between scheduled deletion and lazy deletion above.

Advantages

1. Delete periodically and perform expired key operations at regular intervals, and reduce the impact of deletion operations on cpu time by limiting the length and frequency of deletion operations.

2. By deleting expired keys, the memory waste caused by expired keys can be effectively reduced.

The disadvantage is that it is difficult to determine how long and how often the delete operation will be performed.

1. If the deletion operation is performed too frequently or takes too long, the periodic deletion policy will degenerate into regular deletion, which takes up too much cpu execution time.

2. If the deletion operation takes too little time to perform, or if the execution time is too short, the periodic deletion policy will be the same as lazy deletion, resulting in memory waste.

Expiration Policy of Redis

Redis uses two strategies: lazy deletion and periodic deletion: by matching these two strategies, the server can strike a good balance between using cpu time properly and avoiding wasting memory space.

Implementation of lazy deletion strategy

The lazy deletion strategy of expired keys is implemented by the db.c/expireIfNeeded function. All Redis commands that read and write to the database will call the expireIfNeed function to check the input keys before execution:

If the key has expired, the expireIfNeeded function deletes the key

If the key does not expire, the expireIfNeeded function does not operate

The process of calling the expireIfNeeded function by the command is as follows

In addition, because each accessed key may be deleted, each command must be able to handle both the existence and non-existence of the key. The following figure shows the execution of the get command

Implementation of periodic deletion policy

The periodic deletion policy of expired keys is implemented by the redis.c/activeExpireCycle function. Whenever the Redis server periodically operates the redis.c/serverCron function, the activeExpireCycle function is called. It traverses each database in the server several times within a specified time.

By default, Redis performs 10 expiration scans per second, which does not traverse all the key in the expired dictionary, but uses a simple greedy strategy, as follows.

(1) 20 key are randomly selected from expired dictionaries.

(2) Delete the expired key in the 20 key.

(3) if the proportion of expired key exceeds 1 key 4, repeat step (1). At the same time, in order to ensure that the expired scan will not be over-cycled, resulting in terminal jam, the algorithm also increases the upper limit of scanning time, which will not exceed 25ms by default.

What happens if all the key in a large Redis instance expires at the same time?

Consume cpu

Redis continues to scan expired dictionaries (multiple cycles) until expired key in expired dictionaries becomes sparse (the number of loops decreases significantly).

Causes the request to stutter or time out

When the client request arrives, if the server happens to enter the expired scan state, the client request will wait for at least 25ms before it will be processed. If the client sets the timeout period to be relatively short, such as 10ms, then a large number of connections will be closed due to timeout, and many exceptions will occur on the business side.

So be sure to pay attention to the expiration time, if there are a large number of key expiration, set a random range of expiration time, and not all expire at the same time.

This is the end of this article on "what is the expiration strategy in Redis?". I hope the above content can be helpful to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.