Shulou(Shulou.com)06/01 Report--

This article briefly explains the expiration mechanism of Redis.

Before we explain, let's throw out a question. We know that many servers often use redis as a cache, and a lot of data is temporarily cached, which may not be used for a long time after use (such as temporarily storing session, or only storing daily stock market data), then the following problems will arise.

Does Redis recycle data that it doesn't use?

If so, how to configure it?

If not, how to prevent the accumulation of data from taking up a large amount of storage space?

I have not been in touch with Redis very deeply before, but recently I encountered a requirement scenario in which some data stored in Redis need to be cleared, mainly through some time to filter and delete those unsatisfied data, but this kind of work needs to be carried out every day, so the workload is relatively large, and it is impractical to clean up manually on time every day. Later, it is found that there is a function in Redis to set the expiration time, that is, you can set an expiration time for the values stored in the Redis database.

As a cached database, this is very practical. This is the Redis expiration mechanism that we will talk about in this article. In fact, this mechanism is used in a wide range of scenarios. For example, token or some login information in our general projects, especially SMS verification codes, are time-limited, or the number of requests is limited. If you follow the traditional database processing method, you usually judge the expiration date by yourself, which will undoubtedly seriously affect the performance of the project.

First, set the expiration time

The expiration of the stored value by Redis is actually handled for the key (key) of that value, that is, the setting of time is also the valid time for setting key. The Expires dictionary holds the expiration time of all keys, and Expires is also known as the expiration field.

Expire key time (in seconds)-this is the most commonly used method

Setex (String key, int seconds, String value)-a unique way of string

Note:

1. Except for the unique method of setting the expiration time of the string, all other methods need to rely on the expire method to set the time.

2. If no time is set, the cache will never expire

3. If you set the expiration time and then want the cache to never expire, use persist key

1. Common ways

Generally speaking, it mainly includes four kinds of overdue parties, in which expire is in seconds and pexpire is in milliseconds.

EXPIRE key seconds / / set the lifetime of key to ttl seconds PEXPIRE key milliseconds / / set the generation time of key to ttl milliseconds EXPIREAT key timestamp / / set the expiration time of key to the timestamp of seconds represented by timestamp / / set the expiration time of key to the timestamp of milliseconds represented by timestamp

Note: timestamp is a unix timestamp (for example, timestamp=1499788800 indicates that it will expire on July 12, 2017)

1, 2 two ways is to set an expiration period, that is, we deal with the most commonly used strategy of CAPTCHA, set it to expire after three or five minutes, and convert the minutes into seconds or milliseconds and store them in Redis.

3 and 4 the two ways are to specify an expiration time, for example, the expiration time of the coupon is a certain year, month and day, but the unit is different.

Let's take EXPIREAT as an example to explain how to use it.

Return value

An integer value of 1 or 0, as follows:

If the timeout is successfully set for the key, return 1

If the key does not exist or cannot set the timeout, return 0

Grammar

The following is the basic syntax of the EXPIREAT command in Redis.

Redis 127.0.0.1 6379 > Expireat KEY_NAME TIME_IN_UNIX_TIMESTAMP

Example

First, create a key in Redis: akey, and set some values in akey.

Redis 127.0.0.1 6379 > SET akey redis OK

Now set the timeout for the key created for the setting to 60 seconds.

127.0.1 EXPIREAT akey 6379 > SET akey redisOK127.0.0.1:6379 > EXPIREAT akey 1393840000 (integer) 1127.0.0.1 integer 6379 > EXISTS akey (integer) 0127.0.0.1 integer 6379 > SET akey redisOK127.0.0.1:6379 > EXPIREAT akey 1493840000 (integer) 1127.0.0.1 integer 6379 > EXISTS akey (integer) 1

The other three uses are similar and are not described one by one here.

2. Unique way of string

The special way to handle strings is the SETEX command, which sets the value and its expiration time for the specified key. If key already exists, the SETEX command replaces the old value.

Return value

Returns OK if the setting is successful.

Grammar

The basic syntax for Redis Setex commands is as follows:

Redis 127.0.0.1 6379 > SETEX KEY_NAME TIMEOUT VALUE

Example

Redis 127.0.0.1 TTL mykey60redis 6379 > SETEX mykey60 redisOKredis 127.0.0.1 GET mykey "redis

Two and three expiration strategies

Scheduled deletion

Meaning: while setting the expiration time of the key, create a timer for the key and let the timer delete the key when the expiration time of the key comes

Advantage: make sure the memory is released as soon as possible

Disadvantages:

If there are a lot of expired key, deleting these key will take up a lot of CPU time. When CPU time is tight, CPU can't spend all its time doing urgent things. It also needs to take time to delete these key.

The creation of timers takes time. If you create a timer for each key that sets the expiration time (there will be a large number of timers), the performance will be seriously affected.

No one uses it.

Lazy deletion

Meaning: key is not deleted when it expires. Check whether the key expires every time you get it from the database. If it expires, delete it and return null.

Advantages: deletion occurs only when the key is fetched from the database, and only the current key is deleted, so it takes up less CPU time, and the deletion has reached the point where it has to be done (if we don't delete it at this time, we will get the expired key)

Disadvantages: if a large number of key are not obtained for a long time after the timeout, memory leaks may occur (useless garbage takes up a lot of memory)

Delete periodically

Meaning: delete at regular intervals (set the frequency of hz,1s refresh in the redis.conf configuration file) expired key operation

Advantages:

By limiting the duration and frequency of delete operations, we can reduce the consumption of CPU time by delete operations-- the disadvantage of dealing with "scheduled deletion"

Periodically remove the disadvantage of overdue key-- handling "lazy deletion"

Shortcoming

In terms of memory friendliness, it is better to "delete regularly"

In terms of CPU time-friendliness, it is better to "delete lazily"

Difficult point

Reasonably set the execution time of the deletion operation (how long each deletion will take) and the frequency of execution (how often the deletion will be done) (this depends on the operation of the server)

After looking at the above three strategies, we can draw the following conclusions:

Regular deletion and regular deletion is active deletion: Redis will take the initiative to eliminate a batch of past key on a regular basis.

Lazy deletion is passive deletion: check whether the key has expired until it is used, and delete it when it expires.

Lazy deletion is the built-in policy for redis server

Periodic deletions can be done by:

First, configure the hz option of redis.conf, which defaults to 10 (that is, 10 times per second, 100ms once. The higher the value, the faster the refresh frequency and the greater the Redis performance loss).

Second, configure the maximum maxmemory of redis.conf. When the used memory exceeds the maxmemory limit, the active cleanup policy will be triggered.

Note:

The database mentioned above refers to the in-memory database. By default, each redis server has 16 databases (for database settings, see the following code). By default, database 0 is used by default. All operations are on database 0. About the storage structure of redis database, see Chapter 8 Redis database structure and principles of reading and writing.

# set the number of databases. The default is 16 libraries, and DB 0 is used by default. You can use "select 1" to select database 1. # Note: since database 0 is used by default, all the caching operations we have done are stored on database 0. # when you search on database 1, you will not find the cache previously passed by set # if you want to move the cache on database 0 to database 1. You can use "move key 1" databases 16

Memcached only uses lazy deletions, while Redis uses both lazy deletes and periodic deletions, which is also a difference between the two (it can be seen that redis is superior to memcached)

For lazy deletions, it is not only when you get key that the key expires is checked, but also on some methods of setting key (eg.setnx key2 value2: this method is similar to memcached's add method, if the set key2 already exists, then this method returns false and does nothing; if the set key2 does not exist, then this method sets the cache key2-value2.

Suppose that when you call this method, you find that key2 already exists in the redis, but the key2 has expired. If you do not delete at this time, the setnx method will directly return false, that is to say, the key2-value2 has not been reset successfully at this time, so it is necessary to check the expiration of the key2 before setnx is executed.)

III. Expiration strategy adopted by Redis

Lazy deletion + periodic deletion

Lazy deletion process

Check whether the key is out of date when performing operations such as get or setnx

If it expires, delete the key, and then perform the appropriate action

If it does not expire, perform the corresponding operation directly.

Periodic deletion process (to put it simply, random deletion of each library with a specified number of libraries is less than or equal to a specified number of expired key)

Traverse each database (that is, the number of "database" configured in redis.conf, default is 16)

Check the specified number of key in the current library (by default, each library checks 20 key, note that the loop is executed 20 times, as described below during the loop body)

If no key in the current library sets the expiration time, the traversal of the next library is performed directly.

Randomly get a key with the expiration time set, check whether the key expires, and delete the key if it expires

Determine whether the periodic deletion operation has reached the specified length of time, and if so, directly exit the periodic deletion.

IV. RDB's handling of expired key

Expired key has no effect on RDB

Persist data from in-memory database to RDB file

Before persisting key, it will check whether it expires. Expired key will not enter the RDB file.

Recover data from a RDB file to an in-memory database

The key will be checked for expiration before the data is loaded into the database. If it expires, the database will not be imported (in the case of the main database).

5. AOF's handling of expired key

Expired key has no effect on AOF

Persist data from an in-memory database to an AOF file:

When the key expires and has not been deleted, the persistence operation is performed at this time (the key will not enter the aof file because no modification command has occurred)

When the key expires, when a delete occurs, the program appends a del command to the aof file (the expired key will be deleted when the data is recovered from the aof file in the future)

AOF rewriting

When rewriting, it will first determine whether the key has expired, and the expired key will not be rewritten to the aof file.

These are the details of the Redis data expiration policy, please pay attention to other related articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.