Shulou(Shulou.com)06/01 Report--

Preface

Earlier, Bai Xin (Momo-Storage engineer of the Technical guarantee Department) summarized the fix records related to expired keys from Redis from 2.8 to 4.0 in the Redis technology exchange group, which is very helpful, but there are some things that are not detailed, which will be explained in detail in this article.

From a point of view, the operating environment is as follows:

Redis: 2.8.19

Db0:keys=10000000,expires=10000000

Master-slave structure

As you can see from the following figure, the slave node get hello is not empty, the master node get hello is empty, and then the slave node get hello is empty. After troubleshooting, the master-slave synchronous offset is basically normal, but there is a master inconsistency.

Not to mention the reason, this article will explore the iteration of the Redis2.8-4.0 version, for the fix of expired keys, to see if the answer can be found.

I. Review of expired functions

When you execute a setex command, Redis inserts data into the internal dict and expires hash structures, respectively:

Dict-dict [key]: valueexpires---expires [key]: timeout

For example:

127.0.0.1 setex hello 6379 > info# 120 worldOK127.0.0.1:6379 > the total amount of expired keys in this database that have not been deleted (if they have been set to expired keys and deleted are not counted) each time expired keys are deleted in db0:keys=1,expires=1,avg_ttl=41989# history, an addition operation is performed to record the total number of deleted expired keys. Expired_keys:0

2. Delete policies for Redis expired keys:

What does Redis do when the key value expires? Considering the single-threaded characteristics of Redis, there are two strategies: lazy deletion and timing deletion.

1. Lazy delete policy:

Every time you execute a key-related command, you will first look for whether the key has expired in expires. Here is the source code (db.c) of 3.0.7:

Here are the entries related to reading and writing key:

Robj * lookupKeyRead (redisDb * db, robj * key) {robj * val; expireIfNeeded (db,key); val = lookupKey (db,key);. Return val;} robj * lookupKeyWrite (redisDb * db, robj * key) {expireIfNeeded (db,key); return lookupKey (db,key);}

You can see that every time you read and write key, all Redis commands call the expireIfNeeded function before execution:

Int expireIfNeeded (redisDb * db, robj * key) {mstime_t when = getExpire (db,key); mstime_t now; if (when

< 0) return 0; /* No expire for this key */ now = server.lua_caller ? server.lua_time_start : mstime(); if (server.masterhost != NULL) return now >

When; / * Return when this key has not expired * / if (now id); return dbDelete (db,key);}

As you can see from the code, the master-slave logic is slightly different:

(1) main library: expireIfNeeded will delete the expiration key if it expires. If deleted successfully, 1 will be returned, otherwise 0 will be returned.

(2) from the library: expireIfNeeded does not delete key, but returns a result of logical deletion. 1 is returned after expiration, and 0 is returned if it does not expire.

However, the deletion of the expiration key from the library is controlled by the synthesized DEL operations of the main library.

two。 Delete policies on a regular basis:

Lazy deletion alone cannot delete all expired key. Considering the single-threaded nature of Redis, Redis uses a periodic deletion strategy, which takes a certain number of random keys from a certain number of expired databases for checking. If it is not empty, delete it. Real-time deletion is not guaranteed. Interested students can take a look at the specific implementation in activeExpireCycle, it is still very interesting, the following figure is a schematic diagram

If (server- > masterhost = = NULL) activeExpireCycle ()

(1) main library: expired keys will be deleted regularly.

(2) from the library: periodic deletions are not performed.

To sum up:

Main library:

(1) call expireIfNeeded lazy delete before performing all operations.

(2) call activeExpireCycle periodically, and delete some keys randomly each time (regular deletion).

From the library:

The deletion of expired keys is controlled by the synthesized DEL operations of the main library.

III. The problem of expired reading and writing

Problems caused by Redis expired deletion policy. We only discuss it from the point of view of user operation.

1. Expired key read operation

The following is the fix record of the expired key read operation of Redis 2.8 through 4.0

(1) Redis2.8 principal is not consistent.

The lookupKeyRead function is called first in all the read operations in 2.8:

Robj * lookupKeyRead (redisDb * db, robs * key) {robj * val; expireIfNeeded (db,key); val = lookupKey (db,key); if (val = = NULL) server.stat_keyspace_misses++; else server.stat_keyspace_hits++; return val;}

For the main library, when expireIfNeeded is executed, the key is deleted when it expires. LookupKey returns NULL.

For slave libraries, when expireIfNeeded is executed, expiration does not delete the key. LookupKey returns value.

Therefore, for the read operation of expired keys, there will be inconsistencies in master-slave returns, which is the problem mentioned at the beginning.

(2) Redis 3.2 master and slave are consistent except exists

Https://github.com/antirez/redis/commit/06e76bc3e22dd72a30a8a614d367246b03ff1312

In the 3.2-rc1 read operation, lookupKeyRead is also called first, but actually the lookupKeyReadWithFlags function is called:

Robj * lookupKeyReadWithFlags (redisDb * db, robj * key) {robj * val; if (expireIfNeeded (db,key) = = 1) {if (server.masterhost = = NULL) return NULL If (server.current_client & & / / current client exists server.current_client! = server.master & & / / the current client is not established by a master request (the client requested by the user) server.current_client- > cmd & & server.current_client- > cmd- > flags & REDIS_CMD_READONLY) {/ / read command return NULL;} val = lookupKey (db,key,flags) If (val = = NULL) server.stat_keyspace_misses++; else server.stat_keyspace_hits++; return val;}

As you can see, the judgment of the result returned by expireIfNeeded is increased compared to 2.8:

For the main library, when expireIfNeeded is executed, the key is deleted and 1 is returned when it expires. NULL is returned if masterhost is empty.

For the slave library, when expireIfNeeded is executed, the expiration does not delete the key and returns 1. NULL is returned when the current client is not master and is a read command.

Unless the program is abnormal. Under normal circumstances, the master-slave return is consistent for the read operation of expired keys.

(2) Redis 4.0.11 resolves exists inconsistencies

Https://github.com/antirez/redis/commit/32a7a2c88a8b8cca8119b849eee7976b8ada8936

3.2 does not solve the problem with the exists command, although it is also a read operation. Then the problem was solved in 4.0.11.

2. Expired key write operation

Before we get into this problem, let's talk about the usage scenarios of writable slave libraries.

(1)。 In the master-slave separation scenario, time-consuming operations can be written from the slave library to improve performance.

The author mentioned in https://redis.io/topics/replication:

For example computing slow Set or Sorted set operations and storing them into local keys is an use case for writable slaves that was observed multiple times.

A more specific example is given at https://github.com/antirez/redis/commit/c65dfb436e9a5a28573ec9e763901b2684eadfc4:

For instance imagine having slaves replicating certain Sets keys from the master. When accessing the data on the slave, we want to peform intersections between

Such Sets values. However we don't want to intersect each time: to cache the intersection for some time often is a good idea.

In other words, in the read-write separation scenario, the expired key mechanism can be used to cache the results of time-consuming operations from the library as a cache to improve the overall performance.

(2)。 When migrating data, you need to make the slave library writable first.

For example, the following scenario: the online Redis service is normal, but you may encounter some hardware situations, and you need to migrate the Redis master-slave cluster on this machine. The way to move the data is to build a new master-slave cluster and let the new master become the follower of the old master.

Do the following:

(1) the master (old master) is synchronized with the new master (new master). After 90s of rdb transmission, the slave library (new master) is writable.

(2) after the master database (the old master) has no business connection at all, the slave library (the new master) executes slaveof no one.

In this scenario, in order to ensure that the data is fully synchronized and minimize the impact on the business, the slave database will be writable first.

Then let's do a test:

The expiration key of the master library expires normally in the operation performed by the master library in version 3.2.

Version 3.2 can write to the slave library to perform the following operations, and the expired keys of the slave library will not expire.

The 4.0rc3 version can write to the slave library to perform the following operations, but the expired keys of the slave library can expire.

In fact, the problem of expired keys from the writable library contains two questions:

(1) the expired keys in the slave library are synchronized by the master database, and the expiration operation is performed by the master library (unchanged).

(2) the setting of the expired key in the slave library is operated from the library.

Prior to redis4.0rc3, there was a problem of expired keys leaking. When expire operates directly on the slave library, the key does not expire. The author also mentioned in https://redis.io/topics/replication:

However note that writable slaves before version 4.0 were incapable of expiring keys with a time to live set. This means that if you use EXPIRE or other commands that set a maximum TTL for a key, the key will leak, and while you may no longer see it while accessing it with read commands, you will see it in the count of keys and it will still use memory. So in general mixing writable slaves (previous version 4.0) and keys with TTL is going to create issues.

The problem of expired key leakage has been resolved in https://github.com/antirez/redis/commit/c65dfb436e9a5a28573ec9e763901b2684eadfc4.

four。 Summary

1. Read operations for expired keys

(1) Redis2.8 principal is not consistent.

(2) Redis3.2-rc1 master and slave are consistent except exists: https://github.com/antirez/redis/commit/06e76bc3e22dd72a30a8a614d367246b03ff1312

(3) Redis4.0.11 master and slave are consistent:

Https://github.com/antirez/redis/commit/32a7a2c88a8b8cca8119b849eee7976b8ada8936

2. Write operations for expired keys:

Redis2.8~4.0 only returns physical results.

3. Expire the key from the library, and the key will not expire.

Redis4.0 rc3 solves the problem that expired keys set from the library do not expire https://github.com/antirez/redis/commit/c65dfb436e9a5a28573ec9e763901b2684eadfc4

4. If the slave is not read-write separated and the above migration is used, the basic problems in this article will not occur. Also, Redis 4 is very reliable, and there will be articles about it later. (Fu Lei)

Summary

The above is the whole content of this article. I hope the content of this article has a certain reference and learning value for everyone's study or work. Thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.