Shulou(Shulou.com)06/01 Report--

How to understand the Redis database, key expiration implementation, I believe that many inexperienced people are helpless about this, this article summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

Implementation of Database

Let's start with server.h/redisServer

struct redisServer{ ...

//save db array

redisDb *db;

//Number of db

int dbnum; ...}

Check out the redisDB code:

typedef struct redisDb { dict *dict; /* The keyspace for this DB */

dict *expires; /* Timeout of keys with a timeout set */

dict *blocking_keys; /* Keys with clients waiting for data (BLPOP)*/

dict *ready_keys; /* Blocked keys that received a PUSH */

dict *watched_keys; /* WATCHED keys for MULTI/EXEC CAS */

int id; /* Database ID */

long long avg_ttl; /* Average TTL, just for stats */

} redisDb;

In general, redis servers contain several (default 16) redisDb databases.

Redis is a database of key-value pairs stored k-v. The dictionary dict holds all the key-value pairs in the database. This place is called keyspace, which literally means "key space."

So we can think of it this way. In redisDb we use dict to maintain key space.

The kay of keyspace is the database key, and each key is a string object. Note that it is not a string, but a string object.

The value of keyspace is the value of the database. This value can be one of redis, string object, list object, hash table object, collection object or ordered object.

Database read and write operations

Therefore, the addition, deletion and revision of data is the addition, deletion and revision of keyspace, a large map.

When we perform:

>redis SET mobile "13800000000"

In effect, we add a key to keyspace, which is a string object containing the string "mobile," and a value to a string object containing the characters "138000000."

See also:

There's nothing to say about deletion. Java map operations, most programmers should be able to understand.

It is important to note that Redis also performs some additional maintenance actions when performing read and write operations on keys:

Maintain hit and miss counters. Used to count Redis cache hit rate.

Update the LRU time of the key and record the last active time of the key.

If the key is found to have expired during the read, Redis deletes the expired key and then performs the rest of the operation.

If a client performs a WATCH operation on this key, it marks the key dirty, letting the transaction notice that the key has been changed.

Dirty increases by 1 without changing it once.

If the server has database notification enabled, the notification will be sent as configured after the key is modified.

key expiration implementation

One of the most important features of Redis as a cache is the ability to set expiration times for key-value pairs. Just look at Redis is how to implement this one of the most important features?

Expiration time related commands in Redis

EXPIRE Set key's survival time in seconds

EXPIREAT Set the expiration time point of key in seconds

PEXPIRE Set the key's survival time in milliseconds

PEXPIREAT Set the expiration time point of key in milliseconds

In fact, these commands, the underlying commands are implemented by REXPIREAT.

dict *expires is used in redisDb to store expiration dates. where key points to the key in keyspace (pointer in c language), value is a long long timestamp, indicating the time point when the key expires, in milliseconds.

If we add an expiration time to the mobile above.

>redis PEXPIREAT mobile 1521469812000

This will add a key-value pair to the expired dictionary. As shown below:

The logic of expiration is simple:

Whether the key exists in the dictionary expires.

If key exists, whether the timestamp of value is less than the current system timestamp.

Next, we need to discuss the deletion strategy for expired keys.

There are three strategies for deleting keys:

Regular deletion, Redis regularly deletes all expired key pairs in memory, so as to ensure memory friendliness, expired keys will be deleted, but if the number of keys is large, a deletion requires CPU operation, CPU is not friendly.

Lazy deletion, only when the key is called to check whether the key pair is expired, but it will cause a large number of expired key pairs stored in memory, memory is not friendly, but greatly reduces the burden on the CPU.

Timing part delete, Redis timing scan expired keys, but only delete part, as for how many keys to delete, according to the current state of Redis decision.

These three strategies have different tendencies towards time and space. In order to balance time and space, Redis adopts the latter two strategies: lazy deletion and timed partial deletion.

Lazy deletion is relatively simple and will not be described too much. The main discussion about the timing of partial deletion.

The policy of timed deletion of expired keys is implemented by the expire.c/activeExpireCycle() function, server.c/serverCron() calls activeExpireCycle () on a timed basis.

The big operating principle of activeExpireCycle is that if there are few expired keys, the number of deleted keys is conservative, and if there are many expired keys, the strategy of deleting keys will be radical.

static unsigned int current_db = 0; /* Last DB tested. */

static int timelimit_exit = 0; /* Time limit hit in previous call? */

static long long last_fast_cycle = 0; /* When last fast cycle ran. */

First of all, three static global parameters record the db subscript currently traversed, whether the last deletion was timed out, and when the last quick operation was performed.

Calculate timelimit = 1000000*ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC/server.hz/100; can be interpreted as 25% of CPU time.

If expire size in db is 0, do not operate

expire accounts for less than 1% of the total key and does not operate

num = dictSize(db->expires);num is the number of keys used by expire.

slots = dictSlots(db->expires); slots is the size of the expire dictionary.

If the used key (num) is greater than ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP, it is set to ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP. This means that only ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP keys are checked at a time.

Random access to expired keys. Calculates whether it expires and deletes it if it expires.

Then various statistics, including the number of key deletions, average expiration time.

Every sixteen iterations, calculate the operation time, if more than timelimit end return.

If the deleted expiration key is greater than 1\4 of ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP, the loop is exited and the loop ends.

The steps are more complicated, so to sum up: (here are all described by default configuration)

Redis takes up to 25% of cpu time to process key expiration.

Go through all the redisDbs

In each redisDb, if there are no expired keys in the data or the proportion of expired keys is too low, go directly to the next redisDb.

Otherwise, traverse the expired keys in redisDb, and if the deleted keys reach 25% of the expired keys, or the operation time is greater than 25% of the cpu time, end the current loop and enter the next redisDb.

After reading the above, do you know how to understand Redis database and key expiration implementation? If you still want to learn more skills or want to know more related content, welcome to pay attention to the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.