Shulou(Shulou.com)05/31 Report--

This article will explain in detail the RDB and AOF of redis and how to deal with expired keys when copying. The editor thinks it is very practical, so I share it with you for reference. I hope you can get something after reading this article.

Generate RDB file

When you execute the SAVE command or the BGSAVE command to create a new RDB file, the program checks the keys in the database, and expired keys are not saved to the newly created RDB file.

For example, if the database contains three keys K1, K2, and K3, and K2 has expired, then when the SAVE command or BGSAVE command is executed, the program will only save the data of K1 and K3 to the RDB file, while K2 will be ignored.

Therefore, the inclusion of expired keys in the database does not affect the generation of new RDB files.

You can refer to the source code of the function rdbSave () in rdb.c:

/ * Iterate this DB writing every entry * * traverses the database and writes data for each key-value pair * / while ((de = dictNext (di))! = NULL) {sds keystr = dictGetKey (de); robj key, * o = dictGetVal (de); long long expire; / / creates a key object initStaticStringObject (key,keystr) in the stack based on keystr / / get the expiration time of the key expire = getExpire (db,&key); / / Save the key value pair data if (rdbSaveKeyValuePair (& rdb,&key,o,expire,now) = =-1) goto werr;}

The rdbSaveKeyValuePair function is implemented as follows:

/ * Save a key-value pair, with expire time, type, key, value. * * write the key, value, expiration time and type of the key-value pair to the RDB. * * On error-1 is returned. * * error returns-1. * * On success if the key was actually saved 1 is returned, otherwise 0 * is returned (the key was already expired). * * 1 is returned for successful saving, and 0 is returned when the key has expired. * / int rdbSaveKeyValuePair (rio * rdb, robj * key, robj * val, long long expiretime, long long now) {/ * Save the expiretime * * Expiration time of save keys * / if (expiretime! =-1) {/ * If this key is already expired skip it * * do not write expired keys * / if (expiretime < now) return 0; if (rdbSaveType (rdb,REDIS_RDB_OPCODE_EXPIRETIME_MS) = =-1) return-1 If (rdbSaveMillisecondTime (rdb,expiretime) = =-1) return-1;} / * Save type, key, value * * Save Type, key, value * / if (rdbSaveObjectType (rdb,val) = =-1) return-1; if (rdbSaveStringObject (rdb,key) = =-1) return-1; if (rdbSaveObject (rdb,val) = =-1) return-1; return 1;}

Load the RDB file

When starting the Redis server, if the server turns on the RDB function, the server will load the RDB file:

If the server runs in master server mode, when loading the RDB file, the program will check the keys saved in the file, the unexpired keys will be loaded into the database, and the expired keys will be ignored, so the expired keys will not affect the master server that loads the RDB file.

If the server runs in slave mode, all keys saved in the file, whether expired or not, are loaded into the database when the RDB file is loaded. However, because the slave server's database is emptied when the master and slave servers synchronize data, generally speaking, expired keys will not affect the slave servers that load RDB files.

This part of the code can view the source code of the rdbLoad () function in rdb.c:

/ * Check if the key already expired. This function is used when loading * an RDB file from disk, either at startup, or when an RDB was * received from the master. In the latter case, the master is * responsible for key expiry. If we would expire keys here, the * snapshot taken by the master may not be reflected on the slave. * * if the server is the primary node, * then when the keys have expired, they are no longer associated with the database * / if (server.masterhost = = NULL & & expiretime! =-1 & & expiretime < now) {decrRefCount (key); decrRefCount (val); / / Skip continue;}

AOF file write

When the server is running in AOF persistence mode, if a key in the database has expired but it has not been lazily deleted or deleted periodically, then the AOF file will not be affected by this expired key.

When the expired key is lazy or periodically deleted, the program appends (append) a DEL command to the AOF file to explicitly record that the key has been deleted.

For example, if the client uses the GET message command to try to access the expired message key, the server performs the following three actions:

1) remove the message key from the database.

2) append a DEL message command to the AOF file. (due to the addition of AOF files, AOF will have this DEL operation only when the client makes a request.)

3) return an empty reply to the client that executes the GET command.

This part is the use of the expireIfNeeded function in the lazy delete strategy in Redis. The section on lazy deletion strategy is discussed in the Redis lazy deletion strategy article. So I'm not going to repeat it here.

It is important to note that the expireIfNeeded function is called in the db.c/lookupKeyRead () function, and the lookupKeyRead function is used to fetch the value of the key key in the database db when performing a read operation.

AOF rewriting

Similar to generating RDB files, during AOF rewriting, the program will check the keys in the database, and expired keys will not be saved to the rewritten AOF file.

For example, if the database contains three keys K1, K2, and K3, and K2 has expired, the program will only rewrite K1 and K3, while K2 will be ignored.

If this part has mastered the method of AOF rewriting, it will be understood naturally.

Copy

When the server is running in replication mode, the deletion of expired keys from the server is controlled by the primary server:

After deleting an expired key, the master server explicitly sends a DEL command to all slave servers telling them to delete the expired key from the server.

When the server executes the read command sent by the client, it will not delete the expired key even if it encounters the expired key, but will continue to deal with the expired key like an unexpired key.

The slave server deletes the expired key only after receiving a DEL command from the master server.

For example, there is a pair of master and slave servers that all have the same three keys message, xxx and yyy in their databases, where message is the expired key, as shown in the figure

If a client sends a command GET message to the slave server at this time, the slave server will find that the message key has expired, but instead of removing the message key from the server, it continues to return the value of the message key to the client as if the message key had not expired.

Suppose that after that, a client sends a command GET message to the master server, then the master server will find that the key message has expired: the master server will delete the message key, return an empty reply to the client, and send a DEL message command to the slave server, as shown in the figure:

After receiving the DEL message command from the master server, the slave server will also delete the message key from the database. After that, the master and slave server will no longer save the expired key message, as shown in the figure:

On "redis RDB, AOF and copy when how to deal with expired keys" this article is shared here, I hope the above content can be of some help to you, so that you can learn more knowledge, if you think the article is good, please share it out for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.