Shulou(Shulou.com)06/01 Report--

Today, what the editor shares with you is the data recovery method of redis to achieve server crash and downtime. I believe many people do not know much about it. In order to make you understand better, I have summarized the following contents for you. Let's take a look at it. I'm sure you'll get something.

Because redis is stored in memory and provides the data structure storage types commonly used in general programming languages, it is often used to recover data from server crashes.

In some specified processes, the server can store the data that needs to be saved in redis in the form of json objects, which is what we often call snapshots. When the server is running, it reads redis to determine whether it needs to recover the data to continue to process the business.

Delete the data of redis when the business processing is finished.

Redis provides two ways to export memory data to a hard disk for data backup:

RDB mode (default)

RDB persistence is done through snapshotting. When certain conditions are met, Redis will automatically snapshot all data in memory and store them on the hard disk. The conditions for taking a snapshot can be customized by the user in the configuration file and consist of two parameters: time and the number of keys changed. Snapshots are taken when the number of keys changed within a specified period of time is greater than the specified value. RDB is the default persistence method adopted by redis. Three conditions have been preset in the configuration file:

Save 9001 # 900s if at least 1 key is changed, snapshot will be taken.

Save 30010 # 300 seconds when at least 10 keys are changed, take a snapshot

Save 60 10000 # Snapshot if at least 10000 keys have been changed within 60 seconds

There can be multiple conditions, and the relationship between the conditions is "or", and as long as one of the conditions is met, a snapshot will be taken. If you want to disable automatic snapshots, simply delete all the save parameters.

By default, Redis stores snapshot files in the dump.rdb file in the current directory (which can be viewed by CONFIG GET dir). You can specify the storage path and file name of snapshot files by configuring dir and dbfilename parameters, respectively.

The process of realizing snapshot with Redis

Redis uses the fork function to make a copy of the current process (parent process) (child process)

The parent process continues to receive and process commands from the client, while the child process begins to write data in memory to temporary files on the hard disk

When the child process has written all the data, it replaces the old RDB file with the temporary file, and the snapshot operation is complete.

When executing fork, the operating system (similar to Unix operating system) will use the copy-on-write policy, that is, the parent and child processes share the same memory data at the moment when the fork function occurs. When the parent process wants to change a piece of data (such as executing a write command), the operating system will copy the piece of data to ensure that the data of the child process will not be affected. So the new RDB file stores in-memory data for a moment of fork execution.

Redis will not modify the RDB file during the snapshot, and will replace the old file with the new one only after the snapshot ends, which means that the RDB file is complete at all times. This allows us to implement Redis database backups by backing up RDB files on a regular basis. The RDB file is a compressed binary format (the rdbcompression parameter can be configured to disable compression and save CPU footprint), so it takes up less space than the data in memory and is more convenient for transmission.

In addition to automatic snapshots, you can also manually send SAVE or BGSAVE commands to Redis to perform snapshots. The difference between the two commands is that the former is performed by the main process and blocks other requests, while the latter performs snapshot operations through the help child process. After Redis starts, it reads the RDB snapshot file and loads the data from the hard drive into memory. This time varies depending on the amount of data and structure and server performance. It usually takes 20 to 30 seconds to load a snapshot file with 10 million string type keys and the size of 1GB into memory. Persistence is achieved through RDB. Once the Redis exits abnormally, all data changed since the last snapshot will be lost. This requires developers to control the possible data loss in an acceptable range by combining and setting automatic snapshot conditions according to the specific application situation. If the data is too important to bear any loss, consider using AOF for persistence.

AOF mode

By default, Redis does not enable persistence in AOF (appendonly file) mode. You can enable it through the parameter appendonly in redis.conf:

Appendonly yes

During startup, Redis will execute the commands in the AOF file one by one to load the data from the hard disk into memory, which is slower than that of RDB.

Every time you turn on AOF persistence and execute a command that changes the data in Redis, Redis writes the command to the AOF file on your hard disk. The AOF file is saved in the same location as the RDB file, both of which are set by the dir parameter. The default file name is appendonly.aof, which can be modified by the appendfilename parameter:

Appendfilename appendonly.aof

Configure conditions for redis to automatically rewrite AOF files

Auto-aof-rewrite-percentage 100 # will be rewritten again when the current AOF file size exceeds how many percent of the AOF file size at the time of the last override. If it has not been rewritten before, it will be based on the AOF file size at startup.

Auto-aof-rewrite-min-size 64mb # minimum AOF file size allowed to be overridden

Configure the mechanism that requires the system to flush the hard disk cache after writing to the AOF file

# appendfsync always # synchronization is performed every time a write is performed, which is the safest and slowest

Appendfsync everysec # performs a synchronization operation every second

# appendfsync no # does not take the initiative to synchronize, but leaves it entirely to the operating system (that is, every 30 seconds), which is the fastest and least secure

Redis allows both AOF and RDB to be turned on at the same time, which not only ensures data security, but also makes backup and other operations very easy. After restarting Redis at this point, Redis will use AOF files to recover data, because AOF persistence may lose less data.

Redis = require ('redis'), / / Import js module RDS_PORT =, / / Port number RDS_HOST ='', / / Server IPRDS_OPTS = {}, / / setting item redisdb = redis.createClient (RDS_PORT, RDS_HOST, RDS_OPTS); / / create connection redisdb.select (20); / / specify partition library redisdb.on ('ready', function (res) {console.log (' ready');}) Redisdb.on ('connect', function () {console.log (' connect');}); exports.redisdb = redisdb; function redis_opt (opt, key, value, callback) {if (opt = = 'get') {redisdb.get (key, function (err, data) {if (err = = null) {callback (data) } else {callback (err);}} else if (opt = = 'set') {redisdb.set (key,value, function (err,result) {if (err = = null) {callback (result)) } else {callback (err);}} else if (opt = = 'del') {redisdb.del (key, function (err, result) {if (err = = null) {callback (result);} else {callback (err) });} else {callback ("error opt!");}} function update (key) {redis_opt ("get", key, null, function (data) {console.log ("the redis data is" + data); if (data) {count = parseInt (data) Redis_opt ("set", key, + + count, function (data) {console.log ("set" + count + "" + data);} else {redis_opt ("set", key, 10000, function (data) {console.log ("set" + 10000 + "" + data);}) }});} function clear (key) {redis_opt ("del", key, null, function (ret) {console.log ("del" + key + "" + ret);} function main () {var key = "count_test"; setInterval (function () {clear (key)}, 5000); setInterval (function () {update (key)}, 1000) } / / testmain (); main ()

The above code is a simple timer function, that is, after the server starts, the data of redis is read regularly, if it exists, it is modified cumulatively, and if it does not exist, it is initialized. At the same time, in order to facilitate explanation, a timer is set up to delete data regularly.

This is the end of the data recovery method for redis to achieve server crash and downtime, of course, it is not just the above analysis, but the editor can ensure its accuracy is absolutely no problem. I hope that the above content can have a certain reference value for everyone, and can be put into practice. If you like this article, you might as well share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.