Shulou(Shulou.com)06/02 Report--

This article focuses on "how to achieve Redis persistence". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Now let the editor to take you to learn "the implementation of Redis persistence" bar!

The reason why Redis is so fast is that Redis is a memory-based database. When reading and writing, redis is completed in memory, and then regularly refreshed to disk. RDB and AOF are two ways to persist data in memory.

The data is persistent in the hard disk, and the restart machine will also exist.

If the data is not persistent in memory, the restart machine will be gone.

Learn from a code of redis operation written by yourself. Here we use Jedis (Java connection Development tool), which is the simplest set.

/ * set byte array * * @ param key byte key * @ param value byte value * @ return OK * / public String set (byte [] key, byte [] value) {Jedis jedis = null; String result = null; try {jedis = getJedis (); result = jedis.set (key, value) } catch (Exception e) {log.error ("set byte [] key: {} value: {} error", key, value, e);} finally {returnSource (jedis);} return result;} Snapshot Storage RDB (redis db)

Redis will save its own data to the hard disk as a file, and automatically restore the hard disk data to memory after the server is restarted.

The operation is to save all the data in redis to the hard disk at once, so if the data is large, it is not suitable to do this operation frequently.

The default file name dump.rdb can be set in redis.conf

There are three ways to trigger a snapshot

The first kind. The save command sent by the redis client takes a snapshot and blocks (the method implementation that needs to call jedis.save () in the code)

Block the process of the redis server until the RDB file is created, during which time redis cannot process other commands

The second kind. The bgsave command sent by the redis client does not block (the method implementation of jedis.bgsave () needs to be called in the code)

The actual implementation process:

Client bgsave command

The server returns ok

The server fork creates a child process to perform the backup

Notify redis after the child process has finished execution

The main process and child process exist at the same time, and the redis server process is not blocked. Creating child processes consumes extra memory, so bgsave is slower than save.

The third kind. Redis automatically executes bgsave according to the configuration in redis.conf

If save is modified once within 1 # 900s, 10 times within 10 # 300 seconds, then save 60 10000 # 60 will be modified 10000 times within 60 seconds.

After each execution of the server, the time counter and number counter set for automatic persistence will be cleared and recalculated.

View the save configuration of redis by executing config get save in the redis command

Use the jedis.lastsave () command to check whether the RDB file is generated successfully, and return the unix timestamp that was successfully saved to disk last time (both save and bgsave will modify the time). The RDB file is overwritten every time. If you need to control the backup, save the backup at a fixed time and at a fixed point.

From the above learning process, it is concluded that the active approach should choose to use bgsave to persist.

Public void bgsave () {Jedis jedis = null; try {jedis = getJedis (); jedis.bgsave ();} catch (Exception e) {log.error ("bgsave error", e);} finally {returnSource (jedis);}}

Add a bgsave persistence method to your utility class. In addition, you have encountered an error if you call save or bgsave twice in a Java method, so if the persistence list is proposed, there may be several redis operations in one method.

Summary:

The creation of RDB files by 1.bgsave child threads will not have a significant impact on the performance of the redis server.

two。 The RDB file generated by the snapshot is a compressed binary file, which can be easily transferred and saved in the network.

3. New data is generated after a backup of RDB, but the conditions for generating RDB files have not yet been reached. When the redis server goes down, the new data will be lost.

4. When there is a large amount of data, triggering RDB persistence, including creating child threads and generating RDB files, will take up a lot of system resources and time, and will have an impact on redis.

Persistent AOF (append only file)

Back up every instruction executed by the operation to the aof file, execute the instruction when restoring the data, and append it each time

Aop rewrite (non-blocking). Redis.conf enables aof corresponding to appendonly yes.

Appendfilename "appendonly.aof" / / backup file name appendfsync always / / write to disk immediately every time you receive a write command, the slowest way, but ensure complete persistence appendfsync everysec / / strongly write to disk once per second, there is a good compromise between performance and persistence. The default appendfsync no / / is not synchronized to the hard disk, which is decided by the operating system.

There are three steps in the aof generation process: after executing a write command, redis appends the executed command to the end of the aof_buf buffer inside the redis

Call the fsync function, and the write command of the buffer will be written to the AOF file to complete the synchronization.

In the current operating system, when the system calls the write function and writes some content to a file, it first puts the content into a memory buffer (buffer) until the buffer is filled, or when the user performs fsync and fdatasync calls, the contents stored in the buffer are actually written to the hard disk, so if something goes wrong in the process, will the data be lost?!

Summary:

Using AOF persistence makes Redis very durable: you can set different fsync policies, such as no fsync, fsync per second, or fsync each time a write command is executed. The default policy of AOF is fsync once per second. In this configuration, Redis can still maintain good performance, and even if a failure occurs, it will only lose data for one second at most.

The aof file contains incomplete commands for some reason. The redis-check-aof tool can also easily fix this problem.

Redis can automatically rewrite AOF in the background when the AOF file becomes too large: the new AOF file after rewriting contains the minimum set of commands needed to recover the current dataset. The entire rewrite operation is perfectly safe because Redis continues to append commands to existing AOF files during the creation of new AOF files, and existing AOF files will not be lost even if downtime occurs during rewriting. Once the new AOF file is created, Redis will switch from the old AOF file to the new AOF file and begin to append the new AOF file

AOF files are highly readable, and write commands can also be operated manually.

AOF files are larger than RDB files

Official documents also point out that in some cases, there are some bug in AOF. For example, when using blocking commands, these bug scenarios RDB do not exist.

-

At this point, I believe you have a deeper understanding of "the implementation of Redis persistence". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.