Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "the persistence of Redis and the detailed explanation of master-slave replication". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

What is Redis persistence?

Redis as a key pair memory database (NoSQL), the data is stored in memory, and when processing client requests, all operations are performed in memory, as shown below:

What's wrong with this?

In fact, as long as anyone with a little basic computer knowledge knows that the data stored in memory will disappear as long as the server shuts down (caused by various reasons), not only the server shutdown will cause the data to disappear, the Redis server daemon will exit, but the data in memory will also disappear.

For projects that only use Redis as a cache, it may not be a problem to disappear the data, just load the data from the data source again, but if you directly store the business data submitted by users in Redis, use Redis as a database, and store important business data in it, the impact of Redis's in-memory data loss may be devastating. In order to avoid data loss in memory, Redis provides persistence support. We can choose different ways to save the data from memory to the hard disk, so that the data can be persisted.

Redis provides two different ways of data persistence: RDB and AOF. Let's take a closer look at these different persistence methods.

RDB

RDB is a persistent method of snapshot storage, which specifically saves the memory data of Redis at a certain time to a file on your hard disk. The default file is dump.rdb, and when the Redis server starts, it reloads the data of the dump.rdb file into memory to recover the data.

Enable RDB persistence mode

It is easy to enable RDB persistence. The client can send save or bgsave commands to the Redis server to let the server generate the rdb file, or specify the trigger RDB condition through the server configuration file.

1. Save command

The save command is a synchronous operation.

# asynchronously save datasets to disk > bgsave

When the client sends the bgsave command from the service, the main process of the Redis server will forks a child process to solve the data synchronization problem, and after saving the data to the rdb file, the child process will exit.

Therefore, compared with the save command, the Redis server uses child threads to write to the bgsave, while the main process can still receive other requests, but the bgsave child process is synchronous, so the bgsave child process is also unable to receive other requests, which means that if the Forks child process takes too long (usually very fast), the Forks command still blocks the requests of other clients.

3. Server configuration automatically triggers

In addition to sending commands through the client, there is another way, that is, the save in the Redis configuration file specifies the conditions that trigger RDB persistence, such as enabling RDB data synchronization in "at least how many writes in seconds".

For example, we can specify the following options in the configuration file redis.conf:

At least one write command within # 900s save 900s within 300s at least 10 write commands save 300 within 1cm 60s at least 10000 write commands save 60 10000

The configuration file is then loaded when the server is started.

# start the server to load the configuration file redis-server redis.conf

This way of triggering RDB through the server configuration file is similar to the bgsave command. When the trigger condition is reached, it will forks a child process for data synchronization, but it is best not to trigger RDB persistence in this way, because the setting trigger time is too short, it is easy to write rdb files frequently, affecting server performance. Setting time is too long will result in data loss.

Rdb file

I described three ways for the server to generate rdb files, whether it is generated by the main process or child processes, the process is as follows:

Generate a temporary rdb file and write the data.

Complete the data writing and replace the official rdb file with a temporary text.

Delete the original db file.

The file name generated by RDB by default is dump.rdb. Of course, I can configure it in more detail through the configuration file. For example, when starting multiple redis server processes under a single machine, you can configure different rdb names with port numbers, as shown below:

# whether to compress the name of rdb file, rdbcompression yes# rdb file, dbfilename redis-6379.rdb# rdb file, save directory dir ~ / redis/

Several advantages of RDB

Compared with the AOF method, recovering data through rdb files is faster.

Rdb files are very compact and suitable for data backup.

Data preparation through RDB has little impact on the performance of the Redis server because it is generated by child processes.

Several shortcomings of RDB

If the server goes down, using RDB will cause data loss within a certain period of time. For example, if we set synchronization once in 10 minutes or 1000 writes in 5 minutes, then if the server crashes before the trigger condition is reached, then the data in this period will be lost.

Using the save command will cause the server to block, and the subsequent requests can be received only after the data synchronization is completed.

When using the bgsave command in the forks child process, if the amount of data is too large, the Forks process will also block. In addition, the Forks child process will consume memory.

AOF

After talking about RDB, let's talk about another persistence method of Redis: AOF (Append-only file).

Unlike RDB, which stores snapshots at a certain time, AOF persistence records every write operation command from the client to the server, and appends these write operations to the end of the suffixed aof file with Redis protocol. When the Redis server restarts, the command of the aof file is loaded and run to restore the data.

Enable AOF persistence mode

Redis does not enable AOF persistence by default. We can enable it in the configuration file and configure it in more detail, such as the following redis.conf file:

# enable aof mechanism appendonly yes# aof file name appendfilename "appendonly.aof" # write policy, always means that every write operation is saved to the aof file, or everysec or noappendfsync always# does not overwrite the aof file no-appendfsync-on-rewrite no# save directory dir ~ / redis/ by default

Three write strategies

In the configuration file above, we can specify the write policy through the appendfsync option, and there are three options.

# appendfsync always# appendfsync everysec# appendfsync no1. Always

Every write on the client is saved to an aof file, which is a safe strategy, but every write has an IO operation, so it is also slow.

2. Everysec

Appendfsync's default write policy is to write the aof file once per second, so up to 1 second of data may be lost.

3. No

The Redis server is not responsible for writing the aof, but leaves it to the operating system to handle when to write the aof file. Faster, but also the least safe option, is not recommended.

AOF file rewriting

AOF appends every write operation on the client to the end of the aof file, such as executing the incr command multiple times on a key. At this time, aof saves each command to the aof file, and the aof file becomes very large.

Incr num 1incr num 2incr num 3incr num 4incr num 5incr num 6...incr num 100000

If the aof file is too large, it will be very slow to load the aof file to recover data. To solve this problem, Redis supports aof file rewriting. By rewriting aof, you can generate a minimum set of commands to restore the current data. For example, there are so many commands in the above example, which can be rewritten as:

Set num 100000

The aof file is a binary file, and instead of saving each command directly, as in the example above, it uses Redis's own format, which is just a convenient demonstration.

Two ways of rewriting

Through the option no-appendfsync-on-rewrite in the redis.conf configuration file, you can set whether to enable rewriting, which will be rewritten every time fsync, affecting the server, so the default value is no, which is not recommended.

# do not rewrite aof file no-appendfsync-on-rewrite no by default

The client sends a bgrewriteaof command to the server, or it can have the server rewrite AOF.

# Let the server asynchronously rewrite the append aof file command > bgrewriteaof

Benefits of rewriting aof files

Compress aof files to reduce disk usage.

Aof commands are compressed to a minimum command set, which speeds up the speed of data recovery.

AOF file is corrupted

When writing the aof log file, if the Redis server goes down, the aof log file will have an incorrect format. When you restart the Redis server, the Redis server will refuse to load the aof file. You can use the following steps to repair the aof and restore the data.

Back up the aof file now, just in case.

Use the redis-check-aof command to repair the aof file in the following format:

# repair the aof log file $redis-check-aof-fix file.aof

Restart the Redis server, load the repaired aof file, and restore the data.

Advantages of AOF

AOF only appends log files, so it has less impact on server performance, is faster than RDB, and consumes less memory.

Shortcomings of AOF

The log file generated by AOF is so large that it is still very large even if it is rewritten by AFO.

Data recovery is slower than RDB.

Do you choose RDB or AOF?

Through the above introduction, we understand the respective advantages and disadvantages of RDB and AOF. How to choose?

Through the following presentation, we can compare RDB and AOF from several aspects. In application, choose RDB or AOF based on your actual needs. In fact, if you want the data to be secure enough, you can enable both. However, if the two persistence methods perform IO operation at the same time, it will seriously affect server performance, so sometimes you have to make a choice.

When both RDB and AOF are enabled, Redis will first use AOF logs to recover data, because the files saved by AOF are more complete than RDB files.

This is the end of the content of "Redis persistence and master-slave replication". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.