Shulou(Shulou.com)06/01 Report--

Redis persistence has always been a common factor affecting redis performance, how to monitor persistence and how to optimize persistence? Let's take a look at it together.

Fork monitoring and optimization

Regardless of which persistence, RDB persistence or AOF rewrite is used, the main process forks out a child process where the rdb file generation or aof rewrite is done. Fork operations are heavy operations for operating systems. During the fork phase, redis blocks for a while. The blocking time is proportional to the memory size occupied by redis data, and each 1G memory fork takes 20 milliseconds.

To find out the blocking time of fork phase, use info stats to check the value of latest_fork_usec option in microseconds. Remember microseconds, not milliseconds.

# redis-cli info stats | grep latestlatest_fork_usec:323

Ways to optimize fork:

Controls the amount of memory redis uses. If the memory footprint is too large, you can split the application and deploy it on multiple servers to share the memory footprint of redis.

Lower the fork frequency appropriately.

Memory monitoring

The logs persisted by RDB are as follows:

……21692:C 15 May 2020 14:17:06.935 * DB saved on disk21692:C 15 May 2020 14:17:06.936 * RDB: 2 MB of memory used by copy-on-write……

You can see that the RDB persistence process consumes 2 MB of memory.

The AOF persistent log is as follows:

……15786:C 23 May 2020 07:39:59.145 * AOF rewrite: 2MB of memory used by copy-on-write10679:M 23 May 2020 07:39:59.201 * Background AOF rewrite terminated with success10679:M 23 May 2020 07:39:59.201 * Residual parent diff successfully flushed to the rewritten AOF (0.02 MB)10679:M 23 May 2020 07:39:59.201 * Background AOF rewrite finished successfully

As you can see, aof rewriting takes up 2MB+0.02MB=2.02MB of memory.

If you want to monitor the memory usage during persistence, you can write shell scripts to count the relevant information in the redis log.

Hard drive monitoring

The Redis persistence process puts pressure on the hard disk, because after persistence, the data in memory is saved to the hard disk.

Linux system monitoring hard disk commands have sar, iostat, etc., such as hard disk io pressure found to exceed the threshold, and then according to redis log comparison under the persistence time, to see if it is due to redis persistence caused by pressure.

The optimization method here mentions two points:

With good performance disks, mechanical hard disks are definitely not better than solid state drives.

If several redis instances are configured on a single machine, they can be written to different disks to reduce the write pressure on the disk.

Single-machine multi-instance deployment

Because redis is a single-threaded architecture, deploying only one instance of redis on a server is wasteful for multicore CPUs. Therefore, multiple redis applications are usually deployed on a server, such as opening three redis services with port numbers 6379, 6380, and 6381. 6379 is used for caching services, 6380 is used for message queues, and 6381 is used for tagging and recommendation systems.

This does allow you to make full use of the CPU, but it is prone to problems. If multiple instances are persisting at the same time, the pressure on cpu, memory, and movies is enormous. A good practice is to isolate them so that only one instance is persisted at a time.

The pseudocode to achieve this effect is as follows:

while (true){ $redisObj = [6379,6380,……]; foreach ($redisObj as $obj) { //Does this instance constitute a rewrite requirement if (rewriteConf($ojb)) { //This instance is persisted } }}

foreach is used to iterate over each redis instance, and then determine whether the instance meets the rewriting conditions, and then start rewriting. This allows multiple redis instances to be persisted in isolation.

The above is the detailed content of monitoring and optimization of Redis persistence process, please pay more attention to other related articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.