Shulou(Shulou.com)06/01 Report--

This article mainly introduces the characteristics of Redis persistent AOF, which has certain reference value. Interested friends can refer to it. I hope you will gain a lot after reading this article. Let Xiaobian take you to understand it together.

Redis Persistence (AOF)

Features:

Record each write operation in the form of log, record all write instructions executed by Redis (read operations are not recorded), only add files but not overwrite files, redis will read the file to reconstruct data at the beginning of startup, in other words, redis will restart according to the contents of the log file to execute a write instruction from front to back to restore the completed data.

AOF saves appendonly.aof files

Configuration:

appendonly no: default off appendonly mode yes on

appendfilename "appendonly.aof": Specifies the log file name

appendfsync ：

every modification synchronization persistence every data change is immediately logged to disk Poor performance but good data integrity

everysec factory default recommendation, asynchronous operation, recording per second if down within one second, there is data loss

no from out of sync

no-appendfsync-on-rewrite no: Whether Appendfsync can be used when rewriting, the default is no, to ensure data security

auto-aof-rewrite-percentage 100 : Sets the baseline for rewriting

auto-aof-rewrite-min-size 64mb: Sets the minimum file size for rewriting

Rewrite triggered when AOF file size is twice the size since last rewrite and file size is greater than 64 MB

Lab: Viewing Log Files

1. Modify the configuration file to enable AOF

appendonly yes

2. setting value

127.0.0.1:9736> set k1 v1OK127.0.0.1:9736> set k2 v2OK127.0.0.1:9736> set k3 v3OK127.0.0.1:9736> set k4 v4OK127.0.0.1:9736> FLUSHALLOK127.0.0.1:9736> shutdownnot connected> exit[root@VM_0_7_centos bin]#

3. View aof file contents

[root@VM_0_7_centos bin]# cat appendonly.aof *2$6SELECT$10*3$3set$2k1$2v1*3$3set$2k2$2v2*3$3set$2k3$2v3*3$3set$2k4$2v4*1$8FLUSHALL[root@VM_0_7_centos bin]#

4. Edit aof file contents to simulate file recovery process

Delete last FLUSHALL, save

5. Start redis and view the data

127.0.0.1:9736> keys *1) "k2"2) "k3"3) "k1"4) "k4"

6. Edit aof file, simulate power failure, aof file corruption

Add garbled characters at the end,

set$2k4$2v4123123123-- INSERT --

7. The dum.rdb file exists because of shutdown

8. Start redis service at this time, there is a problem

[root@VM_0_7_centos bin]# redis-server /myredis/redis_aof.conf 18695:C 23 Sep 2020 15:18:43.773 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo18695:C 23 Sep 2020 15:18:43.774 # Redis version=6.0.5, bits=64, commit=00000000, modified=0, pid=18695, just started18695:C 23 Sep 2020 15:18:43.774 # Configuration loaded[root@VM_0_7_centos bin]# redis-cli -p 9736Could not connect to Redis at 127.0.0.1:9736: Connection refusednot connected>

Conclusion aof is preferred when aof and rdb coexist.

9. Fix aof redis-check-aof --fix appendonly.aof

[root@VM_0_7_centos bin]# redis-check-aof --fix appendonly.aof0x 8b: Expected prefix '*', got: '1'AOF analyzed: size=150, ok_up_to=139, diff=11This will shrink the AOF from 150 bytes, with 11 bytes, to 139 bytesContinue? [y/N]: ySuccessfully truncated AOF

10. Launch redis to view repair results

127.0.0.1:9736> keys *1) "k4"2) "k2"3) "k3"4) "k1"

11. Description in configuration file

# AOF and RDB persistence can be enabled at the same time without problems.# If the AOF is enabled on startup Redis will load the AOF, that is the file# with the better durability guarantees.

12. Relite rewriting mechanism

Function: AOF adopts file appending mode, and the file will become larger and larger. In order to avoid this situation, a rewriting mechanism is added. When the size of AOF file exceeds the set threshold, Redis will start the content compression of AOF file, and only the minimum instruction set that can recover data will be retained. You can use the command bgrewriteaof

Principle: When the AOF file continues to grow and is too large, it will fork out a new process to rewrite the file (also write the temporary file first and finally rename), traverse the data in the memory of the new process, and each record has a Set statement. Rewriting an aof file does not read the old aof file, but rather rewrites the entire contents of the database in memory into a new aof file by command, somewhat similar to a snapshot.

Trigger mechanism: Redis records the size of the AOF at the time of the last rewrite. The default configuration is to trigger when the AOF file size is twice the size of the last rewrite and the file is larger than 64 MB.

auto-aof-rewrite-percentage 100auto-aof-rewrite-min-size 64mb

13. advantages

Synchronization per second, synchronization per modification, synchronization out of sync can be set

14. disadvantage

aof files are much larger than rdb files for the same dataset, and recovery is slower than rdb files

AOF is slower than rdb, synchronization per second is better, and asynchronous is the same as rdb.

15. AOF Summary

16. Redis Persistence Overall Summary

1. The RDB persistence method can snapshot your data at specified intervals

2. AOF persistent way to record each write to the server operation, when the server restarts will re-execute these commands to restore the original data,AOF command with redis protocol to save each write operation to the end of the file.Redis can also AOF file background rewriting, so that the size of the AOF file is not too large

3. Caching only: If you only want your data to exist while the server is running, you don't have to use any persistence methods.

4. both persistence modes are enabled at the same time.

In this case, when redis restarts, AOF files are loaded first to recover the original data, because AOF files usually hold more complete datasets than RDB files.

RDB data is not real-time, and when both are used, the server restarts only to find AOF files. Should I just use AOF? The authors recommend not to, as RDB is better suited for backing up databases (AOF is constantly changing and not good for backups), quick restarts, and no potential bugs that AOF might have, save it as an in case means.

17. performance recommendations

Because RDB files are only used for backup purposes, it is recommended to persist RDB files only on Slave, and only backup once in 15 minutes is enough, leaving only the save 900 1 rule.

If Enalbe AOF, the advantage is that in the worst case, no more than two seconds of data will be lost, and the startup script is simpler. Just load your own AOF file. The cost is the constant IO, and the almost inevitable blocking of AOF rewrite by writing new data to new files at the end of the rewrite process. As long as the hard disk is allowed, the frequency of AOF rewrite should be reduced as much as possible. The default base size of AOF rewrite is 64M, which is too small. It can be set to 5G or more. Default size exceeds 100% of original size Overwrite can be changed to an appropriate value.

If AOF is not enabled, it is possible to achieve high availability with Master-Slave Replication alone. Saving a large amount of IO also reduces system volatility when rewriting. The cost is that if the Master/Slave is destroyed at the same time, it will lose ten minutes of data, and the startup script will also compare the RDB files in the two Master/Slave and load the newer one. Sina Weibo chose this architecture

Thank you for reading this article carefully. I hope that the article "What are the characteristics of Redis persistent AOF" shared by Xiaobian will be helpful to everyone. At the same time, I hope that everyone will support you more, pay attention to the industry information channel, and more relevant knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.