Shulou(Shulou.com)06/03 Report--

I. redis characteristics

What is 1.redis?

Redis is a memory-based persistent key-value database

Type of 2.redis data structure?

Value supports five data types, string, string list, string collection, ordered collection, hashs

3.redis persistent storage mode

Two persistence methods of redis, RDB, and AOF

RDB: persisting data at a certain time to disk is a snapshot-based persistence method. In the process of persistence, redis will first write the data to a temporary file, and when the persistence process ends, it will replace the last persisted file with this temporary file. It is this feature that allows us to back up at any time, because snapshot files are always fully available.

Principle: when persisting in rdb mode, redis will fork a child process for persistence, and the main process will not perform any io operations to ensure that redis performance will not be degraded by persistence. If it is insensitive to data and requires large-scale data recovery, you can use this method.

AOF: record the executed write instructions and put the instructions on the execution side in order when the data is recovered.

We can turn on the AOF function by configuring appendonly yes in redis.conf. If there is a write operation, it will be appended to the end of the aof file. The default aof persistence policy is to fsync the data from the cache area to disk once every second, because in this case, redis can still maintain good processing performance. Even if redis fails, only the last second of data will be lost.

Question1: what if the aof file is broken?

Back up bad AOF files

Run redis-check-aof-fix to repair

Use diff-u to look at the differences between the two files and identify the problem points.

Restart redis and load the repaired AOF file

Question: how to deal with the growing size of aof append files?

Redis provides a rewrite rewriting mechanism. When the aof file size exceeds the set threshold, aof file compression will be started, the minimum instruction set for recovering data will be retained, and temporary files will be written first when rewriting.

Rewrite principle: from the beginning of writing, any redis will fork a child process, which will first read the existing AOF file, analyze and compress the instructions it contains and write it to a temporary file.

At the same time, the main worker process will accumulate the newly received write instructions into the memory buffer while continuing to write to the original AOF file, which ensures the availability of the original AOF file and avoids accidents in the rewriting process. When the "rewrite child process" completes the rewriting work, it will send a signal to the parent process. After receiving the signal, the parent process appends the write instruction cached in memory to the new AOF file. When the append is over, redis will replace the old AOF file with the new AOF file. After that, the new write instruction will be appended to the new AOF file.

4.redis master and slave

Master-slave advantages: redundant backup, improve performance, read-write separation, master-slave architecture asynchronously, do not reduce processing performance master-slave principle: send sync instructions from the server to the master server, when the master server receives the instruction, it will call the bgsave instruction to create a child process specifically for persistence work, that is, the data on the master server will be written to rdb. During data persistence, the master server caches all write instructions executed in memory. After the bgsave instruction is completed, the master server sends the rdb file to the slave server, saves the file to disk from the slave server, and then reads it into memory. After this action is completed, the master server will cache the write instructions for this period of time and send them to the slave server in redis protocol. Note: even if there are multiple sync instructions sent from the slave server, the master server will only execute bgsave once

5.redis things

Redis four instructions form the basis of redis transaction processing: 1.MULTI is used to assemble a transaction; 2.EXEC is used to execute a transaction; 3.DISCARD is used to cancel a transaction; 4.WATCH is used to monitor some key and cancel the execution of the transaction once these key are changed before the transaction is executed.

II. Redis architecture

Standalone:

Limited capacity, limited processing capacity, simple deployment, suitable for development, or unimportant data.

Master-slave replication:

1. Master/slave role

2. Master/slave data is the same.

3. Reduce the master reading pressure before transferring it to the slave library.

Problems: (1) High availability cannot be guaranteed; (2) the pressure of master writing is not solved.

Sentinel:

Redis sentinel is a distributed system that monitors redis master and slave servers and automatically fails over when the master server goes offline. Three of these characteristics

Monitoring: Sentinel will constantly check whether your master server and slave server are working properly

Reminder (Notification): when there is a problem with a monitored Redis server, Sentinel can send a notification to the administrator or other application through API

Automatic failover (Automatic failover): when a primary server does not work properly, Sentinel starts an automatic failover operation

Features:

1. Ensure high availability

2. Monitor each node

3. Automatic failover

Disadvantages: master-slave mode, switching takes time to lose data

Did not solve the pressure of master writing

Cluster (proxy)

Twemproxy is a Twitter open source redis and memcache fast / lightweight proxy server; Twemproxy is a fast single-threaded agent that supports Memcached ASCII and redis protocols.

Features: 1. Multiple hash algorithms: MD5, CRC16, CRC32, CRC32a, hsieh, murmur, Jenkins

2. Support automatic deletion of failed nodes

3. The back-end Sharding sharding logic is transparent to the business, and the reading and writing mode of the business side is the same as that of a single Redis.

Cons: a new proxy has been added and its high availability needs to be maintained.

Failover logic needs to be implemented on its own, and it can not support automatic fault transfer. The scalability is poor, and manual intervention is needed for capacity expansion and reduction.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.