Shulou(Shulou.com)06/01 Report--

Master replication is simply to synchronize the data from one redis database to another redis database, and according to the data flow, the sender of the data is called master, and the receiver of the data is called slave (the division of master/slave is not so certain, for example, B can be used as the slave of A, but also as the master of C). The master-slave replication process is explained from the perspective of slave and master, respectively.

The first is the slave terminal. For the slave terminal, the master-slave replication mainly goes through four stages:

Phase 1: establish a connection with master

Phase 2: initiating a synchronization request (SYNC) to master

The third stage: accept the RDB data from master

Phase 4: loading the RDB file

Let's use a diagram to outline exactly what slave does at each stage:

With regard to the figure above, one note is as follows: redis does not establish a connection with master immediately after receiving the slaveof master_host master_port command. Instead, when it executes the server routine task serverCron and finds that it is in the REDIS_REPL_CONNECT state, it really initiates a connection to maser, pseudo code:

Python code

Def serverCron ():

# Server is in REDIS_REPL_CONNECT state

If redisServer.repl_state = = REDIS_REPL_CONNECT:

# initiate a connection to master

ConnectWithMaster ()

# other routine tasks (omitted).

Next, let's take a look at the process of master in the process of master-slave replication. Before looking at the details, let's take a comprehensive look at the main things that master does:

After reading this picture, you may have the following questions:

1. Why send PING commands to slave periodically after master sends RDB files?

two。 What is the "change" command sent by master after sending the RDB file, and what is the use?

Before answering question 1, let's answer question 2:

Master saves RDB files through a child process, so master can still process client requests without blocking, but this also leads to changes in the "key space" during saving RDB files (such as receiving a client request and executing the "set name diaocow" command), so in order to ensure the consistency of data synchronization, master will save the RDB file during the period Save these commands that may change the "key space" of the database, and then put them in the reply list of each slave. When the RDB file is sent, master will send the contents of these reply lists, and after that, if the database changes, master will still append the changed commands to the reply list and send them to slave, thus ensuring the consistency of master and slave data! Related pseudo code:

Python code

Def processCommand (cmd, argc, argv):

# processing commands

Call (cmd, argc, argv)

# if the command causes the database key space to change and the current redis is a master, then synchronize the change command

If redisServer.update_key_space and len (redisServer.slaves) > 0:

ReplicationFeedSlaves (cmd, argc, argv)

Def replicationFeedSlaves (cmd, argc, argv):

# send the change command to each slave node in the: REDIS_REPL_WAIT_BGSAVE_END state

For slave in redisServer.slaves:

If slave.replstate = = REDIS_REPL_WAIT_BGSAVE_START:

Continue

Slave.updateNotify (cmd, argc, argv)

Since master sends "change" commands to slave from time to time after sending RDB files, perhaps after 1 second or 1 hour, in order to prevent slave from waiting meaninglessly (such as when master is dead), master needs to send the "keep alive" command PING regularly to tell slave: I am still alive, do not break the connection with me.

Now let's take a look at what happens when master receives the sync synchronization command from slave:

The above image looks like a complex branch, but let's just grasp the following points:

1. Saving the RDB file is done in a child process

two。 If master is already saving the RDB file, but no client is waiting for this BGSAVE, the newly added slave will have to wait until the next BGSAVE, instead of using the generated RDB file directly (as explained in the reason diagram)

3.master periodically checks whether the RDB file has been saved (time event serverCron)

Let's take a look at how master sends RDB files to each slave:

Well, now we have analyzed how master and slave are handled separately in the process of master-slave replication. Finally, I draw a diagram that summarizes the process of master-slave replication (we can recall the details while looking at the figure):

PS: in the process of master-slave replication, any error in any step will cause the whole process to start all over again, so if the RDB file is very large or is at the peak of business at this time, it will have a great impact on system performance!

Summary:

1. Understand the concept of master-slave replication master and slave

two。 Understand the execution process of master-slave replication, especially the key steps

3. To understand the shortcomings of the current master-slave replication process.

View picture attachments

Oyhk study notes

The number of visitors to the website is slowly coming up. For the performance of the website, we began to use redis as a caching strategy. At the beginning, redis is a single point, when a machine rock machine, the service of redis stops completely, which will affect the normal operation of other services. Needless to say, let's use redis sentinel to do a master-slave cluster management. When I was doing this cluster management, I checked a lot of information before I fully understood how he did it.

For Java clients, please see:

Http://blog.mkfree.com/posts/52b146e6479e5a64742fddd0

Reference: http://redis.io/topics/sentinel I also read this article.

Environment configuration:

Since I didn't have too many machines in my configuration this time, I used vagrant to turn on multiple virtual machines. And set up the environment.

For the installation of redis, please refer to redis simple official script installation method (linux)

Cluster configuration requires at least three machines, so I will have three virtual machines, and three virtual machines will install the same redis environment.

Ip respectively:

192.168.9.17 (redis sentinel cluster monitoring)

192.168.9.18 (redis Master)

192.168.9.19 (redis from)

Redis configuration:

The main redis configuration file, use the default configuration file, if you need to design other parameters

From the redis configuration file, add

# from the redis configuration file, you need to add vim / etc/redis/6379.confslaveof 192.168.9.18 6379

Start the master-slave redis

# launch Master Redis (192.168.9.18) / etc/init.d/redis_6379.conf start# launch slave redis (192.168.9.19) / etc/init.d/redis_6379.conf start

View primary redis information

# View the information of the main redis redis-cli-h 192.168.9.18 info Replication# Replicationrole:master # represents 192.168.9.18 info Replication# Replicationrole:master 6379 this redis is the main connected_slaves:1slave0:192.168.9.18,6379,online

View information from redis

# View the information of the main redis redis-cli-h 192.168.9.19 info Replication# Replicationrole:slave # represents 192.168.9.18 info Replication# Replicationrole:slave 6379 this redis is the main master_host:192.168.9.18master_port:6379master_link_status:upmaster_last_io_seconds_ago:4master_sync_in_progress:0slave_priority:100slave_read_only:1connected_slaves:0

Configure redis sentinel cluster monitoring service

1. Add a redis sentinel profile

The communication port port 26379#master1sentinel monitor master1 192.168.9.18 6379 1sentinel down-after-milliseconds master1 5000sentinel failover-timeout master1 900000sentinel can-failover master1 yessentinel parallel-syncs master1 2#master2 between vim / etc/redis/sentinel.conf##redis-0##sentinel instances can add multiple sets of master-slave redis snooping.

two。 If you have a configuration file, start redis sentinel to do redis cluster listening.

Redis-sentinel sentinel.conf-sentinel

All right, all the environments are set up. Let's start the formal demonstration.

1. Normal demonstration.

Start the master's redis

Start the slave redis

Start redis sentinel cluster snooping

Observe redis sentinel log information

It is clear here that we have joined the cluster from redis.

[4925] 15 Oct 03 master1 42 master1 21.889 * + slave slave 192.168.9.19 1979 192.168.9.19 6379 @ master1

Execute the following command to view redis master and slave information

[root@localhost vagrant] # redis-cli-h 192.168.9.17-p 26379 info Sentinel# Sentinelsentinel_masters:1sentinel_tilt:0sentinel_running_scripts:0sentinel_scripts_queue_length:0master0:name=master1,status=ok,address=192.168.9.18:6379,slaves=1,sentinels=1

Then everything is all right. Your redis sentinel cluster has been configured successfully!

two。 Fault demonstration

2.1What happens when the main redis server rock machine is installed?

Execute the following command to stop using the main redis service

Redis-cli-h 192.168.9.18-p 6379 shutdown # means to shut down the 192.168.9.18 redis

After the shutdown, we will check the log of redis sentinel.

This picture clearly reflects that the redis sentinel monitor stops the master redis service, and then automatically switches the slave redis to the master.

Execute the following command again to view the master and slave information of redis

[root@localhost vagrant] # redis-cli-h 192.168.33.111-p 26379 info Sentinel# Sentinelsentinel_masters:1sentinel_tilt:0sentinel_running_scripts:0sentinel_scripts_queue_length:0master0:name=master1,status=ok,address=192.168.9.19:6379,slaves=1,sentinels=1

The subordinate has been promoted to priority. Then the automatic switching has been successful!

2.2 when we have found that a redis has malfunctioned and may receive some fault messages, what happens if we restore the service state of the redis where the service has been shut down?

The redis sentinel cluster service rejoins the last master redis into the service, but it is no longer the master redis and becomes the slave reids.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.