Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "what is the implementation method of redis master-slave replication". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

1 introduction

This article is mainly about the master-slave replication function of Redis. It will be introduced in turn from the aspects of environment construction, functional testing and principle analysis.

2 preparation work

The server architecture diagram is as follows

Start the master server 101and use the info replication command to check the status. You can see that the role is master (that is, the role is the master server) and the value of connected_salaves is 0 (the number of slave servers is 0)

Next, the 102 machine is added to the master-slave replication by modifying the configuration file.

Then the 103 machine is also added to the master-slave copy by command.

2.1 add the 102 machine to the master and slave by modifying the configuration file

Add slaveof 192.168.17.101 6379 to the Redis profile of the machine with the ip address 192.168.17.102

Start the redis of 102with the following status

You can see that role becomes slave (role is slave server), and master_host (master server IP address) is 192.168.17.101 master master port (master server port) is 6379.

At this time, the master and slave status of the 101master server is as follows. You can see that the value of connected_salaves has changed to 1, and a line of slave0 has been added (the status of the slave server).

2.2 add 103 machine to master and slave by command

The master-slave status of the unexecuted slaveof command is as follows

Start executing the slaveof command

192.168.17.103 slaveof 6379 > 192.168.17.101 6379OK

Looking at the status again, you can see that the role has changed from the server

Now if we look at the status of the master server, we can see that the number of servers has changed to 2, and there is another message about the slave server.

So far, the master has set up from the environment, now let's test a wave.

2.3 Test

Now the main server 101 enters the command

192.168.17.101 101OK 6379 > set 101

Then look at all the keys from the server 102, find that there is a key 101, and then set the key 102

192.168.17.102 set 6379 > keys * 1) "101" 192.168.17.102 set * 1) > set 102102 (error) READONLY You can't write against a read only slave.

An error (error) READONLY You can't write against a read only slave. The reason for the error is described later.

Now look at all the keys from server 103 and find that there are also 101

192.168.17.103 6379 > keys * 1) "101"

Then enter the command to the main server 101

192.168.17.101 6379 > set ip ipOK

Then go to view all the keys from server 103

192.168.17.103 6379 > keys * 1) "101" 2) "ip"

You can see that there is one more key, indicating that the data of the master service is synchronized to the slave server. The operation process is shown in the following figure

2.4 other 2.4.1 (error) READONLY You can't write against a read only slave.

An error occurred (error) READONLY You can't write against a read only slave. It's because

The slave node is read-only by default. If you need to modify it, you can modify the following attribute in the configuration file.

Slave-read-only yes2.4.2 master server sets password

When the main service sets the password, the configuration file needs to add parameters as needed.

Implementation principle of masterauth 3

When I enter the slaveof command from server 103, the following log appears

In general, the detailed steps of the master-slave replication function can be divided into seven steps:

Set the address and port of the primary server

Establish a socket connection

Send PING command

Authentication

Send Port Information

Synchronization

Command propagation

Next, describe each step separately.

3.1 set the address and port of the primary server

The first step of master-slave replication is to set the address and port of the master server. When entering the slaveof command or configuring information in the configuration file, the slave server will save the ip address and port number of the master server to the properties of the server state.

3.2 establish a socket connection

After the slaveof command is executed, the slave server resumes the socket connection to the master server based on the set ip and port.

3. Send PING command

After the socket connection is successful, the slave server sends a PING command to the master server.

At this point, the PING command can check whether the read and write status of the socket is normal, and it can also check whether the master server can handle command requests properly.

The situation that the slave server may encounter when sending PING commands is shown in the following figure

3.4 Authentication

After receiving a PONG reply from the master server from the slave server, the slave server is checked to see if masterauth is set and authentication is performed. If not, this step is skipped. The situations that may be encountered by the slave server during authentication are as follows

3.5 send Port Information

After authentication is passed, the slave server sends its own listening port number to the master server. When received by the master server, the port number is recorded in the status attribute corresponding to the slave server. You can see the port of the slave server by calling info replication on the master server, as follows

3.6 synchronization

After sending the port information, the slave server sends a PSYNC command to the master server, performs a synchronization operation, and synchronizes its own database to the current state of the master server database.

Synchronization will be described in detail later.

3.7 Command propagation

When the synchronization operation is completed, the master-slave server enters the command propagation phase. At this time, the data of the master-slave server is consistent. When the master server has a new write command, it will send the change command to the slave server. Receiving the command from the server and executing it can ensure that it is consistent with the data of the master server.

So how does Redis ensure that the master and slave servers are connected consistently and whether commands are lost?

A: during the command propagation phase, the slave server will use the heartbeat detector to send messages to the master service.

The command sent from the server is as follows

REPLCONF ACK

Replication_offset represents the current replication offset from the server

Next, let's look at the heartbeat mechanism.

3.7.1 heartbeat detection mechanism

The heartbeat detection mechanism has three functions:

Check the network connection status of the master-slave server

Auxiliary implementation of min-slaves options

Detection command is missing

3.7.1.1 check the network connection status of the master-slave server

You can see the connection information of the slave server in the master server information. State indicates slave server status, offset indicates replication offset, and lag indicates delay value (heartbeat detection mechanism existed a few seconds ago)

3.7.1.2 Auxiliary implementation of min-slaves options

There are two parameters in the Redis.conf configuration file

# if the following two conditions are not met, the write operation will not be performed # the minimum slave server # min-slaves-to-write delay value # min-slaves-max-lag 10

If you uncomment the two parameters, the master server will refuse to execute the write command if the number of slaves is less than 3, or if the lag of the three slaves is greater than or equal to 10 seconds.

3.7.1.3 detecting command loss

The replication offset can be seen in the connection information of the slave server. If the replication offset of the master server is inconsistent with that of the slave server, the master server will reissue the missing data.

4 synchronization principle

Synchronization is divided into full resynchronization and partial resynchronization. So what decides to take full resynchronization or partial resynchronization?

4.1 full resynchronization

The steps for full resynchronization are as follows

When the master node receives the full resynchronization request from the slave server, the master server executes the bgsave command and records all write commands executed from now on with a buffer.

When the bgsave command of the master server finishes, the generated RDB file is sent to the slave server. When a RDB file is received from the server, the data file is saved to the hard disk and then loaded into memory.

The master server sends all cached commands in the buffer to the slave server, receives and executes these commands from the server, and will synchronize from the server to the same state of the master server.

4.2 partial resynchronization

To understand the steps of partial resynchronization, you need to understand several properties required for partial resynchronization

Copy offset

Copy buffer

Run ID

4.2.1 replication offset

From the replication information of the master server, you can see that both the slave server slave0 and the slave1 have a parameter offset, which is the replication offset of the slave server. The parameter master_repl_offset is the offset of the primary server. The figure below is as follows

The replication offset of the master server stores the byte data sent to the slave server.

The replication offset from the server holds the bytes of data received from the master server.

By comparing the replication offset of the master server and the slave server, we can know whether the command is lost or not. If the command is lost, the byte command with the difference in replication offset is reissued.

So where are these bytes of data stored?

4.2.2 copy buffer

These bytes of data are stored in the replication buffer of the main server. The copy buffer is a fixed-length (fixed-size) first-in-first-out (FIFO) queue with a default size of 1MB. The default size can be modified to the parameters below.

# repl-backlog-size 1mb

So when was the data from the copy buffer added?

A: in the command propagation phase, the master node not only sends the write command to the slave node, but also sends a copy to the copy backlog buffer.

The copy buffer holds some of the most propagated write commands and the corresponding copy offset for each byte.

Because the size of the replication buffer is limited, there is also a limit to the data saved. If the difference between the replication offset of the slave server and the master server is greater than the data stored by the replication buffer, partial resynchronization will not be performed.

For example, the replication offset of the master server is 20000, the buffer can hold only 5000 data, and the replication offset of the slave server is 10000. In this case, if the offset is 10000 between the slave server and the master server, and the buffer is only 5000, then the full resynchronization will be performed. Partial resynchronization is performed only if the replication offset of the difference is less than 5000.

4.2.3 run ID

When each Redis server starts, it automatically generates its own running ID.

When the slave server replicates the master server for the first time, the master server sends its own running ID to the slave server.

When the slave server is disconnected and reconnected, the running ID of the previous master server is sent to the currently connected master server. At this time, the following two situations will occur.

Running ID is consistent with the primary server, which can attempt to perform a partial resynchronization operation.

Running ID is inconsistent with the primary server, indicating that the previous connection to the primary server is different from this connection, and the full resynchronization operation begins.

5 related configuration # # REPLICATION # # slaveof # slaveof # masterauth # masterauth # when slave loses master or synchronization is in progress If a service request for slave # yes occurs, slave still provides normal service # no, then slave returns client error: "SYNC with master in progress" slave-serve-stale-data yes# specifies whether slave is read-only slave-read-only yes# without hard disk replication function repl-diskless-sync no# without hard disk replication function interval time repl-diskless-sync-delay "cycle of sending PING commands to the master server from the slave server # repl -ping-slave-period 1 "timeout # repl-timeout 6" whether to disable the NO_DELAY option of socket repl-disable-tcp-nodelay no# sets the master-slave replication capacity size This backlog is the lifetime of a buffer# repl-backlog-size 1mb# master that is used to store slave data when the slaves is disconnected and the slave is no longer connected. If the priority slave-priority 10 of # repl-backlog-ttl 360 slave slave does not meet the following two conditions, the write operation will not be performed # the minimum slave server # min-slaves-to-write slave delay value # min-slaves-max-lag 10 "what is the implementation of redis master-slave replication" is introduced here, thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.