Shulou(Shulou.com)06/02 Report--

How to analyze the principle of Zookeeper? in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Zookeeper role

Leader: responsible for initiating voting and system status update, completing cluster write operation and data synchronization

Followers: participate in the voting, responsible for forwarding the write request to the leader, and receive the client request, and the corresponding client query

Observer: do not participate in the election, forward write requests to leader, receive client requests, improve cluster reading and corresponding speed (negligible)

Zookeeper core

ZAB (Zookeeper Actomic BoardCost) protocol: there are two modes (recovery mode and broadcast mode)

There is only one node in the entire zookeeper cluster, the Leader, that converts client writes into things (or proposal proposal). After the Leader node has finished writing the data, it will send a data broadcast request (or data replication) to all follower nodes and wait for feedback from all follower nodes. In the ZAB protocol, as long as more than half of the follower nodes give feedback, the OK,Leader node will send commit messages to all follower servers. Synchronize the data on the leader node to the follower node

Principle:

1. The ZAB protocol requires that each leader go through three stages, namely, discovery, synchronization, and broadcasting.

two。 Discovery: that is, the zookeeper cluster is required to select a leader process, while leader maintains a list of follower available. In the future, the client can communicate with the nodes in the follower.

3. Synchronization: leader is responsible for synchronizing its own data with follower to achieve multi-copy storage. This also reflects the high availability and partition fault tolerance in CAP. After follower completes the consumption of outstanding requests in the queue, it writes it to the local transaction log.

4. Broadcast: leader can accept new proposal requests from the client and broadcast new proposal requests to all follower.

Recovery model (primary node election: in the case of startup and crash recovery)

Election:

1. Serverid: when configuring server, the marked id of the given server.

2. Zxid: the larger the id,zxid of the data generated by the server at run time, the newer the data.

3. Epoch: the number of rounds elected, that is, the logical clock. With the number of rounds in the election

4. Server status: LOOKING,FOLLOWING,OBSERVING,LEADING

Steps:

When 1.Server has just started (crash recovery or just started) and is ready to join the cluster, read its own zxid and other information.

two。 All Server will recommend themselves as leader when they join the cluster, and then broadcast (leader id, zixd, epoch) as broadcast information to all the servers in the cluster (Server). Then wait for the server in the cluster to return information.

3. When you receive the information returned by other servers in the cluster, it is divided into two categories: the server is in looking state, or other state.

(1) the server is in looking state

First determine the logical clock Epoch:

A) if you receive an Epoch larger than your current logical clock (indicating that the logical clock you saved is out of date). Update the native logical clock Epoch while Clear the election data sent by other services (which has already been OUT). Then determine whether you need to update your current election situation (the leader id you chose at the beginning)

Judgment rule rules judging: save the maximum value of zxid and leader Serverid to judge. First look at the data zxid, the data zxid wins, then judge the leaderServerid, the leaderServerid wins, and then broadcast your latest election results (that is, the three data mentioned above (leaderServerid, Zxid,Epoch) to other server).

B) if the received Epoch is less than the current logical clock. Indicates that the other party is in a relatively OUT number of election rounds, and you only need to send your own (leader Serverid,Zxid,Epoch) to him at this time.

C) if the received Epoch is equal to the current logical clock. Then, according to the judgment rules in a), broadcast your latest election results to other server.

At the same time, Server also has to deal with two situations:

A) if the Server receives election information from all other servers, then determine its own status (Following,Leading) based on the election information, end the Looking, and withdraw from the election.

B) even if you have not received the election information from all the servers, you can determine whether the latest election leader after the above process is supported by more than half of the servers. If so, try to accept the latest data. If there is no latest data, it means that everyone has acquiesced to this result, and also set roles to withdraw from the election process.

(2) the server is in other states (Following, Leading)

A) if the logical clock Epoch is the same, save the data to recvset, and if the receiving server claims to be leader, it will determine whether more than half of the servers elect it, and if so, set the election status to exit the election process.

B) otherwise, this is a message that does not match the current logical clock, then it means that there is already an election result in another election process, so add the election result to the outofelection collection, and then judge whether the election can be ended according to the outofelection. If you can also save the logical clock, set the election state and exit the election process.

Crash recovery:

The newly elected leader cannot contain an unsubmitted proposal, that is, the newly elected leader must all be the follower server nodes of the submitted proposal. Meanwhile, the newly elected leader node contains the highest ZXID.

Broadcast mode (data synchronization-class two-phase commit (2pc), but half of them are available)

Data consistency: the core of zookeeper's adoption of the ZAB protocol is to ensure that as long as one server submits the proposal, all servers eventually submit the proposal correctly. This is also a reflection of the ultimate consistency of CAP/BASE.

Performance: there is a separate queue between the leader server and each follower to send and receive messages, which can be decoupled asynchronously using queue messages. Between leader and follower, as long as a message is sent to the queue. If you use synchronization, it is easy to cause blocking. There is a lot of decline in performance.

Proposal and ZXID:ZXID are 64-bit digits in length, where the lower 32 bits are incremented by numbers, that is, each time the client initiates a proposal, the lower 32-bit numbers are simply added by 1. The high 32-bit number is the epoch number of the leader cycle. As for how this number is generated (I don't understand it), whenever a new leader is elected, the new leader takes the ZXID from the local transaction log, then parses the high 32-bit epoch number, adds 1, and sets all the low 32-bit numbers to 0. This ensures the uniqueness and increment of the leader after each new election.

The specific steps are as follows:

1. The client initiates a write request

2. The Leader server converts the client's request request into a transactional proposql proposal, while assigning a globally unique ID, namely ZXID, to each proposal.

3. There is a queue between the leader server and each follower, to which the leader sends messages

4. After the follower machine has finished processing the message from the queue (writing to the local transaction log), it sends ACK confirmation to the leader server.

5. After receiving more than half of the ACK of follower, the leader server thinks that it can send commit.

6. Leader sends commit messages to all follower servers.

The answer to the question on how to analyze the principle of Zookeeper is shared here. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.