Shulou(Shulou.com)06/02 Report--

This article mainly explains "what is the concept and function of zookeeper". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "what is the concept and function of zookeeper".

11.1.1. Zookeeper concept

Zookeeper is a distributed coordination service that can be used for service discovery, distributed locking, distributed leadership election, configuration management, etc. Zookeeper provides a tree structure similar to the Linux file system (can be thought of as a lightweight in-memory file system, but only suitable for storing a small amount of information, not suitable for storing a large number of files or large files), and provides a monitoring and notification mechanism for each node.

11.1.1. Zookeeper role

A Zookeeper cluster is a highly available cluster based on master-slave replication. Each server assumes one of the following three roles

Leader

1. A Zookeeper cluster will have only one actual working Leader at a time, which initiates and maintains heartbeats with each Follwer and Observer. two。 All writes must be done through Leader and then Leader broadcasts them to other servers. As long as more than half of the nodes (excluding observeer nodes) are successfully written, the write request will be submitted (2PC-like protocol).

Follower

1. There may be multiple Follower in a Zookeeper cluster, which will respond to the heartbeat of the Leader. 2. Follower can directly process and return the read request from the client, and forward the write request to the Leader for processing. And is responsible for voting on the request when the Leader processes the write request.

Observer role

Similar to Follower, but with no voting rights. Zookeeper needs to ensure high availability and strong consistency. In order to support more clients, more Server;Server needs to be added, and voting delay increases, which affects performance; Observer is introduced and Observer does not participate in voting; Observers accepts client connections and forwards write requests to leader nodes; more Observer nodes are added to improve scalability without affecting throughput.

ZAB protocol

Transaction number Zxid (transaction request counter + epoch)

In the Zxid design of the transaction number of ZAB (ZooKeeper Atomic Broadcast, ZooKeeper Atomic message broadcast Protocol) protocol, Zxid is a 64-bit number, of which the lower 32-bit is a simple monotonously increasing counter, which is added by 1 for each transaction request of the client. The high 32 bits represent the number of the Leader cycle epoch, and each time a new Leader server is elected, the ZXID of the largest transaction in its local log is taken from this Leader server, and the epoch value is read from it, then added 1 as the new epoch, and the low 32 bits are counted from 0. Zxid (Transaction id) is similar to the transactional ID in RDBMS and is used to identify the Proposal (proposed) ID of an update operation. In order to ensure sequence, the zkid must be monotonously incremented.

Epoch

Epoch: it can be understood as the age or cycle of the current cluster. Each leader, like the emperor, has its own year name, so each change of dynasty and leader will add 1 to the previous era. In this way, even after the old leader crashes and recovers, no one will listen to him, because follower only listens to the orders of the leader of the current era.

There are two modes of Zab protocol-recovery mode (primary) and broadcast mode (synchronization).

There are two modes of Zab protocol, which are recovery mode (primary mode) and broadcast mode (synchronization). When the service starts or after the leader crashes, the Zab enters the recovery mode, and when the leader is elected and most of the Server finishes synchronizing with the leader, the recovery mode ends. State synchronization ensures that leader and Server have the same system state.

ZAB protocol 4-phase Leader election (election phase-election of quasi-Leader)

1. Leader election (election phase):

Nodes are in the election stage at the beginning, as long as one node gets more than half of the votes, it can be elected quasi-leader. Only when you reach the broadcast stage (broadcast) will the quasi-leader become a real leader. The purpose of this stage is to select a quasi-leader and then move on to the next stage.

Discovery (Discovery Phase-accept offer, generate epoch, accept epoch)

2. Discovery (Discovery Phase): in this phase, the followers communicates with the quasi-leader and synchronizes the transaction proposal recently received by the followers. The main purpose of this phase is to find the latest proposal received by most of the current nodes, and the quasi-leader generates a new epoch for followers to accept, and updates their accepted Epoch a follower will only connect to one leader. If a node f thinks that another follower p is leader,f, it will be rejected when trying to connect to p, and after f is rejected, it will enter the re-election stage. Synchronization (synchronization phase-synchronizing follower copies)

3. Synchronization (synchronization phase): the synchronization phase mainly uses the latest proposal history obtained in the previous stage of leader to synchronize all replicas in the cluster. Quasi-leader will become a true leader only when most of the nodes are completed synchronously. Follower will only accept offers that zxid is larger than its own lastZxid. Broadcast (broadcast phase-leader message broadcast)

4. Broadcast (broadcast phase): only at this stage can the Zookeeper cluster formally provide transaction services to the outside world, and leader can broadcast messages. At the same time, if there are new nodes to join, the new nodes need to be synchronized.

ZAB commit transactions do not require all follower to be ACK like 2PC, but only need to get the ACK of more than half of the nodes.

JAVA implementation of ZAB protocol (FLE- discovery phase and synchronization merge into Recovery Phase (recovery phase))

The implementation of the Java version of the protocol is somewhat different from the definition above, and the election phase uses Fast Leader Election (FLE), which includes the discovery responsibility of the election. Because FLE elects the node with the latest proposal history as the leader, this eliminates the step of discovering the latest proposal. The actual implementation combines the discovery phase and synchronization into a Recovery Phase (recovery phase). Therefore, the implementation of ZAB has only three stages: Fast Leader Election;Recovery Phase;Broadcast Phase.

Voting mechanism

Each sever first votes for itself, and then compares its own ballot with other sever ballots. The one with great weight wins, and the one with more weight updates its ballot box. The specific election process is as follows:

1. After each Server starts, it asks the other Server who it wants to vote for. For other server queries, server replies to the id of the leader recommended by him and the zxid of the last transaction according to his own status (each server will recommend himself when the system starts)

two。 After receiving all the Server responses, the Server with the largest zxid is calculated and the Server-related message is set to the Server to vote next time.

3. The sever that gets the most votes in this process is the winner. If the winner has more than half of the votes, then change server to leader. Otherwise, continue the process until leader is elected. Leader will start waiting for the server connection.

5. Follower connects to leader and sends the largest zxid to leader

6. Leader determines the synchronization point according to follower's zxid, and the election phase is completed.

7. After completing the Leader synchronization in the election phase, notify follower that it has become uptodate status 8. After receiving the uptodate message, Follower can re-accept the client request for service.

At present, there are 5 servers, and each server has no data. Their numbers are 1magin, 2pr, 3pr, 4p, respectively. They are started according to the serial number. The process of selecting them is as follows:

1. Server 1 starts, votes for itself, and then sends a vote message. Because other machines have not been started, it does not receive feedback information. The status of server 1 always belongs to Looking.

two。 Server 2 starts, votes for itself, and exchanges results with server 1 started before. Server 2 wins because of its large number, but at this time the number of votes is not more than half, so the status of the two servers is still LOOKING.

3. Server 3 starts up, votes for itself, and exchanges information with the previously started server 1Magin2. Server 3 wins because the number of server 3 is the largest, and the number of votes happens to be more than half, so server 3 becomes the leader. Server 1focus 2 becomes the younger brother.

4. Server 4 starts to vote for itself, while exchanging information with the previously started server 1Magazine 3. Although server 4 has a large number, server 3 has won before, so server 4 can only become a younger brother.

5. Server 5 starts, and the logic behind it becomes a younger brother with server 4.

Thank you for your reading, the above is the content of "what is the concept and function of zookeeper". After the study of this article, I believe you have a deeper understanding of the concept and function of zookeeper, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.