Shulou(Shulou.com)06/02 Report--

This article introduces the relevant knowledge of "A brief introduction to the working mechanism and data structure of Zookeeper". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

I. the working mechanism of Zookeeper

Compared with centralized system, distributed system has many advantages, such as stronger computing power, storage capacity, avoiding single point of failure and so on. However, how to ensure the consistency and availability of each node data is a key problem when we encounter problems such as network failure in the way of distributed deployment.

So, for distributed clusters, we need a middleman who can coordinate and serve between services and nodes-Zookeeper.

Zookeeper is understood from the perspective of design pattern: it is a distributed service management framework based on the observer pattern, which is responsible for storing and managing the data that everyone cares about, and then accepts the registration of the observers. Once the state of the data changes, Zookeeper will be responsible for notifying the observers who have registered on the Zookeeper to respond accordingly.

2. Data structure

The data structure of Zookeeper is similar to the directory structure of linux, as well as the tree in the data structure, as shown below:

Zookeeper's data storage is based on nodes, which are called Znode. Znode is referenced as a reference to a path, and each Znode can be uniquely identified by its path.

The Znode contains: data, child node references, access permissions, etc., as shown below:

Data information stored by data:Znode

ACL: record the access rights of Znode, that is, who or which IP can access this node

Child: the child node reference of the current node, similar to the left child and the right child of the binary tree

Stat: contains all kinds of Znode metadata, such as transaction ID, version number, timestamp, size, etc.

Stat views the details of the root directory:

[zk: localhost:2181 (CONNECTED) 0] stat / cZxid = 0x0 ctime = Thu Jan 01 08:00:00 CST 1970 mZxid = 0x0 mtime = Thu Jan 01 08:00:00 CST 1970 pZxid = 0x0 cversion =-1 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 0 numChildren = 1

III. Election mechanism

Zookeeper cluster is a multi-master and multi-slave mode, with leader as master and follower as slave, in which leader is obtained by election.

Zookeeper cluster has the following characteristics:

-Zookeeper: a cluster of leader and multiple followers (follower)

-Leader is responsible for initiating and deciding votes and updating system status.

-Follower is used to receive customer requests and return the results to the client, and participate in voting during the election of Leader

-as long as more than half of the nodes in the cluster are alive, the Zookeeper cluster can serve normally, so Zookeeper is suitable for installing odd number of servers.

-Global data consistency: each server keeps a copy of the same data. No matter which server the client connects to, the data is consistent.

-Update requests are performed sequentially, and update requests from the same client are executed in the order in which they are sent.

-data update atomicity, a data update is either successful or failed

-Real-time, client can read the latest data within a certain time range

Leader election is the key to ensure the consistency of distributed data. When Zookeeper enters the following two states, you need to enter leader election:

Server initialization startup

Leader crashed and hung up.

1. Election when the server initializes the startup

(1) take a cluster of three servers as an example. During the initialization phase of the cluster, when server1 is started, it cannot complete the election alone; when server2 is started, the two machines can communicate with each other, and each machine tries to find leader, so it enters the election state.

(2) each server first votes for itself: in the initial stage, each server votes as a leader, and each vote contains (myid,ZXID,epoch). In this case, the vote of Server1 is (1,0), and the vote of Server2 is (2,0). Then each server sends the vote to other machines in the cluster.

Epoch is used to determine whether multiple votes are in the same round of election cycle. The value is a self-increasing sequence on the server. After entering the new round of voting, the value will be added by one.

(3) each server accepts votes from various servers: after each server in the cluster receives the vote, it first judges the validity of the vote, such as checking whether it is the current round of voting and whether it comes from the server in LOOKING status.

(4) to deal with voting. For each vote, the server needs to PK,PK other people's votes and their own votes as follows:

Check ZXID first. Servers with larger ZXID are preferred as Leader.

If the ZXID is the same, compare the myid. The server with larger myid acts as the Leader server

For Server1, the vote is (1,0), and the vote for receiving Server2 is (2,0). First, the ZXID of both is 0, and then the myid is compared. At this time, the myid of Server2 is the largest, so update its vote to (2,0), and then revote. For Server2, it does not need to update its vote, just send the last vote information to all the machines in the cluster again.

(5) Statistical voting. After each vote, the server will count the voting information to determine whether more than half of the machines have received the same voting information. For Server1 and Server2, it is counted that two machines in the cluster have accepted the voting information of (2,0). At this time, it is considered that Leader has been selected. Once leader is selected, the machines behind, no matter how old the myid and ZXID are, automatically become the younger brothers of leader.

(6) change the server state. Once Leader is determined, each server will update its status. If it is Follower, it will be changed to FOLLOWING, and if it is Leader, it will be changed to LEADING

2. Voting mechanism in which leader servers hang up

What is different from startup is that there is historical data on each server. Before the election, the non-leader server first changes its state to LOOKING state, because each server has a different ZXID during operation and will be revoted the same as the startup election.

IV. Monitoring mechanism

First, there must be a main () thread

When a Zookeeper client is created in a main thread, two threads are created, one for network connection communication (connet) and one for listening (listener)

Send registered listening events to Zookeeper through the connect thread

Add registered listeners to the list of registered listeners in Zookeeper

When Zookeeper detects a change in data or path, it sends this message to the listener thread.

The process () method is called inside the listener thread

5. API application

The common API of Zookeeper is as follows:

Create creates a node delete deletes a node exists determines whether a node exists getData gets a node's data setData sets a node's data getChildren gets all the child nodes under a node

Among them, exists,getData,getChildren belongs to read operation. When requesting a read operation, the Zookeeper client can choose whether or not to set Watch.

What does Watch mean?

We can understand it as a trigger registered on a particular Znode. When the Znode changes, that is, when the create,delete,setData method is called, the corresponding event registered on the Znode will be triggered, and the client requesting the Watch will receive an asynchronous notification.

The specific interaction process is as follows:

The client calls the getData method, and the watch parameter is true. The server receives the request, returns the node data, and inserts the Znode path to be Watch and the Watcher list in the corresponding hash table.

When the Znode of the Watch is deleted, the server looks up the hash table, finds all the Watcher corresponding to the Znode, notifies the client asynchronously, and deletes the corresponding Key-Value in the hash table

VI. Application scenarios

The services provided by Zookeeper include: unified naming service, unified configuration management, unified cluster management, dynamic up and down of server nodes, soft load balancing and so on.

This is the end of the content of "A brief introduction to the working mechanism and data structure of Zookeeper". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.