Shulou(Shulou.com)05/31 Report--

Today, I would like to talk to you about the principle of the zookeeper mechanism, which may not be well understood by many people. in order to make you understand better, the editor has summarized the following for you. I hope you can get something according to this article.

Zookeeper machine principle (long connection):

Zookeeper is not used to store data specifically, it is mainly used to maintain and monitor the state changes of the data you store. By monitoring the changes in the state of these data, the data-based cluster management can be achieved. When monitoring the state of the directory node is turned on, once the state of the directory node changes, the process method of the Watcher object is called.

(any implementation org.apache.zookeeper.Watcher class, which holds the zookeeper instance object, simply calls the method of zookeeper and monitors the corresponding node type

Then the zookeeper server will notify the observer when the node changes (the observer rewriting method process will be called)

Zk = new ZooKeeper (zookeeperQuorum, sessionTimeout, this)

ZookeeperQuorum=IP+ port (xxx.xxx.xxx.xxx:2181,xxx.xxx.xxx.xxx:2181,xxx.xxx.xxx.xxx:2181) separated by multiple commas

You can set the action to observe: exists,getChildren,getData

Action that can trigger observation: create,delete,setData

Zookeeper observation mechanism

The server only stores the information of events, and the client stores the information of events and the execution logic of Watcher. ZooKeeper client is thread-safe. Each application only needs to instantiate a ZooKeeper client, and the same ZooKeeper client instance can be used in different threads. The ZooKeeper client stores the Path path corresponding to the Watcher in ZKWatchManager and notifies the ZooKeeper server to record the event types registered under Path in the Session corresponding to the Client. When the specified event occurs on the ZooKeeper server, the ZooKeeper server will notify the ZooKeeper client of the event type under which node, and the ZooKeeper client will find the corresponding Path from the ZKWatchManager and execute its callback function process from the corresponding watcher reference.

Source code analysis such as:

Zookeeper.java

Wcb = new DataWatchRegistration (watcher, clientPath); / / bind the observer reference to the node path

ZKWatchManager.java

Public void register (int rc) {

If (shouldAddWatch (rc)) {

Map watches = getWatches (rc)

Synchronized (watches) {

Set watchers = watches.get (clientPath)

If (watchers = = null) {

Watchers = new HashSet ()

Watches.put (clientPath, watchers); / / key is path,value and watcher reference

}

Watchers.add (watcher)

}

}

}

Znode stores the largest amount of data: jute.maxbuffer defaults to 1m

Notice note:

1. One-time trigger (you can get the trigger event by calling the method get () or exists () regularly and access the trigger manually)

Client sets watch on a node, and then the node content changes, and client gets the event. When the content of the node changes again, client will not get this event unless it performs another read operation and sets the watch

two。 Send to client,watch event delay

The watch event is sent asynchronously to the observer. For example, when client performs a write operation, the data content of the node changes, and after the operation returns, the watch event may still be on its way to client. In this case, zookeeper provides an orderly guarantee: client will not know about data changes until it gets the watch event. Network delays or other factors may cause different client to get the return values of watch events and actions at different times.

Circular call:

Watcher watcher = loopWatch?this:null

Byte [] newdata = keeper.getData (path, watcher, null); / / change reminder again!

Application scenarios:

From the design pattern point of view, Zookeeper is a distributed service management framework based on the observer pattern. It is responsible for storing and managing the data that everyone cares about, and then accepts the registration of the observers. Once the state of the data changes, Zookeeper will be responsible for notifying the observers who have registered on the Zookeeper to respond accordingly.

Notifier mode:

The publisher holds the handle reference of the subscriber, a single state change, and calls the subscriber interface to realize the notification pattern call.

Zookeeper operation API:

The client can connect to the Zookeeper server by creating org.apache.zookeeper. An instance object of ZooKeeper, and then call the interface provided by this class to interact with the server

ZooKeeper is mainly used to maintain and monitor the status of the data stored in a directory node tree.

Directory node type:

PERSISTENT: persists the directory node. The data stored by this directory node will not be lost.

PERSISTENT_SEQUENTIAL: a sequentially auto-numbered directory node that automatically adds 1 according to the number of nodes that currently exist

EPHEMERAL: temporary directory node. Once the client and server ports of this node are created, that is, session timeout or fracture, this node will be deleted automatically.

EPHEMERAL_SEQUENTIAL: temporary automatic numbering nod

Node status (Stat):

Private long czxid

Private long mzxid

Private long ctime

Private long mtime

Private int version

Private int cversion

Private int aversion

Private long ephemeralOwner

Private int dataLength

Private int numChildren

Private long pzxid

Watch event type

ZOO_CREATED_EVENT: a node creation event is required to watch a node that does not exist, which is triggered when a node is created. This watch is set by zoo_exists ()

ZOO_DELETED_EVENT: node deletion event, this watch is set by zoo_exists () or zoo_get ()

ZOO_CHANGED_EVENT: node data change event, this watch is set by zoo_exists () or zoo_get ()

ZOO_CHILD_EVENT: child node list change event, this watch is set by zoo_get_children () or zoo_get_children2 ()

ZOO_SESSION_EVENT: session failure event, triggered when the client is disconnected or reconnected from the server

ZOO_NOTWATCHING_EVENT:watch removes the event, triggered when the server is no longer a client watch node for some reason

1: cluster management:

1 Monitoring server health

They are all implemented by creating an EPHEMERAL type directory node on Zookeeper, and then each Server calls the getChildren (String path, boolean watch) method on the parent directory node where they create the directory node and sets watch to true. Because it is an EPHEMERAL directory node, when the Server that created it dies, the directory node will be deleted, so the Children will be changed, and the Watch on the getChildren will be called So the other Server knew that some Server had died. The same principle applies to the new Server.

2 Master election

How Zookeeper implements Leader Election, that is, selects a Master Server. As before, each Server creates an EPHEMERAL directory node, except that it is also a SEQUENTIAL directory node, so it is an EPHEMERAL_SEQUENTIAL directory node. The reason why it is an EPHEMERAL_SEQUENTIAL directory node, because we can give each Server number, we can choose the current is the lowest number of Server for Master, if this minimum number of Server dies, because it is an EPHEMERAL node, the dead Server corresponding node is also deleted, so the current node list of a minimum number of nodes, we choose this node as the current Master. In this way, the dynamic selection of Master is realized, which avoids the problem that a single Master is prone to single point failure in the traditional sense.

Second: distributed lock

It is easy to implement in the same process, but not easy to implement across processes or between different Server. However, Zookeeper is easy to implement this function, which also requires the Server to acquire the lock to create an EPHEMERAL_SEQUENTIAL directory node, and then call the getChildren method to get whether the smallest directory node in the current directory node list is a self-created directory node. If it is created by itself, then it acquires the lock. If not, it calls the exists (String path, boolean watch) method and monitors the changes in the list of directory nodes on the Zookeeper until the node it creates is the lowest numbered directory node in the list, thus obtaining the lock, releasing the lock is very simple, as long as it deletes the directory node created by itself, n > Server is the same principle.

After reading the above, do you have any further understanding of the principle of the zookeeper mechanism? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.