Shulou(Shulou.com)06/01 Report--

Big data is dealing with Zookeeper, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain for you in detail, people with this need can come to learn, I hope you can gain something.

Overview of Zookkeeper

Is an open source framework for distributed coordination services

It is essentially a distributed small file storage system that provides data storage based on a directory tree similar to a file system.

Problem solved: application system consistency in distributed clusters

Properties:

Global consistency of data

Reliability: messages received by one server can be received by other servers

Orderability: there are two kinds of global order and partial order, and the order of message release.

Atomicity of data update: one-time data update was successful (more than half of the machines / nodes)

Real-time: get the updated or invalid information of the server within a time range

Zookeeper cluster role

Master node (Leader): 1. The core of cluster work and the only scheduler and processor of transaction requests (writes) to ensure the sequence of cluster transactions. 2. The scheduler of each server within the cluster

Slave node (Follower): 1. Handle client non-transaction requests. 2. Forward the transaction request to the master node. 3. Participate in the election of the cluster master node.

Observer (Observer): 1. Update the latest state changes of the cluster and synchronize these states, respond to non-transactional requests, 2

Forward the transaction request to the master node. 3. Do not participate in the primary node election and vote, which is usually used to improve the ability of non-transaction processing without affecting the transaction processing capacity of the cluster, or when the number of cluster visits is large.

Zookeeper cluster is built.

Zookeeper's data model:

Similar to the standard file system, the nodes in the tree are called Znode, and each node can have child nodes.

Different:

1. Znode has both file and directory characteristics.

2. Znode has atomic operation.

3. There is a limit on the size of the stored data. The maximum is 1m.

4. Znode is referenced by path. Path unique

5. Serialization features. When a node is created, an increasing sequence number is added after the node name, which is unique to the parent node and can be used to identify the order in which the node is created. It is a 10-digit number.

Node composition:

Stat: status information, version describing its Znode, permission information, and so on.

Data: the data associated with this Znode

Children: the child node under the Znode

Node types: temporary and permanent nodes

Temporary nodes: child nodes are not allowed

There are four options when creating a Znode: permanent node, temporary node, permanent serialization node, temporary serialization node

Attributes of the node:

DataVersion: data version number

Cversion: child node version number, value + 1 when the child node changes

AcVersion: version control number

CZxid:Znode creates the ID of the transaction

MZid:Zode modified transaction ID,zk produces a unique transaction ID (zxid) each time it changes, with sequence

Ctim: node creation timestamp

Mtim: the timestamp of the node when it was last updated

EphemeralOwner: if this node is a temporary node, the value is the sessionid bound to the node, if not, the value is 0

Node operation:

Client link: zkCli.sh-sever ip

Create [- s] [- e] path data,s for permanent node, e for temporary node

Set path data

Delete path data

Get path

Quota: add restrictions to nodes. Setquota [- n |-b] val path,n indicates the maximum number of child nodes, b represents the maximum length of data value, maximum number of valu child nodes or maximum length of data value, and path node path

Listquota: check whether the limit is set, that is, the quota of the specified node, and the soft limit (output warning log)

Delquota:delquota [- n |-b] path deletes the quota of the specified node

History: list history command

Redo: re-execute the command, used in conjunction with hostory. Redo command number

Zookeeper Watcher (data subscription / publication mode)

The watcher (monitoring) mechanism is used to realize the distributed notification function. Zookeeper allows the client to register a watcher snooping with the server. When some events on the server trigger the watcher, the server will send an event notification to the client to achieve the distributed notification function.

Process: 1. The client registers a listening watcher,2 with the server, the server triggers the registered watcher,3, and the client callback watcher to get the trigger event.

Characteristics of watcher mechanism:

One-time trigger: after the event triggers listening, a watcher event (listening event) is sent to the corresponding client. This effect only happens once. The same event occurs again later and will not be triggered again.

Event encapsulation: server-side events are encapsulated and delivered using WatcherEvent objects, including three attributes, notification status (keeperState), event type (EventType), and node path (Path)

Event sends asynchronously:

Register first in trigger: connection events do not need to be registered first, others need to be registered in advance

Shell client settings monitoring (Watcher):

Get path watch registration listening event

Zookeeper Java API (Zookeeper operation API)

Connection operation, Watcher event action::

Zookeeper election mechanism

The FastLeaderElection algorithm is adopted by default, and the logic that the vote is greater than half of the winner is adopted.

Server ID: server number. The larger the number, the greater the weight in the algorithm.

Election status: LOOKING-> campaign status, FOLLOWIING--- > follower status (participating in voting), OBSERVING--- > observing status (not participating in voting), LEADDING--- > leader status

Data ID: the version version of the latest data stored in the server. The larger the data, the newer the version, and the greater the weight in the election algorithm.

Logical clock: the number of votes, the same value in the same round of election.

New cluster election (new cluster just built): each machine votes for itself, more than half of the votes are over, and the server ID has a high priority.

Elections that are not entirely new clusters (elections will be held after the cluster provides services for a certain period of time):

Election criteria: 1. Election results with small logic clock are ignored and revoted. 2. Unified logic clock and data ID win. 3. The server with the same data ID ID wins.

Typical Zookeeper applications:

Data publication and subscription (configuration Center):

1. Small data.

2. When you get the data, you also need to set up monitoring. After you get the data, set up monitoring again to ensure that the monitoring is effective at all times.

Naming service: due to the global uniqueness of the Zookeeper node path, the object providing the service is named, and the client can obtain the corresponding resource, service address, provider and other information through the application name. For example, dubbo stores the provided service address on the zk, maps it to the corresponding service name, and maintains the global service address list.

Distributed lock: because of the consistency of its data, the lock service is divided into: maintaining monopoly and controlling timing.

Keep exclusive: singleton lock. Treat the node of zk as a lock, and all clients go to the specified directory to create nodes, which is the same as creating nonserialized temporary nodes to achieve exclusivity.

Control timing: sequential access, such as queues. Similarly, the zk node is treated as a lock, but this time a serialized temporary node is created to achieve global timing.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.