Shulou(Shulou.com)06/02 Report--

This article shows you what the basic knowledge of Zookeeper is, it is concise and easy to understand, and it will definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

Brief introduction

Apache ZooKeeper is a distributed, open source distributed application coordination service, which is composed of Client and Server. Server provides consistent replication and storage services. Client contains a simple set of primitives, based on which distributed applications can implement synchronization services, configuration maintenance and naming services. ZooKeeper is designed to be very easy to program. ZooKeeper maintains a hierarchal (hierarchical) namespace, which uses a tree-shaped data structure, similar to a standard file system. Because it is very difficult to implement a distributed collaboration service from scratch. The most common problems are competitive conditions and deadlocks. The goal of Apache ZooKeeper is to encapsulate complex and error-prone key services and provide users with simple and easy-to-use interfaces and systems with efficient performance and stable functions.

Zookeepr's data is stored in memory (updated data is also persisted to disk), so its throughput will be very high and latency will be very low. The implementation of ZooKeeper pays more attention to high performance (high performance), highly available (high availability) and strictly ordered access (strict orderly access). The performance of Zookeeper allows it to be used in large distributed systems, high availability avoids a single point of failure, and strictly orderly access allows Client to implement complex synchronization primitives.

Zookeeper system model

Which of the Server shown above make up the Zookeeper service, and each Server is aware of each other's existence. These server are mirrored in memory and persisted to the hard disk through transaction logs and snapshots. As long as most of the Server in the cluster is accessible, the ZooKeeper service is available.

The Clients connects to a ZooKeeper Server. Client and Server maintain a TCP persistent connection, through which Client can send requests, get response, get watch event, and send heartbeats (clients and servers keep connected through heartbeats, that is, session). If the TCP long connection to the Server is broken, then the Client will connect to another Server.

Zookeeper is ordered: Zookeeper uses stamps (numbers) as the order of all transactions.

Zookeeper is very fast: especially in read-oriented situations, Zookeeper applications can run on thousands of machines and perform best when the read-write ratio is 10:1.

Zookeeper data model

The structure of the ZooKeeper data model is very similar to that of the Unix file system, which can be regarded as a tree as a whole, and each node is called a ZNode. A small amount of data can be stored on each ZNode (default is 1m, which can be modified through configuration, and it is usually not recommended to store a large amount of data on ZNode). Here are some important concepts of Zookeeper:

One, Znode

The 1st Zookeeper node is called Znode, and each node has a unique path tag. Znode is classified into two categories: 1, ordinary Znode;2,ephemeral (temporary) Znode

2Jing Znode maintains the stat structure, which contains data, the version number of ACL changes, and a timestamp to allow cache validation and coordination of updates. When the data of znode changes, the version number increases

3Jing Znode can have child nodes, and Znode can store data, but ephemeral (temporary) Znode cannot have child nodes.

4Jing Znode data can have multiple versions, and the client can obtain the data of the node according to the version.

5. If the client and server that created the ephemeral (temporary) Znode lose their connection, the 00:00 node is also automatically deleted.

6Znode can be numbered automatically.

7 watch can be added to the Znode, which is used to monitor whether the data stored by the node has been modified, the changes in the child node directory, etc., and notify the client that sets the monitoring once the change occurs.

8 Znode Znode's read and write operations are atomic, and each Znode has an access control list (ACL) controlling who can do what.

Second, Session

For communication between Client and ZooKeeper, you need to create a Session, and this Session will have a timeout. Because the ZooKeeper cluster persists the Session information of Client, the connection between Client and ZooKeeper Server can be moved transparently between ZooKeeper Server before the Session times out.

In practical applications, if the communication between Client and Server is frequent enough, the maintenance of Session does not need other additional messages. Otherwise, ZooKeeper Client will send a heartbeat to Server every 3 ms, and if Client 2T/3 ms does not receive a heartbeat response from Server, it will switch to a new ZooKeeper Server. Where T is the timeout of the user-configured Session.

Three, Watcher

ZooKeeper supports a Watch operation, and Client can set a Watcher on a ZNode to Watch changes on that ZNode. If there is a corresponding change on the ZNode, the Watcher is triggered and the corresponding event is notified to the Client that sets the Watcher. It should be noted that the Watcher in ZooKeeper is one-time, that is, the trigger will be cancelled once, and if you want to continue Watch, you need to reset the Watcher on the client side.

Fourth, transaction logs and snapshots

The dataDir directory specifies the data directory of Zookeeper, which is used to store snapshot files (snapshot) of Zookeeper.

DataLogDir defines the transaction log directory of Zookeeper, which stores the transaction log of Zookeeper. Normally, before all update operations are returned to the client, Zookeeper must have written this update operation to the transaction log (that is, disk). The filename of the transaction log is log.,zxid, which is the first transaction id written to this file. After several transactions are completed, there is a data snapshot that dump the status of all nodes on the current Server to disk in the form of a snapshot file, that is, the snapshot file.

Zookeeper role

Zookeepr roles can be divided into four categories:

Role

Description

Leader

The leader is responsible for initiating and deciding the vote and updating the status of the system.

Follower

1MagneFollower is responsible for receiving Client requests and returning results to the client.

2. Participate in voting in the process of electing Leader

Observer

ObServer can receive client connections and forward write requests to Leader nodes, but ObServer does not participate in voting and elections, only receiving voting and election results. Its function is mainly used to expand the system and improve the reading speed. ObServer is a new role for zookeeper-3.3.0.

Client

Request initiator

The status of Server can be divided into the following four categories:

Leading: the current Server is Leader.

Following: the current Server is Follower.

Observing: the current Server is Observer.

Looking: currently Server doesn't know who leader is and is searching for it.

Zookeeper characteristics

Order consistency: updates the data in the order in which the client sends the request. Zookeeper does not belong to strong consistency, because watcher cannot capture every change.

Atomicity: updates either succeed or fail, and there are no partial updates.

Single system image: no matter which server the client connects to, you will see the same view.

Reliability: with simple, robust, good performance, if the message is accepted by one server, then it will be accepted by all servers.

Timeliness: Zookeeper guarantees that the client will get updated data from the server, or information about server failure, within a time interval. However, due to network delay and other reasons, Zookeeper can not guarantee that two clients can get the newly updated data at the same time. If the latest data is needed, the sync () interface should be called before reading the data.

ZooKeeper principle

The core of Zookeeper is atomic broadcast, which ensures the synchronization of data among Server through Zab protocol. There are two modes of Zab protocol, namely, recovery mode (election Leader) and broadcast mode (synchronization). After the service starts or the Leader crashes, the Zab enters the recovery mode, and when the Leader is elected and most of the Server finishes synchronizing the state with the leader, the recovery mode ends. State synchronization ensures that Leader and Server have the same system state.

In order to ensure the order consistency of transactions, Zookeeper uses an incremental transaction id number (zxid) to identify transactions. All proposals (proposal) are made with zxid when they are made. In the implementation, zxid is a 64-bit number, and its high 32-bit epoch is used to identify whether the leader relationship has changed. Each time a leader is selected, it will have a new epoch that identifies the current period of leader rule. The lower 32 bits are used to increment the count.

Each Server has four states during its operation:

LOOKING: currently Server doesn't know who leader is and is searching for it.

LEADING: the current Server is the elected leader.

FOLLOWING:leader has been elected and the current Server is synchronized with it.

OBSERVING: do not elect, only synchronize status from Leader.

Zookeeper API

One of the design goals of Zookeeper is to provide a simple programming interface that only supports the following operations:

Create: create a Znode. Path is its path, data is the data to be stored on the Znode, and createMode includes: PERSISTEN,PERSISTENT_SEQUENTAIL,EPHEMERAL,EPHEMERAL_SEQUENTAIL.

Delete: delete a Znode. You can delete a specified version of Znode, or delete all versions if version is set to-1.

Exists: determine whether Znode exists and set whether to Watch the Znode.

Get data: reads the data on the specified Znode and sets whether to watch the Znode.

Set data: updates the data of the specified Znode and sets whether to Watch the Znode.

Get children: updates the data of the specified ZNode and sets whether to Watch the Znode.

Sync: synchronize all previous updates to sync.

Set acl: sets the Acl information for the specified ZNode

Get acl: gets the Acl information of the specified ZNode

Other

Read, write (update) mode:

In Zookeeper cluster, clients can read from any ZooKeeper server, which ensures that ZooKeeper has better read performance.

The written request will first Forwarder to Leader, and then Leader will broadcast the request to all Follower,Leader through the atomic broadcast protocol in ZooKeeper. After receiving more than half of the successful Ack, it will persist the write and tell the client that the write is successful.

WAL (Write-Ahead-Log) and Snapshot:

ZooKeeper also has WAL. For each update operation, ZooKeeper will first write WAL, then update the data in memory, and finally notify Client of the update result.

ZooKeeper also periodically Snapshot the directory tree in memory and land it on disk. In fact, it is similar to fsimage and edits log in HDFS. The main purpose of this is, of course, to persist the data, and to speed up the recovery after restart, which will be slow if it is all recovered in the form of Replay WAL.

FIFO:

For every ZooKeeper client, all operations follow the FIFO order, which is guaranteed by the following two basic features:

First, the network communication between ZooKeeper Client and Server is based on TCP,TCP, which ensures the order of packet transmission between Client/Server.

Second, ZooKeeper Server executes client requests in strict FIFO order.

Linearization:

ZooKeeper includes global order and partial order:

Global ordering is for the server side. For example, if message An is published before message B on a server, then message An on all servers will be published before message B.

The partial order is for the client. For example, if message B is sent on the same client and released after message A, the order of execution must be message A first and then message B.

All update operations have a strict partial order relationship, and update operations are carried out serially, which is the key to ensure the correctness of ZooKeeper functions.

Working with scen

1. Data publishing and subscription

2, name space service

3, distributed notification / coordination

4, distributed lock

5. Cluster management

The above is what is the basic knowledge of Zookeeper. Have you learned any knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.