Shulou(Shulou.com)06/02 Report--

What is Zookeeper?

ZooKeeper is a distributed, open source distributed application coordination service, an open source implementation of Google's Chubby, and an important component of Hadoop and Hbase.

= > ZooKeeper contains a simple set of primitives that provides an interface between Java and C.

In the version of ZooKeeper code, the interfaces of distributed exclusive lock, election and queue are provided in zookeeper-3.4.3\ src\ recipes. There are two versions of Java and C for distribution lock and queue, and only Java version for election.

= = > Zookeeper application scenarios (features):

= > it is a software that provides consistent services for distributed applications, and provides features such as:

Configuration maintenance, domain name service, distributed synchronization, group service, etc.

The goal of ZooKeeper is to encapsulate complex and error-prone key services and provide users with simple and easy-to-use interfaces and systems with efficient performance and stable functions.

= > Zookeeper architecture

How Zookeeper works:

The core of Zookeeper is atomic broadcast, which ensures synchronization between Server. The protocol that implements this mechanism is called the Zab protocol.

There are two modes of Zab protocol: recovery mode (primary) and broadcast mode (synchronization).

When the service starts or after the leader crashes, the Zab enters the recovery mode, and when the leader is elected and most of the Server finishes synchronizing with the leader, the recovery mode ends.

(★) State synchronization ensures that leader and Server have the same system state.

In order to ensure the order consistency of transactions, zookeeper uses an incremental transaction id number (zxid) to identify transactions. All proposals (proposal) are made with zxid when they are made.

In the implementation, zxid is a 64-bit number, and its high 32-bit epoch is used to identify whether the leader relationship has changed. Each time a leader is selected, it will have a new epoch that identifies the current period of leader rule.

The lower 32 bits are used to increment the count.

(★) for clients, ZooKeeper is an ensemble. Connecting to a ZooKeeper cluster actually feels like enjoying the services of the entire cluster. So, you can establish a connection to the service cluster on any node.

= = > Zookeeper installation

= > build a cluster environment for ZooKeeper

1. Three: bigdata12 bigdata13 bigdata14

Time synchronization

2. Install on bigdata12

(*) decompress tar-zxvf zookeeper-3.4.10.tar.gz-C ~ / training/

(*) set environment variables: on each machine

ZOOKEEPER_HOME=/root/training/zookeeper-3.4.10

Export ZOOKEEPER_HOME

PATH=$ZOOKEEPER_HOME/bin:$PATH

Export PATH

(*) Core configuration file: conf/zoo.cfg

Mkdir / root/training/zookeeper-3.4.10/tmp

Edit zoo.cfg

DataDir=/root/training/zookeeper-3.4.10/tmp

Server.1=bigdata12:2888:3888

Server.2=bigdata13:2888:3888

Server.3=bigdata14:2888:3888

Create a file in the / root/training/zookeeper-3.4.10/tmp directory: myid

Enter 1

3. Copy the ZK of bigdata12 to other nodes

Scp-r zookeeper-3.4.10/ root@bigdata13:/root/training

Scp-r zookeeper-3.4.10/ root@bigdata14:/root/training

4. Modify the myid file on bigdata13 and bigdata14

5. On each machine, start ZK: zkServer.sh start

= > configuration file

= > under the zookeeper-3.3.3/conf folder (you can copy a copy of the default zoosample.cfg to zoo.cfg). The configuration file zoo.cfg needs to be written on each server. The following is a sample configuration file:

# Filename zoo.cfg

TickTime=2000

DataDir=/home/zookeeper/data

ClientPort=2181

InitLimit=5

SyncLimit=2

Server.1=202.115.36.251:2888:3888

Server.2=202.115.36.241:2888:3888

Server.3=202.115.36.242:2888:3888

InitLimit: this configuration item is used to configure the Zookeeper accept client (the client here is not the client that the user connects to the Zookeeper server, but the Follower server in the Zookeeper server cluster that connects to the Leader) the maximum number of heartbeat intervals that can be tolerated when initializing the connection. When the time of more than 10 heartbeats (that is, tickTime) is over, the Zookeeper server has not received the return message from the client.

Then it indicates that the client connection failed. The total length of time is 5 "2000" 10 seconds.

SyncLimit: this configuration item identifies the length of time for sending messages, requests and replies between Leader and Follower. The maximum length of time cannot exceed the number of tickTime. The total time length is 2 "2000" 4 seconds.

Server.A=B:C:D

An is a number indicating which server this is.

B is the ip address of this server

C represents the port on which this server exchanges information with the Leader servers in the cluster

D means that in case the Leader server in the cluster dies, a port is needed for re-election to select a new Leader, and this port is the port through which the servers communicate with each other during the election.

If it is a pseudo-cluster configuration, because B is all the same, the communication port numbers of different Zookeeper instances cannot be the same, so they should be assigned different port numbers.

= > create a myid file

In addition to modifying the zoo.cfg configuration file, a file myid is also configured in cluster mode. This file is in the dataDir directory, and there is only one data in this file, that is, the value of A, which will be read when Zookeeper starts.

To determine which server it is, compare the data in it with the configuration information in zoo.cfg. The value of this myid should be the same as the x in server.x=ip:port:port in the configuration file.

Echo 1 > myid

= = > Zookeeper operation

= > View the status of zookeeper: zkServer.sh status

= > connect through the zookeeper client:

ZkCli.sh-server 192.168.10.221purl 2181

= > View the znode node:

Ls /

= > create a znode node (data):

Create / test "hello"

= > get znode node data:

Get / test

= > modify znode node data:

Set / test "word"

= > Delete znode node:

Delete / test

= = > clear zk statu

When a lot of things have been done on zk and the configuration is messy, now you want to reinitialize it, that is, to delete all states. The steps are as follows:

= > close all zk nodes and use zkServer.sh status to see if they are closed

= > Delete all files under the data folder defined by zoo.cfg, except myid files

= > start the zk node

Now the configuration is reset.

= = > FAQ

When starting the client with zkCli.sh, Will not attempt to authenticate using SASL (unable to locate login configuration)

This is due to the failure to enable user authentication, which does not affect usage, but affects security. SASL is: Simple Authentication and Security Layer

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.