Shulou(Shulou.com)06/01 Report--

This article mainly explains "what features Zookeeper has". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's take you to learn "what features Zookeeper has"!

About Zookeeper

ZooKeeper is dedicated to providing a high-performance, high-availability, and distributed coordination service with strict sequential access control capabilities. It is created by Yahoo, an open source implementation of Google Chubby, and an important component of Hadoop and Hbase.

design goal

Simple data structure: shared tree structure, similar to file system, stored in memory;

Cluster can be built: avoid single point failure, 3-5 machines can form a cluster, more than half of the normal work can provide services to the outside;

Sequential access: For each read request, zk assigns a globally unique incremental number, which can be used to implement advanced coordination services;

High performance: Based on memory operations, it serves non-transactional requests and is suitable for business scenarios dominated by read operations. 3 zk clusters can reach 13w QPS;

What are the common uses of ZK?

Data Publishing Subscriptions: The following basics Load Balancer: Client-side Load Balancer Naming Service Master Election Cluster Management Configuration Management Distributed Queue Distributed Lock

ZK download and installation

Baidu can

Zookeeper Profile

ZK characteristic

Conversation (session)

A session connection between the client and the server is essentially a TCP long connection. Heartbeat detection and data transmission can be performed through the session. After the successful connection between the ZK client and the server, a session is created. The ZK session switches between different session states throughout its life cycle. These states include: CONNECTING, CONNECTED, RECONNECTING, RECONNECTED, CLOSE Once the client starts creating Zookeeper objects, the client state changes to CONNECTING, and the client starts trying to connect to the server. After the connection is successful, the client state changes to CONNECTED. Usually, due to network disconnection or other reasons, There will be a disconnection between the client and the server. Once this situation is encountered, Zookeeper client will automatically reconnect with the service, and the client state will change to CONNECTING again. Until the server is reconnected, the state will change to CONNECTED again. Under normal circumstances, the client state will always be between CONNECTING and CONNECTED. However, if there is a session timeout, permission check, or the client actively exits the program, the client state will change directly to CLOSE state.

data model

ZooKeeper's view structure is similar to the standard Unix file system, where each node is called a data node or ZNode, and each znode can store data and mount child nodes, so it can be called a tree. Znode typically stores less than 1M of data. Each znode must have a value, and without a value, the node cannot be created successfully.

In Zookeeper, a znode is a node similar to a Unix file system path to which data can be stored or retrieved.

The client can add, delete and check znodes, and register watcher to monitor znode changes.

Zookeeper Node Type

Node type is very important and is the basis for actual combat in the following projects. There are two types of Znode: short （ephemeral）(create -e /app1/test1 "test1" client disconnect zk delete ephemeral type node) persistent （persistent）(create -s /app1/test2 "test2" client disconnected zk does not delete persistent type node) b. Znode There are four forms of directory nodes (default is persistent) PERSISTENT PERSISTENT_SEQUENTIAL (Persistent Sequence/test 00000019) EPHEMERAL EPHEMERAL_SEQUENTIAL c. Set the sequence identifier when creating znode. A value will be appended to the znode name. The sequence number is a monotonically increasing counter maintained by the parent node d. In distributed systems, the sequence number can be used to sort all events globally. This allows the client to infer the sequence of events from the sequence number

Common commands for clients

Use zkCli.sh-server 127.0.0.1:2181 to connect to ZooKeeper service. After successful connection, the system will output ZooKeeper related environment and configuration information. Some simple operations of command-line tools are as follows:

Show root directory, file: ls /Use ls command to view the contents of current ZooKeeper

Display root directory, file: ls2 /View current node data and update times and other data

Create a file and set the initial content: create /zk "test" Create a new znode node " zk " and its associated string [-e] [-s][-e zero node][-s sequential node]

Get file content: get /zk Confirm whether znode contains the string we created [watch][watch listening]

Modify file content: set /zk "zkbak" Set the string associated with zk

Delete file: delete /zk Delete the znode created just now, if there are child nodes, delete failed

Recursive deletion: rmr /zk deletes the znode created just now, and deletes the child nodes at the same time

Exit client: quit

Help Command: Help

ACL secures data

ACL mechanism, represented as scheme:id:permissions, the first field indicates which mechanism is used, the second id indicates the user, and permissions indicates the relevant permissions (such as read-only, read-write, management, etc.).

It is usually represented as: scheme:id:permission

schema: Represents authorization policy

id: represents the user

permission: Delegate permission

Scheme world: default mode, equivalent to universal access auth: for authenticated users (you can add authorized users by addauth digest user:pwd) digest: username: password authentication, which is also the most commonly used ip address authentication in business systems

id id is the authentication mode, different schemes, id values are also different. When scheme is auth: username:password When scheme is digest:username:BASE64(SHA1(password)) When scheme is ip: IP address of client. Scheme is anyone when world.

Permission CREATE, READ, WRITE, Deleted, ADMIN, namely add, delete, change, check, administrative permissions, these five permissions abbreviated as crwda(that is: the initial character abbreviation of each word) CREATE(c): the right to create child nodes Delete(d): the right to delete nodes READ(r): the right to read node data WRITE(w): the right to modify node data ADMIN(a): the right to set child node permissions

ACL command

getAcl

Get ACL information of a specified node create /testDir/testAcl deer #Create a child node getAcl /testDir/testAcl #Get acl permission information of this node

setAcl

Set ACL information for a specified node setAcl /testDir/testAcl world:anyone:crwa #Set acl permissions for this node getAcl /testDir/testAcl #Acquire acl permission information for this node. After success, this node will lose d permissions create /testDir/testAcl/xyz xyz-data #Create child node delete /testDir/testAcl/xyz #Because there is no d permission, prompt cannot delete

addauth

addauth digest user1:123456 #You need to add a user setAcl /testDir/testAcl auth:user1:123456:crwa #before you can use this user to set permissions getAcl /testDir/testAcl #Passwords are stored in ciphertext

create /testDir/testAcl/testa aaa

delete /testDir/testAcl/testa #Unable to delete due to lack of d permission

After exiting the client: ls /testDir/testAcl #No permission to access create /testDir/testAcl/testb bbb #No permission to access addauth digest user1:123456 #Add permission to access

ACL Permission Supplement

When practicing setAcl permission, due to a mistake, the node cannot be deleted create /enjoy1/node1 enjoy setAcl /enjoy1 world:anyone:r At this time, neither delete nor rmr has permission to delete. Solution: Enable super permission. Use DigestAuthenticationProvider.generateDigest("super:admin"); get password 1. Modify zkServer startup script to add "-Dzookeeper.DigestAuthenticationProvider.superDigest=super:xQJmxLMiHGwaqBvst5y6rkB6HQs="2. Start client and log in as administrator addauth digest super:admin Common four-word commands

ZooKeeper supports interaction with certain four-character command letters. Used to obtain the current status and related information of ZooKeeper service. You can submit commands to ZooKeeper via telnet or nc: provided nc is installed, of course.

echo stat| nc 127.0.0.1 2181 to see which node is selected as a follower or leader using echo ruok| nc 127.0.0.1 2181 Tests whether the Server is started, if the reply imok indicates that it has been started. echo dump| nc 127.0.0.1 2181 , lists unprocessed sessions and temporary nodes. echo kill |nc 127.0.0.1 2181 , Turn off server echo conf| nc 127.0.0.1 2181 , outputs the details of the relevant service configuration. echo cons |nc 127.0.0.1 2181 , lists full connection/session details of all clients connected to the server echo envi| nc 127.0.0.1 2181 , outputs detailed information about the service environment (distinct from the conf command). echo reqs |nc 127.0.0.1 2181 , lists unprocessed requests. echo wchs |nc 127.0.0.1 2181 , lists details of server watch. echo wchc |nc 127.0.0.1, Lists the details of the server watch by session, and its output is a list of sessions related to watch. echo wchp |nc 127.0.0.1 2181 , lists details of server watch by path. It outputs a path associated with the session.

ZooKeeper Log Visualization

Two very important configurations have been discussed earlier: dataDir, which stores snapshot data, and dataLogDir, which stores transaction log files.

LogFormatter java -cp../../ zookeeper-3.4.6.jar;../../ lib/slf4j-api-1.6.1.jar org.apache.zookeeper.server.LogFormatter log.xxxx Data Snapshot Visualization (SnapshotFormatter) java -cp ././ zookeeper-3.4.6.jar;../../ lib/slf4j-api-1.6.1.jar org.apache.zookeeper.server.SnapshotFormatter snapshot.xxxx

ZooInspector

Download and unzip it into ZooInspector\build and run zookeeper-dev-ZooInspector.jar;

java -jar zookeeper-dev-ZooInspector. jar//After successful execution, java ui client will pop up. Enter zk service address: ip or host name:2181

At this point, I believe everyone has a deeper understanding of "what features Zookeeper has," so let's actually operate it! Here is the website, more related content can enter the relevant channels for inquiry, pay attention to us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.