Shulou(Shulou.com)06/01 Report--

This article shows you how to do Zookeeper analysis and cluster building, the content is concise and easy to understand, absolutely can make you shine, through the detailed introduction of this article I hope you can gain something.

About Zookeeper

Zookeeper is an efficient distributed coordination service that can provide configuration information management, naming, distributed synchronization, cluster management, database switching and other services. It is not suitable for storing large amounts of information, but can be used to store small amounts of information such as configuration, publications and subscriptions. Hadoop, Storm, messaging middleware, RPC service framework, distributed database synchronization system, these are Zookeeper application scenarios.

The number of nodes in a Zookeeper cluster is generally odd (>=3). If the Master in the cluster is suspended and the number of remaining nodes is more than half, a new master node can be elected to continue providing services to the outside world.

Here I still want to recommend my own big data learning group:199427210, the group is learning big data development, if you are learning big data, Xiaobian welcomes you to join, everyone is a software development party, from time to time to share dry goods (only big data software development related), including a 2018 latest big data advanced materials and advanced development tutorials compiled by myself, welcome advanced and want to go deep into big data small partners to join.

The client initiates a transaction request, and the result of the transaction request is applied consistently across all machines in the Zookeeper cluster. There is no case where some machines in the cluster apply the transaction and some machines in the cluster do not. Any machine in the Zookeeper cluster sees a consistent data model for all servers. Zookeeper can guarantee the order of client requests, and each request is assigned a globally unique incremental number to reflect the order of transaction operations. Zookeeper keeps the full amount of data in memory and directly serves all non-transactional requests, which is very outstanding in read-based scenarios.

Zookeeper uses a tree structure with root node "/." The nodes in Zookeeper cluster are divided into leader, follower and observer according to their identity characteristics. Leader is responsible for client-side writer type requests;follower is responsible for client-side reader type requests and participates in leader election;observer is a special follower that can receive client-side reader requests, but will not participate in elections, and can be used to expand system support capabilities and improve reading speed.

Zookeeper is a distributed service management framework designed based on the observer pattern, responsible for storing and managing relevant data and receiving registration of observers. Once the state of this data changes, zookeeper is responsible for notifying observers who have registered with zookeeper cluster and are concerned about these state changes, so that observers can perform relevant actions.

Zookeeper uses the ZAB atomic message broadcast protocol, and the consistency algorithm between nodes is Paxos, which can guarantee the consistency of data in distributed environments. High availability in distributed scenarios is a feature of Zookeeper, which can be implemented by third-party clients, namely Curator framework.

Paxos solves the problem of how a distributed system can agree on a value (resolution). A typical scenario is that in a distributed database system, if the initial state of each node is consistent, each node performs the same sequence of operations, and they eventually arrive at a consistent state. To ensure that each node executes the same sequence of commands, a "consistency algorithm" needs to be executed on each instruction to ensure that each node sees the same instruction. A general consistency algorithm can be applied in many scenarios and is an important problem in distributed computing. Therefore, since the 1980s, research on consistency algorithms has not stopped. There are two models of node communication: shared memory and messages passing. Paxos algorithm is a consistency algorithm based on message passing model.

Paxos algorithm can be used not only in distributed systems, but also in any situation where multiple processes need to reach some agreement. Consistency algorithms can be implemented by shared memory (requiring locks) or message passing, which is the case with Paxos. Paxos algorithm applies to several situations: multiple processes/threads in a machine to achieve data consistency; distributed file systems or distributed databases in multiple clients read and write data concurrently; distributed storage in multiple copies of the consistency of response to read and write requests.

Zookeeper cluster building

In this article, the number of Zookeeper nodes (odd) is 3. Zookeeper defaults to port 2181 for external services. By default, communication between the three nodes in the Zookeeper cluster uses 2888:3888

192.168.0.217 192.168.0.218 192.168.0.219

Download the tar package for zookeeper

Upload tar packages to 192.168.0.217 192.168.0.218 192.168.0.219

Execute downlink instructions on zookeeper nodes 217, 218, and 219 respectively, and then modify the decompressed file name zookeeper-3.4.10 to zookeeper.

tar -zxvf zookeeper-3.4.10.tar.gz -C /usr/local/

Configure environment variables in three zookeeper nodes

vim /etc/profile

Add export ZOOKEEPER_HOME=/usr/local/zookeeper

Add $ZOOKEEPER_HOME/bin to path

Execute source /etc/profile to make environment variables take effect immediately

Change zoo_sample.cfg to zoo.cfg under the conf directory in zookeeper

Modify zoo.cfg file in three zookeeper nodes, modify dataDir, add server.0, server.1, server.2

Under the zookeeper directory, create the data directory. Create myid files under the data directory in the three zookeeper nodes and add contents 0, 1 and 2 respectively.

Start zookeeper

zkServer.sh start

zkServer.sh status

At any node in zookeeper, execute zkCli.sh

Execute the command zkCli.sh help to view help information

ls /Find root directory

create /test abc Create node and assign values

get /test Gets the value of the specified node

set /test cb Set the value of an existing node

rmr /test Recursively delete nodes

delete /test/test01 Delete a node that does not have children

View Zookeeper cluster nodes in Eclipse

You can also use ZooInspector.

At this point, execute zkCli.sh in Xshell to see the contents of the tree structure in Zookeeper cluster

You can add and delete Zookeeper cluster nodes in Eclipse and ZooInspector.

That's how Zookeeper analysis and clustering work. Have you learned anything? If you want to learn more skills or enrich your knowledge reserves, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.