Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to install and configure ZooKeeper cluster. It is very detailed and has a certain reference value. Friends who are interested must finish reading it.

one。 ZooKeeper introduction

ZooKeeper is a distributed, open source distributed application coordination service, an open source implementation of Google's Chubby, and an important component of Hadoop and Hbase. It is a software that provides consistency services for distributed applications, including configuration maintenance, naming services, distributed synchronization, group services, etc.

two。 Usage scenarios of ZooKeeper

Data publish and subscribe (configuration Center)

Publish and subscribe model, the so-called configuration center, as the name implies, is that publishers publish data to ZK nodes for subscribers to obtain data dynamically, and achieve centralized management and dynamic update of configuration information. For example, the global configuration information, the service address list of the service framework, etc., are very suitable for use.

Note: in the above-mentioned application scenarios, there is a default premise: scenarios where the amount of data is small, but the data may be updated quickly.

Load balancing

Load balancing here refers to soft load balancing. In a distributed environment, in order to ensure high availability, the same application or provider of the same service will usually deploy multiple copies to achieve peer-to-peer services. Consumers have to choose one of these peer-to-peer servers to execute the relevant business logic, among which the typical one is the producer and consumer load balance in message middleware.

In the event of a consumer failure or restart, other consumers will perceive the change (through the zookeeper watch consumer list) and then rebalance the load to ensure that consumers are available for consumption in all sectors.

Naming Service (Naming Service)

Naming service is also a common scenario in distributed systems. In a distributed system, by using naming services, client applications can obtain information such as the address and provider of resources or services according to the specified name. Named entities can usually be machines in a cluster, service addresses provided, remote objects, and so on-all of which we can collectively call names (Name). One of the more common is the list of service addresses in some distributed service frameworks. By calling the API provided by ZK to create the node, you can easily create a globally unique path, which can be used as a name.

Note that all addresses registered with ZK are temporary nodes, which ensures that service providers and consumers are automatically aware of changes in resources.

Distributed notification / coordination

ZooKeeper has a unique watcher registration and asynchronous notification mechanism, which can well realize the notification and coordination between different systems in the distributed environment, and realize the real-time processing of data changes. The method of use is usually that different systems register the same znode on the ZK and listen for changes in the znode (including the contents of the znode itself and its child nodes). If one system update the znode, the other system can receive the notification and deal with it accordingly.

1)。 Another heartbeat detection mechanism: the detection system and the detected system are not directly related, but through the association of a node on the zk, the system coupling is greatly reduced.

2)。 Another system scheduling mode: a system consists of a console and a push system, and the duty of the console is to control the push system to carry out the corresponding push work. Some of the operations that managers do in the console actually modify the status of some nodes on the ZK, and ZK notifies them of these changes to the client that registers Watcher, that is, the push system, and then makes the corresponding push task.

3)。 Another work reporting mode: some similar to the task distribution system, after the sub-task starts, register a temporary node to zk, and report their progress regularly (write the progress back to this temporary node), so that the task manager can know the progress of the task in real time.

In short, using zookeeper for distributed notification and coordination can greatly reduce the coupling between systems.

Cluster Management and Master Election

Master election is the most classic application scenario in zookeeper.

Distributed lock

Distributed lock, which is mainly due to the fact that ZooKeeper ensures strong data consistency for us. Lock services can be divided into two categories, one is to keep exclusive, the other is to control the timing.

Distributed queue

In terms of queues, there are two kinds of queues, one is the conventional first-in-first-out queue, and the other is to wait for the queue members to gather before they are executed in order. For the first FIFO queue, which is consistent with the basic principle of the control timing scenario in the distributed lock service, I will not repeat it here. The second queue is actually an enhancement based on the FIFO queue. Usually, you can set up a / queue/num node in advance under the znode of / queue, and assign a value of n (or directly assign n to / queue) to indicate the queue size. After that, each time a queue member joins, you can determine whether the queue size has reached the queue size and decide whether execution can begin. A typical scenario for this usage is that in a distributed environment, a large task Task A can only be done when many subtasks are completed (or conditionally ready). At this time, when one of the subtasks is completed (ready), then set up your own temporary timing node (CreateMode.EPHEMERAL_SEQUENTIAL) under / taskList. When / taskList finds that the number of child nodes below it meets the specified number, it can proceed to the next step in sequence processing.

three。 Cluster installation and deployment of ZooKeeper

* * the number of nodes in the ZK cluster should be odd (2n+1: for example, 3, 5, 7 nodes). * * Server 1RV 192.168.1.81 Port: 2181, 2881, 3881 Server 2RV 192.168.1.82 Port: 2182, 2882, 3882 Server 3RU 192.168.1.83 Port: 2183, 2883, 38831. Modify / etc/hosts192.168.1.81 zk-01192.168.1.82 zk-02192.168.1.83 zk-03 2. Extract zookeeper and create the data,logs directory in the installation directory. 3. Zoo_sample.cfg cp zoo_sample.cfg zoo.cfg4 under conf/ in the copy installation file. Configure the zoo.cfg for each node, for example: # heartbeat time tickTime=2000 # Follower server connected to Leader initLimit=10 syncLimit=5 dataDir=/$ {ZK_HOME} / node-01/data dataLogDir=/$ {ZK_HOME} / node-01/logs clientPort=2181 server.1=zk-01:2881:3881 server.2=zk-02:2882:3882 server.3=zk-03:2883:38835 in the Zookeeper server cluster. Creating the myid file under dataDir=/$ {ZK_HOME} / data is the value of x in the configuration server.x 6. Filter exposed port 7 in iptables. Start / ${ZK_HOME} / bin/zkServer.sh start8. Listen to tail-500f / ${ZK_HOME} / bin/zookeeper.out above is all the content of this article "how to install and configure ZooKeeper clusters". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.