Shulou(Shulou.com)06/03 Report--

What is Zookeeper

Zookeeper is an open source distributed service framework, which is a sub-project of the Apache hadoop project. It is mainly used to solve some problems in distributed application scenarios, such as unified naming service, state synchronization service, cluster management, distributed application configuration management and so on. In the distributed mode, it can provide high performance and reliable coordination services for distributed applications, and the implementation of distributed coordination services can be simplified by using Zookeeper.

The functions provided by Zookeeper include: configuration maintenance, domain name service, distributed synchronization, group service and so on.

For more information, see this blog: http://blog.csdn.net/liuyuehu/article/details/52120546

And http://blog.csdn.net/u012152619/article/details/52908878

2. What is the occasion where Zookeeper is usually used?

Data publication and subscription (configuration Center):

Publish and subscribe is that the publisher publishes the data to the zk node, and the worker subscriber acquires the data dynamically to realize the centralized management and dynamic update of configuration information. For example, the global configuration information, the service address of the service framework.

(1) some configuration information used in the application is put on ZK for centralized management. Usually, the application will take the initiative to obtain the configuration at startup, register a Watcher on the node, and notify the subscribed crying protection in real time every time there is an update in the configuration, so as to achieve the purpose of obtaining the latest configuration information.

(2) in the distributed search server, the cloud information of the index and the node status of the server cluster machine store some designated nodes of ZK for each client subscription.

Load balancing

Load balancing here refers to soft load balancing. In a distributed environment, in order to ensure high availability, the same application or the provider of the same service will usually deploy multiple copies to achieve peer-to-peer services. Consumers need to choose one of these peer-to-peer servers to execute the relevant business logic, which is typical of producer / consumer load balancing in message middleware.

The load balancing of message middleware publishers and subscribers, linkedin open source KafkaMQ and Ali open source metaq all achieve load balancing between producers and consumers through zookeeper. Let's take metaq as an example:

Producer load balancing / consumer load balancing:

When metaq sends a message, the producer must select a partition on a broker to send the message. Therefore, during the operation of metaq, all broker and corresponding partition information will be registered with the designated node of ZK. The default policy is a process of polling in turn. After the producer obtains the partition list through ZK, it will be organized into an ordered partition list in the order of brokerId and partition. When sending, select a partition to send the message in a cycle from beginning to end.

During the consumption process, a consumer will consume messages in one or more partitions, but a partition will only be consumed by one consumer. MetaQ's consumption strategy is:

Each partition mounts only one consumer for the same group.

If the number of consumers in the same group is greater than the number of divisions, the extra consumers will not participate in the consumption.

If the number of consumers in the same group is less than the number of divisions, some consumers need to take on additional consumption tasks.

In the event of a consumer failure or restart, other consumers will perceive the change (through the zookeeper watch consumer list) and then rebalance the load to ensure that consumers are available for consumption in all sectors.

(3) naming Service (Naming Service)

Naming service is also a common scenario in distributed systems. In a distributed system, by using naming services, client applications can obtain information such as the address and provider of resources or services according to the specified name. Named entities can usually be machines in a cluster, service addresses provided, remote objects, and so on-all of which we can collectively refer to as their names (Name). One of the more common is the list of service addresses in some distributed service frameworks. By calling the API provided by ZK to create the node, you can easily create a globally unique path, which can be used as a name.

Zookeeper is used as a naming service in Dubbo to maintain a global list of service addresses.

In Dubbo (next topic):

Upon startup, the service provider writes its URL address to the specified node / dubbo/$ {serviceName} / providers directory on the ZK, which completes the publication of the service.

When the service consumer starts, subscribe to the provider URL address in the / dubbo/$ {serviceName} / providers directory and write its own URL address to the / dubbo/$ {serviceName} / consumers directory.

Addresses registered with ZK are temporary nodes, which ensures that service providers and consumers are automatically aware of changes in resources.

(4) distributed notification / coordination

Zookeeper holds watcher registration and asynchronous notification mechanism, which can well realize the notification and coordination between different systems in the distributed environment, and realize the real-time processing of data changes. The method of use is usually that different systems register the same znode on the ZK and listen for changes in the znode (including the contents of the znode itself and its child nodes). If one system updates the znode, the other system can receive the notification and deal with it accordingly. (http://www.cnblogs.com/tommyli/p/3766189.html)

(5) Cluster management and Master election:

Cluster machine monitoring: it is usually used in scenarios with high requirements for the status of machines in the cluster, and can quickly respond to changes in machines in the cluster.

(6) distributed lock

Distributed lock, which is mainly due to the fact that ZooKeeper ensures strong data consistency for us. Lock services can be divided into two categories, one is to keep exclusive, the other is to control the timing.

(7) distributed queue

In terms of queues, there are two kinds of queues, one is the conventional first-in-first-out queue, and the other is to wait for the queue members to gather before they are executed in order.

The second queue is actually an enhancement based on the FIFO queue. Usually, you can set up a / queue/num node in advance under the znode of / queue, and assign a value of n (or directly assign n to / queue) to indicate the queue size. After that, each time a queue member joins, you can determine whether the queue size has reached the queue size and decide whether execution can begin. A typical scenario for this usage is that in a distributed environment, a large task Task A can only be done when many subtasks are completed (or conditionally ready). At this time, when one of the subtasks is completed (ready), then set up your own temporary timing node (CreateMode.EPHEMERAL_SEQUENTIAL) under / taskList. When / taskList finds that the number of child nodes below it meets the specified number, it can proceed to the next step in sequence processing.

3. Download and install Zookeeper in windows and start:

(1) download and enter Zookeeper in the search box to enter the official website. Download the zip package and extract it as follows:

(2) the bin directory is as follows:

(3) Open the console window and change to the bin directory:

Run zkServer.cmd:

Note that if the startup fails, shuts down unexpectedly, and prompts zoo.conf that it can't find it, then zoo_sample.conf is OK, but you need to modify the file partially and replace it with your own parameters. As shown in the figure:

After Zookeeper starts, the following information is displayed, as shown in the figure:

The last few lines look like this: indicates that the startup was successful

(4) start the client to run and check:

Indicates that the client has successfully connected and the Zookeeper has started successfully!

/ / you can refer to this post: http://blog.csdn.net/morning99/article/details/40426133

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.