Shulou(Shulou.com)06/03 Report--

Introduction to 0x01

Zookeeper is a good cluster management tool, which is widely used in distributed computing. Such as Hadoop and Storm systems.

Apache ZooKeeper is an open source coordination service designed for distributed applications, which is designed to reduce the coordination tasks undertaken by distributed applications. Can provide users with synchronization, configuration management, grouping and naming services.

0x02 environment description

Install ZooKeeper on three centos6.5 (64-bit) servers. The official website recommends at least 3 nodes. In this experiment, there are 3.

(number of hosts: 3 (requires 3 hosts and must be odd because of the ZooKeeper election algorithm).

Jdk needs to be installed in advance, and the chosen version is jdk-8u111-linux-x64.

0x03 installation and configuration process

Download ZooKeeper and select the stable version, namely stable

Download address: http://apache.fayea.com/zookeeper/

Decompression

$tar-zxvf zookeeper-3.4.6.tar.gz// moves the file to / usr/cloud/$ mv zookeeper-3.4.6 / usr/cloud/zookeeper

Configure environment variables

# append # set zookeeper environmentexport ZOOKEEPER_HOME=/usr/cloud/zookeeperexport PATH=$PATH:$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf to the last line of vim / etc/profile//

Make the environment variable effective immediately

$source / etc/profile

Cluster deployment

In a Zookeeper cluster environment, Zookeeper services will be available as long as more than half of the machines are started properly. Therefore, it is best to use an odd number of machines to deploy Zookeeper on the cluster, so that if there are five machines, as long as three are working, the service will work properly.

Next, we will set the parameters of Zookeeper's configuration file zoo.cfg:

$cd / usr/cloud/zookeeper/conf$ cp zoo_sample.cfg zoo.cfg$ vim zoo.cfg

Parameters can be found in the following figure

Note the host names of master,slave1 in the configuration above. For the corresponding hosts, please see the previous blog on installation and configuration of Hadoop.

In the configuration file above, the first port in server.id=host:port:port is the port number from the slave machine (follower) to the master machine (leader), and the second port is the port number on which the leadership election is made.

Next, create a file named myid in the directory specified by dataDir. There is only one line in the file, which is the id value of the host, that is, the id in the server.id in the figure above. For example, the contents of the myid in server 1 should be written to 1.

Parameter description

① tickTime: heartbeat time in milliseconds.

② initLimit: this configuration item is used to configure the Zookeeper accept client (here the client is not the client that the user connects to the Zookeeper server, but the Follower server in the Zookeeper server cluster that connects to the Leader) the maximum number of heartbeat intervals that can be tolerated when initializing the connection. When the Zookeeper server has not received a return message from the client after the length of more than 10 heartbeats (that is, tickTime), the client connection failed. The total length of time is 102000,20 seconds.

③ syncLimit: this configuration item identifies the length of time for sending messages, requests and replies between Leader and Follower. The maximum length of time is not more than how many tickTime times. The total length of time is 5, 000, 000, 10 seconds.

④ dataDir: the location where database snapshots are stored in memory.

⑤ clientPort: the port that listens for client connections

⑥ server.A=B:C:D: where An is a number indicating which server this is, B is the ip address of this server, and C represents the port on which the server exchanges information with the Leader servers in the cluster D means that in case the Leader server in the cluster dies, a port is needed for re-election to select a new Leader, and this port is the port through which the servers communicate with each other during the election. If it is a pseudo-cluster configuration, because B is all the same, the communication port numbers of different Zookeeper instances cannot be the same, so they should be assigned different port numbers.

Create a myid file under the dataDir directory and set the content to the A value in the ⑥ to identify different servers.

Copy the installation file to another node

$scp-r / home/hadoop/zookeeper hadoop@slave1:/home/hadoop/$ scp-r / home/hadoop/zookeeper hadoop@slave1:/home/hadoop/

Modify the myid on the corresponding machine

$echo "1" > / home/hadoop/zookeeper/data/myid$ cat / home/hadoop/zookeeper/data/myid0x04 start the ZooKeeper cluster

On each node of the ZooKeeper cluster, execute the script to start the ZooKeeper service, as follows:

Start on each node (here the startup order is master > slave1 > slave2)

Boot sequence:

Hadoop-- > Zookeeper-- > HBase

Stop order

HBase-- > Zookeeper-- > Hadoop

Hadoop@master:/home/hadoop/zookeeper/bin/$ zkServer.sh starthadoop@slave1:/home/hadoop/zookeeper/bin/$ zkServer.sh starthadoop@slave2:/home/hadoop/zookeeper/bin/$ zkServer.sh start

Where QuorumPeerMain is a zookeeper process and starts normally.

After starting Zookeeper on all machines in turn, you can check the startup status through the script of ZooKeeper, including the roles (either Leader or Follower) of each node in the cluster, as shown below, which is the result of the query on each node in the ZooKeeper cluster:

As can be seen from the status query results above, slave1 is the Leader of the cluster, and the remaining two nodes are Follower.

In addition, you can connect to the ZooKeeper cluster through client script. For clients, ZooKeeper is an ensemble, and connecting to a ZooKeeper cluster actually feels like enjoying the services of the entire cluster, so you can establish a connection to the service cluster at any node.

0x05 stops the ZooKeeper process

$zkServer.sh stop

Reference link

Http://blog.csdn.net/cruise_h/article/details/19046357

Https://yq.aliyun.com/articles/36329

Frequently asked questions about 0x06

Error contacting service. It is probably not running.

I looked at 'Error contacting service. It is probably not running.'' on the Internet. There are no more than three answers to class errors:

1, the failure to create the datadir folder in the configuration file zoo.cfg results in

2. The firewall is not closed. It is recommended to permanently disable the firewall-- > chkconfig iptables of

3. Modify the parameter of a nc in the sh script to solve the problem, but the call to nc is not found in your own version. -- > nc belongs to the old version, but the new version does not have nc

However, none of mine is the above problem, my problem is that the myid file is misconfigured.

The file of myid, which contains only one line, is the id value corresponding to the host, that is, the id in the server.id in the image above.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.