Shulou(Shulou.com)06/03 Report--

Installation environment:

CentOS 7 memory 1GB

JDK version: 1.8.0,112

Configure the following environment variables for JDK:

Edit / etc/profile.d/jdk.sh

#! / bin/bashJAVA_HOME=/usr/local/jdk1.8.0_112export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

Then run the following command:

Note: in order to avoid frequent memory swapping in and out of production environments, it is recommended that you set the JAVA heap size a little larger, depending on your physical memory size.

About cluster availability: if you can tolerate N ZK downtime, then your cluster needs to have 2N+1 ZK servers. Three clusters allow one to fail, and five clusters allow two to fail. The number of ZK in the cluster should be kept odd or even, except that the number of ZK made up of 3 is the same as that of 4 that allows failure.

Stand-alone installation

Download the stable version of zookeeper. I'm using 3.4.11 here. I unzipped it under / usr/local. In fact, any path is fine.

Configure environment variables for ZK, edit / etc/profile.d/zk.sh file

#! / bin/bashZOOKEEPER_HOME=/usr/local/zookeeper-3.4.11export PATH=$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf:$PATH

Execute the following command

Prepare the configuration file

Create a zoo.cfg configuration file in the conf directory in the program directory. Zoo_sample.cfg is a template file. Copy and modify the name directly, and then modify the contents inside.

This configuration file can set many parameters, and only the most basic ones are set by default. We'll talk about other parameters later.

Parameter describes the tickTime basic event unit (in milliseconds). Used to set the heartbeat, the minimum session expires twice as long as tickTime, and the rest of the time in ZK is multiplied by this. DataDir

The location where the snapshot of the database in memory is stored is / tmp/zookeeper by default. This is just an example. We changed it to data in the ZK home directory, but this data does not exist either. We need to build it manually. Snapshot files are not real-time and will only be available after a period of time.

DataLogDir

The log path, which is the transaction log. We know that reading and writing to ZK is done in memory, so it is very fast, but if you stop ZK and start the data, you still need to guarantee, so there will be such a path to save the transaction log, when ZK starts again, it is loaded into memory to recover the data. This directory will have a directory called version-2, which determines the version number of the current transaction log, which changes the next time a version of ZK modifies it. The log file size is 64m, and if there is more data, there will be multiple files of this size.

It is recommended that you save the transaction log to a separate disk and it is a high-speed disk. For consistency, ZK's write request to the client writes this operation to the transaction log before returning. LogDir

Log path of logDirzookeeper service clientPort

Listen on the port to which the client connects

Modify the zkEnv.sh script, after zk starts, there will be a zookeeper.out file, this file will become larger and larger over time, the default will be generated in the location where the zkServer.sh is executed, so we need to change it to the specified path.

After configuring the above settings, you can start.

Test connection

Nc is a network command, and its full name is netcat. In fact, it is the same with telnet.

ZK client

ZkCli.sh-server IP:PORT

You can also not enter IP and port. By default, 127.0.0.1 2181 will be connected.

View the transaction log:

Java-classpath / usr/local/zookeeper-3.4.11/zookeeper-3.4.11.jar:../../lib/slf4j-api-1.6.1.jar org.apache.zookeeper.server.LogFormatter log.1

Cluster installation

The cluster configuration is the same as the stand-alone installation, except that the content of the configuration file will be added as follows:

"server.id=host:port:port" indicates the configuration of different ZK servers. Id represents different servers. You need to manually create a file called myid in the directory set by the dataDir of the server configuration file. This file has only one line of content and identifies yourself, that is, your ID value. The value range can be between 1 and 255.

Echo 1 > / usr/local/zookeeper3.4.11/data/myid

Host: hostnam

The first port: the port in the cluster where the slave server (follower) connects to the master server (leader), which is used as a leader, and where all other slave servers connect to the master server.

Second port: the port used in the leader election

Cluster startup is the same as stand-alone startup, using the same command, it is important to note that if the cluster is not available, then you will not be able to connect to the ZK server, that is to say, if you only start one of the three, you will not be able to Telnet to ZK, at least you have to start two.

It is possible that zkServer.sh start# starts the cluster by running the following command to clean up the firewall rules. It is possible that some security mechanism affects cluster startup, mainly in the election process iptables-F

Cluster startup log description

Srv01's log

If the peer is not ready, it will repeat this prompt all the time.

Srv02's log

Srv03's log

Server statu

The status indicates that LOOKING is looking for Leader. When the server appears this state, it will think that the current cluster does not have Leader, so it needs to enter the status of electing FOLLOWING followers, indicating that it is the leader state of Follower role LEADING, and that it is currently Leader role OBSERVING.

Observer status, indicating that the current server is an Observer role

The following is the election information, where (my state) is the status of the current server, and represents its final state for the last time. You can know that the server is in a state change by looking at the changes before and after the change.

Configure Zookeeper

Basic configuration: to run ZK at least need to configure content clientPort listens for client connection port dataDir database snapshot storage location tickTime client single to server heartbeat frequency, also known as basic time unit (in milliseconds). Clients send heartbeat messages to each other after connecting to ZK, this is the time. All time used in the configuration is based on this, that is, its multiple, or how many heartbeats can be tolerated.

Advanced configuration

Save the dataLogDir transaction log path. It is recommended to save the log path and snapshot path on a separate disk in the production environment to avoid disk IO busy resulting in performance degradation. If necessary, you can also turn off the logging feature maxClientCnxns to limit the number of clients connected to the ZK and limit the number of concurrent connections. It distinguishes different clients through IP. A value of 0 means that no restrictions are imposed. Note that the restrictions here are for a single client-to-server and do not control all client connections. The default 60.minSessionTimeout minimum session timeout is 2 times that of tickTime. It is not recommended to set this value lower than tickTime. When the client connects to the ZK, it is marked as a timeout if it does not contact the ZK within this minimum time, that is, it will be disconnected. MaxSessionTimeout maximum session timeout, which defaults to 20 times the minimum session timeout. It is not recommended to set this value lower than tickTime. When the client connects to the ZK, it is marked as a timeout if it does not contact the ZK during this maximum time. So the above parameter and this parameter form a time range, that is, when the client connects to ZK, if the connection is not successful within this time range, it will be marked as a timeout. If the time range set by the client is not within the range set by this server, it will be forced to apply the range set by the server. Autopurge.snapRetainCount

Automatically clean up the log. This parameter sets the number of snapshot files and corresponding transaction log files. The default is 3. If you set it less than 3, it will be automatically adjusted to 3.

Autopurge.purgeInterval

Automatically clean the log, this parameter sets the automatic cleaning frequency, and the above parameters are used together. During the interaction between the client and the ZK server, the server will generate a lot of logs, and ZK will save the data in memory as snapshots, and the data will not be deleted automatically, so the disk space will be occupied. You can set these two parameters to clean up automatically, but if the ZK server is busy and catches up with the log deletion task, it will affect performance, so this automatic cleanup is generally not set. Instead, it is handled by the scheduled task of Linux when the number of ZK visits is low. 0 means that the automatic cleaning feature is not enabled.

GlobalOutstandingLimit

The maximum number of requests piled up in ZK, and there are more requests from the client. In order to prevent the client from consuming too much resources, the server must limit the number of requests processed at the same time. PreAllocSize

Used to configure the pre-allocated space for ZK transaction logs, which defaults to 64MsnapCount

Used to configure the number of transaction logs between adjacent snapshots, which defaults to 100000. That is, after 100000 transactions, take a snapshot and carry forward the transaction log. ClientPortAddres

This parameter is for ZK servers with multiple network cards and allows you to specify a different listening port for each IP address. Fsync.warningthresholdms

It is used to set the alarm threshold of the time consumed by the ZK server transaction log synchronization operation, and the log will be recorded if the actual consumption time exceeds this time. ElectionAlg is used to configure the Leader election algorithm. Currently, there is only one election algorithm, so there is no need to configure it. CnxTimeout is used to time out the creation of TCP connections between servers during the Leader election. The default is 5.forceSync. This parameter is used to configure whether the ZK server is forced to write to disk when things are submitted (delayed write of LINUX). The default is YES. Jute.maxbuffer

Used to configure the maximum number on a single data node, which defaults to 1MB. You don't usually need to change this parameter, but sometimes you need to reduce the value because Zookeeper is not suitable for storing too much data. SkipACL

Whether to skip ACL check, the default is no, that is, acl check is performed on all client connections.

Cluster configuration

InitLimit indicates the time it takes to allow a slave server (the client as opposed to leader) to connect to the leader and complete data synchronization. It is expressed in multiples of tickTime, that is, how many heartbeats can be tolerated for the slave server to complete the initialization connection and data synchronization with the master server. If the establishment of the initialization connection cannot be completed beyond this time, the connection fails. The default is 10. If you have a large amount of data and a large number of servers, then this value can be set a little higher.

SyncLimit represents the frequency at which heartbeat requests and responses are sent between the master server (leader) and the slave server (follower). If the slave server cannot communicate with the master server during this time, the slave server fails. The default is 5. If the network of the cluster environment is not good, you can adjust it a little bit. LeaderServes

It is used to configure whether the Leader server accepts connections from the client and whether to allow Leader to provide services directly to the client. Server.x=

Used to configure the list of cluster servers

Query commands such as Zookeeper service status and configuration

Command description conf displays currently loaded profile information cons lists client session information currently connected to the server, including the number of packets received and sent, session ID and other dump lists all session information in the cluster, as well as unprocessed sessions and each session creates a temporary node envi

List current environment information, such as reqs such as JAVA version, OS information, hostname, etc.

List unprocessed request ruok

Test whether the server is normal. If it is normal, put it back to "imok". If it is not normal, nothing is realistic, stat.

Display and performance as well as a list of clients, including Zookeeper version, operation mode, latest ZXID, number of connections, number of nodes srvr and stat command, except that client connection information is not listed, but only server information mntr is used to output more detailed server statistics than stat, request latency, memory database size, cluster synchronization status, and so on. Wchs

List the details of the server watch wchc

List the details of the server's watch through session, and its output is a list of watch-related sessions wchp

Lists the details of the server watch by path, and its output is a path related to session

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.