Shulou(Shulou.com)06/01 Report--

Editor to share with you Kafka cluster configuration and cluster management tool Kafka Manager installation example analysis, I hope you will learn something after reading this article, let's discuss it!

We use the following three machines to build a Kafka cluster. This document is valid on the premise that we have installed a zookeeper cluster. In this article, zookeeper is all represented by zk abbreviations.

192.168.1.96

192.168.1.45

192.168.1.34

The following operations are all done on the 96 machine without explanation.

Download Kafka

Wget http://apache.fayea.com/kafka/0.9.0.1/kafka_2.11-0.9.0.1.tgz

Decompress Kafka

Tar xzvf kafka_2.11-0.9.0.1.tgz

Modify the configuration file config/server.properties

Broker.id=0

Advertised.host.name=192.168.1.96

Log.dirs=/data/store/log/kafka-logs

Zookeeper.connect=192.168.1.96:2181192.168.1.45:2181192.168.1.34:2181/kafka

1. Broker.id, each cluster machine is unique, increasing from 0. After we synchronize the kafka to other machines, we need to modify the ID of each machine.

2. As shown in the figure below, it means that we can specify a chroot address at the end of the zookeeper.connect, of course, it is not necessary. By default, the root directory is used. Modifying this is tantamount to specifying the data directory that the kafka cluster needs to store in zk, but we need to manually establish such a directory in zk before the cluster starts.

3. Advertised.host.name is configured as the ip of the current server. If it is not configured, the default value is java.net.InetAddress.getCanonicalHostName (). I did not modify this value at that time. During the local test, there was a connection timeout problem, and the ip of broker in kafka manager is all localhost. After modification, it is estimated that it is related to getCanonicalHostName (). This problem will be examined later.

4. How to create this chroot path? Connect to any zk server using the following command

Bin/zkCli.sh

Create / kafka''

Synchronize kafka to other machines and modify the broker.id in config/server.properties. Don't forget

Scp-r kafka_2.11-0.9.0.1 192.168.1.34:/data/strawbird/kafka_2.11-0.9.0.1

Scp-r kafka_2.11-0.9.0.1 192.168.1.45:/data/strawbird/kafka_2.11-0.9.0.1

Start the kafka of three machines respectively

Bin/kafka-server-start.sh config/server.properties &

Install Kafka Manager

Git clone https://github.com/yahoo/kafka-manager

Cd kafka-manager

/ / the next step really takes a long time, wait patiently, to tell you the truth, I spent a total of 2752s here, so. Ha, blah, blah.

Sbt clean dist

Cd target/universal/ in the universal directory, we can see the following two files

/ / extract the zip file in the image above

Unzip kafka-manager-1.3.0.8.zip

Cd kafka-manager-1.3.0.8

/ / modify the value of kafka-manager.zkhosts of the Chinese Academy of Sciences for conf/application.properties

/ / start kafka manager

Bin/kafka-manager

/ / after startup, the default startup port is 9000, so we can access this address http://192.168.1.96:9000, in the browser and then add a cluster ourselves to see the following information

After reading this article, I believe you have some understanding of "sample analysis of Kafka cluster configuration and cluster management tool Kafka Manager installation". If you want to know more about it, welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.