Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to install and deploy KafkaOffsetMonitor. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

KafkaOffsetMonitor function:

1. You can see from the title, Kafka Offset Monitor, is to monitor consumer consumption, and can list each consumer offset, lag data.

two。 List of consumer groups

3. All parition lists (topic,pid,offset,logSize,lag,owner) for each topic

4. View the historical consumption information of topic.

1 download

When installing the KafkaOffsetMonitor management platform, we need to download its installation package first, and its resources can be found on Github

2 installation and deployment

The installation and deployment of KafkaOffsetMonitor is relatively simple, and all the resources are packaged into a JAR file, so it can be run directly, saving us the need to configure. Here we can create a new directory for the monitoring directory of Kafka, and I create a new kafka_monitor file directory here, and then we are preparing the startup script, which is as follows:

#! / bin/bash

Java-cp KafkaOffsetMonitor-assembly-0.2.0.jar\

Com.quantifind.kafka.offsetapp.OffsetGetterWeb\

-- zk ZookeeperAddress\

-- port 8089\

-- refresh 10.seconds\

-- retain 1.days

To explain the meaning of the following startup command, first we need to specify the class under which Web monitoring is run, then we need to use ZooKeeper, all of which need to fill in the ZK cluster information, followed by the Web running port, the time the page data is refreshed and the time value for the data to be retained.

3. Start

The startup steps are as follows:

Step 1: start ZK zkServer.sh start

Step 2: start the Kafka service (the cluster starts by entering the following command)

. / kafka-server-start.sh config/server.properties &

Step 3: start the Web monitoring service

Java-cp KafkaOffsetMonitor-assembly-0.2.0.jar\

Com.quantifind.kafka.offsetapp.OffsetGetterWeb\

-- zk dn1:2181,dn2:2181,dn3:2181\

-- port 8089\

-- refresh 10.seconds\

-- retain 1.days

After the Web service starts successfully, it is shown in the following figure:

4.KafkaOffsetMonitor run Preview

Next, let's use Kafka code to produce and consume some messages, and use Web monitoring to browse the messages. Here is a direct preview of the demonstration result, as shown in the following figure:

5. Summary

When running KafkaOffsetMonitor's JAR package, you need to make sure that the startup parameters are configured correctly to avoid startup errors. In addition, some of the static resources in KafkaOffsetMonitor's JAR on Github use Google hyperlinks, so if you only use them directly, if there is no local agent software, you will start an error. Here use the JAR I provide, this JAR is recompiled using local static resources after static resource revision.

In addition, some of the parameters in the figure have the following meanings:

Topic: create Topic name

Partition: partition number

Offset: indicates how much Message has been consumed by the Parition

LogSize: indicates how many Message have been produced by the Partition

Lag: indicates how many Message are not consumed

Owner: represents the consumer

Created: indicates the time when the Partition was created

Last Seen: indicates the latest time when consumption status is updated

This is the end of the article on "how to install and deploy KafkaOffsetMonitor". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.