Shulou(Shulou.com)06/02 Report--

This article mainly explains the "check kafka broker topic and consumer group status related commands", the article explained the content is simple and clear, easy to learn and understand, the following please follow the editor's ideas slowly in-depth, together to study and learn "View kafka broker topic and consumer group status-related commands"!

Recently, I have encountered a scenario in which I need to use kafka. After the test consumer program starts, it will take dozens of seconds or even minutes to successfully obtain the partition and offset of topic, and then start consuming data. So I learned the commands to check the status of topic and consumer group in kafka broker, and record them here.

The zookeeper configuration address used in the following command is 127.0.0.1 zookeeper 2181 bootstrap Murray server (that is, broker) address is 127.0.0.1 9292

1. View the kafka topic list and use the-- list parameter

Bin/kafka-topics.sh-- zookeeper 127.0.0.1-- list__consumer_offsetslx_test_topictest

2. Check the details of the specific topic of kafka, and use-- topic and-- describe parameters

Bin/kafka-topics.sh-- zookeeper 127.0.0.1 topic lx_test_topic-- describeTopic:lx_test_topic PartitionCount:1 ReplicationFactor:1 Configs: Topic:lx_test_topic Partition: 0 Leader: 0 Replicas: 0 Isr: 0

Lists the number of parition of lx_test_topic, replica factor, and leader and replica information of each partition

3. View the consumer group list and use the-- list parameter

To view the consumer group list, there are new and old commands to view the new version (information saved in broker) consumer list and the old version (information saved in zookeeper) consumer list, so you need to distinguish between the specified bootstrap--server and zookeeper parameters:

Bin/kafka-consumer-groups.sh-- new-consumer-- bootstrap-server 127.0.0.1 bootstrap-server 9292-- listlx_testbin/kafka-consumer-groups.sh-- zookeeper 127.0.0.1 bootstrap-server 2181-- listconsole-consumer-86845console-consumer-11967

4. View specific consumer group details and use-- group and-- describe parameters

Also specify the bootstrap-server and zookeeper parameters according to the new / old version of consumer:

Bin/kafka-consumer-groups.sh-new-consumer-bootstrap-server 127.0.0.1 bootstrap-server 9292-group lx_test-describeGROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG OWNERlx_test lx_test_topic 0465 465 0 kafka-python-1.3.1_/127.0.0.1bin/kafka-consumer-groups.sh-- zookeeper 127.0.0.1 group console-consumer-11967 2181-- describeGROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG OWNERCould not fetch offset from zookeeper for group console-consumer-11967 partition [lx_test_topic 0] due to missing offset data in zookeeper.console-consumer-11967 lx_test_topic 0 unknown 465 unknown console-consumer-11967_aws-lx-1513787888172-d3a91f05-0

The group name, the topic name of the consumption, the partition id, the offset last submitted by consumer group, the production message offset submitted last, the difference between the consumption offset and the production offset, and the group member id of the current consumption topic-partition (not necessarily including hostname) are shown in turn.

In the above example, console-consumer-11967 is used to test a temporary console consumer, and the current_offset information saved in zookeeper is missing.

Thank you for your reading, the above is the content of "viewing topic and consumer group status commands in kafka broker". After the study of this article, I believe you have a deeper understanding of the problem of viewing topic and consumer group status commands in kafka broker, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.