Shulou(Shulou.com)06/01 Report--

This article mainly shows you "what are the common Kafka development and operation and maintenance commands", which are easy to understand and clear, and hope to help you solve your doubts. Let the editor lead you to study and learn this article "what are the common development and operation and maintenance commands of Kafka?"

Common commands:

1. View help

/ usr/hdp/2.6.3.0-235/kafka/bin/kafka-topics.sh-- help

two。 View all topic of the current cluster

. / kafka-topics.sh-zookeeper salver31.hadoop.unicom:2181-list

3. View topic details (kafkawordcount)

. / kafka-topics.sh-- zookeeper salver31.hadoop.unicom:2181-describe-topic kafkawordcount

4. View kafka cluster details

. / kafka-topics.sh-zookeeper salver31.hadoop.unicom:2181-- describe

5. Create a new topic

. / kafka-topics.sh-- zookeeper salver31.hadoop.unicom:2181-- create-- topic topic001-- partitions 1-- replication-factor 1-- config max.message.bytes=64000-- config flush.messages=1

Note:

Theme-related configurations use global defaults by default, or you can override global defaults in the form of-- config key=value. The topic1 above customizes the maximum message size and refresh speed.

6. Modify the topic configuration:

You can use the alter topic command to change or set overrides.

Update number of partitions:

. / kafka-topics.sh-- zookeeper salver31.hadoop.unicom:2181-- alter-- topic topic001-- partitions 2

Maximum message size for updating my-topic:

. / kafka-topics.sh-zookeeper salver31.hadoop.unicom:2181-alter-topic topic001-config max.message.bytes=128000

Delete custom configuration (delete custom maximum number of message bytes):

. / kafka-topics.sh-zookeeper salver31.hadoop.unicom:2181-alter-topic topic001-delete-config max.message.bytes

7. View the amount of data in each partition message of topic:

. / kafka-run-class.sh kafka.tools.GetOffsetShell-- broker-list salver31.hadoop.unicom:6667,salver32.hadoop.unicom:6667-- topic kafkawordcount-- time-1

8. View the contents of the log log message:

. / kafka-run-class.sh kafka.tools.DumpLogSegments-- files / kafka-logs/kafkawordcount-0/00000000000000000000.log-- print-data-lo

9.Producer sends a message:

. / kafka-console-producer.sh-broker-list salver31.hadoop.unicom:6667,salver32.hadoop.unicom:6667-topic topic001

10.Consumer receives messages:

. / kafka-console-consumer.sh-- bootstrap-server salver31.hadoop.unicom:6667,salver32.hadoop.unicom:6667-- from-beginning-- topic kafkawordcount

There are three common parameters for receiving messages:

The consumer-property parameter specifies the consumer-level configuration in the form of key-value pairs.

From-beginning sets the starting position of the message to start consumption. The default is to start consumption from the new location latest.

Delete-consumer-offsets deletes the offset that records the consumed in the zookeeper

Consumer receives messages with parameters:

/ kafka-console-consumer.sh-- bootstrap-server salver31.hadoop.unicom:6667,salver32.hadoop.unicom:6667-- topic kafkawordcount-- consumer-property group.id=consumerGroup1-- consumer-property consumer.id=consumerId1-- from-beginning-- delete-consumer-offsets

11. View the offset of a consumer group in topic:

. / kafka-run-class.sh kafka.tools.ConsumerOffsetChecker-- topic kafkawordcount-- zookeeper localhost:2181-- group consumerGroup1

twelve。 View topic maximum offset:

/ usr/hdp/2.6.3.0-235/kafka/bin/kafka-run-class.sh kafka.tools.GetOffsetShell-broker-list salver31.hadoop.unicom:6667,salver32.hadoop.unicom:6667-topic test511-time-2

View minimum offset

/ usr/hdp/2.6.3.0-235/kafka/bin/kafka-run-class.sh kafka.tools.GetOffsetShell-broker-list salver31.hadoop.unicom:6667,salver32.hadoop.unicom:6667-topic test511-time-1

. Delete topic

. / kafka-topics.sh-- zookeeper salver31.hadoop.unicom:2181-- delete-- topic topic001

Note:

Deleting topic here doesn't really delete it, just marks topic as "deleted". If you need to delete topic completely, please refer to the article in kafka Advanced: "kafka Advanced: how to completely delete topic and data."

These are all the contents of this article entitled "what are the common development and operation and maintenance commands of Kafka?" Thank you for your reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.