Basic configuration and simple use of Kafka under Windows

First download zookeeper (version zookeeper-3.4.13) and kafka (version kafka_2.12-2.0.0), and decompress it.

1 、 zookeeper:

In the config directory, there is a zoo.conf configuration file, you can configure the port and log storage location, you can change according to your own habits.

Start zookeeper first. If zookeeper is configured in the environment variable, you can directly open the cmd window and start it with zkserver, as follows:

If all the printed logs are INFO or WARN and enter the daemon thread state, the startup is successful.

2 、 Kafka:

In the config directory, there is a server.properties configuration file that configures the log directory: log.dirs=D:/nomalAPP/kafka_2.12-2.0.0/kafka-logs, and the address of the zookeeper that configures the connection: zookeeper.connect=localhost:2181. Others are basically unnecessary and can be modified according to specific needs.

The startup command is:.\ bin\ windows\ kafka-server-start.bat.\ config\ server.properties. As follows:

Some basic configuration information will also be printed below. A successful startup will also enter the daemon thread state.

Do not close the two started above, and then continue with the following:

3. Topic: create a theme

The command is:.\ bin\ windows\ kafka-topics.bat-- create-- zookeeper localhost:2181-- replication-factor 1-- partitions 1-- topic test_topic, as follows:

To view the existing theme, the command is:.\ bin\ windows\ kafka-topics.bat-- list-- zookeeper localhost:2181, as follows:

3. Producer: producer

The command to create the producer is:.\ bin\ windows\ kafka-console-producer.bat-- broker-list localhost:9092-- topic test_topic, as follows:

4. Consumer: consumer

Command to create a consumer:.\ bin\ windows\ kafka-console-consumer.bat-- zookeeper localhost:2181-- topic test_topic

But when I used this command, the following happened:

This is because the version of kafka used is relatively new, and the above method has been outdated and deleted, so it cannot be started.

This can be switched to another:.\ bin\ windows\ kafka-console-consumer.bat-- bootstrap-server localhost:2181-- topic test_topic-- from-beginning. Start as follows:

Garbled problems can be dealt with in specific producer and consumer code.

The above are just some entry-level basic operations, advanced construction of clusters, integration of other technologies, which need to be learned!

Personal understanding: kafka is like a transitional channel, used to transfer messages (message agents), may be like a Wechat official account, the producer is the manager of the official account, put articles into the official account, and consumers are like people who subscribe to this official account. As long as they subscribe to this official account, when new messages are pushed to the official account, the official account will push new articles to subscribers.

The benefits of kafka are:

(1) Reliability-Kafka is distributed, with partitioning, replication, and fault tolerance.

(2) scalability-the Kafka messaging system scales easily without downtime.

(3) durability-Kafka uses distributed commit logs, which means that messages remain on disk as quickly as possible, so it is persistent.

(4) performance-Kafka has high throughput for both publish and subscribe messages. It maintains stable performance even though many TB messages are stored.