Shulou(Shulou.com)06/03 Report--

This article mainly shows you "how to solve kafka errors", the content is easy to understand, clear, hope to help you solve your doubts, the following let the editor lead you to study and learn "how to solve kafka errors" this article.

Today, due to the improper configuration of the kafka cluster, and due to the death of a main consumer (the server crashed and needs to be repaired), all new consumers (the new version of offset is stored in kafka) cannot pull messages.

A mistake of kafka

Because it is an online problem, it must not affect the user, use the old client (offset stored in zk) for consumption, and then migrate the kafka to the standby service.

Let's talk about the specific ways to deal with the accident.

1. The first thing is to make sure that the information you get can't be lost, so the old consumers spend.

two。 Online services switch kafka services by balancing the load one by one, and switch consumers to standby services when the original producer data is consumed.

3. Start configuring online kafka services

How to solve kafka error

The main changes are as follows

Offsets.topic.replication.factor=3

Set the number of copies to 3, so that when one consumer goes down, other consumers can also spend.

In order to make replicas possible for topic, a more crude method is used to empty topic and offset directly.

It is divided into the following steps

. / zookeeper-shell.sh227.0.0.1:24500

Rmr/brokers/topics/topic-name

Rmr/config/topics/topic-name

Rmr/admin/delete_topics/topic-name

At this point, the topic has been completely deleted, and then the new topic is created.

Bin/kafka-topics--create--zookeeperlocalhost:2181--replication-factor3--partitions3--topictopic-name

At this time, producers and consumers can be started to view, and there is no longer the original exception.

Connectiontonode-1couldnotbeestablished.Brokermaynotbeavailable.

At this time, I tested any of the three broker and went offline, and the service automatically selected other nodes for production and consumption.

The above is all the contents of the article "how to solve kafka errors". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.