Shulou(Shulou.com)05/31 Report--

How to carry out Kafka access control and monitoring alarm, I believe that many inexperienced people do not know what to do. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

Authority Control and Monitoring alarm of Kafka

(1) access control

At the beginning of the introduction, we said that the early Kafka cluster did not set security verification to run naked, so as long as you know the connection address of Broker, you can produce and consume it, and there are serious data security problems.

Generally speaking, users who use SASL will choose Kerberos, but as far as the usage scenario of platform Kafka cluster is concerned, the user system is not complex, and the use of Kerberos is overqualified. At the same time, Kerberos is relatively complex, which may lead to other problems. In addition, in terms of Encryption, SSL encryption is not used because it runs in an intranet environment.

Finally, the Kafka cluster of the platform uses SASL as the authentication method, which is based on the lightweight combination of SASL/SCRAM+ACL to create users dynamically and ensure data security.

(2) Monitoring and alarm

In the past, in the use of clusters, we often found that the performance of consumer applications deteriorated for no reason. The reason for analyzing the problem is usually that the data that lags behind reading by Consumer misses Page-cache. As a result, the kernel of the Broker machine has to read the data from disk and load it into Page-cache before returning the result to Consumer. This is equivalent to the fact that the disk that could have served the write operation is now going to read the data, which affects the performance of the cluster that is degraded by the user at the same time.

At this time, it is necessary to find the applications that lag behind Consumer to intervene in advance to reduce the occurrence of problems, so monitoring alarms are of great significance to both platforms and users. The following is to introduce our practical ideas.

Overall plan:

The overall solution is mainly based on the open source component KafkaJMXMetrics+OpenFalcon+Grafana:

The internal indicators of KafkaJMXMetrics:Kafkabroker are exposed to the outside in the form of JMXMetrics. Version 1.1.1 provides a wealth of monitoring metrics to meet monitoring needs

OpenFalcon: Xiaomi's open source enterprise-level, highly available, scalable open source monitoring system

Grafana:Metrics visualization system, we are familiar with, can dock a variety of Metrics data sources.

About the alarm:

Radar system: self-developed monitoring system, through Falcon and Eagle to obtain Kafka indicators, combined with the setting threshold for alarm. Take consumption pattern as an example, Lag is an important indicator of whether consumption is normal or not, and if Lag keeps increasing, it must be dealt with.

When a problem occurs, not only the Consumer administrator should know, but also its users, so the alarm system also needs to notify the user. The specific way is to automatically remind the person in charge of the corresponding consumer group or users and the manager of the Kafka cluster through the WeCom alarm robot.

About monitoring:

Falcon-agent: deploy to each Broker to parse KafkaJMX metrics and report data

Grafana: used to visualize FalconKafkaMetrics data and create a monitoring market for Cluster, Broker, Topic and Consumer4 roles.

Eagle: obtain the Active status and Lag backlog of the consumer group, and provide API to provide monitoring data for the monitoring alarm system "radar".

After reading the above, have you mastered how to control the authority of Kafka and how to monitor the alarm? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.