Shulou(Shulou.com)06/03 Report--

This article introduces the relevant knowledge of "what are the advantages of Kafka 2.8.0". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Friends who usually pay attention to Kafka should pay attention. On April 19, 2021, Kafka 2.8.0 is officially released!

This upgrade includes many important changes, the most striking of which is that kafka replaces ZooKeeper through self-managed arbitration. Popularly speaking, Kafka will no longer need ZooKeeper and officially break up!

In fact, as early as 19 years, someone in the community put forward the idea of removing Kafka's dependence on Zookeeper, which was considered almost impossible at the time, but as everyone worked together and complacent, it was realized step by step.

Version 2.8.0 will be the first version to run Kafka without ZooKeeper, and this is also known as Kafka Raft Metadata mode (Kafka Raft metadata Schema), which may be remembered by future generations.

There may be some friends who have just come into contact with Kafka who don't understand what this means.

One of the advantages of Kafka is that it can provide high efficiency and throughput, and the underlying implementation of submitting logs is often the first task to learn for a partner who has just come into contact with before.

A large part of Kafka's code base is also responsible for arranging logs, assigning leadership, handling failures, and so on in multiple clusters. This makes Kafka a reliable and reliable distributed system. ZooKeeper is the key to the work of distributed code. In previous versions, ZooKeeper provided authoritative metadata storage that stored the most important things in the system, such as where partitions can exist, which components are dominant, and so on.

In any case, ZooKeeper is a special file system / trigger API based on consistent logs. Kafka is a publish / subscribe system based on consistent logs.

This unalterable reality makes it possible for operators to tune, configure, monitor, protect, and evaluate communication and performance across two log implementations, two network layers, and two security implementations (each with different tools and monitoring hooks). This makes the system quite complex!

So a friendly breakup with ZooKeeper may have labor pains in the short term, but the advantages outweigh the disadvantages for the long-term development of Kafka.

In addition to being separate from ZooKeeper, three new features have been added to this update:

[KAFKA-10500]-add API to start and stop stream threads

[KAFKA-10700]-supports mutual TLS authentication using SASL_SSL listeners

[KAFKA-10749]-add IP limits by connection rate

There are as many as a hundred bug optimizations and modifications.

Some important updates such as:

[KAFKA-5488]-KStream.branch should not return an array of streams that must be accessed through a known index

[KAFKA-6687]-allows multiple readings of topics

[KAFKA-6943]-if any thread crashes, or if all threads crash, you can choose to shut down KS cleanly

[KAFKA-9023]-producer network exception response should record more information

[KAFKA-12327]-remove MethodHandle usage from CompressionType

[KAFKA-12365]-Block API is not supported by kip-500 agent / controller (currently)

[KAFKA-12394]-consider subject id presence and authorization errors

[KAFKA-4748]-requires a way to shut down all worker processes in the Streams application at the same time

[KAFKA-10722]-use timestamp storage even if you don't need it

[KAFKA-10723]-LogManager leaks internal thread pool activity during shutdown

This is the end of "what are the advantages of Kafka 2.8.0". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.