Shulou(Shulou.com)06/02 Report--

This article mainly explains "the principle and usage of Kakfa". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "the principle and usage of Kakfa".

Kafka is a distributed, publish / subscribe based messaging system

1. Why the throughput of kafka queue is large

A. When producing messages, kafka is written sequentially to the hard disk.

B. Kafka data is not written to the hard disk in real time, but uses memory mapped file technology (mmap).

Kafka provides a parameter, producer.type, to control whether the active flush is synchronous and asynchronous.

C. When consumers consume messages, when they read data, they enter zero copy directly from the kernel space into the port in the form of files.

D, Kafaka message storage is based on partiton and can be extended horizontally.

2 、 kafaka Broker Server

Leader election

Prevent kafka from losing messages

1. The production end uses synchronous transmission and sets the required.acks attribute to-1 (master and slave confirm at the same time)

0: no confirmation, 1: master confirmation

2. Set message ID to ensure that messages are not duplicated.

3. The consumer sets the delivery attribute to at least one to consume at least once

Spend first and then confirm. Guarantee idempotency through business primary key

Thank you for your reading, the above is the content of "the principle and use of Kakfa". After the study of this article, I believe you have a deeper understanding of the principle and use of Kakfa, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.