Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

What is the main role in KAFKA architecture design

2025-01-18 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >

Share

Shulou(Shulou.com)06/01 Report--

This article mainly explains "what is the main role in KAFKA architecture design". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "what is the main role in KAFKA architecture design".

[Topic]

Topic, which we call the theme. In a word, it summarizes its function: Topic is the macro concept of our production / consumption.

That is, when we producers are producing messages, we have to specify a certain Topic;. When we consumers consume messages, we should also specify certain Topic.

Let's take a closer look at the above words, why do we say that Topic is a macro concept?

[Partition]

Partition, which we call partitioning. In a word, it sums up its function: Partition is the real entity in which we produce / consume.

That is, when we producers produce messages, we actually write to some partition of a certain Topic; when we consumers consume messages, we actually consume some partitions of a certain Topic.

From the figure above, we can see one thing: a Topic is a logical collection of messages of a certain kind, which contains partitioned entities, and the actual objects we produce and consume are all partitions of a Topic.

Like the figure above, [A-Topic-0], what does this mean, that is, [topic name-partition number]

The topic name generally represents what type of data the Topic is used to store, such as Order-Topic, which is used to store order-related data.

The partition number means that we need to specify the number of partitions when we create the Topic. There are two Partition in the above figure, and then the partition numbers start from 0, so we can see such a representation.

[Replica]

Replica, which we call a replica. To sum up its role in a sentence: Replica is data redundancy for Partition to achieve high reliability of data, as well as high availability of the entire service (detailed in this subsequent chapter).

A copy is to make N copies of all the data from a partition and then distribute them to different machines, so as to prevent data loss after a machine goes down.

As an example in the above figure, there are the following roles:

Topic name: A-Topic

Number of partitions: 2

Number of copies: 3

The above figure is an overview of a complete role model of a KAFKA cluster. Finally, let's give a unified overview of the above three roles:

Topic is a logical concept of messages, but in fact, production and consumption are carried out by partitions under Topic, and each partition has a share of data redundancy distributed on different Broker in the cluster. This data redundancy is called Replica (replica).

Thank you for reading, the above is the content of "what is the main role in KAFKA architecture design". After the study of this article, I believe you have a deeper understanding of what the main role in KAFKA architecture design is, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Servers

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report