Shulou(Shulou.com)06/01 Report--

This article mainly introduces "what is ISR in KAFKA". In daily operation, I believe many people have doubts about what is ISR in KAFKA. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful for you to answer the doubt of "what is ISR in KAFKA?" Next, please follow the editor to study!

[what is ISR]

First of all, the full name of ISR is: In-Sync Replicas (synchronous replica set), which we can understand as a collection of all replicas that are synchronized with leader.

The set of all copies of a partition is called AR (Assigned Repllicas)

A replica set that cannot be synchronized with leader-replica is called OSR (Out-Sync Relipcas).

So we can get such a representation: AR = ISR + OSR, which means that a replica set of a partition is divided into two parts: synchronous collection and asynchronous collection.

So we can assume a scene in which the AR set of a partition is [0mem1pr 2pr 3pr 4p5], where leader-replica is 0.

Among them, the data of follower and leader are kept in sync, while the data of 4Jet 5 are not synchronized with leader.

So at this point, ISR= [0mem1re2jue 3], OSR= [4pje 5]

If replica 4 catches up with leader-replica at this time, it is synchronized with leader.

So at this point, ISR= [0mem1re2je 3jin4], OSR= [5]

As we can see from the above scenario, ISR dynamically maintains a collection of replicas that are synchronized with leader replicas, and all replicas in ISR are synchronized with leader data.

[what is the role of ISR]

Let's think about what we can do when we know the replica set that is synchronized with leader.

1. When we produce messages, how many copies do we have to write to be considered successful?

2. When leader is dead, which follower should we choose to become the new leader?

So correspondingly, through ISR, we know which follower is synchronized with leader.

Then we can set all replicas in ISR to be written successfully when the message is written.

Then when we switch between leader, we can select the corresponding follower from the ISR to become the new leader.

This is what ISR is for: it is an indispensable part of the replica mechanism when messages are highly reliable and services are highly available; this is why ISR has to be mentioned when it comes to replicas.

[why design ISR mechanism]

In some middleware, there is the concept of copy. When writing data in different scenarios, the number of copies required to be written is not the same.

For example, in zk, it is necessary to write more than half of the nodes to be considered successful, or in some scenarios that require high reliability, it is necessary to write all copies to be considered successful.

Kafka's ISR allows you to configure the desired effect according to your business scenario when producing messages:

Acks=0:fire and forget, that is, even if I send it, I don't care whether it succeeds or not. Under this setting, there is almost no guarantee for the high reliability of the message, but it has great throughput.

Acks=1: even if writing to the master node is successful, this setting can ensure a certain degree of high reliability and good throughput.

Acks=all: it is only successful if all copies in ISR are written. In this setting, high reliability can be provided, but the throughput is relatively low.

When we consider the production message, the ISR mechanism can be friendly to allow users to set parameters according to their own business needs to choose what degree of reliability they want to achieve, instead of just providing a reliability choice.

Add: our ISR is dynamically scalable. It may occur that all follower is dead and only leader is left in ISR, so setting acks=all is equivalent to acks=1 at this time.

This poses a risk to scenarios with high reliability requirements, so kafka provides the parameter: min.insync.replicas

This parameter can be used to configure at least how many copies are required in ISR to continue to provide write services. If set to 2, once the number of ISR is less than 2, the write service will no longer be provided, at the expense of some availability, to ensure this highly reliable scenario.

Finally, we answer the question in this section: the existence of the ISR mechanism is that in order to balance reliability and availability, kafka does not specify the provision of highly reliable or highly available services, but leaves the decision to the user, allowing the user to control through parameters, to what extent of high reliability and high availability to be achieved.

At this point, the study of "what is ISR in KAFKA" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.