Shulou(Shulou.com)05/31 Report--

This article mainly explains "what is the Acker mechanism of Storm". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "what is the Acker mechanism of Storm".

Analysis of basic concepts

For Storm, a relatively important concept is "Guarantee no data loss"-- reliability.

Obviously, to achieve this feature, it is necessary to tracker the whereabouts and results of each data, and how Storm does it.

-"that's the Acker mechanism that we're going to talk about. Let's first summarize the workflow that Acker is involved in.

1 Spout when creating a new Tuple, a message is sent to inform acker to trace

2 Bolt will also send a message to notify Acker when it succeeds or fails to process Tuple

3 Acker will be so good that the Spout that launches the Tuple and returns its Ack, fail method

A tuple that is fully processed means:

The tuple and all subsequent tuple caused by the tuple are processed successfully, and a tuple is considered to have failed if this

The message was not processed successfully within the time specified by timeout

That is to say, for any Spout-tuple and its descendants, we will be informed of the success or failure of processing.

A new tuple is generated by a tuple called anchoring. When you launch a tuple, you complete an anchoring.

There is a special kind of task called acker in Storm. Please note that Acker also belongs to a kind of task. If you are not familiar with Task, please refer to another document: about the relationship between Storm-executor-task, acker is responsible for tracking the tuple tree of every tuple sent by spout. When Acker finds that a tuple tree has been processed, it will send a message to the task that generated the tuple.

Acker task component to set the number of acker in a topology, the default is one, if your topoogy has more tuple, then please set the number of acker a little more, it will be more efficient.

The way to understand the reliability of Storm is to look at the life cycle of the tuple,tuple tree. When a tuple is created, whether created by Spout and bolt, it is assigned an one-bit ID, and acker uses this ID to track all tuple. Every tuple knows his ancestor's iD, spits Stomr and detects that a tuple has been completely processed, then Storm will call the ACk method of the source of the message with the original message-id as a parameter, whereas Storm will call the fail method of Spout

It is worth noting that Storm calls Ack or the task of fail is always the task that generated this tuple, so if a Spout is divided into many task to execute, the success or failure of message execution will always notify the task that originally issued the tuple.

Reliability production scene of Storm

As a user of Storm, there are two things to do to make better use of the reliability characteristics of Storm. First, you should notify Storm when you generate a tuple, and secondly, notify Storm after a tuple is fully processed, so that Storm can detect whether the entire tuple tree has been processed or not, and notify the source Spout of the processing result.

1 because the corresponding task is dead, a tuple is not Ack:

Storm's timeout mechanism marks the tuple as a failure after the timeout, so it can be reprocessed.

2 Acker is dead: in this case, all spout tuple tracked by this Acker will time out and be reprocessed

3 Spout is dead: in this case, the source that sent the message to Spout is responsible for resending the message

Three basic mechanisms ensure that Storm is fully distributed, scalable and highly fault-tolerant.

Thank you for your reading, the above is the content of "what is the Acker mechanism of Storm". After the study of this article, I believe you have a deeper understanding of what the Acker mechanism of Storm is, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.