Shulou(Shulou.com)06/01 Report--

This article mainly explains "what are the types of Time in Flink". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "what are the types of Time in Flink".

* * Processing Time**

Processing Time refers to the system time of the machine when the event is processed.

When the streaming program runs on Processing Time, all time-based operations (such as time windows) will use the system time of the machine at that time. The hourly Processing Time window will include all events that reach a specific operation during the entire hour indicated by the system clock.

For example, if the application starts running at 9:15, the first hourly Processing Time window will include events handled between 9:15 and 10:00, and the next window will include events handled between 10:00 and 11:00.

Processing Time is the simplest "Time" concept that does not require coordination between streams and machines. It provides the best performance and lowest latency. However, in both distributed and asynchronous environments, Processing Time cannot provide certainty because it is vulnerable to the speed at which events arrive at the system (for example, from message queues), the speed at which events flow through the system, and interrupts.

* * Event Time**

Event Time is the time when the event occurs, which is generally the time carried by the data itself. This time is usually determined before the event reaches the Flink, and the event timestamp can be obtained from each event. In Event Time, time depends on the data and has nothing to do with others. The Event Time program must specify how to generate the Event Time watermark, which is the mechanism for indicating the progress of the Event Time.

Perfectly speaking, no matter when the events arrive or how they are sorted, the final processing of the Event Time will produce exactly the same and definite results. However, unless events arrive in a known order (in terms of their time), there will be some delay in processing Event Time because you have to wait for some unordered events. Because you can only wait for a limited period of time, it is difficult to guarantee that processing Event Time will produce completely consistent and definitive results.

Assuming that all the data has arrived, the Event Time operation will run as expected, producing correct and consistent results even when dealing with unordered events, deferred events, and reprocessing historical data. For example, the hourly event time window will contain all records with event timestamps that fall into that hour, regardless of the order in which they arrive.

Note that sometimes when Event Time programs process real-time data, they use some Processing Time operations to ensure that they do so in a timely manner.

* * Ingestion Time**

Ingestion Time is the time when the event enters the Flink. At the source action, each event takes the current time of the source as the timestamp, and time-based operations, such as the time window, take advantage of this timestamp.

Ingestion Time is conceptually located between Event Time and Processing Time. It is slightly more complex than Processing Time, but the results are more predictable. Because Ingestion Time uses a stable timestamp (allocated once at the source), different window operations on events will reference the same timestamp, while in Processing Time, each window operator can assign the event to a different window (based on machine system time and arrival delay).

Compared to Event Time, the Ingestion Time program cannot handle any unordered events or delayed data, but the program does not have to specify how to generate the watermark.

In Flink, Ingestion Time is very similar to Event Time, but Ingestion Time has the functions of automatically assigning timestamps and automatically generating watermarks.

Thank you for your reading, the above is the content of "what are the Time types in Flink". After the study of this article, I believe you have a deeper understanding of what the Time types in Flink have, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.