Shulou(Shulou.com)06/01 Report--

This article mainly talks about "what are the time characteristics of Flink". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Now let the editor take you to learn "what are the time characteristics of Flink"?

The first part of the Flink DataStream program usually sets the basic time characteristics.

This setting defines how data stream sources behave (for example, whether they will allocate timestamps) and what time concepts should be used for window operations such as KeyedStream.timeWindow (Time.seconds (30)).

Flink supports different concepts of time in stream handlers.

ProcessingTime

Default, no need to specify

Refers to the system time of the machine performing the corresponding operation.

When the stream processor runs based on processing time, all time-based operations, such as time windows, use the system clock of the machine running the corresponding operator. The hourly processing time window will include all records that reach a specific operator during the entire hour indicated by the system clock. For example, if the application starts running at 9:15, the first hourly processing time window will include events processed between 9:15 and 10:00, the next window will include events handled between 10:00 and 11:00, and so on.

Processing time is the simplest concept of time and does not require coordination between the flow and the machine. It provides the best performance and the lowest latency. However, in both distributed and asynchronous environments, processing time does not provide certainty because it is vulnerable to the speed at which records arrive at the system (for example, from message queues), the speed at which operators are recorded within the system, and power outages (planned or otherwise).

EventTime

Event time is the time at which each event occurs on its production equipment.

It is commonly understood that the time specified by a certain field in the data received by Flink, such as the received Map object, has an eventtime field or a createdate field.

Since the system does not know which one represents EventTime, it should be specified when the business is required to write code.

The mechanism used to measure the time progress of events in Flink is watermarking. The watermark flows as part of the data stream and is time-stamped. Watermark (t) declares that the event time has reached the time t in the stream, which means that there should be no timestamped t 'from the stream

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.