Shulou(Shulou.com)06/01 Report--

This article mainly explains "how to use Flink Apply to deal with the data flow in the window". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "how to deal with the data flow in the window with Flink Apply".

Apply operator: processing the data flow within the window

Sample environment

Java.version: 1.8.xflink.version: 1.11.1

Sample data source (project code cloud download)

Building Development Environment and data of Flink system example

Apply.java

Import com.flink.examples.DataSource;import org.apache.flink.api.java.functions.KeySelector;import org.apache.flink.api.java.tuple.Tuple3;import org.apache.flink.streaming.api.datastream.DataStream;import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;import org.apache.flink.streaming.api.functions.windowing.WindowFunction;import org.apache.flink.streaming.api.windowing.windows.GlobalWindow;import org.apache.flink.util.Collector;import java.util.Iterator;import java.util.List / * * @ Description Apply method: processes the data flow in the window * / public class Apply {/ * traverses the collection and prints the sum of the total number and age of different genders * @ param args * @ throws Exception * / public static void main (String [] args) throws Exception {final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment (); List tuple3List = DataSource.getTuple3ToList () DataStream dataStream = env.fromCollection (tuple3List) .keyby ((KeySelector) k-> k.f1) / / scroll by quantity window, data flow for every 3 input windows Calculate .countWindow (3) / / can only be called based on Windowed window Stream. Apply (/ / WindowFunction new WindowFunction () {/ * processing window Data set * @ param s key value returned from keyBy * @ param window window type * @ param input all partitioned data streams obtained from the window * @ param out Output data flow object * @ throws Exception * / @ Override public void apply (String s GlobalWindow window, Iterable input, Collector out) throws Exception {Iterator iterator = input.iterator () Int total = 0; int I = 0; while (iterator.hasNext ()) {Tuple3 tuple3 = iterator.next (); total + = tuple3.f2 I + +;} out.collect (s + "total:" + I + ", cumulative total age:" + total);}}); dataStream.print () Env.execute ("flink Filter job");}}

Print the result

4 > girl total: 3 people, cumulative total age: 742 > man total: 3 people, cumulative total age: 79 Thank you for your reading, the above is "how to use Flink Apply to deal with the data flow in the window", after the study of this article, I believe you have a deeper understanding of how to use Flink Apply to deal with the data flow in the window, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.