Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to use Flink Process". In daily operation, I believe many people have doubts about how to use Flink Process. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "how to use Flink Process". Next, please follow the editor to study!

Process operator: handles the batch data streams that each keyBy (partition) inputs to the window (for KeyedStream type data streams)

Sample environment

Java.version: 1.8.xflink.version: 1.11.1

Sample data source (project code cloud download)

Building Development Environment and data of Flink system example

Process.java

Import com.flink.examples.DataSource;import org.apache.flink.api.java.functions.KeySelector;import org.apache.flink.api.java.tuple.Tuple3;import org.apache.flink.streaming.api.datastream.DataStream;import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;import org.apache.flink.streaming.api.functions.windowing.ProcessWindowFunction;import org.apache.flink.streaming.api.windowing.windows.GlobalWindow;import org.apache.flink.util.Collector;import java.util.Iterator;import java.util.List / * * @ Description process operator: handles the batch data streams (for KeyedStream type data streams) input to the window by each keyBy (partition) * / public class Process {/ * traverses the collection and prints the sum of the total number of people and ages of different genders * @ param args * @ throws Exception * / public static void main (String [] args) throws Exception {final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment () List tuple3List = DataSource.getTuple3ToList () DataStream dataStream = env.fromCollection (tuple3List) .keyby ((KeySelector) k-> k.f1) / / scroll by the quantity window, calculating every 3 input data streams. CountWindow (3) / processes the window data flow after each keyBy The process method is usually applied to data flow processing of type KeyedStream. Process (new ProcessWindowFunction () {/ * handle window data set * @ param s returned key value * @ param context window context from keyBy * @ param input all partitioned data streams obtained from the window * @ param out output data flow objects * @ throws Exception * / @ Override public void process (String s Context context, Iterable input, Collector out) throws Exception {Iterator iterator = input.iterator () Int total = 0; int I = 0; while (iterator.hasNext ()) {Tuple3 tuple3 = iterator.next (); total + = tuple3.f2; I + + } out.collect (s + "total:" + I + "people, average age:" + total/i);}}); dataStream.print (); env.execute ("flink Process job");}}

Print the result

4 > girl total: 3 people, average age: 242 > man total: 3 people, average age: 26 at this point, the study on "how to use Flink Process" is over, hoping to solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.