Shulou(Shulou.com)06/01 Report--

This article mainly explains "how to use Flink Fold". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn how to use Flink Fold.

Fold operator: every output of the data stream is scrolled and superimposed, and the output results are merged.

Sample environment

Java.version: 1.8.xflink.version: 1.11.1

Sample data source

Building Development Environment and data of Flink system example

Fold.java

Import com.flink.examples.DataSource;import org.apache.flink.api.common.functions.FoldFunction;import org.apache.flink.api.java.functions.KeySelector;import org.apache.flink.api.java.tuple.Tuple3;import org.apache.flink.streaming.api.datastream.KeyedStream;import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;import java.util.List / * * @ Description Fold operator: scrolls and overlays each output of the data stream and merges the output results * (unlike Reduce, Reduce takes the results of the previous aggregation and outputs the data stream Fold directly appends the current data object to the result of the previous overlay and outputs the data stream) * / public class Fold {/ * traverses the collection, and the partition prints the result of each rolling overlay (example: partition by sex, sort by sex Append output without bits) * @ param args * @ throws Exception * / public static void main (String [] args) throws Exception {final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment () Env.setParallelism (4); List tuple3List = DataSource.getTuple3ToList () / / Note: when using Integer for partition, the partition result will be incorrect. Convert to String type output key to correctly output KeyedStream keyedStream = env.fromCollection (tuple3List) .keyby (new KeySelector () {@ Override public String getKey (Tuple3 tuple3) throws Exception {/ / F1) as the gender field, and partition return String.valueOf (tuple3.f1) with the same F1 value (gender). }}); SingleOutputStreamOperator result = keyedStream.fold ("classmates:", new FoldFunction () {@ Override public String fold (String s, Tuple3 tuple3) throws Exception {if (s.startsWith ("male") | | s.startsWith ("female")) {return s + tuple3.f0 + "," } else {return (tuple3.f1.equals ("man")? Male: female) + s + tuple3.f0 + ",";}); result.print (); env.execute ("flink Fold job");}}

Print the result

2 > male students: Zhang San, 2 > male students: Zhang San, Wang Wu, 2 > male students: Zhang San, Wang Wu, Wu Ba, 1 > female students: Li Si, 1 > female students: Li Si, Liu Liu, 1 > female students: Li Si, Liu Liu, Wu Qi, thank you for your reading. These are the contents of "how to use Flink Fold". After the study of this article, I believe you have a deeper understanding of how to use Flink Fold. The specific use situation still needs to be verified by practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.