Shulou(Shulou.com)06/01 Report--

This article mainly explains "how to use Split in Flink". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "how to use Split in Flink".

Split operator: split a data stream into multiple data streams (obsolete and cannot be split twice, not recommended)

Sample environment

Java.version: 1.8.xflink.version: 1.11.1

Sample data source (project code cloud download)

Building Development Environment and data of Flink system example

Split.java

Package com.flink.examples.functions;import com.flink.examples.DataSource;import org.apache.flink.api.common.functions.MapFunction;import org.apache.flink.api.java.tuple.Tuple3;import org.apache.flink.api.java.tuple.Tuple4;import org.apache.flink.streaming.api.collector.selector.OutputSelector;import org.apache.flink.streaming.api.datastream.DataStream;import org.apache.flink.streaming.api.datastream.SplitStream;import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;import java.util.ArrayList Import java.util.List / * * @ Description Split operator: split the data stream into multiple data streams (outdated and cannot be split twice, so it is not recommended) * / public class Split {/ * traverse the collection, split the data stream into multiple streams and print * @ param args * @ throws Exception * / public static void main (String [] args) throws Exception {final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment () Env.setParallelism (1); List tuple3List = DataSource.getTuple3ToList (); / / Datastream DataStream dataStream = env.fromCollection (tuple3List); / / split by gender / / flink.1.11.1 shows that SplitStream class is obsolete. It is recommended to use keyBy for window processing or SideOutput side output stream processing. Note that the split stream using split cannot be split twice, otherwise an exception SplitStream split = dataStream.split (new OutputSelector () {@ Override public Iterable select (Tuple3 value) {List output = new ArrayList (); if (value.f1.equals ("man")) {output.add ("man")) will be thrown. } else {output.add ("girl");} return output;}}) / / query the data stream with the specified name DataStream dataStream1 = split.select ("man") .map (new MapFunction () {@ Override public Tuple4 map (Tuple3 T3) throws Exception {return Tuple4.of (t3.f0, t3.f1, t3.f2, "male") }}); DataStream dataStream2 = split.select ("girl") .map (new MapFunction () {@ Override public Tuple4 map (Tuple3 T3) throws Exception {return Tuple4.of (t3.f0, t3.f1, t3.f2, "female") }); / / print: male dataStream1.print (); / / print: female dataStream2.print (); env.execute ("flink Split job");}}

Print the result

(Zhang San, man,20, male) (Li Si, girl,24, female) (Wang Wu, man,29, male) (Liu Liu, girl,32, female) (Wu Qi, girl,18, female) (Wu Ba, man,30, male) Thank you for your reading. This is the content of "how to use Flink Split". After studying this article, I believe you have a deeper understanding of how to use Flink Split. The specific use situation still needs to be verified by practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.