Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to use Flink Connect". In daily operation, I believe many people have doubts about how to use Flink Connect. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "how to use Flink Connect". Next, please follow the editor to study!

Connect operator: functions like union, merging two streams (union supports two or more) into one stream, but the difference is that connect does not require consistent data types

Sample environment

Java.version: 1.8.xflink.version: 1.11.1

Sample data source (project code cloud download)

Building Development Environment and data of Flink system example

Connect.java

Import com.flink.examples.DataSource;import org.apache.flink.api.java.tuple.Tuple3;import org.apache.flink.api.java.tuple.Tuple4;import org.apache.flink.streaming.api.datastream.DataStream;import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;import org.apache.flink.streaming.api.functions.co.CoMapFunction;import java.util.Arrays;import java.util.List / * * @ Description Connect operator: similar to union, merging two streams (union supports two or more streams) into one stream, but the difference is that connect does not require consistent data types * / public class Connect {/ * merge two data streams that do not distinguish between data types into one data stream And print * @ param args * @ throws Exception * / public static void main (String [] args) throws Exception {final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment () Env.setParallelism (1); List tuple3List = DataSource.getTuple3ToList (); / / dataStream1 DataStream dataStream1 = env.fromCollection (tuple3List) / / dataStream2 DataStream dataStream2 = env.fromCollection (Arrays.asList ("Doctor", "Shanghai", 2), new Tuple3 ("teacher", "Beijing", 4), new Tuple3 ("worker", "Guangzhou", 9) / / close two data streams DataStream dataStream = dataStream1.connect (dataStream2) .map (new CoMapFunction () {/ / represents the stream input of dataStream1 @ Override public Tuple4 map1 (Tuple3 value) throws Exception {return Tuple4.of (value.f0, value.f1, value.f2, "user") } / / indicates dataStream2 stream input @ Override public Tuple4 map2 (Tuple3 value) throws Exception {return Tuple4.of (value.f0, value.f1, value.f2, "occupation");}}) / / print dataStream.print (); env.execute ("flink Split job");}}

Print the result

(Zhang San, man,20, user) (Li Si, girl,24, user) (Wang Wu, man,29, user) (Liu Liu, girl,32, user) (Wu Ba, girl,18, user) (doctor, Shanghai, 2, occupation) (teacher, Beijing, 4, occupation) (worker, Guangzhou, 9, occupation) this ends the study on "how to use Flink Connect" I hope I can solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.