Shulou(Shulou.com)06/01 Report--

This article mainly introduces the functions of Flink, which have a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

1. Map: converts the data in the data stream to form a new data stream, consumes an element, and produces an element.

Specific code implementation

Package com.wudl.core;import org.apache.flink.api.common.functions.MapFunction;import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment / * @ version v1.0 * @ ProjectName Flinklearning * @ ClassName WordMap * @ Description TODO map operator instance * @ Date 10:15 on 2020-10-29 * / public class WordMap {/ * * @ param args * function usage * Mapping: transforms the data in the data stream to form a new data stream and consumes an element And generate an element * parameter: Lambda expression or new MapFunction implementation class * return value: DataStream * / public static void main (String [] args) throws Exception {StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment () Env.setMaxParallelism (1); env.socketTextStream ("10.204.125.140", 8899) .map (new MapFunction () {@ Override public String map (String s) throws Exception {String [] split = s.split (",") Return split [0] + "- -" + split [1];}}) .print (); env.execute ();}}

2. FlatMap:

Split the whole of the data flow into individual uses, consume one element and produce zero to multiple element package com.wudl.core;import org.apache.flink.api.common.functions.FlatMapFunction;import org.apache.flink.streaming.api.datastream.DataStreamSource;import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;import org.apache.flink.util.Collector;import java.util.Arrays;import java.util.List / * * @ version v1.0 * @ ProjectName Flinklearning * @ ClassName TransformFlatMap * @ Description TODO FlatMap * * FlatMap: a flat mapping that splits the whole of a data stream into individual uses The consumed element produces zero to multiple elements * @ Author wudl * @ Date 10:46 on 2020-10-29 * the function FlatMap * splits the whole of the data stream into one for individual use Consume an element and produce zero to multiple element * parameters: lambda expression or FlatFunction implementation class * return value: DataStream * / public class TransformFlatMap {public static void main (String [] args) throws Exception {StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment () Env.setParallelism (1); / / DataStreamSource listDs = env.fromCollection (Arrays.asList (/ / Arrays.asList (1,2,3), / / Arrays.asList (3,4,5), / / Arrays.asList / / listDs.flatMap (new FlatMapFunction () {/ / @ Override// public void flatMap (List list, Collector collector) throws Exception {/ for (Integer number: list) {/ / collector.collect (number + 100); / /} /} / /}) .print () DataStreamSource strDs = env.socketTextStream ("10.204.125.140", 8899); strDs.flatMap (new FlatMapFunction () {@ Override public void flatMap (String s, Collector collector) throws Exception {String [] split = s.split (","); collector.collect (split [0] + split [1]);}}) .print () Env.execute ();}}

Third: Filter filters the data stream according to the specified rules, the data that meets the condition (true) will be retained, and the data that does not hide the condition (false) will be discarded.

Package com.wudl.core;import org.apache.flink.api.common.functions.FilterFunction;import org.apache.flink.streaming.api.datastream.DataStream;import org.apache.flink.streaming.api.datastream.DataStreamSource;import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment / * @ version v1.0 * @ ProjectName Flinklearning * @ ClassName TransformFilter * @ filtering of Description TODO streams * @ Date 10:26 on 2020-11-5 * / public class TransformFilter {/ * function filtering in Filter * filtering: data that meets the criteria (true) is retained according to the specified rules (false) will discard * return value: DataStream * / public static void main (String [] args) throws Exception {/ / 1. Get context environment StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment (); / / 2. Set parallelism env.setParallelism (1); / / 3. Get data flow DataStreamSource SourceDs = env.socketTextStream ("10.204.125.140", 8899); / / 4. Filter data flow DataStream filter = SourceDs.filter (new FilterFunction () {@ Override public boolean filter (String value) throws Exception {String [] split = value.split (","); return split [1] .length () > 3;}}); filter.print (); env.execute () }} Thank you for reading this article carefully. I hope the article "what are the functions of Flink" shared by the editor will be helpful to you. At the same time, I also hope that you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.