Shulou(Shulou.com)06/01 Report--

How to analyze KeyBy in Flink, for this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more small partners who want to solve this problem find a simpler and easier way.

KeyBy operator: partition (group) the data stream according to the specified key

example environment

java.version: 1.8.xflink.version: 1.11.1

Sample Data Source (Project Code Cloud Download)

Flink System Example: Building Development Environment and Data

KeyBy.java

package com.flink.examples.functions;import com.flink.examples.DataSource;import org.apache.flink.api.java.functions.KeySelector;import org.apache.flink.api.java.tuple.Tuple3;import org.apache.flink.streaming.api.datastream.DataStream;import org.apache.flink.streaming.api.datastream.KeyedStream;import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;import java.util.List;/** * @ KeyDescription By operator: partition the data stream according to the specified key */public class KeyBy { /** * Traversing the collection, separating users into two categories by gender * @param args * @throws Exception */ public static void main(String[] args) throws Exception { final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); //If there are multiple partitions, set parallelism greater than 1, or set setParallelism(2) on the operator, otherwise the operator has only one parallelism, and the calculation result always has only one partition// env.setParallelism(4); List tuple3List = DataSource.getTuple3ToList(); DataStream dataStream = env.fromCollection(tuple3List); //Note: When using Integer for partitioning, the partitioning result will be incorrect. Convert it to String type and output key correctly. KeyedStream keyedStream = dataStream.keyBy(new KeySelector() { @Override public String getKey(Tuple3 tuple3) throws Exception { //f1 is the gender field, partitioned with the same f1 value (gender) return String.valueOf(tuple3.f1); } }); //lambda// KeyedStream keyedStream = dataStream.keyBy((KeySelector) t3 -> t3.f1); //Specify which field is used as key to calculate// KeyedStream keyedStream = dataStream.keyBy(1); keyedStream.print().setParallelism(4); env.execute("flink keyBy job"); }}

print result

2> (Zhang San,man,20)4> (Li Si,girl,24)2> (Wang Wu,man,29)4> (Liu Liu,girl,32)2> (Wu Ba,man,30)4> (Wu Qi,girl,18) About how to analyze the KeyBy problem in Flink to share here, I hope the above content can be of some help to everyone, if you still have a lot of doubts, you can pay attention to the industry information channel to learn more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.