Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to build the development environment and data in Flink. The content of the article is of high quality, so the editor will share it with you for reference. I hope you will have some understanding of the relevant knowledge after reading this article.

Sample environment

Java.version: 1.8.xflink.version: 1.11.1

Create a project

Create a project in idea. Project name: flink-examples

Pom.xml

4.0.0 com.flink.examples flink-examples pom 1.0-SNAPSHOT stream connectors UTF-8 UTF-8 1.8 1.11.1 8 org.apache.flink flink-streaming-java_2.11 ${flink.version} Org.apache.flink flink-clients_2.11 ${flink.version} org.apache.flink flink-table-api-java-bridge_2.11 ${flink.version} org.apache.flink flink-table-planner-blink_2.11 ${flink.version} org.apache.flink flink-table-planner_2.11 ${flink.version} org.apache.flink flink-json ${flink.version} com.google.code. Gson gson 2.8.6 org.apache.logging.log4j log4j-core 2.11.1 org.apache.maven.plugins maven-compiler-plugin UTF-8 ${compiler.version} ${compiler.version} org.apache.maven.plugins maven-shade-plugin package Shade

The project is divided into two modules.

Flink-examples |-connectors (sample module for inter-component connectors) |-examples (simulating e-commerce order data and pushing it to kafka, and flink core data stream processing client) |-stream (data flow and operator, method, window and other sample modules) |-tableapi (example module between table&sql and middleware) |-web (get the storage result calculated by flink operator Provided to the front-end display module)

Connectors module

Pom.xml

Com.flink.examples flink-examples 1.0-SNAPSHOT 4.0.0 connectors org.apache.flink flink-jdbc_2.11 1.10.1 mysql mysql-connector-java 5.1.47 Org.apache.flink flink-connector-kafka_2.11 ${flink.version} org.apache.bahir flink-connector-redis_2.11 1.0 org.apache.flink flink-connector-rabbitmq_2.11 ${flink.version} org.apache.flink flink-connector-elasticsearch7_2.11 ${flink.version} org.apache.flink flink-sql-connector-elasticsearch7_2.11 ${flink.version}

Stream module

Pom.xml

Flink-examples com.flink.examples 1.0-SNAPSHOT 4.0.0 stream org.apache.flink flink-table-common ${flink.version}

Project global sample diagram

Sample data class

The data demonstrated by all subsequent operators come from this class to provide public data.

Import org.apache.flink.api.java.tuple.Tuple3;import java.util.Arrays;import java.util.List / * * @ Description Public sample data * / public class DataSource {/ * sample data collection * Tuple3 is an entity class with fixed three attribute variables F0 return f1jue f2 is used to denote three structural transmission participating variables * @ return * / public static List getTuple3ToList () {/ / Tuple3 = Tuple3 return Arrays.asList (new Tuple3 ("Zhang San", "man", 20), new Tuple3 ("Li Si", "girl", 24), new Tuple3 ("Wang Wu", "man", 29). New Tuple3 (Liu Liu, girl, 32), new Tuple3 (Wu Qi, girl, 18), new Tuple3 (Wu Ba, man, 30)) }} on how to build a development environment in Flink and data sharing here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.