Shulou(Shulou.com)06/01 Report--

Editor to share with you how to install Flink in the Linux system, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Introduction to Flink: in the current era of the proliferation of Internet users, devices, services, etc., the amount of data generated is no longer the same. A large amount of data will be generated in a variety of business scenarios, and how to deal with these data effectively is a problem that many enterprises need to consider. Frameworks such as Map Reduce,Storm,Spark that we are familiar with in the past may not be able to fully meet the needs of users in some scenarios, or the cost of implementing the requirements, no matter the amount of code or the complexity of the architecture may not be able to meet the expected requirements. The emergence of new scenarios promotes the production of new technologies, and Flink provides a new choice for the processing of real-time streams. Apache Flink is a distributed processing framework that is more active in the community in recent years. Coupled with the promotion of Ali in China, it is believed that it will have more advantages in the future competition. The background of Flink is not introduced too much, those who are interested can Google it. Flink's relatively simple programming model, coupled with its high throughput, low latency, high performance and support for exactly-once semantics, makes it outstanding in industrial production. I believe that as many blog materials have written, "Flink will become the mainstream data processing framework within the enterprise, and eventually become the next generation of big data processing standard."

1. Download and deploy. Flink can run on Linux, MacOS, and Windows systems and requires JDK1.7+ support. This article documented the deployment process on linux (CentOS 7), and JDK1.8 has been installed by default.

Download the binary Flink from the link below and choose the corresponding Hadoop/Scala version according to your favorite Flink version.

Http://flink.apache.org/downloads.html

1.2 deployment the version I chose is flink-1.6.2-bin-hadoop24-scala_2.11.tgz under the itcast directory

[root@itcast01] # wget http://mirrors.hust.edu.cn/apache/flink/flink-1.6.2/flink-1.6.2-bin-hadoop24-scala_2.11.tgz-- 2018-12-03 22-22 root@itcast01 39-http://mirrors.hust.edu.cn/apache/flink/flink-1.6.2/flink-1.6.2-bin-hadoop24-scala_2.11.tgz is parsing the host mirrors.hust .edu.cn (mirrors.hust.edu.cn). 202.114.18.160 mirrors.hust.edu.cn (mirrors.hust.edu.cn) is being connected | 202.114.18.160 |: 80. Connected. A HTTP request has been issued Waiting for response. 200 OK length: 291631283 (278m) [application/octet-stream] is being saved to: "flink-1.6.2-bin-hadoop24-scala_2.11.tgz" 28% [= >] 82004863 642KB/s remaining 9m 7s unzipped to the specified directory 2 [root@itcast01~ ] # tar-zxvf flink-1.3.2-bin-hadoop27-scala_2.10.tgz-C / itcast [root@itcast01 bin] #. / start-cluster.sh # Start Flink is followed by the startup method before version 1.3. Bin/start-local.sh [root@itcast01 bin] #. / stop-cluster.sh stop if you want to stop Flink Use the command: bin/stop-cluster.sh.

1.3 visit the Flink monitoring interface, access the Flink front-end interface, confirm that the system is started, address: http://192.168.1.128:8081 this interface is the result of I have run the test task code, there are jobs tasks

two。 Running WordCount in the field of data processing, WordCount is HelloWorld. Flink comes with WordCount examples that read text data through socket and count the number of occurrences of each word. If you want to see the example source code, please click on the link: SocketWindowWordCount source code.

2.1First of all, start a local monitoring using the nc command, which is:

Nc-l 9000 if the error:-bash: command not found, please install nc first, online installation command: yum-y install nc.

2.2 submit WordCount programs Open a new shell window to submit commands for WordCount programs:

Switch to flink-1.6.2. Do not enter the bin directory to run the code.

[root@itcast01 flink-1.6.2] # bin/flink run examples/streaming/SocketWindowWordCount.jar-- if the port 9000 Starting execution of program is running normally, you can already see the task

2.3 send data in the shell window that launches nc, enter

[root@itcast01 flink-1.6.2] # nc-l 9000 cpu cpu cpu lmc lmc lmc jiao jiao jiao opens a new shell window, go to the / itcasst/ flink-1.6.2/log directory, execute tail-f flink*.out, and you can view the results of word count.

[root@itcast01 log] # tail-f flink-root-taskexecutor-0-itcast01.out cpu: 3 jiao: 3 lmc: 3: 1 [root@itcast01 bin] #. / start-cluster.sh # Start Flink start the service [root@itcast01 bin] #. / stop-cluster.sh stop

The above is all the contents of the article "how to install Flink in Linux system". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.