Shulou(Shulou.com)05/31 Report--

This article will explain in detail how to use Flink for you. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

Flink installation preparation

Flink runs on mainstream platforms such as Linux, Apple and Windows. But it's best to use Linux. The preparation before installation is given below:

Install Jdk1.7.X or above

Download the corresponding Hadoop precompiled version on the Flink official website.

Decompress the precompiled version and enter the unzipped file. For convenience, this directory is called FLINK_HOME later.

Start a quick attempt to install a single machine

It is very easy to try a stand-alone machine and execute the command directly:

Linux user: sh bin/start-local.sh

Windows user, enter: bin\ start-local.bat in the command window

Wait for the following prompt:

D:\ Java\ flink\ flink-0.10.1 > bin\ start-local.batStarting Flink job manager. Webinterface by default on http://localhost:8081/.Don't close this batch window. Stop job manager by pressing Ctrl+C.

Enter in the browser: http://localhost:8081/ http://localhost:8081/ Flink listens on port 8081 by default to prevent other processes from occupying this port. The following administrative interface appears:

You can find that the logic of this interface is similar to that of Spark's management interface, mainly managing running Job, completed Job, and Task management and Job management. Task should manage Job, and then carefully analyze the logic inside.

The first example of running

I can't wait to run a classic example of a distributed system: WordCount. Take the README.txt file of FLINK_HOME as an example file, test the WordCount program, run the code on Windows and run the process as shown below:

D:\ Java\ flink\ flink-0.10.1 > bin\ flink.bat run.\ examples\ WordCount.jar file:/D:/Java/flink/flink-0.10.1/README.txt file:/D:/Java/flink/flink-0.10.1/wordcount-result.txtlog4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory) .log4j:WARN Please initialize the log4j system properly.log4j:WARN See http://logging.apache.org/log4j/1 .2 / faq.html#noconfig for more info.01/15/2016 16:30:51 Job execution switched to status RUNNING.01/15/2016 16:30:51 CHAIN DataSource (at getTextDataSet (WordCount.java:142) (org.apache.flink.api.java.io.TextInputFormat))-> FlatMap (FlatMap at main (WordCount.java:69))-> Combine (SUM (1)) At main (WordCount.java:72) (1amp 1) switched to SCHEDULED01/15/2016 16:30:51 CHAIN DataSource (at getTextDataSet (WordCount.java:142) (org.apache.flink.api.java.io.TextInputFormat))-> FlatMap (FlatMap at main (WordCount.java:69))-> Combine (SUM (1)) At main (WordCount.java:72) (1Compact 1) switched to DEPLOYING01/15/2016 16:30:52 CHAIN DataSource (at getTextDataSet (WordCount.java:142) (org.apache.flink.api.java.io.TextInputFormat))-> FlatMap (FlatMap at main (WordCount.java:69))-> Combine (SUM (1), at main (WordCount.java:72) (1gamble 1) switched to RUNNING01/15/2016 16:30:52 Reduce (SUM (1) At main (WordCount.java:72) (1) switched to SCHEDULED01/15/2016 16:30:52 Reduce (SUM (1), at main (WordCount.java:72) (1) switched to DEPLOYING01/15/2016 16:30:52 CHAIN DataSource (at getTextDataSet (WordCount.java:142) (org.apache.flink.api.java.io.TextInputFormat))-> FlatMap (FlatMap at main (WordCount.java:69))-> Combine (SUM (1)) At main (WordCount.java:72) (1) switched to FINISHED01/15/2016 16:30:52 Reduce (SUM (1), at main (WordCount.java:72) (1) switched to RUNNING01/15/2016 16:30:53 DataSink (CsvOutputFormat (path: file:/D:/Java/flink/flink-0.10.1/wordcount-result.txt) Delimiter:) (1) switched to SCHEDULED01/15/2016 16:30:53 DataSink (CsvOutputFormat (path: file:/D:/Java/flink/flink-0.10.1/wordcount-result.txt, delimiter:)) (1) switched to DEPLOYING01/15/2016 16:30:53 Reduce (SUM (1)) At main (WordCount.java:72) (1amp 1) switched to FINISHED01/15/2016 16:30:53 DataSink (CsvOutputFormat (path: file:/D:/Java/flink/flink-0.10.1/wordcount-result.txt, delimiter:)) (1Accord 1) switched to RUNNING01/15/2016 16:30:53 DataSink (CsvOutputFormat (path: file:/D:/Java/flink/flink-0.10.1/wordcount-result.txt) Delimiter:) (1) switched to FINISHED01/15/2016 16:30:53 Job execution switched to status FINISHED.

You can see that the output log is very detailed, and it is convenient to understand the whole running process. The first 10 items of the output file wordcount-result.txt are as follows:

1113 15d002 1740 1about 1account 1administration 1algorithms 1and 7another 1any 2 this is the end of the article on "how to use Flink". I hope the above content can be helpful to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.