Shulou(Shulou.com)06/02 Report--

This article mainly introduces "how to run Flink tasks on IDEA". In daily operation, I believe many people have doubts about how to run Flink tasks on IDEA. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts of "how to run Flink tasks on IDEA". Next, please follow the editor to study!

Welcome to my GitHub

Https://github.com/zq2599/blog_demos

Content: classification and summary of all original articles and supporting source code, involving Java, Docker, Kubernetes, DevOPS, etc.

IDEA is a commonly used IDE. If the flink task code we write can be run directly in IDEA, it will bring great convenience to learning and development, such as no deployment, breakpoints, single-step debugging and so on.

Environmental information

Computer: 2019 version 13-inch MacBook Pro,2.3 GHz quad-core Intel Core i5jing8 GB 2133 MHz LPDDR3

Operating system: macOS Catalina 10.15.3

JDK:1.8.0_211

Maven:3.6.0

IDEA:2018.3.5 (Ultimate Edition)

Flink version: 1.9.2

About authentic IDEA

How to use genuine IDEA for free, you can refer to the "Free Application and use of IntelliJ IDEA Commercial License Guide"

Development

First, make sure that maven is installed on your computer.

Create a maven project for the flink task, find a clean directory and execute the following command:

Mvn\ archetype:generate\-DarchetypeGroupId=org.apache.flink\-DarchetypeArtifactId=flink-quickstart-java\-DarchetypeVersion=1.9.2

Enter groupId, artifactId, version, package, etc., as prompted:

4. Now that the maven project has been generated, import the project with IDEA, as shown below:

5. Import as maven:

6. What it looks like after a successful import:

7. Modify StreamingJob.java, add the function is to listen to the local port 18081, the resulting string to do word count operation, the complete code can be downloaded here: https://raw.githubusercontent.com/zq2599/blog_demos/master/files/StreamingJob.java

8. Now that the code is running, but the flink page is not accessible yet, the following error message will be displayed and some settings need to be made:

9. Download the flink-1.9.2 installation package, which contains a jar used by the flink web service at https://www-eu.apache.org/dist/flink/flink-1.9.2/flink-1.9.2-bin-scala_2.11.tgz 10. Flink-1.9.2-bin-scala_2.11.tgz is downloaded and decompressed, and there is a flink-dist_2.11-1.9.2.jar file in the lib directory Remember the location of this file, which will be used later 11. Go back to IDEA, right-click on the project, and click the menu Open Module Settings:

twelve。 Do the following in the pop-up window to add jar to the project:

13. In a window that pops up to select a file, select the file flink-dist_2.11-1.9.2.jar that you just prepared:

14. The setting work has been completed, because the job of StreamingJob is to read the data of local port 18081, so we need to start the service of port 18081, otherwise the StreamingJob will not be connected to the port. Open a console and execute the command: nc-l 18081 15. Now you can run StreamingJob, as shown in the following figure. Right-click StreamingJob and select Run 'StreamingJob.main ()': to start the flink task. If you want to interrupt debugging, select Debug 'StreamingJob.main ()'.

16. Go back to the console where you just executed the nc command, enter some strings, such as "aaa bbb aaa", and enter enter; 17. Going back to IDEA, you can see that StreamingJob has read the data from port 18081 and output the statistical results:

18. Search for the keyword localhost in the IDEA console. The red box below shows the port of the flink web page. Here is 62641:

19. The browser accesses http://localhost:62641, as shown in the following figure, which shows that the flink web page has been displayed normally, and the running tasks can also be seen:

So far, the simplest IDEA to run the flink task is completed. If you are also learning flink, I hope this article can give you some reference.

At this point, the study on "how to run Flink tasks on IDEA" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.