Shulou(Shulou.com)06/01 Report--

This article mainly introduces "what are the running modes of Flink submission jobs". In the daily operation, I believe that many people have doubts about the running mode of Flink submission jobs. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "what is the mode of Flink submission jobs running?" Next, please follow the editor to study!

0. Background

Based on Flink-1.12.x

1.Session Mode

Needless to say, there is a session, and then multiple programs will be submitted to this session.

Benefits: cluster resources are allocated only once, making full use of resources, program App starts faster

Disadvantages: there may be a chain restart, and the jobManager load is heavy.

2.Per-Job Mode

Use a lot, one application and one flink cluster

Benefits: resource isolation, finer granularity, and ease of managing a single job

Disadvantages: when there is more than one client on a machine, there will be a higher network load (download jar, transmit jar) and consume a large amount of CPU to execute the main method

3.Application Mode

Application Mode is similar to Per-Job Mode, mainly to solve the shortcomings of Per-Job Mode, through yarn.provided.lib.dirs

In addition, client is executed on JobManager, which can avoid the hot issues of bandwidth and CPU.

And compared to Per-Job Mode, it is more powerful and can submit multiple job

At this point, the study on "what are the modes for Flink to submit homework to run" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.