Shulou(Shulou.com)06/02 Report--

The main content of this article is to explain "what is the role of the computing framework running on YARN". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "what is the role of the computing framework running on YARN"?

1. Benefits of running the framework on YARN

With the maturity and stability of YARN, all kinds of applications can run in a YARN cluster for unified resource management and scheduling, which brings about the following changes:

(1) Application deployment becomes easier

Administrators only need to deploy YARN services. All kinds of application frameworks no longer have their own services and do not need to implement deployment. They have become client programming libraries (library), and the distributed caching mechanism provided by YARN is distributed to each node.

(2) Service deployment becomes simple

Users can deploy a set of services by running an application, such as Storm services, and copying jar packages is done automatically by YARN. After deployment, users use Storm-On-YARN as if they were using a normal Storm cluster.

(3) sharing cluster resources with multiple versions (simple isolation)

Because YARN is only responsible for resource management and scheduling, it is entirely up to users to decide what applications or services to run on it, which allows users to run multiple similar service instances on YARN, such as running multiple Storm instances for different types of applications. YARN itself can provide isolation mechanism (Cgroups) for these instances. With YARN, users don't have to think about resource isolation when developing new frameworks or applications.

(4) flexible management of resources

Because many kinds of applications run in a YARN cluster, such as offline computing, real-time computing, DAG computing, etc., YARN can adjust the corresponding resource usage according to the pressure of different types of applications to achieve flexible resource management.

two。 A computing framework that can currently run on YARN

Frameworks running on YARN, including MapReduce-On-YARN, Spark-On-YARN, Storm-On-YARN and Tez-On-YARN.

(1) offline computing on MapReduce-On-YARN:YARN is included in the YARN distribution. With the stability of YARN, the operation mode of MRv1 will be eliminated.

(2) memory calculation on Spark-On-YARN:YARN

(3) Real-time / streaming computing on Storm-On-YARN:YARN

(4) DAG calculation on Tez-On-YARN:YARN

At this point, I believe you have a deeper understanding of "what is the role of the computing framework running on YARN". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.