Shulou(Shulou.com)06/02 Report--

This article mainly explains "what is the spark deployment model". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn what the spark deployment model is.

Spark has local mode, pseudo-distributed, standalone,spakr on yarn,spark on messos.

Local pattern is mostly used for development and testing.

Pseudo-distribution is a rarely used pattern, which simulates distribution on a node.

Spakr on messos is seldom used in China.

Standalone mode is suitable for use when the data scale is small, the big data computing framework deployed in the cluster is single, and the entire cluster computing resources can be provided to spark computing without considering the problem of resource allocation.

Spark on yarn is widely used in China. It has the following advantages: 1. Avoid building spark clusters. Spark only needs to be deployed on one node, through which the application is submitted to the yarn cluster for processing. In the case of having a yarn cluster, it is convenient to run spark applications 2. 5 without building a spark cluster. Unified resource management and scheduling. Yarn is the implementation of resource management and scheduling of many kinds of big data computing engines, including mr and spark. Yarn has a resource isolation mechanism, so that all kinds of big data computing tasks can run stably in the cluster without resource competition. 3. Flexible management of resources. Through queuing, yarn flexibly schedules resource usage according to the resource pressure of different types of applications.

Thank you for reading, the above is the content of "what is the spark deployment model", after the study of this article, I believe you have a deeper understanding of what the spark deployment model is, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.