Shulou(Shulou.com)06/01 Report--

This article mainly introduces what is the use of Spark-submit script, the article is very detailed, has a certain reference value, interested friends must read it!

The submission of the spark program is achieved through the spark-submit script, from which we unravel the steps of the spark submission cluster step by step.

Spark-submit 's main command line: exec "${SPARK_HOME}" / bin/spark-class org.apache.spark.deploy.SparkSubmit "$@"

Is to execute the spark-class script with the spark.deploy.SparkSubmit class as the first parameter.

1 、 spark-class

The most important thing is the following:

CMD= () while IFS= read-d ''- r ARG; do CMD+= ("$ARG") done <

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.