Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to install Spark 1.5.1, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

1. Hadoop 2.x installation

Hadoop 2.x install http://my.oschina.net/u/204498/blog/519789

II. Spark1.5.1 installation

1. Download spark1.5.1

Http://spark.apache.org/downloads.html

Select the version of spark

[hadoop@hftclclw0001 ~] $pwd/home/hadoop [hadoop@hftclclw0001 ~] $wget [hadoop@hftclclw0001 ~] $lltotal 480004drwxr-xr-x 11 hadoop root 4096 Jan 17 04:54 hadoop-2.7.1-rw- 1 hadoop root 210606807 Jan 17 04:09 hadoop-2.7.1.tar.gzdrwxr-xr-x 13 hadoop root 4096 Jan 18 08:31 spark-1.5.1-bin-hadoop2.6-rw- 1 hadoop root 280901736 Jan 17 04:08 spark-1.5.1-bin-hadoop2.6.tgz [hadoop@hftclclw0001 conf] $pwd/home/hadoop/spark-1.5.1-bin-hadoop2.6/conf [hadoop@hftclclw0001 conf] $cp slaves.template slaves-p [hadoop@hftclclw0001 conf] $vi slaves = > slaves node hfspark0003.webex.com hfspark0007.webex.com [hadoop@hftclclw0001 conf] $cp spark-env.sh.template spark- Env.sh-p [hadoop@hftclclw0001 conf] $vi spark-env.sh = > configure the spark environment variable... export SPARK_CLASSPATH=$SPARK_CLASSPATH:/home/hadoop/spark-1.5.1-bin-hadoop2.6/lib/mysql-connector-java-5.1.25-bin.jar:/home/hadoop/spark-1.5.1-bin-hadoop2.6/lib/ojdbc6.jar = > use spark sql...export HADOOP_HOME=/home/ Hadoop/hadoop-2.7.1 export HADOOP_CONF_DIR=/home/hadoop/hadoop-2.7.1/etc/hadoopexport SPARK_MASTER_IP=hftclclw0001.webex.comexport SPARK_WORKER_MEMEORY=4gexport JAVA_HOME=/usr/java/...

Copy it to other machines

[hadoop@hftclclw0001 ~] $pwd/home/hadoop [hadoop@hftclclw0001 ~] $scp-r spark-1.5.1-bin-hadoop2.6 hadoop@ {ip}: / home/hadoop

III. Start

[hadoop@hfspark0003 spark-1.5.1-bin-hadoop2.6] $. / sbin/start-all.sh.

IV. Verification

A. Jps = = > master,worker

B.webui = = > http://${ip}:8080

Thank you for reading this article carefully. I hope the article "how to install Spark 1.5.1" shared by the editor will be helpful to you. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.