Shulou(Shulou.com)06/03 Report--

1. Cluster environment

Master 192.168.2.100

Slave1 192.168.2.101

Slave2 192.168.2.102

two。 Download the installation package

# Master

Wget http://mirrors.shu.edu.cn/apache/hadoop/common/hadoop-2.8.4/hadoop-2.8.4.tar.gz

Tar zxvf hadoop-2.8.4.tar.gz

3. Modify Hadoop configuration file

# Master

Cd hadoop-2.8.4/etc/hadoop

Vim hadoop-env.sh

Export JAVA_HOME=/usr/local/src/jdk1.8.0_152

Vim yarn-env.sh

Export JAVA_HOME=/usr/local/src/jdk1.8.0_152

Vim slaves

Slave1

Slave2

Vim core-site.xml

Fs.defaultFS hdfs://master:9000 hadoop.tmp.dir file:/usr/local/hadoop-2.8.4/tmp

Vim hdfs-site.xml

Dfs.namenode.secondary.http-address master:9001 dfs.namenode.name.dir file:/usr/local/hadoop-2.8.4/dfs/name dfs.datanode.data.dir file:/usr/local/hadoop-2.8.4/dfs/data Dfs.repliction 3

Vim mapred-site.xml

Mapreduce.framework.name yarn

Vim yarn-site.xml

Yarn.nodemanager.aux-services mapreduce_shuffle yarn.nodemanager.aux-services.mapreduce.shuffle.class org.apache.hadoop.mapred.ShuffleHandler yarn.resourcemanager.address master:8032 yarn.resourcemanager.scheduler.address Master:8030 yarn.resourcemanager.resource-tracker.address master:8035 yarn.resourcemanager.admin.address master:8033 yarn.resourcemanager.webapp.address master:8088

# create temporary directories and file directories

Mkdir / usr/local/hadoop-2.8.4/tmp

Mkdir-p / usr/local/hadoop-2.8.4/dfs/name

Mkdir-p / usr/local/hadoop-2.8.4/dfs/data

4. Configure environment variables

# Master 、 Slave1 、 Slave2

Vim / .bashrc

HADOOP_HOME=/usr/local/src/hadoop-2.8.2

Export PATH=$PATH:$HADOOP_HOME/bin

# refresh environment variables

Source / .bashrc

5. Copy installation package

# Master

Scp-r / usr/local/src/hadoop-2.8.4 root@slave1:/usr/local/src/hadoop-2.8.4

Scp-r / usr/local/src/hadoop-2.8.4 root@slave2:/usr/local/src/hadoop-2.8.4

6. Start the cluster

# Master

# initialize Namenode

Hadoop namenode-format

# start the cluster

. / sbin/start-all.sh

6. Cluster status

Jps

# Master

# Slave1

# Slave2

7. Monitoring web pages

Http://master:8088

test

Shut down the cluster

. / sbin/hadoop stop-all.sh

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.