Shulou(Shulou.com)06/03 Report--

Environment: CentOS7 64-bit 3

Centos7-1 192.168.190.130 master

Centos7-2 192.168.190.129 slave1

Centos7-3 192.168.190.131 slave2

Hadoop download address: http://mirrors.hust.edu.cn/apache/hadoop/common/ option version 3.1.0

1. Install three virtual machines (centos7) and install jdk to achieve password-free login between nodes.

# install the virtual machine without instructions, you can view the relevant articles. # install jdk, configure JAVA_HOME environment variables, and use Baidu without instructions. # all three virtual machines need to modify hostname, modify / etc/hosts # vim / etc/hostname / / write as master,slave1,slave2, and restart reboot # vim / etc/hosts # 192.168.190.130 master # 192.168.190.129 slave1 # 192.168.190.131 slave2 # set the face password for the three machines ssh login # ssh-keygen-t rsa / / generate key All three machines need to execute the same command

# authorized_keys

# on the touch / root/.ssh/authorized_keys / / master node

# cat / root/.ssh/id_rsa.pub > > / root/.ssh/authorized_keys

# copy the id_rsa.pub on slave1 and slave2 to the master node and append it directly to authorized_keys

# scp-r / root/.ssh/id_rsa.pub root@masterIP:/root/.ssh/slave1.id_rsa.pub / / slave1

# scp-r / root/.ssh/id_rsa.pub root@masterIP:/root/.ssh/slave2.id_rsa.pub / / slave2

# cat / root/.ssh/slave1.id_rsa.pub > > / root/.ssh/authorized_keys / / master

# cat / root/.ssh/slave2.id_rsa.pub > > / root/.ssh/authorized_keys / / master

# authorized_keys is as follows:

# copy the authorized_keys of master to / root/.ssh/ of slave1\ slave2 through the scp command

# scp-r / root/.ssh/authorized_keys root@Slave1IP:/root/.ssh/

# scp-r / root/.ssh/authorized_keys root@Slave2IP:/root/.ssh/

# ssh target domain names / / master, slave1, slave2, you need to confirm that you enter yes for the first time, and then you don't need to log in without a password.

two。 Load hadoop and extract the tar.gz file

# cd / usr # mkdir hadoop # cd hadoop # wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-3.1.0.tar.gz # tar-zxvf hadoop-3.1.0.tar.gz after decompression, the hadoop-3.1.0 working directory appears, which can be used. # cd / usr/hadoop/hadoop-3.1.0 # mkdir tmp #. / bin/hadoop-version / / verify whether it can be used

two。 Set environment variables, global (/ etc/profile)

# vim / etc/profile set HADOOP_HOME and add it to HADOOP_HOME=/usr/hadoop/hadoop-3.1.0 PATH=$PATH:$HADOOP_HOME/bin # source / etc/profile under the PATH path

3. Configure hadoop

# hadoop main file, hadoop-env.sh / / core-site.xml / / hdfs-site.xml / / mapred-site.xml / / yarn-site.xml / / workers / / just configure the namenode node under the path hadoop-3.1.0/etc/hadoop/ URI used to discover slave node a.hadoop-env.sh configuration export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk-1.8.0.171-0.b13.el7_3.x86_64 b.core-site.xml configuration fs.default.name hdfs://master:9000 HDFS File system: / / namenode ID: local temporary hadoop folder c. Usr/hadoop/tmp namenode on port number hadoop.tmp.dir / hadoop. Configure the number of copies of hdfs-site.xml dfs.replication 1. The default configuration is 3, which should be less than the number of datanode machines d. Configure mapred-site.xml mapreduce.framework.name

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.