Shulou(Shulou.com)06/02 Report--

How to build a hadoop cluster in centos8? in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Modify hosts vim / etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4:: 1 localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.1.107 dsj2020master 192.168.1.108 dsj2020slave1 192.168.1.109 dsj2020slave2

Change hostname. Vim / etc/sysconfig/network

NETWORKING=yes HOSTNAME=master

Configure static IP. Vim / etc/sysconfig/network-scripts/ifcfg-ethx

BOOTPROTO=static ONBOOT=yes IPADDR=192.168.1.101 NETMASK=255.255.255.0 GATEWAY=192.168.1.1

Restart the network

Nmcli c reload

Java

Wget https://repo.huaweicloud.com/java/jdk/8u201-b09/jdk-8u201-linux-x64.rpm

Rpm-ivh jdk-8u201-linux-x64.rpm

Hadoop

Wget https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-2.9.2/hadoop-2.9.2.tar.gz

Tar-zxvf hadoop-2.9.2.tar.gz

Mv hadoop-2.9.2.tar.gz hadoop

Add environment variables for hadoop

#

Export HADOOP_HOME=/home/user1/software/hadoop export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

Cd / soft/hadoop/etc/hadoop/-- Edit core-site.xml fs.defaultFS hdfs://master:9000-- Edit hdfs-site.xml dfs.replication 3

-- Edit slaves file dsj2020master dsj2020slave1 dsj2020slave2-- copy mapred-site.xml.template cp mapred-site.xml.template mapred-site.xml-- Edit mapred-site.xml mapreduce.framework.name yarn- Edit yarn-site.xml yarn.nodemanager.aux-services mapreduce_shuffle yarn.resourcemanager.resource-tracker.address master

Add the JAVA_HOME environment variable to hadoop-env.sh

JAVA_HOME=/soft/bin

Copy the past

Scp-r hadoop user2@slave1:~/software/

Secret-free landing

Ssh-keygen-t rsa ssh-copy-id user@master ssh-copy-id user@slave1 ssh-copy-id user@slave2

JAVA_HOME=/root/jdk JRE_HOME=/root/jdk/jre CLASSPATH=.:$JAVA_HOME/lib:/dt.jar:$JAVA_HOME/lib/tools.jar PATH=$PATH:$JAVA_HOME/bin

Server.1=dsj2020master:2888:3888 server.2=dsj2020slave1:2888:3888 server.3=dsj2020slave2:2888:3888

This is the answer to the question about how to build a hadoop cluster in centos8. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel to learn more about it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.