Shulou(Shulou.com)06/03 Report--

Hardware and software planning ID host type hostname IP application software operating system hardware configuration 1 physical machine namenode01192.168.1.200jdk1.8.0_171, hadoop-2.7.3, hive-2.3.3, hbase-1.2.6, zookeeper-3.4.10, spark-2.3.1CentOS7.46 core CPU, 8G memory, 250memory 2physical machine datanode01192.168.1.201jdk1.8.0_171, hadoop-2.7.3, hive-2.3.3, Hbase-1.2.6, zookeeper-3.4.10, spark-2.3.1CentOS7.46 core CPU, 8G memory, 250storage 3 physical machines datanode02192.168.1.202jdk1.8.0_171, hadoop-2.7.3, hive-2.3.3, hbase-1.2.6, zookeeper-3.4.10, spark-2.3.1CentOS7.46 core CPU, 8G memory, 250storage 4physical machines datanode03192.168.1.203jdk1.8.0_171, hadoop-2.7.3, Hive-2.3.3, hbase-1.2.6, zookeeper-3.4.10, spark-2.3.1CentOS7.46 core CPU, 8GB of memory, 250g of storage

The role I plan here is not very good. If there are four machines, I suggest you assign them according to our below.

ZooKeeper

ZooKeeper Server:namenode01HDFS

NameNode:namenode01

SecondaryNameNode:namenode01

DataNode:datanode [01-03] YARN

ResourceManager:namenode01

NodeManager:datanode [01-03] HBase

Master:namenode01

RegionServer:datanode [01-03] Hive

HiveServer2:namenode01

Hive Metastore Server:namenode01Spark

Master:namenode01

Worker:datanode [01-03] II. Hostname resolution

Modify the hostnames of the four servers.

192.168.1.200 namenode01 hbase01 spark01 hive01 cluster1192.168.1.201 datanode01 hbase02 spark02 zk01 mysql01192.168.1.202 datanode02 hbase03 spark03 zk02192.168.1.203 datanode03 hbase04 spark04 zk03

3. Configure SSH mutual trust

Operate on top of the namenode01 node.

Ssh-keygen-t rsassh-copy-id namenode01ssh-copy-id datanode01ssh-copy-id datanode02ssh-copy-id datanode03 IV. Create users

Create a user (4 machines operate the same)

Useradd hadoop & & echo '123456' | passwd-- stdin hadoop

Use the command visudo to set user permissions (the same operation on all four machines) and add the following:

Hadoop ALL= (ALL) ALL V. Installation of JDK

To download JDK1.8, you can choose to download the rpm package and install it directly.

Https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.