Shulou(Shulou.com)06/01 Report--

1. Check that the JDK version is the same and copy the JDK, and configure the JAVA_HOME and CLASSPATH properties of the / etc/profile file

two。 Configure / etc/hosts file, add the computer name of the newly added machine, including the existing machine and the new machine, all need to modify this file

3. Copy hadoop-2.6.0 and hbase-1.0.1.1, and change the file owner and file group of the hadoop-2.6.0 and hbase-1.0.1.1 folders to hadoop

4. Configure password-less login from Master to Slave

5. Set the number of machine handles, modify / etc/security/limits.conf, see http://beyond3518.blog.51cto.com/1360525/1764190 for details

6. Install snappy

Installation software: Gcc caterpillar, autoconf, automake, libtool, Java 6, JAVA_HOME set, Maven 3, glibc

Installation method: yum install xxx

6.2 install glibc method:

6.2.1 to 192.168.9.110 / data/server/glibc-2.14.tar.gz copy file, copy the file to the / usr/local directory

6.2.2 enter / usr/local directory, cd / usr/local

6.2.3 decompress glibc-2.14.tar.gz, tar-zxvf glibc-2.14.tar.gz

6.2.4 create a new directory / usr/local/glibc, mkdir / usr/local/glibc

6.2.5 enter the / usr/local/glibc directory, cd / usr/local/glibc

6.2.6 run the command export CFLAGS= "- g-O2"

6.2.7 run the command / usr/local/glibc-2.14/configure-prefix=/usr-disable-profile-enable-add-ons-with-headers=/usr/include-with-binutils=/usr/bin

6.2.8 run the command make

6.2.9 run the command make install

6.2.10 verify that the installation is successful:

Run the command: ll / lib64/libc.so.6. If the result / lib64/libc.so.6-> libc-2.14.so is displayed, the installation is successful

7. Modify hadoop's slaves configuration file, add a new node, and send the profile to all other nodes

8. Add Datanode

For the newly added Datanode node, you need to start the datanode process to add it to the cluster

8.1 on the new node, run sbin/hadoop-daemon.sh start datanode to start the datanode process

8.2 on the new node, run sbin/yarn-daemon.sh start nodemanager to start the nodemanager process

8.3 then check the cluster status through hdfs dfsadmin-report in namenode

Finally, hdfs load balancing needs to be set, because the default data transmission bandwidth is relatively low, which can be set to 64m, that is, hdfs dfsadmin-setBalancerBandwidth 67108864.

8.5 the default threshold of balancer is 10%, that is, the difference in total storage utilization between each node and the cluster is no more than 10%. We can set it to 5%.

8.6 then start Balancer,sbin/start-balancer.sh-threshold 5 and wait for the cluster self-balancing to complete.

9. Add Nodemanager

Because Hadoop 2.x introduces the YARN framework, each compute node can be managed through NodeManager. Similarly, after starting the NodeManager process, it can be added to the cluster.

9.1 when you add a new node, you can run sbin/yarn-daemon.sh start nodemanager

9.2 in ResourceManager, check the cluster situation through yarn node-list

10. Modify the regionservers configuration file of hbase to add new nodes. And send the configuration file to all other nodes

11. On the new node, start regionserver. Exe. Just run hbase-daemon.sh start regionserver.

twelve。 Modify the / etc/hosts file of the InputService/StoreService server to add the domain name of the new node

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.