Shulou(Shulou.com)06/01 Report--

This article mainly shows you "how to build a cluster in Hadoop". The content is easy to understand and clear. I hope it can help you solve your doubts. Let me lead you to study and learn this article "how to build a cluster in Hadoop".

Main environment:

CentOS6.4

JDK 1.6

Hadoop 1.2.1

Use three machines to build.

Master (NameNode, SecondaryNameNode, JobTracker): 10.0.0.13

Slave (DataNode, TaskTracker): 10.0.0.15

Slave (DataNode, TaskTracker): 10.0.0.16

1. All three machines are configured with / etc/hosts files

two。 Configure the ssh password-free connection to run ssh-keygen-t rsa on each machine, followed by the

Id_rsa.pub is merged into authorized_keys and copied to each machine

3. Configure the java environment variable in conf/hadoop-env.sh under the hadoop directory

4. Modify hdfs-site.xml modify the number of copies of data blocks to be copied in hdfs

5. Modify mapred-site.xml configuration jobtracker

6. Modify the core-site.xml specified data directory (create the / usr/hadoop/tmp directory using mkdir)

7. Modify masters and slaves

8. Use the scp command to copy the hadoop directory to each node

9. And configure the hadoop environment variable on each node (edit / etc/profile take effect immediately source / etc/profile)

10. Formatting

11. Start hadoop and run start-all.sh

twelve。 Finally, use jps to verify whether the appropriate java process is started

Master:

Slave:

The above is all the contents of the article "how to build a cluster in Hadoop". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.