Shulou(Shulou.com)06/02 Report--

This article will explain in detail the sample analysis of Hadoop cluster construction for everyone. Xiaobian thinks it is quite practical, so share it with you for reference. I hope you can gain something after reading this article.

Hadoop cluster building

I. Linux system configuration

1. Create the same directory on all machines, or you can create the same user and use the user's home path to do the installation path of hadoop. For example, a user named fan is created on all machines: /home/fan.

2. Add IP-host mapping relationship at the end of Etc/hosts file, for example:

192.168.146.161fan001

192.168.146.162fan002

192.168.146.163fan003

192.168.146.164fan004

II. Uninstall and install jdk in RedHat when building Hadoop cluster

(i) jdk1.4 unloading

Since redhat Enterprise 5 comes with jdk1.4 installed, so before installing jdk1.6 I removed jdk1.4, the steps are as follows:

1. Open the terminal and enter #rpm-qa| grepgcj, its real content is:

java-1.4.2-gci-compat.........

java-1.4.2-gcj-compat-devel..... (I don't know what it is, but there are two Java files.)

2. Unloading

#rpm-e--nodepsjava-1.4.2-gci... (Use the rpm-e--nodeps command to delete the contents found above) jdk1.4 has been uninstalled at this time.

jdk1.6 Installation

1. Download the jdk1.6 package (jdk-6u18-ea-bin-b04-linux-i586-28_oct_2009.bin) from the website and place it in the/usr directory.

2. In linux, switch to/usr directory and modify jdk1.6 permissions:

#chmodu+xjdk-6u18-ea-bin-b04-linux-i586-28_oct_2009.bin

3. Unzip and install: #./ jdk-6u10-linux-i586-rpm.bin(installed by default in/usr/java) Press ctrl+c to skip prompts

4. Jdk1.6 installation environment variable configuration in Hadoop cluster construction:

File directory: /etc/profile

Add to the profile:

JAVA_HOME=/usr/java/jdk1.6.0_10

JRE_HOME=/usr/java/jdk1.6.0_10/jre

PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/binCLASSPATH=.:$ JAVA_HOME/lib/jt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib In Linux, the number is:

exportJAVA_HOMEJRE_HOMEPATHCLASSPATH

Add complete Save exit

#source/etc/profile

#java-version

Display javaversion"1.6.0_10"(jdk1.6 installed successfully)

5. Uninstall rpm version of jdk: (unused)

#rpm-qa|grepjdk

Display: jdk-1.6.0_10-fcs

Uninstall: #rpm-e--nodepsjdk-1.6.0_10-fcs

3. Configure Hadoop

Take a look at the Hadoop cluster setup for Hadoop configuration. First download Hadoop, extract it to Master, and put it in the user root directory. Here I am downloading version 0.19.2. Hadoop is installed at/home/fan/hadoop-0.19.2. Then modify the Hadoop configuration file, extract it and enter the conf directory. You mainly need to modify the following files: hadoop-env.sh, hadoop-site.xml, masters, slaves.

1. Modify conf/hadoop-env.sh and add jdk support for exportJAVA_HOME=/usr/jdk1.6.0_18

2. Modify the conf/site-site.xml file in the hadoop directory.

3. Masters increased:

fan001

Slaves added:

fan002

fan003

fan004

About "Hadoop cluster build sample analysis" This article is shared here, I hope the above content can be of some help to everyone, so that you can learn more knowledge, if you think the article is good, please share it to let more people see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.