Shulou(Shulou.com)06/01 Report--

Editor to share with you a sample analysis of Hadoop2.4.1 distributed installation. I hope you will get something after reading this article. Let's discuss it together.

1. Do all the configuration below: Hadoop1.2.1 distributed installation-1-preparation.

2. There is a small problem in the distribution of Hadoop2.x: libhadoop.so.1.0.0 has a problem in 64-bit OS because it is 32-bit. When hadoop starts up in 64-bit OS, it will report a log of WARN. The function of this package is to call the api of native, which can improve the performance of hadoop. If the package fails, it will use jvm to do compression and other work, and the efficiency will be very low. The way to do this is to recompile Hadoop, see xxx (link article).

3. On the machine that is going to do namenode, download the hadoop package by wget or other means, and extract it to the local specified directory. Download the decompression commands and refer to the Linux common commands.

4. Various configuration files and hadoop1 will be different, with a total of seven files, which are described below.

/ hadoop-2.4.1/etc/hadoop/hadoop-env.sh

# The java implementation to use.export JAVA_HOME=$ {JAVA_HOME}

/ hadoop-2.4.1/etc/hadoop/yarn-env.sh

# some Java parameters# export JAVA_HOME=/home/y/libexec/jdk1.6.0/if ["$JAVA_HOME"! = ""]; then # echo "run java in $JAVA_HOME" JAVA_HOME=$JAVA_HOMEfiif ["$JAVA_HOME" = ""]; then echo "Error: JAVA_HOME is not set." The default heap_max of exit 1fiJAVA=$JAVA_HOME/bin/javaJAVA_HEAP_MAX=-Xmx512m# is 1000m, and my virtual machine doesn't have so much memory, so it's smaller.

/ hadoop-2.4.1/etc/hadoop/slaves

# write to the node of your slave. If there are multiple, one per line, write the host name bd24bd25

/ hadoop-2.4.1/etc/hadoop/core-site.xml

Fs.defaultFS hdfs://bd23:9000 io.file.buffer.size 131072 hadoop.tmp.dir file:/home/wukong/a_usr/hadoop-2.4.1/tmp Abase for other temporary directories. Hadoop.proxyuser.hduser.hosts * hadoop.proxyuser.hduser.groups

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.