Shulou(Shulou.com)06/01 Report--

This article mainly introduces the example analysis of hadoop1.1.2 pseudo-distribution installation, which has a certain reference value, and interested friends can refer to it. I hope you can learn a lot after reading this article.

1. Pseudo-distributed installation 1.1 modify ip

(1) Open the virtual network card of VMWare or VirtualBox

(2) set the network connection method to host-only in VMWare or VirtualBox

(3) in linux, modify ip. There is an icon in the upper corner. Right-click and select Edit Connections.....

* the ip must be on the same IP address range as the ip of the virtual ENI under the windows, and the gateway must exist.

(4) restart the Nic and execute the command service network restart

* error, such as no suitable adapter error

(5) Verification: execute the command ifconfig

1.2 turn off the firewall

(1) execute the command: service iptables stop closes the firewall

(2) Verification: execute the command service iptables status

1.3 turn off the automatic opening of the firewall

(1) execute the command chkconfig iptables off

(2) Verification: execute the command chkconfig-- list | grep iptables

1.4 modify hostname

(1) execute the command hostname cloud4 to modify the hostname in the session

(2) Verification: execute the command hostname

(3) execute the command vi / etc/sysconfig/network to modify the hostname in the file

(4) Verification: execute the command reboot-h now to restart the machine

1.5 set the binding between ip and hostname

(1) execute the command vi / etc/hosts

Add a line of 192.168.80.100 cloud4 at the end of the text

(2) Verification: ping cloud4

(3) configure the ip corresponding to the hostname in window

C:\ Windows\ System32\ drivers\ etc\ hosts

1.6 ssh password-free login

(1) execute the command ssh-keygen-t rsa (then Enter all the way) to generate the key at / root/.ssh/

(2) execute the command cp / root/.ssh/id_rsa.pub / root/.ssh/authorized_keys to generate the authorization file

(3) Verification: ssh localhost (ssh hostname)

1.7 install jdk

(1) use winscp to copy jdk and hadoop to / root/Downloads of linux

(2) cp / root/Downloads/* / usr/local

(3) cd / usr/local

Give the execute permission chmod Ubunx jdk-6u24-linux-i586.bin

(4) / jdk-6u24-linux-i586.bin

(5) rename mv jdk1.6.0_24 jdk

(6) execute the command vi / etc/profile to set the environment variable

Add two lines of export JAVA_HOME=/usr/local/jdk

Export PATH=.:$JAVA_HOME/bin:$PATH

Save exit

Execute the command source / etc/profile

(7) Verification: execute the command java-version

1.8 install hadoop

(1) execute the command tar-zxvf hadoop-1.1.2.tar.gz to decompress.

(2) execute the command mv hadoop-1.1.2 hadoop

(3) execute the command vi / etc/profile to set the environment variable

Add a line of export HADOOP_HOME=/usr/local/hadoop

Modify a row of export PATH=.:$HADOOP_HOME/bin:$JAVA_HOME/bin:$PATH

Save exit

Execute the command source / etc/profile

(4) Verification: execute the command hadoop

(5) modify the configuration files hadoop-env.sh, core-site.xml, hdfs-site.xml and mapred-site.xml located in conf/

Line 9 of the file hadoop-env.sh

Export JAVA_HOME=/usr/local/jdk/

File core-site.xml

Fs.default.name

Hdfs://cloud4:9000

Change your own hostname

Hadoop.tmp.dir

/ usr/local/hadoop/tmp

File hdfs-site.xml

Dfs.replication # indicates to set the number of copies. Default is 3.

one

Dfs.permissions # indicates whether to set permission control

False

In the case of super-user (superuser), it is the identity of the nameNode process. The system does not perform any permission checks

File mapred-site.xml

Mapred.job.tracker

Cloud4:9001

Change your own hostname

(6) execute the command hadoop namenode-format to format

(7) execute the command start-all.sh to start hadoop

(8) Verification:

Execute the command jps to view the java process and find five processes, namely NameNode, SecondaryNameNode, DataNode, JobTracker and TaskTracker

View through the browser: http://cloud4:50070 and http://cloud4:50030

* modify the hosts file in the C:/Windows/system32/drivers/etc/ directory of windows

1.9 if the warning prompt is removed:

[root@cloud4 ~] # hadoop fs-ls /

Warning: $HADOOP_HOME is deprecated. (remove warning)

The methods are as follows:

[root@cloud4 ~] # vi / etc/profile (add a sentence)

# / etc/profile

Export HADOOP_HOME_WARN_SUPPRESS=1

Export JAVA_HOME=/usr/local/jdk

Export HADOOP_HOME=/usr/local/hadoop

Export PATH=.:$HADOOP_HOME/bin:$JAVA_HOME/bin:$PATH

[root@cloud4 ~] # source / etc/peofile (effective immediately)

Thank you for reading this article carefully. I hope the article "sample Analysis of hadoop1.1.2 pseudo-Distribution installation" shared by the editor will be helpful to you. At the same time, I also hope that you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.