Shulou(Shulou.com)06/01 Report--

Editor to share with you how to achieve pseudo-distributed Hadoop2.4.1 building, I believe that most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to understand it!

1. Prepare the Linux environment

Click the VMware shortcut, right open the location of the file-> double-click vmnetcfg.exe-> VMnet1 host-only-> modify the subnet ip setting IP address range: 192.168.1.0 Subnet Mask: 255.255.255.0-> apply-> ok

Go back to windows-> Open Network and sharing Center-> change Adapter Settings-> right-click VMnet1-> Properties-> double-click IPv4-> set the IP:192.168.1.100 subnet mask for windows: 255.255.255.0-> Click OK

On virtual software-- My Computer-> Select virtual machine-> right-click-> settings-> network adapter-> host only-> ok

1.1 modify hostname

Vim / etc/sysconfig/network

NETWORKING=yes

HOSTNAME=itcast #

1.2 modify IP

There are two ways:

First: modify it through the Linux graphical interface (highly recommended)

Enter the Linux graphical interface-> right click on the two minicomputers on the top right-> Click Edit connections-> Select the current network System eth0-> click the edit button-> Select IPv4-> method Select manual-> click the add button-> add the IP:192.168.1.101 subnet mask: 255.255.255.0 Gateway: 192.168.1.1-> apply

Second: modify the configuration file (for losers and programmers only)

Vim / etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE= "eth0"

BOOTPROTO= "static" #

HWADDR= "00:0C:29:3C:BF:E7"

IPV6INIT= "yes"

NM_CONTROLLED= "yes"

ONBOOT= "yes"

TYPE= "Ethernet"

UUID= "ce22eeca-ecde-4536-8cc2-ef0dc36d4a8c"

IPADDR= "192.168.1.101" #

NETMASK= "255.255.255.0" #

GATEWAY= "192.168.1.1" # #

1.3 modify the mapping relationship between hostname and IP

Vim / etc/hosts

192.168.1.101 itcast

1.4 turn off the firewall

# View firewall status

Service iptables status

# turn off the firewall

Service iptables stop

# check the boot status of the firewall

Chkconfig iptables-list

# turn off the firewall and boot

Chkconfig iptables off

1.5 restart Linux

Reboot

two。 Install JDK

2.1 upload

2.2 decompress jdk

# create a folder

Mkdir / usr/java

# decompression

Tar-zxvf jdk-7u55-linux-i586.tar.gz-C / usr/java/

2.3 add java to the environment variable

Vim / etc/profile

# add at the end of the file

Export JAVA_HOME=/usr/java/jdk1.7.0_5

Export PATH=$PATH:$JAVA_HOME/bin

# refresh configuration

Source / etc/profil

3. Install hadoop2.4.1 (itcast01 is the hostname)

Note: hadoop2.x configuration file $HADOOP_HOME/etc/hadoop

Pseudo-distribution requires five configuration files to be modified

3.1Configuring hadoop

The first one: hadoop-env.sh

Vim hadoop-env.sh

# Line 27

Export JAVA_HOME=/usr/java/jdk1.7.0_65

The second one: core-site.xml

Fs.defaultFS

Hdfs://itcast01:9000

Hadoop.tmp.dir

/ itcast/hadoop-2.4.1/tmp

The third one: hdfs-site.xml

Dfs.replication

one

Fourth: mapred-site.xml (mv mapred-site.xml.template mapred-site.xml)

Mv mapred-site.xml.template mapred-site.xml

Vim mapred-site.xml

Mapreduce.framework.name

Yarn

The fifth: yarn-site.xml

Yarn.resourcemanager.hostname

Itcast01

Yarn.nodemanager.aux-services

Mapreduce_shuffle

3.2 add hadoop to the environment variable

Vim / etc/proflie

Export JAVA_HOME=/usr/java/jdk1.7.0_65

Export HADOOP_HOME=/itcast/hadoop-2.4.1

Export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

Source / etc/profile

3.3.Formenting namenode (initializing namenode)

Hdfs namenode-format (hadoop namenode-format)

3.4 start hadoop

Start HDFS first

Sbin/start-dfs.sh

Restart YARN

Sbin/start-yarn.sh

3.5 verify that the startup is successful

Use the jps command to verify

27408 NameNode

28218 Jps

27643 SecondaryNameNode

28066 NodeManager

27803 ResourceManager

27512 DataNode

Http://192.168.8.118:50070 (HDFS Management Interface)

Http://192.168.8.118:8088 (MR Management Interface)

4. Configure ssh login-free

# generate ssh login-free key

# go to my home directory

Cd / .ssh

Ssh-keygen-t rsa (four carriage returns)

After executing this command, two files, id_rsa (private key) and id_rsa.pub (public key), are generated.

Copy the public key to the machine that will not be logged in

Ssh-copy-id localhost

These are all the contents of the article "how to achieve pseudo-distributed Construction in Hadoop2.4.1". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.