Shulou(Shulou.com)06/01 Report--

This article mainly introduces "hadoop pseudo-distributed installation method". In daily operation, I believe many people have doubts about hadoop pseudo-distributed installation method. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts about "hadoop pseudo-distributed installation method". Next, please follow the editor to study!

Hadoop pseudo-distribution is generally used for learning and testing. The production environment is generally not used.

(if there are any mistakes, you are welcome to criticize and correct them)

1. Installation environment

Install linux system on windows system, take CenOS as an example. Hadoop version is hadoop1.1.2

two。 Configure the linux virtual machine

2.1 make sure that the network card WMnet1 on the window and the network card of the Linux virtual machine are on the same network segment (try ping to make sure you can ping)

2.2 modify the hostname

You'd better revise it (unified management, easy to remember). Instruction: vim / etc/sysconfig/network

2.3 modify ip

Students who are not familiar with linux instructions had better use graphical interface to change it (recommended)

Instruction: vim / etc/sysconfig/network-scripts/ifcfg-eth0 (per human machine

Different, the network card on the virtual machine may be different, change it to the network card on your own machine)

Modify ipaddress,netmask,gateway.

2.4 modify the mapping between hostname and ip

Instruction: vim / etc/hosts

2.5 turn off the firewall (turn off boot)

Instruction: chkconfig iptables off

2.6 restart linux

Instruction: reboot

3. Install jdk

3.1 upload jdk

3.2 add execution permissions

Instruction: chmod Ubunx jdk (self-uploaded jdk)

3.3 decompression

Extract it to the specified directory (please manage all files)

Instruction: tar-zxvf jdk (self-uploaded jdk)-C specified directory

3.4 add environment variables

Instruction: vim / etc/profile

3.5 Refresh (to make the configured environment variable effective)

Instruction: source / etc/profile

4. Install hadoop pseudo-distributed

4.1 upload hadoop

4.2 ensure that you have the right to execute and extract it (again, it is best to manage documents in a unified way)

Directive: tar-zxvf hadoop (hadoop version uploaded)-C specified directory

4.3 configure hadoop (modify 4 configuration files) and go to the hadoop-1.1.2/conf directory

If you are not familiar with the instructions (please use tools to modify them, such as Notepad++, etc.)

4.31hadoop-env.sh

On line 9, remove the comments and configure JAVA_HOME

4.32core-site.xml

Fs.default.name

Hostname for hdfs:// configuration: 9000

Hadoop.tmp.dir

/.. / hadoop-1.1.2/tmp

4.33hdfs-site.xml

Dfs.replication

one

4.34mapred-site.xml

Mapred.job.tracker

Configured hostname: 9001

4.4 add hadoop environment variable

Instruction: vim / etc/profile

4.5 hdfs for formatting hadoop

Instruction: hadoop namenode-format

4.6 start hadoop

Instruction: start-all.sh

4.7 verify that hadoop starts successfully

Instruction: jps

If it succeeds, the following five will appear.

NameNode

SecondaryNameNode

DataNode

JobTracker

TaskTracker

It can also be verified by browser.

Ip:50070 of http://linux (hdfs management interface)

Ip:50030 of http://linux (mapreduce management interface)

But it must be under the windows system first.

C:\ Windows\ System32\ drivers\ etc add the mapping between linux hostname and ip

5. Configure ssh login-free

Ssh is secure shell.

Generate ssh key instruction: ssh-keygen-t rsa can press enter continuously (4 times).

There is a hidden file .ssh under / root. Under / root/.ssh/, there are 2 more files.

File (id_rsa id_rsa.pub) private key and public key, execute the following instructions:

Cat ~ / .ssh/id_rsa.pub > > ~ / .ssh/authorized_keys

Hello hadoop, it's done. Ready for development!

At this point, the study on the "pseudo-distributed installation method of hadoop" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.