Shulou(Shulou.com)06/03 Report--

1. Set up ip

Service network restart restart Nic takes effect

two。 Turn off the firewall (many ports are used in hadoop, to avoid hassle)

Service iptables stop command to turn off firewall

Service iptables status View Firewall status Command

Turn off the automatic operation of the firewall:

View auto-run settings: chkconfig-- list can display all settings (firewall iptables)

Filter to view firewall auto-startup settings

Chkconfig-- list | (vertical bar) grep (filter) iptables (filter object)

Pipe: use the output on the left as the input on the right

Set all the automatic startup settings of the firewall to off

Chkconfig iptables off

From then on, the firewall will not turn on automatically with the restart of the system.

3. Set hostname

View hostname command: hostname

Modify the hostname: hostname crxy (Superman Academy). This method is only valid for the current session and remains unchanged after rebooting the system.

Modify the disk file and completely change the hostname: vi / etc/sysconfig/network

Effective after restarting the machine: reboot-h now (you don't have to restart the system here)

Configuration operation of 4.ssh: ssh is a tool similar to telnet for remote access between machines. Telnet is plaintext and ssh is encrypted.

Ssh uses asymmetric encryption. There is a problem with public key. As shown in the figure, password login is required to access ip.

In order to log in without a password, you need to configure as follows:

Ssh-keygen-t rsa (generate key first). Enter many times and then find your public key has been saved in.

/ root/.ssh/id_rsa.pub (the public key generated is saved in a directory)

Cd / root/.ssh or cd / root/.ssh,

Cp id_rsa.pub authorized_keys (s missing in the figure)

Or directly cp ~ / .ssh/id_rsa.pub ~ / .ssh/authorized_keys

You can also ssh localhost (you need to enter a password when you visit for the first time, then you don't need to enter a password after that)

For solving the problem of ssh hostname (the error occurs because the hostname depends on the domain name, and the reason why the hostname can be entered in the browser for access is because the domain name is bound), the method of binding ip address:

5.ip binds to hostname

Vi / etc/hosts (add 192.168.42.100 crxy on the next line) (for ip and hostname of linux, respectively)

6. File transfer between windows and linux using WinSCP tool software

Transfer file: find the file on the left, open the file on the right, modify / root to / usr/local in the path (where / usr/local is a custom installation directory, or you can define it yourself), click add session bookmark below, then click share bookmark, click add, and finally determine

After this, there will be files in the local directory. We can select them all and right-click to delete them.

Then drag the hadoop file and jdk file into it.

After dragging, you can use the command to view the files under local.

7. Install jdk

Permissions need to be modified before installing jdk, so that you can use the tab key to automatically complete

Through. / jdk... Decompress

8. Install hadoop

Extract the command in .tar.gz format: tar-xzvf hadoop-1.1.2.tar.gz

X for decompression, z for compression format, v for display information, f for file

Configure environment variables

9. Install pseudo distribution

Modify several configuration files under hadoop/conf

1.hadoop-env.sh

2.core-site.xml

3.hdfs-site.xml

4.mapred-site.xml

a. Modify hadoop-env.sh

B.core-site.xml

C.hdfs-site.xml

D.mapred-site.xml

In this way, the configuration of hadoop is complete, and all that is left is to start hadoop.

Startup of 10.hadoop

a. The hadoop file system needs to be formatted before starting hadoop (hadoop includes both storage and computing, and like other file systems, it needs to be formatted when a storage space is needed)

Formatted command: hadoop namenode-format

B.hadoop file system startup command:

Start-all.sh

Verification: under the command window of window (windows has configured the environment variables of jdk), you can view the running process of java by running the command jps (the command to view the java process). If you can view the java process by running jps in linux, the hadoop file system starts successfully (because hadoop is developed based on java).

Possible reasons why there is no NameNode at startup:

(1) not formatted

(2) incorrect setting of environment variables

(3) failed to bind ip to hostname

Finally, we can observe the status of hadoop startup through a browser, that is, we can access it through windows:

Enter the http://192.168.42.100:50070(http://192.168.42.100:50070(li linux URL + port number in the windows browser or enter 192.168.42.100ping 50070 directly (because you can also ping the linux network in windows), and you can see the same effect in linux browsers (ip addresses can be replaced with hostnames in linux browsers because they are bound to each other)

In browser viewing, 50070 views the information of NameNode and 50030 views the information of MapReduce.

11. Command summary:

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.