Shulou(Shulou.com)06/03 Report--

I have always wanted to write about my experience of learning hadoop to help me understand the whole learning process and help more beginners to learn.

All the information introduced in this article is on ubuntu14.04, jdk1.8,hadoop2.7.4

The following is divided into several parts to introduce the whole learning process.

1. Install hadoop on ubuntu system

Hadoop is a java program running under the linux system, so it is necessary to install a linux system first. The author chooses the vmwork+ubuntu14.04 64-bit system as the development environment. For information about how to use vmwork to install ubuntu system, please contact Baidu

Vmwork download address vmWork password: ohsl

Ubuntu download address ubuntu14.04 password: 9stz

Now the default is for everyone to install the ubuntu system.

1. Install jdk on ubuntu

Because hadoop is ultimately developed and compiled using java, you need to use jdk. In the ubuntu system, you can install jdk using apt-get install jdk, but because you need to use some files in the jdk directory when installing hadoop, use the jdk installed by apt-get install, in which the files such as bin and conf will always be divided into different places, so it is recommended to download jdk directly for configuration.

You can go to the orcale official website to download, the following is the Baidu network disk path of jdk: JDK8. Password: dqk7.

Copy the jdk-8u151-linux-x64.tar.gz to the user folder corresponding to ubuntu.

Use the command

$tar-xvzf jdk-8u151-linux-x64.tar.gz

Extract it to the folder. As shown in the picture

After decompression, get the corresponding file. Since the user directory is generally only used to store the user's data, we need to save the installed software in / usr/soft.

Use

$cd / usr

Go to the usr directory and create a folder in the usr directory. Administrator permission is required, so you need to use sudo

$sudo mkdir soft

It was created successfully after entering the password.

After successful creation, you need to modify the permissions of the folder so that the user has write and execute permissions.

$sudo chmod 777 soft

And use ls-al to check whether the corresponding permissions are successful.

$ls-al

Copy jdk to soft

$mv / home/zhaofan/Downloads/jdk1.8.0_151 / usr/soft

Configure environment variables

$sudo gedit / etc/environment

Add JAVA_HOME to the file to point to the jdk installation directory, and add the corresponding bin to PATH

After editing, the source will change the configuration file to take effect immediately

$source / etc/environment

$java-version

You can see the version number of java, and the jdk installation is complete.

2. Install hadoop

Go to apache to download hadoop, or click the link below to download Baidu online disk, download password: 2vc9

Hadoop download address

After the download is complete, extract it to the soft folder as well.

$tar-xvzf hadoop-2.7.4.tar.gz

Two folders jdk and hadoop will appear in the soft folder

Also add environment variables to enviroment, as shown in the following figure.

$sudo gedit / etc/environment

$source / etc/environment

Use the above command to make the environment variable work.

$hadoop version

Check the version information of hadoop. If prompted that JAVA_HONE is missing, check whether JDK is configured. If configured, check the version of hadoop after restart.

If you see the information in the figure above, the installation of hadoop is complete.

In the next section, we will explain how to configure hadoop

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.