Shulou(Shulou.com)06/01 Report--

This article mainly introduces how CentOS7 compiles Hadoop-2.7.2. It is very detailed and has certain reference value. Friends who are interested must finish reading it.

First of all, the 32-bit Linux operating system can download the compiled Hadoop directly, but the 64-bit Linux operating system needs to compile itself.

Later, the official website has provided 64-bit since the hadoop2.5 version. It can be downloaded and used directly.

However, if you do it yourself and have plenty of food and clothing, you can regard it as learning. Here's how to start:

1. Tool preparation, the most reliable are the tools required in the hadoop documentation.

Go to the hadoop website and click source to download hadoop-2.7.2-src.tar.gz.

Decompress it

Tar-zxvf hadoop-2.7.2-src.tar.gz

Get the hadoop-2.7.2-src folder.

Go to the hadoop-2.7.2-src folder and view BUILDING.txt

Cd hadoop-2.7.1-src

Vim BUILDING.txt

You can see the libraries or tools required for compilation:

These include:

JDK1.7+

Maven 3.0 or later

Findbugs 1.3.9

ProtocolBuffer 2.5.0

Cmake 2.6

Zlib-devel

Openssl-devel

In addition to the above, in order not to make mistakes, you actually need autoconf automake gcc and so on.

Let's start preparing these tools:

First use the su command to get root permissions to avoid various restrictions

two。 Download jdk-8u65-linux-x64.tar.gz, extract it and move it to the / opt directory

Tar-zxvf jdk-8u65-linux-x64.tar.gz

Mv jdk1.8.0_65 / opt

Then open / etc/profile to configure the jdk environment variable

Vim / etc/profile

Press I to enter insert mode and add at the end of the file

Export JAVA_HOME=/opt/jdk1.8.0_65

Export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/tools.jar

Export PATH=$PATH:$JAVA_HOME/bin

Export JRE_HOME=/opt/jdk1.8.0_65/jre

Export PATH=$PATH:$JRE_HOME/bin

Press Esc, Shift+:, wq, and press enter to save and exit editing.

Enter source / etc/profile and enter to save the changes.

Run javac-version to check the status:

[root@server1 java] # javac-version

Javac 1.8.0_65

[root@server1 java] #

3. Install various libraries below

Yum-y install svn ncurses-devel gcc*

Yum-y install lzo-devel zlib-devel autoconf automake libtool cmake openssl-devel

4. Install protobuf-2.5.0.tar.gz (note that version must be 2.5.0)

Tar zxvf protobuf-2.5.0.tar.gz

Enter protobuf-2.5.0 and execute in turn.

Cd protobuf-2.5.0

. / configure

Make

Make install

Verify that the installation is complete

Protoc-version

Appear

[root@server1 protobuf-2.5.0] # protoc-- version

Libprotoc 2.5.0

[root@server1 protobuf-2.5.0] #

5. Install maven

Download apache-maven-3.3.9-bin.tar.gz

Extract and configure environment variables

Decompress:

Tar-zxvf apache-maven-3.3.9-bin.tar.gz

Move to the / opt directory:

Mv apache-maven-3.3.9 / opt

Configure environment variables:

Vim / etc/profile

Add at the end:

Export MAVEN_HOME=/opt/apache-maven-3.3.9

Export MAVEN_OPTS= "- Xms256m-Xmx512m"

Export PATH=$PATH:$MAVEN_HOME/bin

Press Esc, Shift+:, wq, and enter to save and edit.

Enter source / etc/profile and enter to save the changes.

Check the installation status: mvn-version

[root@server1 ~] # mvn-version

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00)

Maven home: / usr/local/maven-3.3.9

Java version: 1.8.0_65, vendor: Oracle Corporation

Java home: / usr/java/jdk1.8.0_65/jre

Default locale: en_US, platform encoding: UTF-8

OS name: "linux", version: "3.10.0-229.el7.x86_64", arch: "amd64", family: "unix"

[root@server1 ~] #

6. Install ant

Download apache-ant-1.9.7-bin.tar.gz

Extract and configure environment variables

Decompress:

Tar-zxvf apache-ant-1.9.7-bin.tar.gz

Move to / opt directory

Mv apache-ant-1.9.7 / opt

Configure environment variables

As above, no addition is made at the end of the / etc/profile file:

Export ANT_HOME=/opt/apache-ant-1.9.7

Export PATH=$PATH:$ANT_HOME/bin

Then save, exit, and make the changes take effect.

View the installation results:

Ant-version

[root@server1 ~] # ant-version

Apache Ant (TM) version 1.9.7 compiled on April 9 2016

[root@server1 ~] #

7. Install findbugs

Download findbugs-3.0.1.tar.gz and select the standard version above

Extract and configure environment variables

Decompress:

Tar-zxvf findbugs-3.0.1.tar.gz

Move to / opt directory

Mv findbugs-3.0.1 / opt

Configure environment variables:

Add at the end of the / etc/profile file:

Export FINDBUGS_HOME=/opt/findbugs-3.0.1

Export PATH=$PATH:$FINDBUGS_HOME/bin

Save the exit and make the changes take effect.

View installation results

[root@server1 ~] # findbugs-version

3.0.1

[root@server1 ~] #

8. Ready to finish, let's start compiling hadoop

Go to the hadoop-2.7.2-src directory

Use the command:

Mvn clean package-Pdist,native-DskipTests-Dtar

Or:

Mvn package-Pdist,native-DskipTests-Dtar

To compile.

Be sure to keep the network open. After a long wait, the following picture shows that the compilation is successful!

Due to problems in the middle of my network, I recompiled it for almost 3 hours.

The compiled file is under / hadoop-dist/target/hadoop-2.7.2.tar.gz.

Note:

1. For all commands, hand tapping is recommended, copy and paste is not recommended, in order to avoid unnecessary errors caused by the space of the page.

two。 Be sure to keep the network open, and if a file is missing, clean up the maven (use the command mvn clean) before recompiling.

3. If the compilation fails due to the same error all the time, it may be that some library or tool is missing. Check that all of the above tools are installed successfully and that the version is correct.

The above is all the content of the article "how CentOS7 compiles Hadoop-2.7.2". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.