Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to compile hadoop2.5.1 source code under centos6.5 64-bit, the article is very detailed, has a certain reference value, interested friends must read it!

Required tool: centos6.5-64bit

Jdk1.7.0.71

Maven3.23 (http://mirrors.cnnic.cn/apache/maven/maven-3/3.2.3/binaries/apache-maven-3.2.3-bin.tar.gz)

Ant1.9 (http://mirrors.cnnic.cn/apache//ant/binaries/apache-ant-1.9.4-bin.tar.gz)

Findbugs (http://cznic.dl.sourceforge.net/project/findbugs/findbugs/2.0.3/findbugs-2.0.3.tar.gz)

Required software: java, svn, autoconf, automake, libtool, ncurses-devel, openssl-devel, gcc, lzo-devel, zlib-devel, znt, maven, protobuf, cmake

Check whether a software command is installed in the system: rpm-qa | grep software name, such as whether svn is installed in the system:

$rpm-qa | grep svn

If it is installed, skip it. If it is not installed, install it with the command: yum install svn, such as:

$yum install svn

If the installation of the yum command fails (the corresponding package cannot be found), you need to manually download the corresponding software package for installation, such as maven and protobuf.

Next, you need to go to the apach official website to download the hadoop2.5.1 source code package and decompress it, and then install jdk, maven, ant. There are a lot of detailed steps online.

Question guide:

1. How to get the hadoop src maven package?

two。 What software does it need to install to compile hadoop?

3. How do I compile hadoop2.4?

Extend:

Compile hadoop Why install these softwares?

Installation of protobuf

To prevent mistakes, here are some additions.

Install grub + first

Sudo apt-get install gathers +

Under centos: yum install gcc-c++

Download protobuf-2.5.0.tar.gz network disk download link: http://pan.baidu.com/s/1dDgWe6P password: xgiu

Decompress tar zxvf protobuf-2.5.0.tar.gz

$sudo. / configure

$sudo make

$sudo make check

$sudo make install

$sudo ldconfig

If a lower version of protobuf already exists in the system and is recognized, make the following settings

Modify the environment variable:

Vi / etc/profile

Export LD_LIBRARY_PATH=~/protobuf-2.5.0

(here in LD_LIBRARY_PATH, fill in the location of the protobuf file)

Source / etc/profile

Note: configuration / etc/profile, the configuration may fail after the virtual machine is restarted, so you need to perform the source operation again after the restart.

Verify protoc-- version

. Install the opensl library

Sudo apt-get install libssl-dev / yum install openssl

CMake installation

Sudo apt-get install cmake / sudo yum install cmake

Install findbugs (choose to install)

$wget http://prdownloads.sourceforge.net/findbugs/findbugs-2.0.3.tar.gz?download$ sudo tar zxf findbugs-2.0.3.tar.gz-C / opt$ sudo vim / etc/profileexport FINDBUGS_HOME=/opt/findbugs-2.0.3export PATH=$PATH:$FINDBUGS_HOME/bin Verification: [root@hadoop001 bin] # fb-version2.0.3

Enter the src package to execute the compilation command

Mvn package-Pdist,native-DskipTests-Dtar-No documents are generated

The one that generates the document has not been tested yet.

Documentation can be generated using the-Pdocs option, of course, provided that Forrest and Findbugs are installed and can be manually specified by referring to the following commands: FORREST_HOME and FINDBUGS_HOME.

Mvn package-Pdocs-DskipTests-Dtar-Dmaven.test.skip-Denv.FORREST_HOME=/usr/local/apache-forrest-Denv.FINDBUGS_HOME=/usr/local/findbugs

The generated documents are in their respective target/site directories.

Record:

Compiling under jdk1.8 will have the problem of dealing with comment tags. It is similar to the following question:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.8.1:jar (module-javadocs) on project hadoop-annotations: MavenReportException: Error while creating archive:

[ERROR] Exit code: 1-/ home/hadoop/Desktop/hadoop-2.5.1-src/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/InterfaceStability.java:27: error: unexpected end tag:

Solution: replace it with jdk1.7

[ERROR] around Ant part. @ 4 141 in / home/hadoop/Desktop/hadoop-2.5.1-src/hadoop-common-project/hadoop-common/target/antrun/build-main.xml

Resolve:

Yum-y install zlib-devel

Debian:apt-get install zlib1g-dev

Yum-y install ncurses-devel

Debian: apt-get install libncurses5-dev

[ERROR] around Ant part. @ 5Rom 131 in / home/hadoop/Desktop/hadoop-2.5.1-src/hadoop-tools/hadoop-pipes/target/antrun/build-main.xml

Solve

Yum install openssl-devel

The above is all the content of this article "how to compile hadoop2.5.1 source code under centos6.5 64-bit". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.