Shulou(Shulou.com)06/02 Report--

This article will explain in detail how to install and build Hadoop 2.7.3 under Linux. The editor thinks it is very practical, so I share it with you for reference. I hope you can get something after reading this article.

Linux Hadoop 2.7.3 installation set-up

Hadoop implements a distributed file system (Hadoop Distributed File System), referred to as HDFS.

HDFS has high fault tolerance and is designed to be deployed on low-cost (low-cost) hardware; and it provides high throughput (high throughput) to access application data, suitable for applications with very large data sets (large data set). HDFS relaxes the requirement of (relax) POSIX to access (streaming access) data in a file system as a stream.

The core design of Hadoop's framework is: HDFS and MapReduce. HDFS provides storage for massive data, while MapReduce provides computing for massive data.

What problems does Hadoop solve?

Massive data need to be analyzed and processed in time.

Massive data needs in-depth analysis and mining.

Data needs to be preserved for a long time.

Problems with massive data storage:

Disk IO is called a bottleneck, not a CPU resource

Network bandwidth is a scarce resource

Hardware failure has become a major factor affecting stability.

Installation proc

Install JAVA 1.8

Download:

Http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

Java environment installation configuration / etc/profile:

Export JAVA_HOME=/usr/jdk/jdk1.8.0_112export JRE_HOME=/usr/jdk/jdk1.8.0_112/jreexport CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib:$CLASSPATHexport PATH=$JAVA_HOME/bin:$PATH

Install Hadoop2.7.3

Download: http://archive.apache.org/dist/hadoop/core/hadoop-2.7.3/

Configure Startup Hadoop

1. Modify hadoop 2.7.3/etc/hadoop/hadoop-env.sh to specify JAVA_HOME

# The java implementation to use.export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-8.b10.el6_9.x86_64

2. Modify the configuration file of hdfs

Fs.defaultFS hdfs://0.0.0.0:9000 hadoop.tmp.dir / tmp/hadoop

3. Start

The first time you start, you have to format.

/ usr/hadoop/bin/hdfs namenode-format

Start hdfs

/ usr/hadoop/sbin/start-dfs.sh

Use the jps command to view the started Java process

Preview address: http://127.0.0.1:50070/dfshealth.html

4. Stop

/ usr/hadoop/sbin/stop-dfs.sh, this is the end of the article on "how to install and build Hadoop 2.7.3 under Linux". I hope the above content can be helpful to you so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.