Shulou(Shulou.com)06/03 Report--

Experimental environment:

Operating system: Ubuntu 16.04LTS

Hadoop version: Hadoop 2.7.1

First, install JAVA environment

Dblab@dblab-VirtualBox:/$ sudo apt-get install default-jre default-jdk

Dblab@dblab-VirtualBox:/$ vim / .bashrc

Export JAVA_HOME=/usr/lib/jvm/default-java

Dblab@dblab-VirtualBox:/$ source ~ / .bashrc # enables variable settings to take effect

Dblab@dblab-VirtualBox:/$ echo $JAVA_HOME

/ usr/lib/jvm/default-java

Dblab@dblab-VirtualBox:/$ java-version

Openjdk version "1.8.0,131"

OpenJDK Runtime Environment (build 1.8.0mm 131-8u131-b11-2ubuntu1.16.04.3-b11)

OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

Second, install Hadoop

Dblab@dblab-VirtualBox:/$ cd ~ / download

Dblab@dblab-VirtualBox:~/ download $wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.6.5/hadoop-2.6.5.tar.gz

Download $sudo tar-zxvf hadoop-2.6.5.tar.gz-C / usr/local from dblab@dblab-VirtualBox:~/

Dblab@dblab-VirtualBox:/usr/local$ sudo mv. / hadoop-2.6.5/. / hadoop # change the folder name

Dblab@dblab-VirtualBox:/usr/local$ sudo chown-R hadoop. / hadoop/ # modify folder permissions

Dblab@dblab-VirtualBox:/usr/local/hadoop$. / bin/hadoop version # displays version information

Hadoop 2.7.1

Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git-r 15ecc87ccf4a0228f35af08fc56de536e6ce657a

Compiled by jenkins on 2015-06-29T06:04Z

Compiled with protoc 2.5.0

From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a

This command was run using / usr/local/hadoop/share/hadoop/common/hadoop-common-2.7.1.jar

III. Hadoop stand-alone configuration

The default mode of Hadoop is non-distributed mode (local mode) and can be run without additional configuration.

You can execute the example to feel the operation of Hadoop. Hadoop comes with a wealth of examples

Dblab@dblab-VirtualBox:/usr/local/hadoop$. / bin/hadoop jar. / share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar

An example program must be given as the first argument.

Valid program names are:

Aggregatewordcount: An Aggregate based map/reduce program that counts the words in the input files.

Aggregatewordhist: An Aggregate based map/reduce program that computes the histogram of the words in the input files.

Bbp: A map/reduce program that uses Bailey-Borwein-Plouffe to compute exact digits of Pi.

Dbcount: An example job that count the pageview counts from a database.

Distbbp: A map/reduce program that uses a BBP-type formula to compute exact bits of Pi.

Grep: A map/reduce program that counts the matches of a regex in the input.

Join: A job that effects a join over sorted, equally partitioned datasets

Multifilewc: A job that counts words from several files.

Pentomino: A map/reduce tile laying program to find solutions to pentomino problems.

Pi: A map/reduce program that estimates Pi using a quasi-Monte Carlo method.

Randomtextwriter: A map/reduce program that writes 10GB of random textual data per node.

Randomwriter: A map/reduce program that writes 10GB of random data per node.

Secondarysort: An example defining a secondarysort to the reduce.

Sort: A map/reduce program that sorts the data written by the random writer.

Sudoku: A sudoku solver.

Teragen: Generate data for the terasort

Terasort: Run the terasort

Teravalidate: Checking results of terasort

Wordcount: A map/reduce program that counts the words in the input files.

Wordmean: A map/reduce program that counts the average length of the words in the input files.

Wordmedian: A map/reduce program that counts the median length of the words in the input files.

Wordstandarddeviation: A map/reduce program that counts the standarddeviation of the length of the words in the input files.

$cd / usr/local/hadoop

$sudo mkdir. / input

$sudo cp. / etc/hadoop/*.xml. / input # use the configuration file as the input file

$. / bin/hadoop jar. / share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar grep. / input. / output 'dfs [a Murz.] +'

$cat. / output/* # View the running result

Dblab@dblab-VirtualBox:/usr/local/hadoop$ cat. / output/*

1 dfsadmin

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.