Shulou(Shulou.com)06/01 Report--

This article mainly shows you "Hadoop stand-alone mode how to deploy", the content is easy to understand, clear, hope to help you solve your doubts, the following let the editor lead you to study and learn "Hadoop stand-alone mode how to deploy" this article.

I. Hadoop deployment model

Stand-alone mode: runs as a separate Java process on a separate machine by default, mainly for debugging environments

Pseudo-distributed mode: simulated as a distributed multi-node environment on a single machine, with each Hadoop daemon running as a separate Java process

Fully distributed model: real production environment, built in a fully distributed cluster environment

Add users and groups

$sudo adduser hadoop # # create user hadoop$ sudo usermod-G sudo hadoop # # add hadoop user to sudo user group

Third, install related software

$sudo apt-get update$ sudo apt-get-y install openssh-server rsync openjdk-7-jdk$ sudo service ssh restart$ java-version

Configure SSH secret-free login

$su-l hadoop # # switch to hadoop user $ssh-keygen-t rsa-P "" # # configure SSH secret-free login $cat ~ / .ssh/id_rsa.pub > > ~ / .ssh/authorized_keys # # append the public key to the authorized_keys $ssh localhost # # authentication

V. Hadoop configuration

① download and install Hadoop

$su-l hadoop$ wget http://apache.fayea.com/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz $tar-zxvf hadoop-2.7.1.tar.gz$ sudo mv hadoop-2.7.1 / usr/local/hadoop

② configuration Hadoop

a. Add environment variabl

$vi / home/hadoop/.bashrc#HADOOP STARTexport JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64export HADOOP_INSTALL=/usr/local/hadoop/export PATH=$PATH:$HADOOP_INSTALL/binexport PATH=$PATH:$HADOOP_INSTALL/sbinexport HADOOP_MAPRED_HOME=$HADOOP_INSTALLexport HADOOP_COMMON_HOME=$HADOOP_INSTALLexport HADOOP_HDFS_HOME=$HADOOP_INSTALLexport YARN_HOME=$HADOOP_INSTALLexport HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/nativeexport HADOOP_OPTS= "- Djava.library.path=$HADOOP_INSTALL/lib" # HADOOP END$. / home/hadoop/.bashrc # # overload to make it effective

VI. Testing

① create input data

$su-l hadoop$ cd / usr/local/hadoop$ sudo mkdir input$ sudo cp / etc/passwd input/

② executes Hadoop WordCount applications (word frequency statistics)

$bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.1-sources.jar org.apache.hadoop.examples.WordCount input output

③ views generated word statistics

$hadoop dfs-ls output-rw-r--r-- 1 hadoop hadoop 0 2015-12-18 13:18 output/_SUCCESS-rw-r--r-- 1 hadoop hadoop 2128 2015-12-18 13:18 output/part-r-00000$ hadoop dfs-cat output/*

These are all the contents of the article "how to deploy Hadoop stand-alone mode". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.