Shulou(Shulou.com)06/01 Report--

Editor to share with you the hadoop stand-alone version of the installation and basic features of the example analysis, I believe that most people do not know much about it, so share this article for your reference, I hope you will learn a lot after reading this article, let's go to understand it!

Hadoop stand-alone installation preparation work create user useradd-m hadooppasswd hadoop download installation package

1.jdk goes to the official download page http://www.oracle.com/technetwork/java/javase/downloads/index.html and downloads jdk. The version I downloaded is jdk-8u151-linux-x64.tar.gz.

The 2.hadoop installation package goes to the hadoop official website to download the hadoop binary package. The version I use is: hadoop-2.7.5.tar.gz.

Upload the packages of jdk and hadoop to linux at the path: / home/hadoop.

Hadoop installation and configuration installation

Log in as a hadoop user and extract the packages for jdk and hadoop, respectively.

Tar-zxvf jdk-8u151-linux-x64.tar.gztar-zxvf hadoop-2.7.5.tar.gz

Rename the directory of hadoop

Mv hadoop-2.7.5/ hadoop/hadoop runtime environment configuration

Go to hadoop's directory / home/hadoop/hadoop vi etc/hadoop/hadoop-env.sh

Set the path of jdk

# The java implementation to use.export JAVA_HOME=/home/hadoop/jdk1.8.0_151 uses hadoop for word statistics to create input file directories and test files mkdir input4testvi input4test/1.txthadoop is an open-source softwarevi input4test/2.txtI want to learn hadoop to perform word statistics operation bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.5.jar wordcount. / input4test. / output4test to view the statistical results cat output4test/part-r-00000I 1an 1hadoop 2is 1learn 1open-source 1software 1to 1want 1hdfs related configuration and operation configuration hdfs

Go to the directory of hadoop / home/hadoop/hadoop to modify the configuration file etc/hadoop/core-site.xml

Fs.defaultFS hdfs://localhost:9000

Modify the configuration file etc/hadoop/hdfs-site.xml

Dfs.replication 1 sets password-free login to $ssh-keygen-t rsa-P''- f ~ / .ssh/id_rsa$ cat ~ / .ssh/id_rsa.pub > > ~ / .ssh/authorized_keys$ chmod 0600 ~ / .ssh/authorized_keyshdfs format and launch

Hdfs formatting

Bin/hdfs namenode-format

Hdfs start

Sbin/start-dfs.sh

View hdfs status enter http://IP:50070 in the browser to view the status of hdfs. If port 50070 is not open, enter the following command to open port 50070.

Iptables-I INPUT-p tcp-- dport 50070-j ACCEPThdfs basic operations view files and folders bin/hadoop dfs-ls-R /

This command means to view all files and folders in the root directory recursively.

Create the directory bin/hadoop dfs-mkdir / input

This command means to create the input directory under the root directory.

Upload files to hdfsbin/hadoop hdfs-put input4test/1.txt / input/bin/hadoop hdfs-put input4test/2.txt / input/

This command means to upload the local input4test/1.txt and 2.txt files to the / input directory in hdfs.

# View file content

Bin/hadoop dfs-cat / input/1.txt configure YARN via YARN scheduling

Modify the configuration file etc/hadoop/mapred-site.xml

Mapreduce.framework.name yarn

Modify the configuration file etc/hadoop/yarn-site.xml

Yarn.nodemanager.aux-services mapreduce_shuffle yarn.nodemanager.env-whitelist JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME starts YARNsbin/start-yarn.sh to view YARN status

Enter http://IP:8088 in the browser to view the status of YARN. If port 8088 is not open, enter the following command to open port 8088.

Iptables-I INPUT-p tcp-- dport 8088-j ACCEPT performs word count operation bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.5.jar wordcount / input / output again

The input and output directories of this execution are all hdfs directories.

Check the statistical results bin/hadoop dfs-cat / output/part-r-00000

The result is the same as before.

I 1an 1hadoop 2is 1learn 1open-source 1software 1to 1want 1 and above are all the contents of this article entitled "sample Analysis of hadoop stand-alone installation and basic functions". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.