Shulou(Shulou.com)06/01 Report--

This article mainly introduces "win7 steps to build a pseudo-distributed hadoop". In daily operation, I believe many people have doubts about the steps to build a pseudo-distributed hadoop in win7. Xiaobian consulted all kinds of materials and sorted out simple and easy to use operation methods. I hope to help you answer the doubts about "win7 steps to build a pseudo-distributed hadoop"! Next, please follow the small series to learn together!

window7 install hadoop [32 bit]

1. Download hadoop

hadoop-2.2.0.tar.gz

decompression

2. Configure the environment variable hadoop

3. Modify hadoop-env.sh for %HADOOP_HOME%\etc\hadoop

export JAVA_HOME=D:\Java\jdk1.6.0_10#Cannot have spaces

At the end add [will make mistakes do not add]

set HADOOP_PREFIX=E:\hadoop\hadoop-2.2.0\

set HADOOP_CONF_DIR=%HADOOP_PREFIX%\etc\hadoop

set YARN_CONF_DIR=%HADOOP_CONF_DIR%

set PATH=%PATH%;%HADOOP_PREFIX%\bin

Download hadoop-common-2.2.0-bin-master

Copy the files inside to hadoop/bin

4. Configure core-site.xml

fs.default.name

hdfs://0.0.0.0:19000

5. Configure hdfs-site.xml

dfs.replication

1

6. Configure mapred-site.xml

mapreduce.job.user.name

%USERNAME%

mapreduce.framework.name

yarn

yarn.apps.stagingDir

/user/%USERNAME%/staging

mapreduce.jobtracker.address

local

7. Create yarn-site.xml file

yarn.server.resourcemanager.address

0.0.0.0:8020

yarn.server.resourcemanager.application.expiry.interval

60000

yarn.server.nodemanager.address

0.0.0.0:45454

yarn.nodemanager.aux-services

mapreduce_shuffle

yarn.nodemanager.aux-services.mapreduce.shuffle.class

org.apache.hadoop.mapred.ShuffleHandler

yarn.server.nodemanager.remote-app-log-dir

/app-logs

yarn.nodemanager.log-dirs

/dep/logs/userlogs

yarn.server.mapreduce-appmanager.attempt-listener.bindAddress

0.0.0.0

yarn.server.mapreduce-appmanager.client-service.bindAddress

0.0.0.0

yarn.log-aggregation-enable

true

yarn.log-aggregation.retain-seconds

-1

yarn.application.classpath

%HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_COMMON_HOME%/share/hadoop/common/lib/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/lib/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/lib/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/lib/*

Switch to E:\hadoop\hadoop-2.2.0\etc\hadoop Run the hadoop-env.cmd script to set the current command window execution environment variable

Execute hdfs namenode -format

16/02/27 11:36:23 INFO util.GSet: VM type = 32-bit

16/02/27 11:36:23 INFO util.GSet: 0.029999999329447746% max memory = 992.3 MB

16/02/27 11:36:23 INFO util.GSet: capacity = 2^16 = 65536 entries

16/02/27 11:36:24 INFO common.Storage: Storage directory \tmp\hadoop-goudcheng\d

fs\name has been successfully formatted.

16/02/27 11:36:24 INFO namenode.FSImage: Saving image file \tmp\hadoop-goudcheng

\dfs\name\current\fsimage.ckpt_0000000000000000000 using no compression

16/02/27 11:36:24 INFO namenode.FSImage: Image file \tmp\hadoop-goudcheng\dfs\na

me\current\fsimage.ckpt_0000000000000000000 of size 201 bytes saved in 0 seconds

.

16/02/27 11:36:24 INFO namenode.NNStorageRetentionManager: Going to retain 1 ima

ges with txid >= 0

16/02/27 11:36:24 INFO util.ExitUtil: Exiting with status 0

16/02/27 11:36:24 INFO namenode.NameNode: SHUTDOWN_MSG:

/** **********************************************************

SHUTDOWN_MSG: Shutting down NameNode at CDCH20100020-5/172.31.168.244

At this point, the study of "win7 steps to build a pseudo-distributed hadoop" is over, hoping to solve everyone's doubts. Theory and practice can better match to help everyone learn, go and try it! If you want to continue learning more relevant knowledge, please continue to pay attention to the website, Xiaobian will continue to strive to bring more practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.