Shulou(Shulou.com)06/01 Report--

This article mainly shows you "how to install and deploy hadoop", which is easy to understand and clear. I hope it can help you solve your doubts. Let me lead you to study and learn this article "how to install and deploy hadoop".

One: preparation work 1: preparation number JDK environment 2: turn off firewall 3:selinux disabled 4: ssh installation 5: modify host file 6: modify hostname

Two: download CDH http://archive.cloudera.com/cdh5/cdh/4/

Three: configure export JAVA_HOME=/home/bigdata/jdk1.7.0_45 with environment variables

Export HADOOP_HOME=/home/hadoop/cdh5.4.0/hadoop-2.0.0-cdh5.4.0

Export HADOOP_COMMOM_HOME=$HADOOP_HOME

Export HADOOP_HDFS_HOME=$HADOOP_HOME

Export HADOOP_MAPRED_HOME=$HADOOP_HOME

Export HADOOP_YARN_HOME=$HADOOP_HOME

Export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop

Export HDFS_CONF_DIR=$HADOOP_HOME/etc/hadoop

Export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop

Export HADOOP_LIB=$HADOOP_HOME/lib

Export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native

Export PATH=$PATH:/etc/haproxy/sbin/:$JAVA_HOME/bin:$JAVA_HOME/jre/bin export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar:$HADOOP_LIB/native/libhadoop.so

Four: configuration file core-site.xml

Fs.default.name

Hdfs://hadoop-001:8020 hadoop.tmp.dir / hadoop/tmp fs.trash.interval 10080 fs.trash.checkpoint.interval 10080 hadoop.proxyuser.hadoop.hosts hadoop-001 hadoop.proxyuser.hadoop.groups hadoop

Hdfs-site.xml

Dfs.replication 2 dfs.namenode.name.dir file:/hadoop/name ture dfs.datanode.data.dir file:/hadoop/data ture dfs.permissions false dfs.namenode.http-address hadoop-001:50070 dfs.secondary.http.address hadoop-001:50090 dfs.webhdfs.enabled true

Yarn-site.xml

Yarn.resourcemanager.resource-tracker.address

Hadoop-001:18025

Yarn.resourcemanager.address

Hadoop-001:18040

Yarn.resourcemanager.scheduler.address

Hadoop-001:18030

Yarn.resourcemanager.admin.address

Hadoop-001:18141

Yarn.resourcemanager.webapp.address

Hadoop-001:8088

Yarn.nodemanager.aux-services

Mapreduce.shuffle

Yarn.nodemanager.aux-services.mapreduce.shuffle.class

Org.apache.hadoop.mapred.ShuffleHandler

Yarn.application.classpath

$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/share/hadoop/common/,$HADOOP_COMMON_HOME/share/hadoop/common/lib/,$HADOOP_HDFS_HOME/share/hadoop/hdfs/,$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/,$HADOOP_YARN_HOME/share/hadoop/yarn/,$HADOOP_YARN_HOME/share/hadoop/yarn/lib/

Mapred-site.xml

Mapreduce.framework.name yarn mapreduce.jobhistory.address hadoop-001:10020 mapreduce.jobhistory.webapp.address hadoop-001:19888 mapreduce.job.tracker hadoop-001:8021 ture mapred.system.dir file:/hadoop/mapred/system Ture mapred.local.dir file:/hadoop/mapred/local ture mapred.child.env LD_LIBRARY_PATH=/usr/local/lib

Five: create the directory in the hdfs configuration file

Six: format namenode hadoop namenode-format

Seven: start their respective services

These are all the contents of the article "how to install and deploy hadoop". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.