In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-01-24 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >
Share
Shulou(Shulou.com)06/03 Report--
First, install Hadoop1. First extract the hadoop to the relevant folder
two。 After entering the extracted folder, modify the relevant files
3. Configure hadoop-env.shvi hadoop-env.sh:export= JAVA_HOME=/opt/inst/jdk181
4. Configure core-site.xmlvi core-site.xml fs.defaultFS hdfs://bigdata:9000 hadoop.tmp.dir / opt/hadoopdata hadoop.proxyuser.root.users * hadoop .proxyuser.root.groups *
5. Configure hdfs-site.xmlvi hdfs-site.xml dfs.replication 1
6. Configure cp mapred-site.xml.template mapred-site.xmlcp mapred-site.xml.template mapred-site.xml
7.vi mapred-site.xml mapreduce.framework.name yarn
8.vi yarn-site.xmlvi
9.vi / etc/profileexport HADOOP_HOME=/opt/bigdata/hadoop260export HADOOP_MAPRED_HOME=$HADOOP_HOMEexport HADOOP_COMMON_HOME=$HADOOP_HOMEexport HADOOP_HDFS_HOME=$HADOOP_HOMEexport YARN_HOME=$HADOOP_HOMEexport HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/nativeexport PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/binexport HADOOP_INSTALL=$HADOOP_HOME
10. Update and format source / etc/profilehdfs namenode-format
11. Start start-all.sh
Jps# View process
twelve。 Import linux into hadoop hdfs dfs-put / opt/a.txt / cm/hdfs dfs-ls / cm
Hadoop understands the three core 1.HDFS: Hadoop Distribute File System distributed file system
NameNode: master node, directory
DataNode: slave nodes, data
SecondaryNameNode: backup of the primary node
2.YARN:Yet Another Resource Negotiator Resource Management and scheduling system
dispatches memory resources and CPU computing power.
is dispatched through ResourceManager (only one)
The main functions of ResourceManager are:
1. Processing client request
2. Monitoring NodeManager
3. Start or monitor ApplicationMaster ()
4. Allocation or scheduling of resources
NodeManager (multiple)
The main functions of NodeManager are:
1. Manage resources on a single node
2. Processing commands from ResourceManager
3. Processing commands from ApplicationMaster
3.MapReduce: distributed Computing Framework
operation
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.