In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-01-29 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >
Share
Shulou(Shulou.com)06/03 Report--
1. Environment selection 1, server selection
Ali CVM: entry-level (postpaid)
Operating system: linux CentOS 6.8
Cpu:1 kernel
Memory: 1G
Hard disk: 40g
Ip:39.108.77.250
2, configuration selection
JDK:1.8 (jdk-8u144-linux-x64.tar.gz)
Hadoop:2.8.2 (hadoop-2.8.2.tar.gz)
3, download address
Official website address:
JDK:
Http://www.oracle.com/technetwork/java/javase/downloads
Hadopp:
Http://www.apache.org/dyn/closer.cgi/hadoop/common
Baidu Cloud:
Link: http://pan.baidu.com/s/1pLqS4kF password: yb79
Second, the configuration of the server
Before configuring Hadoop, you should do the following configuration
1, change the hostname
Change the hostname first in order to facilitate management.
Enter:
Hostname
View the name of this machine
Then enter:
Vim / etc/sysconfig/network
Modify host name
Change the name of HOSTNAME to the name you want to set
Note: after the host name is changed, reboot will not take effect until it is restarted.
Input
Vim / etc/hosts
Add the host IP and the corresponding host name to do the mapping.
Note: this mapping must be done if the hostname is used in the configuration file!
2. Turn off the firewall
Turn off the firewall to facilitate external access.
The following inputs for CentOS 7 version:
Turn off the firewall
Service iptables stop
Input for versions above CentOS 7:
Systemctl stop firewalld.service3, time settin
Enter:
Date
Check whether the server time is the same, and if not, change it.
Change time command
Date-s' MMDDhhmmYYYY.ss' III, Hadoop environment installation 1, download jdk, hadoop
Put the downloaded jdk and hadoop packages in the home directory
And create new java and hadoop folders
3.1.1 extract the file
Enter:
Tar-xvf jdk-8u144-linux-x64.tar.gz
Tar-xvf hadoop-2.8.2.tar.gz
Extract jdk and hadoop, and move files to java and hadoop files, respectively
And rename the folder to jdk1.8 and hadoop2.8
2Det JDK environment configuration
First enter
Java-version
Check to see if JDK is installed, and if so, but the version is not suitable, uninstall it
3.2.1 profile file changes
Edit / etc/profile file
Enter:
Vim / etc/profile
Overall profile:
Export JAVA_HOME=/home/java/jdk1.8export JRE_HOME=/home/java/jdk1.8/jreexport CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/libexport PATH=.:$ {JAVA_HOME} / bin:$PATH
Note: the path of JAVA_HOME is the path of JDK
Remember to enter after editing with vim
Source / etc/profile
Make the configuration effective
After configuration, type java-version to view the version information
3Get Hadoop environment configuration 3.3.1 profile file changes
Edit / etc/profile file
Enter:
Vim / etc/profile
Overall profile:
Export HADOOP_HOME=/home/hadoop/hadoop2.8export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/nativeexport HADOOP_OPTS= "- Djava.library.path=$HADOOP_HOME/lib" export PATH=.:$ {JAVA_HOME} / bin:$ {HADOOP_HOME} / bin:$PATH
Enter:
Source / etc/profile
Make the configuration effective
Modify the configuration file
3.3.2 create a new folder
Before modifying the configuration file, now set up some folders in the root directory.
Enter:
Mkdir / root/hadoop mkdir / root/hadoop/tmp mkdir / root/hadoop/var mkdir / root/hadoop/dfs mkdir / root/hadoop/dfs/name mkdir / root/hadoop/dfs/data
Note: create a new folder under the root directory to prevent it from being deleted inexplicably.
First change to the / home/hadoop/hadoop2.8/etc/hadoop/ directory
3.3.3 modify core-site.xml
Enter:
Vim core-site.xml
Before adding:
Hadoop.tmp.dir / root/hadoop/tmp Abase for other temporary directories. Fs.default.name hdfs://test1:9000
Description: test1 can be replaced with the ip of the host.
3.3.4 modify hadoop-env.sh
Enter:
Vim hadoop-env.sh
Modify ${JAVA_HOME} to your own JDK path
Export JAVA_HOME=$ {JAVA_HOME}
Modified to:
Export JAVA_HOME=/home/java/jdk1.8
3.4.5 modify hdfs-site.xml
Enter:
Vim hdfs-site.xml
Before adding:
Dfs.name.dir / root/hadoop/dfs/name Path on the local filesystem where theNameNode stores the namespace and transactions logs persistently. Dfs.data.dir / root/hadoop/dfs/data Comma separated list of paths on the localfilesystem of a DataNode where it should store its blocks. Dfs.replication 2 dfs.permissions false need not permissions
Description: after dfs.permissions is configured as false, you can generate files on dfs without checking permissions, which is convenient, but you need to prevent accidental deletion, please set it to true, or delete the property node directly, because the default is true.
3.4.6 modify mapred-site.xml
If the file is not mapred-site.xml, copy the mapred-site.xml.template file and rename it to mapred-site.xml.
Enter:
Vim mapred-site.xml
Modify the newly created mapred-site.xml file and add the configuration to the node:
Mapred.job.tracker test1:9001 mapred.local.dir / root/hadoop/var mapreduce.framework.name yarn
Commands used:
At this point, the configuration of the stand-alone mode of Hadoop is complete.
IV. Hadoop start
Initialization is required to start Hadoop for the first time
Change to / home/hadoop/hadoop2.8/bin directory to enter
. / hadoop namenode-format
After successful initialization, you can add a new current directory and some files in the / root/hadoop/dfs/name directory (the path is configured in the hdfs-site.xml file and created a new folder).
Starting Hadoop is mainly to start HDFS and YARN.
Change to the / home/hadoop/hadoop2.8/sbin directory
Start HDFS
Enter:
Start-dfs.sh
Login will ask if you want to connect, enter yes, and then enter the password.
Start YARN
Enter:
Start-yarn.sh
You can enter jps to see if it starts successfully.
Enter in the browser:
Http://39.108.77.250:8088/cluster
Enter:
Http://39.108.77.250:50070
Note: I forgot to take a screenshot at that time. This is a picture of a local environment built with a virtual machine.
At this point, the stand-alone configuration of Hadoop is over.
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 241
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.