Shulou(Shulou.com)05/31 Report--

Today, the editor will share with you the relevant knowledge points about how to build Hadoop services in Centos7 in Linux. The content is detailed and the logic is clear. I believe most people still know too much about this, so share this article for your reference. I hope you can get something after reading this article. Let's take a look at it.

Download hadoop

Decompress to / usr/local after download

Tar-zxvf hadoop-2.8.0.tar.gz-c / usr/local

Change hadoop-2.8.0 to hadoop for convenience

Mv / usr/local/hadoop-2.8.0 / usr/local/hadoop

View hostname

Hostname / / the host name of the first parameter

Check whether it is password-free.

Ssh localhost / / localhost hostname here

Note: password is usually required for the first installation.

Ssh-keygen-t dsa-p''- f ~ / .ssh/id_dsacat ~ / .ssh/id_dsa.pub > > ~ / .ssh/authorized_keys

Verify again

Ssh-keygen-t dsa-p''- f ~ / .ssh/id_dsacat ~ / .ssh/id_dsa.pub > > ~ / .ssh/authorized_keys

Note: if there is no enter password, it means no password.

Configure environment variables

Vim / etc/profile

Add at the end

Export hadoop_home=/usr/local/hadoop export path=$java_home/bin:$path:$home/bin:$hadoop_home/bin

Make the environment variable effective immediately

Source / etc/profile

Create a temporary file store for hadoop

Mkdir / usr/local/hadoop/tmp

Edit the configuration file for hadoop

Cd / usr/local/hadoop/etc/hadoop/vim hadoop-env.sh

Add at the end

Export java_home=/usr/local/java/jdk1.8.0_131/ export hadoop_conf_dir=/usr/local/hadoop/etc/hadoop

Note: the path should be correct.

Make the environment variable effective immediately

Source hadoop-env.sh

Configure another file

Vim core-site.xml

Add the following to the

Fs.defaultfs hdfs://localhost:9000 hadoop.tmp.dir file:/usr/local/hadoop/tmp

Configure the next file hdfs-site.xml

Vim hdfs-site.xml

Add the following to the

Dfs.replication 1 dfs.namenode.name.dir file:/usr/local/hadoop/tmp/dfs/name dfs.datanode.data.dir file:/usr/local/hadoop/tmp/dfs/data

Configure the next file, mapred-site.xml, because this file does not exist by default. We configure mapred-site.xml.template as a template.

Cp mapred-site.xml.template mapred-site.xml

Vim mapred-site.xml

Add the following to the

Mapreduce.framework.name yarn

Configure the next file yarn-site.xml

Vim yarn-site.xml

Add the following to the

Yarn.nodemanager.aux-services mapreduce_shuffle

Configure the last file, yarn-env.sh

Vim yarn-env.sh

Note: export java_home around line 23 (delete # uncomment) and change the path of the correct jdk

Format namenode

Cd / usr/local/hadoopbin/hdfs namenode-format

Note: if successful, you will see the prompt of "successfully formatted" and "exitting withstatus 0". If it is "exitting withstatus 1", it is an error.

Start hdfs through a script

Sbin/start-dfs.sh

Open a browser to access http://localhost:50070 and verify that hdfs is configured successfully

And then start yarn.

Sbin/start-yarn.sh

Open a browser to access http://localhost:8088 and verify that yarn is configured successfully

Note: due to a cache problem in the browser, opening the address may fail to close the browser and reopen it once or twice. If it still fails, check whether the configuration file is incorrect.

These are all the contents of the article "how to build Hadoop Services in Centos7 in Linux". Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.