Shulou(Shulou.com)06/03 Report--

Install hadoop on mac

1 java under the Magi Mac

Ll / usr/bin | grep'java'

Default:

/ System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/java

Specific installation path:

/ Library/Java/JavaVirtualMachines/ under

JAVA_HOME is / Library/Java/JavaVirtualMachines/jdk1.7.0_75/Contents/Home.

2 ssh under the mac

Test ssh localhost first

If it is not successful, see if you can start allow remotelogin, and if it fails, continue with the following

Ssh-keygen-t dsa-P'- f ~ / .ssh/id_dsa

Ssh-keygen means generate secret key.

-t represents the secret key type;-P is used to provide the password; and-f specifies the generated key file.

This command creates two files, id_dsa and id_dsa.pub, under the "~ / .ssh /" folder, which are a pair of private and public keys for ssh.

Next, append the public key to the authorized key and type:

Cat~/.ssh/id_dsa.pub > > ~ / .ssh/authorized_keys

Try again

Ssh localhost

If it cannot be opened, the solution is as follows:

Go to Mac system preferences-> share-> check remote login and set allow access for all users.

3. Download the image, mirror, and select a tar.gz

Http://mirror.bit.edu.cn/apache/hadoop/common/

4, set the environment variable

Vi ~ / .bash_profile or / etc/profile

ExportJAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_75/Contents/Home

ExportJRE_HOME=$JAVA_HOME/jre

ExportHADOOP_HOME=/Users/xiaolong.zhao/ittools/hadoop

# is to prevent the warning error of Warning:$HADOOP_HOME is deprecated.

ExportHADOOP_HOME_WARN_SUPPRESS=1

ExportPATH=$JAVA_HOME/bin:$JRE_HOME/bin: $HADOOP_HOME/ sbin:$HADOOP_HOME/bin:$PATH

Note: the only difference between the installation of linux and mac is that there is a default jdk path under mac and a configuration of ssh

5, configure hadoop-env.sh

In the / Users/xiaolong.zhao/ittools/hadoop/etc/hadoop directory, find hadoop-env.sh, open the edit and set it as follows:

Export JAVA_HOME=$ {JAVA_HOME} (remove comments)

Export HADOOP_HEAPSIZE=2000 (remove comments)

Export HADOOP_OPTS= "- Djava.security.krb5.realm=OX.AC.UK-Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk" (remove comments)

6, 4 core file configuration

6.1. Configure core-site.xml-- (defaultFS) specifies the hostname and port of the NameNode

Fs.defaultFS hdfs://localhost:8000 hadoop.tmp.dir / Users/time/Documents/hadoop-2.7.3/tmp A base for othertemporary directories.

6.2. Configuration hdfs-site.xml-- specifies the number of default parameter copies for HDFS

Dfs.replication 1 dfs.namenode.name.dir file:/home/hdfs/name dfs.namenode.http-address localhost:57000 dfs.datanode.data.dir file:/home/hdfs/data

Notice the file here.

If it is configured as: file:/path/a/c, the specific path is: / path/a/c

If it is configured as: file://path/a/c, the specific path is: / a _ c

6.3 mapred-site.xml configuration

Mapreduce.framework.name yarn

6.4 Yarn-site.xml configuration

Yarn.resourcemanager.address localhost:18040 yarn.resourcemanager.scheduler.address localhost:18030 yarn.resourcemanager.resource-tracker.address localhost:18025 yarn.resourcemanager.webapp.address localhost:18088 yarn.nodemanager.aux-services mapreduce_shuffle

7, format namenode

Hdfs namenode-format

8, start hadoop

8.1sbin/start-all.sh

8.2 web Test

8.2.1 namenode-web

Http://localhost:50070

8.2.2resourcemanager-web

Http://localhost:18088

8.2.3 nodemanager-web

Http://localhost:8042

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.