Shulou(Shulou.com)06/03 Report--

principle

NFS allows users to access remote file systems as if they were local, while the introduction of NFS to HDFS greatly simplifies HDFS usage by allowing users to read and write files on HDFS as if they were local, by introducing an NFS gateway service that converts the NFS protocol to the HDFS access protocol, as shown in the following figure.

Portmap and Nfs3 processes are processes that exist only after the hdfs nfs gateway is successfully started

HDFS NFS Gateway Installation Configuration Details

http://blog.csdn.net/rzliuwei/article/details/38388279

Start NFS Gateway Service

official website link

http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html

verification service

If not, you can check the/etc/exports file.

Mount HDFS

Mount the HDFS file system to the local/mnt/hdfs directory

On the NFS GATEWAY host, type

sudo mount -t nfs -o vers=3,proto=tcp,nolock 192.168.1.105:/ /mnt/hdfs

May 8 15:53:18 cdh2 rpc.mountd[3689]: refused mount request from 192.168.1.105 for / (/): unmatched host

If an error is reported because NFSv3 is not turned on by default, modify the/etc/sysconfig/nfs file.

Remove the #before RPCNFSDARGS="-N 4"(4 is compatible with 2 and 3) and modify to restart all services.

View error details:

cat /var/log/messages|grep mount

After a successful mount, you can see that the HDFS file system has been mounted to the local directory as shown below

create a directory

You can see that the corresponding directory on HDFS system has been successfully created

Upload files locally to HDFS

File size is 500m, testing performance

time cp /opt/cloudera-manager-el6-cm5.3.3_x86_64.tar.gz /mnt/hdfs/test

My environment takes about 48s, 10M/s, and it's OK.

Unhook HDFS

Error umount.nfs: /mnt/hdfs: device is busy

fuser -m /mnt/hdfs to see which processes are accessing this file system

yum -y install lsof

Looking at the files in use in this file system, you can see that the/data directory is in use

kill -9 15355 kill access process can, input umount /mnt/hdfs unhook successfully.

Additional:

Linux is easy to use: tree

Install tree package

yum -y install tree

After installation, type tree /mnt/hdfs/hbase. Is it intuitive to display the directory structure in tree form?

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.