Shulou(Shulou.com)06/03 Report--

Execute the following command and find an error

[hdfs@cdhm1 ~]$ hadoop distcp ftp://ftp01:ftppasswd@132.110.64.186/test hdfs://user/dataError: java. net.SocketException: Network unreachable at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.commons.net.SocketClient.connect(SocketClient.java:171) at org.apache.commons.net.SocketClient.connect(SocketClient.java:192) at org.apache.hadoop.fs.ftp.FTPFileSystem.connect(FTPFileSystem.java:133) at org.apache.hadoop.fs.ftp.FTPFileSystem.getFileStatus(FTPFileSystem.java:405) at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:218) at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

Why is that? The order should be correct.

Check the network, normal, ftp can also be transferred normally.

Firewall, selinux have been closed, but the execution error.

No matter how much he thought about it, he could not understand.

After researching a lot of information, he discovered that:

Hadoop does not officially provide ftp interface!

Only personal development tools, feel the installation is quite troublesome, maybe there will be bugs, reference links:

http://www.linuxidc.com/Linux/2013-01/78302.htm

I didn't install it. Now I download the file from ftp to local first, and then download it through

[hdfs@cdhm1 ~]$ hadoop fs -put test /user/data

Upload to hadoop, although troublesome, but win in stability, later can also be simplified through scripts.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.