Example Analysis of hdfs File Operation
This article mainly shows you the "hdfs file operation example analysis", the content is easy to understand, clear, hope to help you solve doubts, the following let Xiaobian lead you to study and learn "hdfs file operation example analysis" this article.
Import org.apache.hadoop.conf.Configuration
Import org.apache.hadoop.fs.*
Import java.io.File
Import java.io.IOException
/ * * examples of hdfs file operations, including uploading files to HDFS, downloading files from HDFS, and deleting files on HDFS * /
Public class HadoopFile {
Private Configuration conf = null
Public HadoopFile () {
Conf = new Configuration ()
Conf.addResource (new Path ("/ hadoop/etc/hadoop/core-site.xml"))
}
Public HadoopFile (Configuration conf) {
This.conf = conf
}
Public boolean sendFile (String path,String localfile) {
File file=new File (localfile)
If (! file.isFile ()) {
System.out.println (file.getName ())
Return false
}
Try {
FileSystem localFS = FileSystem.getLocal (conf)
FileSystem hadoopFS = FileSystem.get (conf)
Path hadPath=new Path (path)
FSDataOutputStream fsOut=hadoopFS.create (new Path (path+ "/" + file.getName ()
FSDataInputStream fsIn=localFS.open (new Path (localfile))
Byte [] buf = new byte [1024]
Int readbytes=0
While ((readbytes=fsIn.read (buf)) > 0) {
FsOut.write (buf,0,readbytes)
}
FsIn.close ()
FsOut.close ()
FileStatus [] hadfiles= hadoopFS.listStatus (hadPath)
For (FileStatus fs: hadfiles) {
System.out.println (fs.toString ())
}
Return true
} catch (IOException e) {
E.printStackTrace ()
}
Return false
}
Public boolean delFile (String hadfile) {
Try {
FileSystem hadoopFS = FileSystem.get (conf)
Path hadPath=new Path (hadfile)
Path p=hadPath.getParent ()
Boolean rtnval= hadoopFS.delete (hadPath, true)
FileStatus [] hadfiles= hadoopFS.listStatus (p)
For (FileStatus fs: hadfiles) {
System.out.println (fs.toString ())
}
Return rtnval
} catch (IOException e) {
E.printStackTrace ()
}
Return false
}
Public boolean downloadFile (String hadfile,String localPath) {
Try {
FileSystem localFS = FileSystem.getLocal (conf)
FileSystem hadoopFS = FileSystem.get (conf)
Path hadPath=new Path (hadfile)
FSDataOutputStream fsOut=localFS.create (new Path (localPath+ "/" + hadPath.getName ()
FSDataInputStream fsIn=hadoopFS.open (hadPath)
Byte [] buf = new byte [1024]
Int readbytes=0
While ((readbytes=fsIn.read (buf)) > 0) {
FsOut.write (buf,0,readbytes)
}
FsIn.close ()
FsOut.close ()
Return true
} catch (IOException e) {
E.printStackTrace ()
}
Return false
}
}
The above is all the contents of the article "sample Analysis of hdfs File Operation". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!
