Hadoop learning-- AP reading HDFS data-- day04

Import java.io.ByteArrayOutputStream

Import java.io.InputStream

Import java.net.URL

Import org.apache.hadoop.conf.Configuration

Import org.apache.hadoop.fs.FSDataInputStream

Import org.apache.hadoop.fs.FileSystem

Import org.apache.hadoop.fs.FsUrlStreamHandlerFactory

Import org.apache.hadoop.fs.Path

Import org.apache.hadoop.io.IOUtils

Import org.junit.Test

/ * *

* Test the API of the hadoop file system

* @ author Administrator

*

, /

Public class TestFileSystemAPI {

/ * *

* read HDFS file data through filesystem object API

*

* @ author Administrator

*

* 1. Create a configuration object. Note that if some parameters are involved, you need to write and specify it in the src directory.

* 2. Using the get method of filesystem to get the filesystem object, there are three get methods (key)

* 3. Use the open method of filesystem to open a data input stream, which requires the parameter transfer of a path object (key).

* 4. Using ioutils tool class to realize the output of files

*

*

*

, /

@ Test

Public void read () throws Exception {

/ / create configuration objects, which have a default loading order, first from core-default.xml to the files in the src directory, here

/ / We have given

Configuration conf = new Configuration ()

/ / the distributed file system fs is created through the configuration object of conf. By default, it is the local file system if no file is specified.

FileSystem fs = FileSystem.get (conf)

/ / define a string of URL

String file = "hdfs://hadoop01:9000/user/hadoop/data/hello.txt"

/ / construct a path object from a string of URL

Path path = new Path (file)

/ / call the open method of fs to create an fsdata input stream to read data.

FSDataInputStream in = fs.open (path)

/ / the first parameter is the input stream, the second parameter is the output stream, the third parameter is the buffer size, and the fourth parameter is the

/ / No data flow is turned off

/ / IOUtils.copyBytes (in, System.out, 1024, true)

Byte [] bys = new byte [1024]

Int len = 0

While ((len=in.read (bys))! =-1) {

System.out.print (new String (bys,0,len))

}

}

}