Shulou(Shulou.com)06/03 Report--

Import java.io.ByteArrayOutputStream

Import java.io.InputStream

Import java.net.URL

Import org.apache.hadoop.fs.FsUrlStreamHandlerFactory

Import org.apache.hadoop.io.IOUtils

Import org.junit.Test

Public class TestFileSystemURL {

Static {

URL.setURLStreamHandlerFactory (new FsUrlStreamHandlerFactory ())

}

/ * *

* read HDFS file data through URL object

*

* @ author Administrator

*

* to read data from HDFS, the first thing we consider is to extract data from WEBUI.

* 1. Define a URL object that encapsulates the URL address information of the HDFS

* 2. Call the openStream () method to open an input stream

* 3. Copy the input stream to the byte array output stream through the IOUtils utility class, and then print the information of the output byte output stream

*

, /

@ Test

Public void readByURL () throws Exception {

/ / configure files with URL addresses to be read

String urlStr = "hdfs://hadoop01:9000/user/hadoop/data/hello.txt"

/ / create URL objects through URL's constructor with string

URL url = new URL (urlStr)

/ / call the openStream () method of the URL object to get an inputstream object

InputStream is = url.openStream ()

/ / define an array of characters

/ / byte [] bys = new byte [1024]

/ / int len = 0

/ / there are three parts, read, assign, judge, read one character array at a time, and the return value is the length of the read character array.

/ / at the end of the file, return-1

/ / while ((len=is.read (bys))! =-1) {

/ / outputs an array of characters, part of the array of output characters, from 0 to the end of the read length, with no addition to print.

/ / ln

/ / System.out.print (new String (bys,0,len))

/ /}

/ / create byte array output stream object

ByteArrayOutputStream baos = new ByteArrayOutputStream ()

/ / IOUtils method provided by hadoop to realize stream copy

IOUtils.copyBytes (is, baos, 1024)

/ / close the stream object

IOUtils.closeStream (is)

System.out.println (new String (baos.toByteArray ()

}

}

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.