Shulou(Shulou.com)06/01 Report--

This article is about how to use the Hadoop archiving tool. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Each file is stored in blocks, and the metadata for each block is stored in namenode memory.

Hadoop archive file or HAR file is a more efficient file archiving tool that stores files in HDFS blocks, allowing transparent access to files while reducing memory usage.

Hadoop archive files can be used as input to MapReduce

Use the Hadoop archiving tool

Hadoop archives are created from a set of files through the archive tool, which runs a MapReduce job to process all input files in parallel

Use archive

Hadoop archive-archiveName files.har / my/files / my

The first option is the name of the archive file, here is the first parameter file.har

The second parameter is the file to be archived

The third parameter is the output directory of the HAR file

List the files in the HAR file

Hadoop fs-ls / my/files.har

Recursively list files in HAR files

Hadoop fs-lsr / my/files.har

If you reference HAR files in other file systems, you need to use different URI path formats, as shown in the following example

Hadoop fs-lsr insufficient to create a new archive file will create a copy of the original file once created, cannot modify the InputFormat do not know the file has been archived thank you for reading! This is the end of this article on "how to use the Hadoop Archiving tool". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.