Shulou(Shulou.com)06/01 Report--

This article shows you how to copy files across clusters with the Hadoop distcp command. The content is concise and easy to understand, and it will definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

Hadoop provides Hadoop distcp commands for data replication and copy between different clusters of Hadoop.

The format is: hadoop distcp-pbc hdfs://namenode1/test hdfs://namenode2/test

Distcp copy, only Map, no Reduce.

Usage: distcp OPTIONS [source_path...]

OPTIONS

-append Reuse existing data in target files and append new

Data to them if possible

-async Should distcp execution be blocking

-atomic Commit all changes or none

-bandwidth Specify bandwidth per map in MB

-delete Delete from target, files missing in source

-diff Use snapshot diff report to identify the

Difference between source and target

-f List of files that need to be copied

-filelimit (Deprecated!) Limit number of files copied to

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.