Shulou(Shulou.com)06/02 Report--

This article will explain in detail how to use common Hadoop commands, and the content of the article is of high quality, so the editor will share it with you for reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

When learning Hadoop, we can operate on Hadoop better only by understanding how to use the Hadoop command. Let's learn the Hadoop command together.

Version

This Hadoop command is used to print version information.

Usage: hadoopversion

CLASSNAME

The hadoop script can be used to call any class.

Usage: hadoopCLASSNAME

Run a class named CLASSNAME.

Management command

Commands commonly used by hadoop cluster administrators.

Balancer

Run the cluster balancing tool. The administrator can simply press Ctrl-C to stop the balancing process. Refer to Rebalancer for more information.

Usage: hadoopbalancer [- threshold]

Command option description

-percentage of threshold disk capacity. This overrides the default threshold.

Daemonlog

This Hadoop command is used to get or set the log level for each daemon.

Usage: hadoopdaemonlog-getlevel

Usage: hadoopdaemonlog-setlevel

Command option description

-getlevel prints the log level of the daemon running at. Http:///logLevel?log= is connected internally to this command.

-setlevel sets the log level of the daemon running at. Http:///logLevel?log= is connected internally to this command.

Datanode

Run a datanode for HDFS.

Usage: hadoopdatanode [- rollback]

Command option description

-rollback rolls back datanode to the previous version. This needs to be used after stopping datanode and distributing the old version of hadoop.

Dfsadmin

This Hadoop command is used to run a dfsadmin client for HDFS.

Usage: Hadoopdfsadmin [generic _ OPTIONS] [- report] [- safemodeenter | leave | get | wait] [- refreshNodes] [- finalizeUpgrade] [- upgradeProgressstatus | details | force] [- metasavefilename] [- setQuota...] [- clrQuota...] [- help [cmd]]

Command option description

-report reports basic information and statistics of the file system.

-safemodeenter | leave | get | wait safe mode maintenance command. Safe mode is a state of Namenode, in which case Namenode

1. Changes to namespaces are not accepted (read-only)

two。 Do not copy or delete blocks

Namenode automatically enters safe mode at startup and leaves safe mode automatically when the minimum percentage of configured blocks meets the minimum number of copies. Safe mode can be entered manually, but safe mode must also be turned off manually.

-refreshNodes reread the hosts and exclude files, updating collections that allow connections to Namenode or Datanode that need to be exited or compiled.

-finalizeUpgrade terminates the upgrade of HDFS. Datanode deletes the previous version of the working directory, and then Namenode does the same. This operation completes the entire upgrade process.

-upgradeProgressstatus | details | force requests the upgrade status of the current system, details of the status, or mandatory upgrade operation.

-metasavefilename saves the main data structure of Namenode to a file in the directory specified by the hadoop.log.dir attribute.

For each of the following items, there will be a line corresponding to it

Datanode's heartbeat signal received by 1.Namenode

two。 Blocks waiting to be copied

3. The block being copied

4. Blocks waiting to be deleted

-setQuota... Set quotas for each directory.

A directory quota is a long integer that forces the number of names under the directory tree to be limited.

The command will work well in this directory, and the following will report an error:

1. N is not a positive integer, or

two。 The user is not an administrator or

3. This directory does not exist or a file, or

4. The directory will immediately exceed the newly set quota.

-clrQuota... Clear quota settings for each directory.

The command will work well in this directory, and the following will report an error:

1. This directory does not exist or a file, or

two。 The user is not an administrator.

If the directory does not have a quota, it will not report an error.

-help [cmd] displays help for a given command, or for all commands if no command is given.

Jobtracker

This Hadoop command is used to run the MapReducejobTracker node.

Usage: hadoopjobtracker

Namenode

Run namenode. For more information about upgrade, rollback, upgrade termination, please refer to upgrade and rollback.

Usage: hadoopnamenode [- format] | [- upgrade] | [- rollback] | [- finalize] | [- importCheckpoint]

Command option description

-format formats namenode. It starts namenode, formats namenode, and then closes namenode.

-after upgrade distributes a new version of hadoop, namenode should be launched with the upgrade option.

-rollback rolls back namenode to the previous version. This option should be used after stopping the cluster and distributing the older version of hadoop.

-finalizefinalize deletes the previous state of the file system. The most recent upgrade will be persisted, the rollback option will no longer be available, and after the upgrade terminates, it will stop namenode.

-importCheckpoint mounts the image from the checkpoint directory and saves it to the current checkpoint directory, which is specified by fs.checkpoint.dir.

Secondarynamenode

Secondarynamenode that runs HDFS. Refer to SecondaryNamenode for more information.

Usage: hadoopsecondarynamenode [- checkpoint [force]] | [- geteditsize]

Command option description

-checkpoint [force] if the size of EditLog > = fs.checkpoint.size, start the checkpoint process of Secondarynamenode. If-force is used, the size of the EditLog is not considered.

-geteditsize prints the EditLog size.

Tasktracker

The taskTracker node running MapReduce.

Usage: hadooptasktracker.

On the common use of Hadoop commands is shared here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.