Shulou(Shulou.com)05/31 Report--

This article focuses on "how to remove a node from a cluster by hadoop". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how hadoop removes a node from a cluster.

1. First check the Average block replication of the whole cluster. If it is greater than 2, even if you pull out a node directly, you will not lose data.

Hadoop fsck / check the file system status of the cluster

The redundancy multiple of the file can be set manually, and for safe backup, hadoop fs-setrep-w 3-R can be used for critical data.

two。 Add configuration to the following two files and fill in the name of the node to be dismissed in the excludes

Mapred-site.xml

Mapred.hosts

Names a file that contains the list of nodes that may

Connect to the jobtracker. If the value is empty, all hosts are

Permitted.

Mapred.hosts.exclude

HADOOP_HOME/conf/excludes

Names a file that contains the list of hosts that

Should be excluded by the jobtracker. If the value is empty, no

Hosts are excluded.

Hdfs-site.xml

Dfs.hosts

Names a file that contains a list of hosts that are

Permitted to connect to the namenode. The full pathname of the file

Must be specified. If the value is empty, all hosts are

Permitted.

Dfs.hosts.exclude

HADOOP_HOME/conf/excludes

Names a file that contains a list of hosts that are

Not permitted to connect to the namenode. The full pathname of the

File must be specified. If the value is empty, no hosts are

Excluded.

Configure the hostname of the machine in the excludes file.

Run on namenode: hadoop dfsadmin-refreshNodes

Run on jobtracker: hadoop mradmin-refreshNodes

The "hadoop dfsadmin-refreshNodes" triggers the Decommission process, and during the Decommission process, the cluster redundancy the data on the Decommission node to several other points

At this point, I believe you have a deeper understanding of "how hadoop removes a node from the cluster". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.