Shulou(Shulou.com)05/31 Report--

This article will explain in detail how to ensure the high availability of hdfs. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

How to ensure High availability of hdfs

As we can see from the above figure, when starting, the master / standby election is realized by using zookeeper. The ZKFailoverController process on the hdfs namenode node is mainly responsible for controlling the active / standby switch, monitoring whether the namenode process is still alive, connecting the zookeeper for the election, and controlling the behavior of the local namenode. If it is found that the lock node of the zookeeper has not been preempted, create a lock node and turn the local namenode into an active node. If you find that the lock node has been created, change the native namenode to the standby node.

If the ZKFailoverController finds that the native namenode is dead, it deletes the node on the zookeeper, and the ZKFailoverController on the standby senses the deletion event and upgrades the native namenode to the master node.

If the active node does not connect to the zookeeper for longer than the time configured by the Zookeeper Session Timeout parameter, the master node is also considered to be dead, and the standby node will also be upgraded to the master node.

How to switch between active and standby by hdfs

Similar to the stand-alone file system, HDFS maintains the directory structure of the file system according to the tree structure, and Namespace stores the attributes of the directory tree and each directory / file node. In addition to being resident in memory, this part of the data will be periodically flush to the persistence device to generate a new FsImage file to facilitate the timely recovery of the entire Namespace from the FsImage when the NameNode is restarted.

In the figure above, we can see that sharing metadata is based on journalnode. Editlog is stored in jounalnode, that is, the operation log of file metadata. For example, when a user (hdfs dfs rm / a.txt) deletes a file, an operation log is generated. After acting on the namespace, the a.txt file is killed. This log is similar to the redo log in the database, and the standby node periodically pulls editlog from the journalnode. Then replay from the location of the last checkpoint, merge the results of the operation into the old fsimage (namespace) in memory, and update the checkpoint location. Finally, upload the merged fsimage to active namenode to save and replace the old fsimage file.

When NameNode is in the active and standby states, the roles played are very different.

The active role, which writes the user's operations on the namespace as editlog to the journalnode cluster

Standby role, the EditLogTailer thread pulls the editlog log from the journalnode, the StandbyCheckpointer thread merges the editlog into the fsimage, and then uploads the fsimage file to the active namenode node

When active is hung up, after standby is aware, it is converted to active role to generate a new epoch (that is, each active / standby switch adds one, indicating who is now the boss and who is in whose term of office). Because standby pulls editlog periodically, it is certain that the latest editlog has not yet acted on fsimage and needs to be pulled and replayed, because when writing editlog to journal, as long as most (for example, a total of 5 points, 3 points must be successfully written A total of 3 points, 2 points must be successfully written) A successful write is considered successful, so when pulling and restoring, the data on each journalnode node may be inconsistent. At this time, we must find the largest id point of most of the transactions as a benchmark. This transaction id is similar to Log sequence number (LSN for short) in the database. It is necessary to replay the log between the last checkpoint (LSN) and the largest (LSN) in the redo log, redo, and merge it into fsimage.

Suppose a situation, the old active node, because of network problems, can not connect to the zookeeper, and then is thought to be dead, in fact, it is a fake death, at this time the network is restored, still think that they are leader, write editlog logs to the journalnode, and find that it is now someone else's world (epoch number has already added 1), their own epoch number has been outdated, demote themselves to standby, do what the standby role should do.

Why merge on secondnamnode, mainly to reduce the pressure on namenode, namenode has a lot of other things to do.

Operation

Hadoop-daemon.sh start journalnode, start the journalnode cluster, and start the journalnode cluster according to the machine configured in your configuration file.

Hdfs namenode-format, formatting the namenode namespace

Start namenode on hadoop-daemon.sh start namenode,nn1

Hdfs namenode-bootstrapStandby, nn2 synchronizes the metadata information of nn1, and copies the fsimage editlog directory on the primary node.

Hadoop-daemon.sh start namenode, launch namenode on nn2

Start zk cluster hdfs zkfc-formatZK format directory hadoop-daemon.sh start zkfc in zkfc zk launch dfszk failover controller on each namenode node, which machine starts first, which machine is active namenode

Hadoop-daemon.sh start datanode uploads files to test reading and writing files

Test ha to active namenode process kill, jps process number, kill, and disconnect active namenode machine from network service network stop

When we do the above tests on the cluster and look at the logs, we will find that what hdfs does is basically the process described earlier.

This is the end of the article on "how to ensure high availability of hdfs". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.