Shulou(Shulou.com)06/02 Report--

This article mainly introduces "what are the Namenode HA knowledge points". In the daily operation, I believe many people have doubts about the Namenode HA knowledge points. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts about "what are the Namenode HA knowledge points?" Next, please follow the editor to study!

HA (High Available), high availability, is an effective solution to ensure business continuity. Generally, there are two or more nodes, which are divided into active node (Active) and standby node (Standby). The uninterrupted or short-term interruption NN used to achieve business is the single point of failure of the HDFS cluster. In the case of different implementation methods of HA, the flow of HA framework is consistent, and the inconsistency is how to store, manage and synchronize edits editing log files.

QJM/Qurom Journal Manager, the basic principle is to use 2N+1 JournalNode to store EditLog. Every time a write data operation is returned successfully, the write is considered successful, and the data will not be lost. In HA mode, datanode needs to ensure that only one and only one NN can command DN at a time.

FailoverController consists of three main components:

HealthMonitor: monitors whether the NameNode is in a unavailable or unhealthy state. This is currently done by calling the corresponding method of NN through RPC.

ActiveStandbyElector: monitors the status of NN in ZK.

ZKFailoverController: subscribe to HealthMonitor and ActiveStandbyElector events and manage the state of NN, while zkfc is also responsible for solving fencing (that is, brain fissure problems).

The main responsibilities of ZKFailoverController are:

Health monitoring: periodically sends health detection commands to the NN it monitors to determine whether a NameNode is in a healthy state. If the machine goes down and the heartbeat fails, the zkfc marks it in an unhealthy state.

Session management: if NN is healthy, zkfc will keep an open session in zookeeper. If NameNode is also in Active state, zkfc will also have a transient type of znode in Zookeeper. When the NN dies, the znode will be deleted, and then the spare NN will get the lock, upgrade to the primary NN, and mark the status as Active.

When the down NN starts up again, it registers zookeper again and automatically changes to Standby state when it finds that it already has a znode lock. This cycle ensures high reliability. It should be noted that only a maximum of 2 NNmaster elections are supported: a preemptive locking mechanism is implemented by maintaining a transient type of znode in the zookeeper to determine which NameNode is in Active status.

At this point, the study of "what are the knowledge points of Namenode HA" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.