Shulou(Shulou.com)05/31 Report--

This article mainly introduces "hadoop 2.4 namenode source code analysis". In daily operation, I believe many people have doubts about hadoop 2.4 namenode source code analysis. Xiaobian consulted all kinds of data and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the doubts of "hadoop 2.4 namenode source code analysis". Next, please follow the editor to study!

In the HA of hadoop nn, the election of active and standby nodes is realized through ActiveStandbyElector. There is an explanation for this class on the source code.

Little brother is not good at English, translate it. This class mainly uses zookeeper to implement the election of the primary node. For the successful primary node, a 00:00 node is created on the zookeeper. If the creation is successful, the NN becomes active and the remaining nn nodes become backup nodes.

Next, let's analyze the role of the ActiveStandbyElector class in detail. ActiveStandbyElector mainly implements the election, and the election process is mainly achieved by creating 00:00 nodes, if the creation is successful. It can be thought of as getting the corresponding LOCK, and the node can be called active. If the node is not created successfully, it can be considered a standby node, and for a standby node, the state of the LOCK node needs to be monitored all the time. If there is an event on the node, try the election. That's the basic process.

Next, take a look at the main methods and processes of the ActiveStandbyElector class. For students who are familiar with zookeeper, zookeeper must implement the watcher interface, which can implement their own logic for handling various events.

In ActiveStandbyElector, the

The inner class implements the Watcher interface, and its process method calls processWatchEvent to implement the specific business processing.

Let's analyze the specific logic of the processWatchEvent:

/ / handle events of zk

Synchronized void processWatchEvent (ZooKeeper zk, WatchedEvent event) {Event.EventType eventType = event.getType (); if (isStaleClient (zk)) return; LOG.debug ("Watcher event type:" + eventType + "with state:" + event.getState () + "for path:" + event.getPath () + "connectionState:" + zkConnectionState + "for" + this) If (eventType = = Event.EventType.None) {/ / the time of the session itself, such as a connection. Lost connection. / / the connection state has changed switch (event.getState ()) {case SyncConnected: LOG.info ("Session connected."); / / if the listener was asked to move to safe state then it needs to / / be undone ConnectionState prevConnectionState = zkConnectionState; zkConnectionState = ConnectionState.CONNECTED; if (prevConnectionState = = ConnectionState.DISCONNECTED & & wantToBeInElection) {monitorActiveStatus () / / Monitoring node} break; case Disconnected: LOG.info ("Session disconnected. Entering neutral mode... "); / / ask the app to move to safe state because zookeeper connection / / is not active and we dont know our state zkConnectionState = ConnectionState.DISCONNECTED; enterNeutralMode (); break; case Expired: / / the connection got terminated because of session timeout / / call listener to reconnect LOG.info (" Session expired. Entering neutral mode and rejoining... "); enterNeutralMode (); reJoinElection (0); / / participating in the election break; case SaslAuthenticated: LOG.info (" Successfully authenticated to ZooKeeper using SASL. "); break; default: fatalError (" Unexpected Zookeeper watch event state: "+ event.getState ()); break;} return } / / a watch on lock path in zookeeper has fired. So something has changed on / / the lock. Ideally we should check that the path is the same as the lock / / path but trusting zookeeper for now / / Node event String path = event.getPath (); if (path! = null) {switch (eventType) {case NodeDeleted: if (state = = State.ACTIVE) {enterNeutralMode (); / / this method is not currently implemented} joinElectionInternal (); / / start electing break Case NodeDataChanged: monitorActiveStatus (); / / continue to monitor the node and try to become active break; default: LOG.debug ("Unexpected node event:" + eventType + "for path:" + path); monitorActiveStatus ();} return;} / / some unexpected error has occurred fatalError ("Unexpected watch error from Zookeeper");}

And joinElectionInternal, the core method of election is

The election is done through the creation of zkLokFilePath nodes. This uses zk's asynchronous callback.

From the definition of this class, we can see that it implements two interfaces of zk.

The methods that StatCallback needs to implement are as follows:

For the implementation of the two methods, the internal implementation of ActiveStandbyElector is almost the same. The source code is no longer posted here, and those who are interested can look at the source code by themselves.

Paste the implementation method, there are comments. He he

Public synchronized void processResult (int rc, String path, Object ctx, String name) {if (isStaleClient (ctx)) return; LOG.debug ("CreateNode result:" + rc + "for path:" + path + "connectionState:" + zkConnectionState + "for" + this); Code code = Code.get (rc) / / for ease of use, a custom set of status if (isSuccess (code)) {/ / is returned successfully, and the zklocakpath node / / we successfully created the znode is created successfully. We are the leader. Start monitoring if (becomeActive ()) {/ / to change the NN on this node into active monitorActiveStatus (); / / continue to monitor node status} else {reJoinElectionAfterFailureToBecomeActive (); / / fail, continue election attempt} return } if (isNodeExists (code)) {/ / node exists, indicating that active,wait already exists and if (createRetryCount = = 0) {/ / znode exists and we did not retry the operation. So a different / / instance has created it. Become standby and monitor lock. BecomeStandby ();} / / if we had retried then the znode could have been created by our first / / attempt to the server (that we lost) and this node exists response is / / for the second attempt. Verify this case via ephemeral node owner. This / / will happen on the callback for monitoring the lock. MonitorActiveStatus (); / / but the effort to be an active cannot be stopped by return;} String errorMessage = "Received create error from Zookeeper. Code: "+ code.toString () +" for path "+ path; LOG.debug (errorMessage); if (shouldRetry (code)) {if (createRetryCount < maxRetryNum) {LOG.debug (" Retrying createNode createRetryCount: "+ createRetryCount); + + createRetryCount; createLockNodeAsync (); return;} errorMessage = errorMessage +". Not retrying further znode create connection errors. ";} else if (isSessionExpired (code)) {/ / This isn't fatal-the client Watcher will re-join the election LOG.warn (" Lock acquisition failed because session was lost "); return;} fatalError (errorMessage);}

For these state changes in becomeStandby,becomeActive, there is ZKFailoverController to achieve.

At this point, the study of "hadoop 2.4namenode source code analysis" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.