Shulou(Shulou.com)05/31 Report--

This article introduces what the solution of HMaster hanging up after HBase startup is, the content is very detailed, interested friends can refer to it, hope to be helpful to you.

Recently, when using HBase, an error was found when using the list command of HBase shell. Jps checked the process and found that HMaster was down. After confirming that the Hadoop status is normal, check the HMaster log and find that the error is as follows:

2015-02-17 05 lock.ZKInterProcessLockBase: Released / hbase/table-lock/hbase:namespace/write-master:6000000000000042015-02-17 05 FATAL [master:master:60000] master.HMaster: Master server abort: loaded coprocessors are: [] 2015-02-17 05 DEBUG 46 master:master:60000 15212 FATAL [master:master:60000] master.HMaster: Unhandled exception. Starting shutdown. Org.apache.hadoop.hbase.TableExistsException: hbase:namespace at org.apache.hadoop.hbase.master.handler.CreateTableHandler.prepare (CreateTableHandler.java:120) at org.apache.hadoop.hbase.master.TableNamespaceManager.createNamespaceTable (TableNamespaceManager.java:232) at org.apache.hadoop.hbase.master.TableNamespaceManager.start (TableNamespaceManager.java:86) at org.apache.hadoop.hbase.master.HMaster.initNamespace (HMaster.java:1049) at org .apache.hadoop.hbase.master.HMaster.finishinitialization (HMaster.java:913) at org.apache.hadoop.hbase.master.HMaster.run (HMaster.java:606) at java.lang.Thread.run (Unknown Source) 2015-02-17 05Frey 46at java.lang.Thread.run 15214 INFO [master:master:60000] master.HMaster: Aborting2015-02-170546VRV 15214 INFO [master 60000Legend 1424180766819 balancer.BalancerChore BalancerChore] balancer.BalancerChore: master,60000,1424180766819-BalancerChore exiting2015-02-1705 master:master:60000 46 master,60000,1424180766819-ClusterStatusChore 15215 INFO [master,60000,1424180766819-ClusterStatusChore] balancer.ClusterStatusChore: master,60000,1424180766819-ClusterStatusChore exiting2015-02-1705 5V 46RV 15215 INFO [CatalogJanitor-master:60000] master.CatalogJanitor: CatalogJanitor-master:60000 exiting2015-02-1705V 46RV 15216 DEBUG [master:master:60000] master.HMaster: Stopping service threads

There are two FATAL (lines 2 and 3), serious errors, intuitive feeling related to Zookeeper, after trying a variety of methods, finally found the correct solution, this solution comes from the polaris god on Stack Overflow (the original URL is attached at the end, interested can take a look).

4 steps to solve the problem:

1. Stop the HBase cluster

2. Use the offline repair command of HBase

Hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair

3. Delete the old information of HBase that already exists in Zookeeper

Enter the zookeeper client, and note that you want to enter the client when the zookeeper cluster starts:

. / opt/zookeeper/bin/zkCli.sh

Use ls / to view the data directory in zookeeper

Delete hbase data in zookeeper using rm / hbase

4. Restart the HBase cluster and the cluster returns to normal.

Think about:

After solving this problem, I have been reflecting on why this situation suddenly appeared in the cluster, and finally figured it out. In the previous test of the cluster, a zookeeper node was deployed on the primary node (no zookeeper was previously deployed on the primary node), and then the zookeeper node was deleted to prevent the total number of zookeeper nodes from becoming even. This may be the reason why there is a problem with the hbase data that already exists in zookeeper, so emptying the hbase data in zookeeper can solve the problem.

About what the solution of HMaster hanging up after HBase startup is shared here, I hope the above content can be of some help to everyone and learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.