Shulou(Shulou.com)05/31 Report--

Editor to share with you the example analysis of HBase architecture, I believe that most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

1 introduction

It is generally believed that Hadoop consists of the following four major components:

Distributed Lock Service Zookeeper

Distributed file system HDFS

Distributed Computing Framework MapReduce

Distributed database HBase

Among them, Zookeeper is the basic component that HDFS, MapReduce and HBase all rely on. It provides a solution to the consistency problem in distributed systems.

In a narrow sense, Hadoop refers to Hadoop in the broad sense of HDFS and MapReduce;, including HDFS, MapReduce, HBase, and even Hive, Spark and so on.

For more information about the Hadoop biosphere, see: http://www.36dsj.com/archives/23504

Hbase is built on top of HDFS and relies on HDFS and Zookeeper.

2 system architecture

The functions of various nodes are described below:

Client

Client contains interfaces to access HBase and maintains some Cache to speed up access to HBase, such as regione location information.

Zookeeper

Ensure that there is only one master; in the cluster that implements the failover of the Master master-slave node at any time.

Monitor the status of Region Server in real time and inform Master of the online and offline information of Region Server in real time.

The addressing entry for storing the Region.

Stores the schema of the HBase, including what table it has and what column family it has for each table.

Master

Responsible for load balancing of RegionServer.

Manage and assign Region, such as assigning a new Region; when region split and migrating the region within it to other Region Server when Region Server exits.

Responsible for schema update requests, implement DDL operations (such as create to table, column familiy, drop, etc.), and synchronize schema updates to Zookeeper.

Access control (ACL).

Region server

Maintains the region assigned to it by Master and processes IO requests to these region.

Responsible for shredding region that becomes too large during operation.

After Client finds the Region Server where the user's talbe and RowKey are located through Zookeeper and Region Server addressing, it reads and writes data directly from the Region Server. This process does not require the participation of Master, Master only maintains the metadata information of table and region, and the load is very low.

3 read-write addressing

HBase has a special table:. META., which stores the Region location information corresponding to all user table, which is not split, while the ZooKeeper node (/ hbase/meta-region-server) stores. META. The location of the table.

The process for the client to access the user table for the first time is as follows:

Get the location of the hbase:meta (the location of the HRegionServer) from ZooKeeper (/ hbase/meta-region-server) and cache the location information.

Query the Region Server where the user table corresponds to the requested RowKey from the Region Server, and cache the location information.

Read the Row from the queried Region Server.

From this process, we find that the customer will cache the location information, but the second step is only to cache the location of the HRegion corresponding to the current RowKey, so if the next RowKey to be checked is not in the same HRegion, we need to continue to query the HRegion where the hbase:meta is located. However, with the passage of time, the location information cached on the client side is more and more, so that there is no need to look up the hbase:meta Table information again. Unless a Region is moved due to downtime or split, you need to re-query and update the cache.

The above is all the content of the article "sample Analysis of HBase Architecture". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.