Shulou(Shulou.com)06/01 Report--

Editor to share with you the example analysis of hbase addressing mechanism, I believe that most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Detailed explanation of hbase addressing Mechanism

How does the system find the region big table where a row key (or a row key range (range)) is located using a three-tier structure similar to a B+ tree to save the region location

The first layer is to save the files in the zookeeper, which holds the location of the root region.

The second layer of root region is .meta. The first region of the table holds the location of the other region of the .META.z table. Through root region, we can visit .meta. The data of the table.

.META. Is the third layer, which is a special table that holds the region location information of all data tables in hbase.

/ / see figure

Description:

1 root region will never be split, ensuring that the most need three jumps, can be located to any region.

2.META. Each row of the table holds the location information of one region, and the row key is encoded by the table name plus the last part of the table.

In order to speed up the visit,. META. All region of the table is stored in memory.

Suppose,. META. One row of the table takes up approximately 1KB in memory. And each region is limited to 128MB.

Then the number of region that can be saved by the above three-tier structure is:

(128MB/1KB) * (128MB/1KB) = = 2 (34) region

4 client will save and cache the queried location information, and the cache will not expire actively, so if all the caches on the client fail, it will take 6 network trips to locate the correct region (three of which are used to detect cache failure and the other three are used to obtain location information).

From the above path, we can see that the user needs 3 requests to reach the real location of the user Table, which brings performance degradation in certain programs. The main reason for using a three-tier design before 0.96 was to consider that metadata might be large. But when the real cluster runs, the size of the metadata is easy to calculate. In BigTable's paper, the storage size of each row of METADATA data is about 1KB. According to a calculation with a Region of 128m, the 3-tier design can support 2 ^ 34 Region and the 2-tier design can support 2 ^ 17 (131072). Then in the case of a two-tier design, a cluster can store 4p data. This is only a case where the Region is only 128m. What if it's 10g? Therefore, through calculation, in fact, the two-tier design can meet the needs of the cluster. So the-ROOT- table was removed after version 0.96.

Tip: more versions are different, divided into the old addressing method and the new addressing method.

The above is all the contents of the article "sample Analysis of hbase addressing Mechanism". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.