Shulou(Shulou.com)05/31 Report--

This article mainly introduces "what is the refresh mode of MemStore". In the daily operation, I believe that many people have doubts about the refresh mode of MemStore. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts about "what is the refresh mode of MemStore?" Next, please follow the editor to study!

About the refresh method of MemStore:

1. When the MemStore reaches the upper limit, the default hbase.hregion.memstore.flush.size is 128MB, the MemStore refresh will be triggered. This parameter represents the threshold for the size of a single MemStore. The write operation is not blocked at this time.

2. When the total MemStore of a Region reaches hbase.hregion.memstore.block.multiplier * hbase.hregion.memstore.flush.size (the default 2*128M=256M), the write operation of the region will be blocked and the write will be forced to HFile. Triggering this refresh will only happen when the put has a huge record when the MemStore is about to be full of 128m, which blocks the write operation and forces the refresh to succeed before the write can continue.

3. A RegionServer will have many Region and a lot of MemStore, so it is possible that a single Region does not exceed the threshold, but the memory of the whole RegionServer has taken up a lot of memory. At this time, there are two other parameters that control memory brushing: hbase.regionserver.global.memstore.upperLimit defaults to 0.4, when all MemStore on the RegionServer occupies more than 40% of the heap (the size of the heap is set in hbase-env.sh, the default is 1G, the 4G is set by us). Forcibly block all writes and write all MemStore to HFile Hbase.regionserver.global.memstore.lowerLimit defaults to 0.35, which means that when all MemStore occupies more than 35% of heap, some memory-intensive MemStore blocking writes will be selected and flush will be performed. This is to reduce the problems caused by blocking all write operations flush.

4. When the HLog reaches the maximum value (hbase.regionserver.maxlogs * hbase.regionserver.hlog.blocksize defaults to 32mm 64m = 2G), the refresh of MemStore will also be triggered to force the update to be solidified into HFile to avoid excessive recovery time during RegionServer crash.

5. MemStore is refreshed regularly. Hbase.regionserver.optionalcacheflushinterval defaults to 3600000, one hour, to ensure that the data of MemStore will not be solidified into HFile for a long time. In order to avoid the problems caused by all MemStore flush at the same time, regular flush operations have a random delay of about 20000.

6. You can perform flush operations manually. When you call flush in hbase shell, you can flush a table or region:

Hbase (main): 010 help 0 > flush'

Flush all regions in passed table or pass a region row to

Flush an individual region. For example:

Hbase > flush 'TABLENAME'

Hbase > flush 'REGIONNAME'

At this point, the study of "what is the refresh mode of MemStore" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.