Shulou(Shulou.com)06/01 Report--

Hbase implementation

The Hbase is coordinated by a Master node to manage one or more RegionServer slave machines. The Master is responsible for starting, allocating the area to the registered RegionServer and recovering the failure of the RegionServer. The Master load is very light. RegionServer is responsible for managing zero or more regions and responding to client read and write requests. RegionServer is also responsible for dividing regions and informing Master that Hbase depends on Zookeeper. If there is a server crash in the process of zone allocation, it is coordinated and allocated through Zookeeper. Allocating the transaction state in Zookeeper helps to continue the allocation from the state left over from the crash during recovery. When initiating a client connection to the cluster, the client must at least get the overall location of the Zookeeper passed by the cluster. In this way, the client can access the Zookeeper hierarchy and understand the cluster properties, such as the location of the server.

Hbase in operation

Hbase retains-ROOT- and .meta. The root table maintains the information of the Meta table, the Meta table maintains the information of the user table, the items in the Meta table use the region name as the primary key, and the region name consists of the table name, the starting row of the region, and the hash result of the created timestamp. When the area changes, it splits, disables / enables. When deleting, redeploying the machine for load balancer or redeploying the area due to Regionserver crash, the directory table will be updated accordingly, so that the information of all regions on the cluster can be kept up-to-date.

The client accesses the remote node three times for each row operation:

1. Get the location of the Master from Zookeeper

two。 Get .Meta. From Master. Information about the table

3. According to .Meta. Table information to get region location information

In order to reduce access to remote nodes, Hbase clients cache the information they get when traversing the ROOT table, the location of the Meta table and the start and end rows of the region in the user space, so that they can know the location of the region without accessing the Meta table. When the client encounters an error, it will check the new location of the Meta fetch area. If the .Meta has also moved, it will query the ROOT table.

Client

1 contains interfaces to access hbase, and client maintains some cache to speed up access to hbase, such as regione location information.

Zookeeper

1 guarantee that there is only one master in the cluster at any time

2 store the addressing entry of all Region.

3 monitor the status of Region Server in real time, and inform Master of the online and offline information of Region server in real time.

4. Store the schema of Hbase, including what table there are and what column family each table has.

Master

1 assign region to Region server

2 be responsible for load balancing of region server

3 find the failed region server and redistribute the region on it

4 garbage file collection on GFS

5 process schema update request

Region Server

1 Region server maintains the region assigned to it by Master and processes IO requests for these region

2 Region server is responsible for shredding region that becomes too large during operation

As you can see, client does not need master to participate in the process of accessing data on hbase (addressing access zookeeper and region server, data read and write access regione server). Master only maintains the metadata information of table and region, and the load is very low.

Region positioning

How does the system find the region where a row key (or a row key range) is located

Bigtable uses a three-tier structure similar to a B+ tree to hold region locations.

The first layer is to save the files in the zookeeper, which holds the location of the root region.

The second layer of root region is .meta. The first region of the table holds the location of the other region of the .META.z table. Through root region, we can visit .meta. The data of the table.

.META. Is the third layer, which is a special table that holds the region location information of all data tables in hbase.

Description:

1 root region will never be split, ensuring that the most need three jumps, can be located to any region.

2.META. Each row of the table holds the location information of one region, and the row key is encoded with the table name plus the last part of the table.

In order to speed up the visit,. META. All region of the table is stored in memory.

Suppose,. META. One row of the table takes up approximately 1KB in memory. And each region is limited to 128MB.

Then the number of region that can be saved by the above three-tier structure is:

(128MB/1KB) * (128MB/1KB) = = 2 (34) region

4 client will save and cache the queried location information, and the cache will not expire actively, so if all the caches on the client fail, it will take 6 network trips to locate the correct region (three of which are used to detect cache failure and the other three are used to obtain location information).

Reading and writing process

As mentioned earlier, hbase uses MemStore and StoreFile to store updates to the table.

When the data is updated, it is first written to Log (WAL log) and memory (MemStore). The data in MemStore is sorted. When MemStore accumulates to a certain threshold, a new MemStore is created, and the old MemStore is added to the flush queue, and a separate thread flush to disk to become a StoreFile. At the same time, the system records a redo point in zookeeper, indicating that the changes made before this time have been persisted. (minor compact)

When an accident occurs on the system, which may result in data loss in memory (MemStore), Log (WAL log) is used to recover the data after checkpoint.

As mentioned earlier, StoreFile is read-only and cannot be modified once created. So the update of Hbase is actually an additional operation. When the StoreFile in a Store reaches a certain threshold, there will be a major compact, merging the changes to the same key together to form a large StoreFile. When the size of the StoreFile reaches a certain threshold, the StoreFile will be split, which is divided into two StoreFile.

Because updates to the table are constantly appended, when processing read requests, you need to access all StoreFile and MemStore in Store and merge them according to row key. Because StoreFile and MemStore are sorted, and StoreFile has in-memory indexes, the merging process is relatively fast.

Write request processing process

1 client submits a write request to region server

2 region server find the target region

3 region checks whether the data is consistent with schema

4 if the client does not specify a version, get the current system time as the data version

5 write updates to WAL log

6 write updates to Memstore

7 to determine whether the flush of Memstore needs to be a Store file.

Region allocation

A region can only be assigned to one region server at any one time. Master keeps track of what region server is currently available. And which region is currently assigned to which region server and which region has not yet been assigned. When there is an unallocated region and there is free space on a region server, the master sends a load request to the region server and assigns the region to the region server. After the region server is requested, it begins to provide services to this region.

Region server is online

Master uses zookeeper to track region server status. When a region server starts, it first creates a file that represents itself in the server directory on zookeeper, and acquires an exclusive lock on the file. Because master subscribes to change messages in the server directory, master can get real-time notification from zookeeper when files are added or deleted in the server directory. So as soon as region server comes online, master can get the news immediately.

Region server offline

When region server goes offline, it disconnects its session with zookeeper, and zookeeper automatically releases the exclusive lock on the file that represents the server. On the other hand, master constantly polls the lock status of files in the server directory. If master finds that a region server has lost its own exclusive lock (or if master has been unable to communicate with region server several times in a row), master is trying to acquire a read-write lock on behalf of that region server. Once successful, it can be determined:

1 the network between region server and zookeeper is disconnected.

2 region server is dead.

In either case, region server can no longer serve its region, and master will delete the file that represents the region server in the server directory and assign the region of this region server to other comrades who are still alive.

If a brief problem with the network causes region server to lose its lock, after region server reconnects to zookeeper, as long as the file representing it is still there, it will keep trying to acquire the lock on that file, and once it has acquired it, it can continue to provide services.

Master is online

Master starts with the following steps:

1 obtain the lock of the only code master from the zookeeper to prevent other master from becoming master.

2 scan the server directory on zookeeper to get a list of currently available region server.

Each region server in 3 and 2 communicates to obtain the corresponding relationship between the currently allocated region and region server.

4 scan the collection of .META.region, calculate the currently unallocated region, and put them in the list of region to be allocated.

Master offline

Since master only maintains the metadata of tables and region, and does not participate in the IO process of table data, master offline only causes all metadata modifications to be frozen (unable to create deleted tables, cannot modify schema of tables, cannot load balance of region, cannot handle region uploading and unloading, cannot merge region, the only exception is that split of region can be carried out normally, because only region server participates), and data reading and writing of tables can be carried out normally. Therefore, master offline has no impact on the entire hbase cluster in a short period of time. As you can see from the launch process, all the information saved by master can be redundant (all can be collected or calculated from other parts of the system). Therefore, in general, there is always a master providing services in a hbase cluster, and there is more than one 'master' waiting for the opportunity to seize its location.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.