Shulou(Shulou.com)05/31 Report--

Xiaobian to share with you the example analysis of big data and Hbase, I believe most people still do not know how to, so share this article for your reference, I hope you have a lot of harvest after reading this article, let's go to understand it together!

Big Data and Hbase

Big data has developed rapidly in recent years, and real-time computing is also an important development trend. Whether it is log data in enterprises, or data generated by sensors, smart devices, etc., countless.

Structured data accounts for only a small portion of this data, and most of it is unstructured data. At this time, for example, pictures and videos cannot be easily stored in relational databases, and big data can be processed for all types of data.

Relational databases, however, have several shortcomings:

Unable to cope with high concurrency, no way to scale out, transaction consistency impact on performance.

Nosql database, which stands for Not Only Sql. Strong scalability, good concurrency performance, flexible data model.

Hbase, or Hadoop Database, is a highly reliable, high-performance, scalable distributed database. Hbase references Google's BigTable modeling, using HDFS as the underlying storage. Use Zookeeper as a collaborative service component.

Hbase is written in Java and is also a NoSQL database. These characteristics determine the unique application scenarios of Hbase.

2. Conceptual characteristics

HBASE is a database----that can provide real-time random access to data.

HBASE differs from relational databases such as mysql, oralce, db2, sqlserver, etc. It is a NoSQL database (non-relational database)

The table model of Hbase differs from that of relational databases:

Hbase tables have no fixed field definitions;

Each row in the Hbase table stores key-value pairs;

The Hbase table has column family divisions, and users can specify which kv to insert into which column family;

Hbase tables are physically stored and divided according to column families, and the data of different column families must be stored in different files;

Each row in the Hbase table has a row key, and the row key of each row cannot be repeated in the table;

The data in Hbase, including row keys, including keys, including values, are all byte[ ] type, hbase is not responsible for maintaining data types for users;

HBASE has poor support for transactions;

Features of HBASE compared to other nosql databases (mongodb, redis, cassendra, hazelcast):

Hbase table data is stored in the HDFS file system

Therefore, hbase has the following characteristics: storage capacity can be linearly expanded; data storage security reliability is extremely high!

3. Core module

Client

The entire Hbase portal, where users operate Hbase through clients. Communicate with HMaster and RegionServer via client. Management operations communicate with HMaster, and read and write operations communicate with RegionServer.

Coordination services zookeeper

Zookeeper is responsible for managing the election of multiple HMasters in the Hbase, and state synchronization between servers.

HMaster

HMaster can start multiple, zookeeper ensures that one always works properly, others as alternatives.

HMaster is mainly responsible for the management of Table and Region.

Node HRegionServer

HRegionServer is mainly responsible for responding to user IO requests and reading and writing data to HDFS. HRegionServer manages a set of HRegion objects. HRegion corresponds to a Region in Table. HRegion consists of multiple HStores, which use a Column Family from Table.

And each HRegionServer has an HLog object for data recovery.

4. Use scene search engine

Generate index, in the query through the stitching of conditions, quickly find the data to be queried.

real-time streaming computing

Whether it is a real-time recommendation system or incremental storage of logs, it is an application of real-time streaming computing.

By storing the incremental data into Hbase and querying Hbase in real time in streaming processing, the final analysis results are obtained by combining history.

The above is all the content of this article "Example Analysis of Big Data and Hbase". Thank you for reading it! I believe that everyone has a certain understanding, hope to share the content to help everyone, if you still want to learn more knowledge, welcome to pay attention to the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.