Shulou(Shulou.com)06/02 Report--

Hive how to manipulate Hbase data through external tables, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain in detail for you, people with this need can come to learn, I hope you can get something.

Overview:

HBase:

Query efficiency is relatively high, often providing services for real-time services, but the query method is relatively simple, only through the row way to get a single piece of data, or through scan plus filter to scan the data table to obtain data.

Hive:

Hive is used to store structured data, and is often used to build data warehouse as data storage for offline analysis, supporting HQL operations, but for complex HQL will be transformed into mapreduce tasks, the execution is relatively slow.

Sometimes our data is stored in HBase, but we need to manipulate hbase table data through sql statements, and we generally use Hive external tables to manipulate HBase data.

The general steps are as follows:

1. Copy dependency package

Copy several packages that the Hive operation HBase depends on to lib under the Hive installation directory (each hive node performs a copy):

/ usr/lib/hive/lib/zookeeper.jar

/ usr/lib/hive/lib/hbase.jar

/ usr/lib/hive/lib/hive-hbase-handler-X.X.X.jar

/ usr/lib/hive/lib/guava-11.0.2.jar

2.HBase operation

Log in to the hbase shell console and create a new hbase table ljs:student (ljs is the namespace and student is the table name)

Execute the command:

Create_namespace 'ljs'

Create 'ljs:student','info'

View the table structure:

! desc 'ljs:student'

Insert data into the HBase table:

Put 'ljs:student','1001','info:name','ljs1001'

Put 'ljs:student','1001','info:age','21'

Put 'ljs:student','1001','info:score','89'

Put 'ljs:student','2001','info:name','ljs1002'

Put 'ljs:student','2001','info:age','24'

Put 'ljs:student','2001','info:score','78'

View table data:

3.Hive operation

Execute / usr/hdp/2.6.3.0-235/hive/bin/hive to log in to the hive console

Create a new hive data Table:

Create external table hive_student (user_no string,user_name string,user_age string,user_score string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties ("hbase.columns.mapping" = ": key,info:name,info:age,info:score") tblproperties ("hbase.table.name" = "ljs:student")

Query the hive table to view the corresponding data in the hbase mapping table:

At this point, the Hive operation of the HBase table data is complete.

Note:

When building a hbase mapping table in 1.hive, the corresponding mapping field after the keyword "hbase.columns.mapping" in the table statement should be the same as the case of the HBase table field. Otherwise, the mapping may fail.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.