Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "what are the characteristics of Hive". In the operation of actual cases, many people will encounter such a dilemma. Then let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Hive is a data warehouse tool built on hadoop, which can map structured data files to a database table, quickly hi data extraction, transformation and loading (ETL), and provide a simple SQL query function, called HQL (here to be different from Hibernate), allowing users who are familiar with SQL to query data. It works on thrift, and the control delimiter also allows the user to specify the data format.

Characteristics of Hive

Support indexing and speed up data query.

There are different storage types, such as files in HBase, plain text files

Saving metadata in a relational database greatly reduces the time it takes to perform semantic checking during a query

Data stored in the Hadoop file system can be used directly

Built-in a large number of user functions UDF to manipulate time, strings and other data mining tools, allowing users to extend UDF functions to complete operations that can not be achieved by built-in functions

SQL-like query method that converts SQL queries into Job of MapReduce and executes on the Hadoop cluster.

Shortcomings of Hive

Statically processed Hadoop has high execution latency, high job submission and scheduling overhead, and minute set latency, so it is not suitable for applications that require low latency, such as training transaction processing (OLTP).

The relationship between HIVE and hadoop

The architecture of Hive, as shown in the figure above, includes user access interfaces (CLI, JDBC/ODBC, WEB GUI, and Thrift Server) metadata storage (Metastore) and driver components (compilation, optimization, execution).

User access interface: a tool interface for users to interact with Hive data warehouse.

CLI (command line interface) command line interface.

Thrift Server can call Hive interface in different languages.

HWI (Hive web interface) provides services provided by Hive through web access

Metastore and Hive drivers are interconnected: 1. Integration mode 2. Remote mode

It is mainly used to store metadata in Hive, usually using Mysql or Derby database.

The driver components include compiler, optimizer and execution engine, which complete lexical analysis, syntax analysis, compilation, optimization and query plan generation respectively. The generated query plan is stored in HDFS and executed by MapperReduce calls.

This is the end of the content of "what are the characteristics of Hive". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.