Shulou(Shulou.com)06/03 Report--

Hive and Impala are both data query tools built on Hadoop, so how do they load and store data in real applications?

Hive and Impala store and load tables and, like all relational databases, have their own data management structures, from its Server to Database to tables and views.

In other databases, tables are stored in their own specific file formats. For example, Oracle has its own storage format, while for Hive, a table is an HDFS directory containing one or more files, which belongs to the contents of the table. The default storage path: / user/hive/warehouse/, supports multiple storage formats.

The above is the data storage, so each table, each structure has its own column or type definition of information, how to save this information? They are stored in Metastore, and all the data is stored on HDFS, so if we want to get table structure information, we need to know the meaning and structure of each table in the metadata of hive. In hive, there is a simple command to roughly view the structure of the table: the structure of the describe formatted tableName;hive metastore table is as follows:

Because Hive and Impala use the same data, tables in HDFS and metadata in Metastore, the above storage and structure introduction also applies to Impala.

Examples of data loading and storage:

Here, we have to distinguish between two concepts: data and metadata. Data refers to the information you store and process, such as billing records, sensor readings, and service logs. Metadata is used to describe the shape of the data, such as field name and order.

The introduction of Hive and Impala is a sharing of our own experience. I hope it can help you understand and understand the functions of Hive and Impala. Normally, we can also pay more attention to big data's information. For example, Wechat service number "big data cn" and "big data era Learning Center" are of great help and promotion in improving our own knowledge structure.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.