Shulou(Shulou.com)06/02 Report--

Although Hcatalog is a sub-project, it brings a clear flow to the entire Hadoop ecological environment, it makes data access is no longer disorganized, so what is the role of Hcatalog? How does it uniformly define data?

HCatalog is a hive sub-project used to access Metastore, and its existence gives a unified definition to the whole Hadoop ecological environment.

Why do you say that? Throughout the ecology, there are many projects that use HDFS, such as Hive, Pig, Spark, and they use the same HDFS data. When they access the data, Hive may parse the data into integers, Pig into String types, and Spark into other types, so this definition is not unified. On the other hand, Hcatalog can unify a definition, for example, the data is defined as an integer, and all projects access the data through Hcatalog, and the data types accessed are all integers.

Hcatalog can be accessed from the command line and RESTAPI, allowing you to use HiveQLDDL syntax to define tables. Access tables from Hive, Impala, MapReduce, Pig, and other tools, included in CDH4.2 and later versions.

Create a table in HCatalog

(1) HCatalog uses Hive's DDL syntax, and you can use the-e option to specify a single command.

(2) Tip: save a long command as a file, using the-f option. If the file contains more than one statement, separate it with semicolons

View metadata in HCatalog

(1) SHOW TABLES command can also view tables directly.

(2) the DESCRIBE command lists the fields of the specified table, allowing you to use HiveQLDDL syntax to define the table.

Remove the table in HCatalog

(1) the DROP TABLE command is the same as in Hive and Impala

Note: both data and metadata will be removed (except for external tables)

In the usual process of learning, we should pay attention to the use of these sub-items and their role. Mastering and understanding them is of great significance for us to better master big data technology. I usually like to follow Wechat information platform "big data cn", pay attention to some of big data's latest information and some knowledge and experience sharing, which has improved my own knowledge structure.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.