Shulou(Shulou.com)06/02 Report--

Hive and Impala as data query tools, how do they query data? What tools do we have to interact with Impala and Hive?

First of all, we make it clear that Hive and Impala provide the corresponding query interfaces respectively:

(1) Command line shell:

1 、 Impala:impala shell

2. Hive:beeline (in the early days, the command line version of hive was hive shell, but now it is basically not used)

(2) Hue Web UI:

Hive query Editor is provided in 1.Hue

Impala query Editor is provided in 2.Hue

3.Hue provides a metadata manager that can access metadata directly.

(3) JDBC and ODBC support are provided.

The following are described in detail:

1. Impala

(1) use Impala shell

Impala shell is an interactive tool similar to MySQL, and you can start Impala shell directly on the terminal, but it's very important to know where to start Impala shell. Impala itself is a distributed architecture, and its Impalad is in each slave node. Then Impala must be running in the slave node if it is executed directly as follows.

If you are not running in the slave node, you can specify its server, as shown below, by specifying port 21000 to execute through the parameter of-I.

Impala shell, like all SQL, enter a semicolon as the end of the statement and use the quit command to exit shell. If you need help, use impala-shell-- help to see the complete list of options. Because any tool has a lot of parameters, you must get used to and learn to use help to find some parameters you want, and then to solve some complex data analysis and processing.

An example of executing a query in Impala shell:

(2) Impala interacts with the operating system

In Impala, we sometimes need to execute some Linux commands, such as querying dates, remote connections, accessing directory data, and so on. We do not need to exit Impala shell, but can operate directly, such as:

However, Impala does not directly support the HDFS command, but you can use shell to run the hdfs dfs command to create a directory, as follows:

(3) run the Impala query from the command line, which can be executed without logging in to Impala shell, such as:

1. Use the-f option to execute the file containing the query

2. Run the query directly on the command line using the-Q option

3. Use-o to output the results to a file

II. Hive

(1) start beeline

Hive shell is an earlier version of Hive and now uses Beeline shell, which is similar to Impala shell, but it is based on JDBC and ODBC. If you need to use Beeline, you need to start Hive2, and start Beeline by specifying URL for the Hive2 server. In this case, you need to specify a user name and password as needed, as follows:

(2) execute the query in Beeline

Like SQL, it ends with a semicolon, and executing a query is similar to Impala shell, but the result format is slightly different, such as:

(3) use Beeline

Unlike other tools, executing queries is normal SQL input, but if it is some administrative commands, such as connect, break, exit, and execute Beeline commands, you need to take "!" No Terminator is required Introduction to common commands:

1.! connect url-Connect to different Hive2 servers

2.! exit-quit shell

3! help-displays a list of all commands

4! verbose-displays the details of the query appended

Example:

(4) execute Hive query from the command line

1. Use the-f option to execute the file containing the HiveQL code

2. Run HiveQL directly on the command line using the-e option

3. Use-- silent to block the message output of notifications, or with the-e or-f options

III. Hue

(1) visit Hive and Impala through Hue, each of which has editors. In Hue's query editors, we can find some tools of Hive and Impala.

It is as follows:

And Metabase we can see some of its tools in Data Browsers, such as:

(2) Hue query editor, query Impala and Hive query editor are almost the same.

To interact with Impala and Hive, these are the tools we can use and use, but in practice we still need to strengthen our actual combat ability in order to better grasp and understand. Big data, as a technology that is still constantly improving and developing, needs everyone who wants to engage in it and has already engaged in it to continue to learn and accumulate, but also needs to communicate and share, and make common progress. " Big data cn "and" big data Times Learning Center "are the Wechat service numbers I usually find in the process of autonomous learning. The knowledge introduced about big data and some of the industry knowledge developed by big data are very good, usually you can have a look.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.