Shulou(Shulou.com)06/01 Report--

This article mainly introduces the example analysis of vectorization query in Hive, which has a certain reference value, and interested friends can refer to it. I hope you will gain a lot after reading this article.

Vectorization

Vectorized query execution in Hive greatly reduces CPU usage for typical query operations such as scanning, filters, aggregations, and joins.

The standard query execution system processes one row at a time, and before processing the next row, the single row of data is processed by all operators in the query, resulting in very inefficient use of CPU. In the execution of a vectorization query, data rows are batched together (default = > 1024 rows) and represented as a set of column vectors.

To execute using a vectorization query, you must store the data in ORC format (CDH 5) and set the following variables.

SET hive.vectorized.execution.enabled=true

Hive query vectorization is enabled by default in CDH 6. After query vectorization is enabled, other properties can be set to adjust the way of query vectorization.

Thank you for reading this article carefully. I hope the article "sample Analysis of Quantification query in Hive" shared by the editor will be helpful to you. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.