Shulou(Shulou.com)06/01 Report--

Editor to share with you how to optimize based on cost in Hive, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Cost-based optimization

Hive optimizes the logical and physical execution plan of each query before submitting it for final execution. Cost-based optimization is further optimized according to the query cost, which may lead to different decisions, such as how to determine the order of JOIN, which type of JOIN to execute, and the degree of parallelism.

You can enable cost-based optimization by setting the following parameters.

Set hive.cbo.enable=true

Set hive.compute.query.using.stats=true

Set hive.stats.fetch.column.stats=true

Set hive.stats.fetch.partition.stats=true

You can use statistics to optimize queries to improve performance. The cost-based optimizer (CBO) also uses statistics to compare query plans and select the best plan. It can be efficient by viewing statistics instead of running queries.

Collect column statistics for the table:

ANALYZE TABLE mytable COMPUTE STATISTICS FOR COLUMNS

View column statistics for the my_id column in my_table in the my_db database:

DESCRIBE FORMATTED my_db.my_table my_id above is all the content of the article "how to optimize based on cost in Hive". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.