Shulou(Shulou.com)06/03 Report--

This article introduces the relevant knowledge of "how to configure spark sql query hive". In the actual case operation process, many people will encounter such difficulties. Next, let Xiaobian lead you to learn how to deal with these situations! I hope you can read carefully and learn something!

Configure idea spark sql connection hive

1. First copy hive-site.xml of/etc/hive/conf/to resources of the project

2. To add maven dependencies to the pom.xml file, you need to add spark-sql, spark-hive, spark version 2.2.0, scala version 2.10

org.apache.spark

spark-sql_2.10

2.2.0

org.apache.spark

spark-hive_2.10

2.2.0

3. Set it in the program

val spark = SparkSession

.builder()

.appName("test_note")

.master("local")

.enableHiveSupport()//Set support hive

.getOrCreate()

To this place, you can directly query the hive data of the server

Configure CDH Cluster

Cluster configuration only needs to copy hive-site.xml from/etc/hive/conf/directory to/etc/spark2/conf/directory to spark-submit.

"How to configure Spark SQL query hive" content is introduced here, thank you for reading. If you want to know more about industry-related knowledge, you can pay attention to the website. Xiaobian will output more high-quality practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.