Shulou(Shulou.com)06/01 Report--

Editor to share with you how to configure spark on hive, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

1. Add dependency 2.4.0 2.11 org.apache.spark spark-core_$ {scala.version} ${spark.version} org.apache.spark spark-sql_$ {scala.version} ${spark.version} Org.apache.spark spark-hive_$ {scala.version} ${spark.version} net.jpountz.lz4 lz4 1.3.0 net.alchim31.maven Scala-maven-plugin 3.2.0 compile Org.apache.maven.plugins maven-assembly-plugin 3.0.0 jar-with-dependencies make-assembly Package single 2. Copy the hive-site.xml to the project resource path 3. Create a SparkSession object and directly read the data in the hive table val session: SparkSession = SparkSession.builder.appName ("etl") .master ("local [*]") .enableHiveSupport.getOrCreateval frame: DataFrame = session.sql ("select * from warningplatform.ini_current_frequency") frame.show () these are all the contents of the article "how to configure spark on hive". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.