Shulou(Shulou.com)05/31 Report--

Based on spark1.3.1 spark-sql actual combat is how, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain for you in detail, people with this need can come to learn, I hope you can gain something.

Copy the HIVE_HOME/conf/hive-site.xml file to SPARK_HOME/conf/

When not configured by the configuration unit site. Xml, the context automatically creates metastore_db and warehouse in the current directory.

/ / sc is an existing SparkContext.val sqlContext = new org.apache.spark.sql.hive.HiveContext (sc) sqlContext.sql ("CREATE TABLE IF NOT EXISTS src (key INT, value STRING)") sqlContext.sql ("LOAD DATA LOCAL INPATH 'examples/src/main/resources/kv1.txt' INTO TABLE src") / / Queries are expressed in HiveQLsqlContext.sql ("FROM src SELECT key, value") .collect () .foreach (println) JDBC To Other Databases

Load mysql database: test, table: t_user_new returns DataFrame

Query data:

Performance Tuning

The spark program operates JDBC:

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.