Shulou(Shulou.com)06/02 Report--

This article mainly introduces "the basic use of Spark-SQL". In the daily operation, I believe many people have doubts about the basic use of Spark-SQL. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts about "the basic use of Spark-SQL"! Next, please follow the editor to study!

Official document address http://spark.apache.org/docs/latest/sql-getting-started.html

The official test data file http://mirrors.tuna.tsinghua.edu.cn/apache/spark/spark-2.4.3/spark-2.4.3-bin-hadoop2.7.tgz

Extract the current folder examples/src/main/resources

# # SparkSession

Created by SparkSession.builder ()

Import org.apache.spark.sql.SparkSessionval spark = SparkSession .builder () .appName ("Spark SQL basic example") .config ("spark.some.config.option", "some-value") .getOrCreate () / / For implicit conversions like converting RDDs to DataFramesimport spark.implicits._

Read json file

Package cn.bx.sparkimport org.apache.spark.sql. {DataFrame, SparkSession} object SparkSessionApp {def main (args: Array [String]): Unit = {val spark: SparkSession = SparkSession.builder (). AppName ("SparkSessionApp"). Master ("local [*]") .getOrCreate () val people: DataFrame = spark.read.json ("resources/people.json") people.show () spark.stop ()}}

Print the result

+-this is the end of the study on "basic usage of Spark-SQL". Hope to solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.