Shulou(Shulou.com)05/31 Report--

How to carry out spark SQL programming hands-on combat, I believe that many inexperienced people do not know what to do, so this paper summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

First create the SparkContext context:

Then a stealth transformation is introduced to convert RDD to SchemaRDD:

Next, to load the data, the test data here is the user.txt file:

We created use.txt to add content and uploaded it to hdfs:

Hdfs command query:

Verify that the data is loaded successfully:

At the moment, user is still a MappedRDD:

At this moment, the teenagers has been transformed into SchemaRDD.

Results:

Also use "user.txt" data:

We use the SQL query operation directly this time:

It can be found that the use of teenagers using DSL has been transformed into SchemaRDD by stealth internally.

You can find that the result is the same as the previous one that registers Table.

After reading the above, have you mastered the practical method of how to do spark SQL programming? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.