Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to analyze SparkCore and SparkSQL. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

Here are seven simple exercises to compare and demonstrate SparkCore and SparkSQL programming methods. With the exception of WordCount word frequency statistics, which is a typical example of dealing with unstructured data, the programming interface using SparkSQL is more concise and easy to understand in most of the examples demonstrated below.

First, find the average.

Problem: given a set of integers, find their average.

(note: these integers may add up to a large number, so consider the possibility of overflow.)

Second, WordCount word frequency statistics

Problem: read the words in the document and count the number of words that appear in each word.

Third, ask for TopN

Question: given the students' names and grades and other information, find out the top three students.

Fourth, find the maximum and minimum value.

Problem: given a set of numbers, find out the maximum and minimum values.

Five, sort and add the serial number

Problem: given a set of numbers, sort them from small to large, and add a sequence number starting at 0.

Six, secondary sorting

Question: given the student's score and age and other information, according to the student's score from the largest to the smallest, if the scores are equal, according to the age from the largest to the smallest.

Seven, connection operation

Problem: given the class information table and the student score table, connect them and find out the classes with an average class score of more than 75.

On how to parse SparkCore and SparkSQL to share here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.