Shulou(Shulou.com)06/02 Report--

1.Spark cluster deployment and development environment building. The content includes Hadoop cluster, Spark cluster building, Intellij IDEA Spark development environment building, Spark Shell use and so on.

The principle of 2.Spark operation. The content includes spark script file parsing, several different operation modes of Spark, RDD principle, wide and narrow dependency, Spark task scheduling and so on.

3.Spark programming model, Spark programming model, common transformation and action operations are introduced.

4.Spark SQL and DataFrame, introduce the operation principle and usage of Spark SQL and DataFrame, and use a case to introduce the use of Spark SQL.

5.Spark streaming computing. The principles of DStream and Spark Streaming are introduced, and their use is illustrated by several cases.

6.Spark machine learning. This paper introduces the Spark MLlib architecture, and explains the use of Spark MLlib through K-Mean algorithm, random forest algorithm and so on.

7.Spark Graphx . This paper introduces the calculation of Spark diagram and related data structure, and explains its use with PageRank algorithm.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.