Shulou(Shulou.com)05/31 Report--

Today, the editor will share with you the relevant knowledge about the functions of Spark1.2. The content is detailed and the logic is clear. I believe most people still know too much about this knowledge, so share this article for your reference. I hope you can get something after reading this article. Let's take a look at it.

This version includes

Core operation and performance improvement of Spark

A new network transmission subsystem has been added and great improvements have been made.

Spark SQL introduces support for an external data source and supports Hive13

Dynamic partition

Fixed-precision decimal type

MLlib adds a new pipeline-oriented package (spark.ml) to combine multiple algorithms

Spark Streaming added a Python API to write error tolerance log in advance.

GraphX officially broke away from the Alpha version and introduced a stable API.

Spark is an open source cluster computing environment similar to Hadoop, but there are some differences between the two. These useful differences make Spark superior in some workloads. In other words, Spark enables in-memory distributed datasets to optimize iterative workloads in addition to interactive queries.

Spark is implemented in the Scala language and uses Scala as its application framework. Unlike Hadoop, Spark and Scala can be tightly integrated, where Scala can manipulate distributed datasets as easily as local collection objects.

Although Spark is created to support iterative jobs on distributed datasets, it is actually a complement to Hadoop and can be run in parallel in the Hadoo file system. This behavior can be supported through a third-party cluster framework called Mesos. Developed by AMP Lab (Algorithms, Machines, and People Lab) at the University of California, Berkeley, Spark can be used to build large, low-latency data analysis applications.

These are all the contents of this article "what are the functions of Spark1.2?" Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.