Shulou(Shulou.com)05/31 Report--

This article mainly explains "what is the partial application and partial application of Spark". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next, let the editor take you to learn "what is the partial application and partial application of Spark"!

Local applications and partial applications are derived from mathematical language techniques (based on the work of the 20th century mathematician Haskell Curry and others). These two technologies exist in various types of languages and can exist alone or at the same time in functional languages. Local and partial applications enable you to handle the number of parameters of a function or method, usually by providing one or more default values (called correction parameters) for some parameters. All Java next-generation languages include local applications and partial applications, but they are implemented in different ways. In this article, I'll introduce the differences between the two technologies and show their implementation details in Scala, Groovy, and Clojure, as well as practical applications.

Pay attention to terminology

For the purposes of this section, methods (method) and functions (function) are interchangeable. Object-oriented language usage that supports local application and partial application. Similarly, function parameters (function parameter) and function parameters (function argument) are interchangeable. Since these concepts originated in mathematics, I used function and argument from beginning to end, but that doesn't mean these two techniques don't work for methods.

Definition and distinction

For amateurs, partial application and partial application have the same effect. When using both techniques, you can create a version of the function with some parameters with pre-supplied values:

Local application is to convert multi-parameter functions into a series of single-parameter functions. It describes the conversion process, not the call to the conversion function. The caller can determine how many parameters are applied, creating an exported function with fewer parameters.

Some applications convert a multi-parameter function into a multi-parameter function with fewer parameters, whose value is the value of the omitted parameter provided in advance. The name of this technique is apt: it applies some parameter parts to the function and returns a function with a signature (consisting of the remaining parameters).

Using local applications and partial applications, you can provide parameter values and return a function that can be called with missing parameters. However, applying a local application to a function returns the next function in the chain, while some applications bind the parameter value to the value provided during the operation, resulting in a function with fewer elements (the number of parameters). This difference is even more pronounced when considering functions with more than two elements. For example, the fully applied version of the process (x, y, z) function is process (x) (y) (z), where process (x) and process (x) (y) are both functions that take one parameter. If only the first parameter is locally applied, the return value of process (x) will be a function that accepts one parameter, so only one parameter is accepted. In contrast, when using some applications, there is a function with fewer elements left. Using a partial application to a parameter of process (x, y, z) generates a function that takes two parameters: process (y, z).

The results of these two technologies are usually the same, but the difference between them is also important, and people often misunderstand the difference between them. To make matters more complicated, Groovy can implement partial applications and local applications, but both of them are called currying. Scala, on the other hand, has partial application functions (partially applied function) and PartialFunction, and although their names are similar, they are two different concepts.

In Scala

Scala supports local and partial applications, as well as features (trait), which can define constraint functions (constrained function).

Local application

In Scala, functions can define multiple parameter lists as parentheses groups. When a function with fewer arguments than it defines is called, a function that takes a list of missing parameters as its argument is returned. Consider an example of an Scala document, as shown in listing 1.

Listing 1. Scala parameters locally apply def filter (xs: List [Int], p: Int = > Boolean): List [Int] = if (xs.isEmpty) xs else if (p (xs.head)) xs.head:: filter (xs.tail, p) else filter (xs.tail, p) def modN (n: Int) (x: Int) = (x% n) = 0) val nums = List (1, 2, 3, 4, 5, 6, 7, 8) println (filter (nums) ModN (2)) println (filter (nums, modN (3)

In listing 1, the filter () function recursively applies the passed filter condition. The modN () function defines two parameter lists. When I call modN with filter (), I pass a parameter. The filter () function is taken as the second argument of the function, with an Int parameter and a Boolean return value, which matches the signature of the local application function that I passed.

Partial application function

Functions can also be partially applied in Scala, as shown in listing 2.

Listing 2. Function def price (product: String): Double = product match {case "apples" = > 140case "oranges" = > 223} def withTax (cost: Double, state: String): Double = state match {case "NY" = > cost * 2 case "FL" = > cost * 3} val locallyTaxed = withTax (_: Double, "NY") val costOfApples = locallyTaxed (price ("apples")) assert (Math.round (costOfApples) = 280)

In listing 2, I first create a price function that returns the mapping between the product and the price. Then I created a withTax () function with arguments to cost and state. However, in the special source file, I know that I have to deal specifically with a country's tax revenue. Instead of applying locally to the extra parameters for each call, I partially applied the state parameter and returned a function with a fixed state value. The locallyTaxed function takes one parameter, cost.

Partial (constrained) function

The Scala PartialFunction feature works seamlessly with patterns (read pattern matching in the "Either Tree and pattern matching" section of the functional thinking series). Although the name is similar, this feature does not create a partial application function. Instead, you can use it to define functions that apply only to a subset of value and type definitions.

Case blocks are a way to apply partial functions (partial function). Listing 3 uses Scala's case and has no traditional match operator.

Listing 3. Use caseval cities = Map without match ("Atlanta"-> "GA", "New York"-> "New York", "Chicago"-> "IL", "San Francsico"-> "CA", "Dallas"-> "TX") cities map {case (k, v) = > println (k + "- >" + v)}

In listing 3, I create a mapping between the city and the state that corresponds to that city. Then, I called the map function on the collection, and map detached the key-value pairs to output them. In Scala, a block of code that contains case declarations is a way to define anonymous functions. Anonymous functions can be defined more succinctly without case, but the case syntax provides the added benefit shown in listing 4.

Listing 4. The difference between map and collect List (1,3,5, "seven") map {case I: Int? I + 1} / / won't work// scala.MatchError: seven (of class java.lang.String) List (1,3,5, "seven") collect {case I: Int? I + 1} / / verifyassert (List (2,4,6) = (List (1,3,5, "seven") collect {case I: Int? I + 1}))

In listing 4, I can't use map on heterogeneous collections with case: I received MatchError because the function tried to increment the seven string. But collect works fine. Why is there this difference? What went wrong?

Case blocks define partial functions, not partial application functions. Partial functions have limited allowable values. For example, the mathematical function 1max x is invalid if x = 0. Partial functions provide a way to define allowable value constraints. In the collect example in listing 4, the constraint for Int instead of String is defined, so no seven strings are collected.

To define a partial function, you can also use the PartialFunction feature, as shown in listing 5.

Listing 5. Define the partial function val answerUnits = new PartialFunction [Int, Int] {def apply (d: Int) = 42 / d def isDefinedAt (d: Int) = d! = 0} assert (answerUnits.isDefinedAt (42)) assert (! AnswerUnits.isDefinedAt (0)) assert (answerUnits (42) = = 1) / / answerUnits (0) / / java.lang.ArithmeticException: / by zero

In listing 5, I export answerUnits from the PartialFunction feature and provide two functions: apply () and isDefinedAt (). The apply () function calculates the value. I used isDefinedAt () (the necessary method of PartialFunction) to define the constraints that determine the applicability of the parameters.

You can also use case blocks to implement partial functions, and the answerUnits in listing 5 can be written in a more concise way, as shown in listing 6.

Listing 6. Another definition of answerUnits def pAnswerUnits: PartialFunction [Int, Int] = {case d: Int if d! = 0 = > 42 / d} assert (pAnswerUnits (42) = = 1) / / pAnswerUnits (0) / / scala.MatchError: 0 (of class java.lang.Integer)

In listing 6, I use a combination of case and guard conditions to constrain values and provide values at the same time. One obvious difference from listing 5 is MatchError (not ArithmeticException), because listing 6 uses pattern matching.

Partial functions are not limited to numerical types. It can use all types of values, including Any. Consider the implementation of the incrementer, as shown in listing 7.

Listing 7. Define the increment def inc in Scala: PartialFunction [Any, Int] = {case I: Int = > I + 1} assert (inc (41) = = 42) / / inc ("Forty-one") / / scala.MatchError: Forty-one (of class java.lang.String) assert (inc.isDefinedAt (41)) assert (! Inc.isDefinedAt ("Forty-one")) assert (List (42) = (List (41, "cat") collect inc))

In listing 7, I define a partial function to accept any type of input (Any), but choose to react to a subset of the type. Note that I can also call the isDefinedAt () function of the partial function. Implementers who use the PartialFunction feature of case can call isDefinedAt (), which is implicitly defined. In listing 4, I show that map and collect behave differently. The behavior of partial functions explains the difference: collect is designed to accept partial functions and call the element's isDefinedAt () function, ignoring those functions that do not match.

In Scala, partial functions and partial application functions have similar names, but they provide different sets of orthogonal properties. For example, nothing can stop you from partially applying partial functions.

In Groovy

In my functional thinking series, "using functional thinking, part 3" describes in detail the local application and partial application of Groovy. Groovy implements local application through the curry () function, which comes from the Closure class. Despite the name, curry () actually implements part of the application by processing the closure blocks below it. However, you can simulate a local application by using a partial application to reduce the function to a series of partially applied single-parameter functions, as shown in listing 8.

Listing 8. Partial application and local application of Groovy def volume = {h, w, l-> return h * w * l} def area = volume.curry (1) def lengthPA = volume.curry (1,1) / / partial applicationdef lengthC = volume.curry (1) .curry (1) / / curryingprintln "The volume of the 2x3x4 rectangular solid is ${volume (2,3,4)}" println "The area of the 3x4 rectangle is ${area (3) 4)} "println" The length of the 6 line is ${lengthPA (6)} "println" The length of the 6 line via curried function is ${lengthC (6)} "

In listing 8, in both cases of length, I partially applied the parameters using the curry () function. However, when using lengthC, I created the illusion of using local application by partially applying parameters until a series of single-parameter functions appeared.

In Clojure

Clojure contains (partial f A1 A2...) Function, which has function f and fewer arguments than required, and returns a partial application function that is called when the remaining parameters are provided. Listing 9 shows two examples.

Listing 9. Partial applications of Clojure (def subtract-from-hundred (partial-100)) (subtract-from-hundred 10); same as (- 10010); 90 (subtract-from-hundred 10 20); same as (- 10010 20); 70

In listing 9, I define the subtract-from-hundred function as a partially applied-operator (operators in Clojure are indistinguishable from functions) and provide 100 as parameters for partial applications. Some of the applications in Clojure apply to single-parameter and multi-parameter functions, as shown in the two examples in listing 9.

Because Clojure is dynamically typed and supports variable parameter lists, local application cannot be implemented as a language function. Some applications will deal with the necessary situations. However, Clojure is added to the namespace private (defcurried.) of the reducers library (see Resources). Function, which makes it easier to define some functions in the library. In view of the flexibility of Clojure's Lisp inheritance, it can be easily expanded (defcurried.) The scope of use of.

Common usage

Although local applications and some applications have complex definitions and a large number of implementation details, they all have a place in actual programming.

Function factory

Local applications (and some applications) are suitable for the location where factory functions are implemented in traditional object-oriented languages. As an example, listing 10 implements a simple adder function in Groovy.

Listing 10. Adder and increment in Groovy def adder = {x, y-> x + y} def incrementer = adder.curry (1) println "increment 7: ${incrementer (7)}" / / 8

In listing 10, I use the adder () function to export the incrementer function. Again, in listing 2, I used some of the applications to create a more concise version of the local function.

Template Method design pattern

One of the Gang of Four design patterns is the Template Method pattern. Its purpose is to help define the algorithm shell, using internal abstract methods to achieve later implementation flexibility. Partial application and local application can solve the same problem. Using some applications to provide known behavior and letting other parameters be used for implementation details free of charge simulates the implementation of this object-oriented design pattern.

Implied value

Similar to listing 2, a common situation is that you have a series of functions called with similar parameter values. For example, when interacting with a persistence framework, you must pass the data source as the first parameter. By using partial applications, you can implicitly provide values, as shown in listing 11.

Listing 11. Use some applications to provide implied values (defn db-connect [data-source query params]...) (def dbc (partial db-connect "db/some-data-source")) (dbc "select * from% 1"cust")

In listing 11, I use the convenient dbc function to access the data function, which can be provided automatically without providing a data source. The essence of object-oriented programming (the implicit this context seems to appear in all functions) can be achieved by using local applications to provide this for all functions, which makes it invisible to the user.

At this point, I believe you have a deeper understanding of "Spark partial application and part of the application method is what", might as well to the actual operation of it! Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.