Shulou(Shulou.com)06/02 Report--

Scala powerful collection data operation example analysis, in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Scala is one of the most powerful programming languages in the field of data mining algorithms, and the language itself is function-oriented, which is also in line with the common scenarios of data mining algorithms: a series of transformations are applied on the original data set, and the language itself provides many powerful functions for set operations. This paper will take the List type as an example to introduce common set transformation operations.

Basic syntax of scala: https://my.oschina.net/u/3991887/blog/2710178

Challenge the Scala stunt of Java programmers: https://my.oschina.net/joymufeng/blog/2251038

1. Common operators (operators are also functions)

+ [B] (that: GenTraversableOnce [B]): List [B] add another list from the end of the list

+ +: [B >: a, That] (that: substitution.Traversale [B]) (implicit bf: CanBuildFrom [List [A], B, That]): That adds a list to the head of the list

+: (elem: a): List [A] add an element to the header of the list

: +: + (elem: a): List [A] add an element to the end of the list

: (X: a): List [A] add an element to the head of the list

: (prefix: List [A]): List [A] add another list to the head of the list

:\: [B] (z: B) (op: (a, B) ⇒ B): B is equivalent to foldRight

Val left = List, val right = List / / the following operations are equivalent: left + + right / / List) left + +: right / / List (1min2, 3, 4, 5) right.++: (left) / / List (1, 2, 3, 4, 4, 5) right.::: (left) / List (1, 2, 3, 4, 5) / the following operations are equivalent 0 +: left / List (0) / List (0) left.+: (0) / List (2) left 3) / / the following operations are equivalent to left: + 4 / / List (1mine2min3) left.:+ (4) / / List (1min2min3) / / the following operations are equivalent to 0:: left / / List (0min1min2jin3) left.:: (0) / / List (0min1d2p3)

See here you should be a little dizzy like me, why so many strange operators, here to give you a hint, any operator that results in a colon is bound to the right, that is, 0:: List (1Magne2Magol 3) = List (1Magne2Magne3).:: (0) = List (0Lie1L2 Magne3) from here we can see the operation:: it is actually the operator of List on the right, not the operator of type Int on the left.

Second, common transformation operations

1.map

Map [B] (f: (a) ⇒ B): List [B]

Define a transformation, apply the transformation to each element of the list, leave the original list unchanged, and return a new list data

Example1 square transformation

Val nums = List (1Mei 2jue 3) val square = (x: Int) = > xonomx val squareNums1 = nums.map (num = > num*num) / / List (1m 4m 9) val squareNums2 = nums.map (math.pow (_, 2)) / / List (1m 4m 9) val squareNums3 = nums.map (square) / / List (1m 4J 9)

Example2 saves some columns in text data

Val text = List ("Homeway,25,Male", "XSDYM,23,Female") val usersList = text.map (_ .split (",") val usersWithAgeList = text.map (line = > {val fields = line.split (",") val user = fields (0) val age = fields (1). ToInt (user,age)})

2.flatMap, flatten

Flatten: flatten [B]: List [B] flatten the list flatMap: flatMap [B] (f: (A) ⇒ GenTraversableOnce [B]): List [B] map followed by flatten of the result

Define a transformation f, apply f to each element of the list, each f returns a list, and finally link all the lists together.

Val text = List ("A Brecinct C", "Drecincter F") val textMapped = text.map (_ .split (",") .toList) / / List (List ("A", "B", "C"), List ("D", "E", "F")) val textFlattened = textMapped.flatten / / List ("A", "B", "C", "D", "E", "F") val textFlatMapped = text.flatMap (_ .split (" ToList) / / List ("A", "B", "C", "D", "E", "F")

3.reduce

Reduce [A1 >: a] (op: (A1, A1) ⇒ A1): A1

Define a transformation f, f combines the elements of two lists into one, traverses the list, and finally merges the list into a single element

Summation of Example list

Val nums = List (1 and 2) val sum1 = nums.reduce (aMagneb) = > ahumb) / / 6val sum2 = nums.reduce (_ + _) / / 6val sum3 = nums.sum / / 6

4.reduceLeft,reduceRight

ReduceLeft: reduceLeft [B >: a] (f: (B, A) ⇒ B): B

ReduceRight: reduceRight [B >: a] (op: (a, B) ⇒ B): B

ReduceLeft applies the reduce function from left to right of the list, and reduceRight applies the reduce function from right to left of the list

Example

Val nums = List (2.0 math.pow) val resultLeftReduce = nums.reduceLeft (math.pow) / / = pow (pow (2.0) / 3.0) = 64.0val resultRightReduce = nums.reduceRight (math.pow) / / = pow (2.0, pow (2.0)) = 256.0

5.fold,foldLeft,foldRight

Fold: fold [A1 >: a] (z: A1) (op: (A1, A1) ⇒ A1): A1 reduce with an initial value, starting with an initial value, merging two elements into one from left to right, and finally merging the list into a single element.

FoldLeft: foldLeft [B] (z: B) (f: (B, A) ⇒ B): B reduceLeft with initial value

FoldRight: foldRight [B] (z: B) (op: (a, B) ⇒ B): B reduceRight with initial value

Val nums = List val sum = nums.fold (1) (_ + _) / / = 9val nums = List (2.0) val result1 = nums.foldLeft (4.0) (math.pow) / / = pow (pow (4.0), 3.0) = 4096val result2 = nums.foldRight (1.0) (math.pow) / / = pow (1.0)

6.sortBy,sortWith,sorted

SortBy: sortBy [B] (f: (A) ⇒ B) (implicit ord: math.Ordering [B]): List [A] sorts the elements produced after applying the function f

Sorted: sorted [B >: a] (implicit ord: math.Ordering [B]): List [A] sorts by element itself

SortWith: sortWith (lt: (a, A) ⇒ Boolean): List [A] uses a custom comparison function to sort

Val nums = List val sorted = nums.sorted / / List) val users = List (("HomeWay", 25), ("XSDYM", 23) val sortedByAge = users.sortBy {case (user,age) = > age} / / List (("XSDYM", 23), ("HomeWay", 25) val sortedWith = users.sortWith {case (user1,user2) = > user1._2

< user2._2} //List(("XSDYM",23),("HomeWay",25)) 7.filter, filterNot filter: filter(p: (A) ⇒ Boolean): List[A] filterNot: filterNot(p: (A) ⇒ Boolean): List[A] filter 保留列表中符合条件p的列表元素 ， filterNot，保留列表中不符合条件p的列表元素 val nums = List(1,2,3,4)val odd = nums.filter( _ % 2 != 0) // List(1,3)val even = nums.filterNot( _ % 2 != 0) // List(2,4) 8.count count(p: (A) ⇒ Boolean): Int 计算列表中所有满足条件p的元素的个数，等价于 filter(p).length val nums = List(-1,-2,0,1,2) val plusCnt1 = nums.count( >

0) val plusCnt2 = nums.filter (> 0) length

9. Diff, union, intersect

Diff:diff (that: collection.Seq [A]): List [A] saves elements in the list that are not in another list, that is, subtracting the intersection with another set from the collection

Union: union (that: collection.Seq [A]): List [A] links to another list

Intersect: intersect (that: collection.Seq [A]): the intersection of List [A] with another set

Val nums1 = List, val nums2 = List, val diff1 = nums1 diff nums2 / / List, val diff2 = nums2.diff (num1) / / List, val union1 = nums1 union nums2 / / List, val union2 = nums2 + + nums1 / List, val intersection = nums1 intersect nums2 / List

10.distinct

Distinct: List [A] retains the non-repeating elements in the list, and the same element will only be retained once

Val list = List ("A", "B", "C", "A", "B") val distincted = list.distinct / / List ("A", "B", "C")

11.groupBy, grouped

GroupBy: groupBy [K] (f: (A) ⇒ K): Map [K, list [A]] groups the list based on the new elements generated by applying f to the element

Grouped: grouped (size: Int): Iterator [list [A]] is grouped by list and by fixed size

Val data = List (("HomeWay", "Male"), ("XSDYM", "Femail"), ("Mr.Wang", "Male")) val group1 = data.groupBy (_. _ 2) / / = Map ("Male"-> List (("HomeWay", "Male"), ("Mr.Wang", "Male")), "Female"-> List (("XSDYM", "Femail")) val group2 = data.groupBy {case (name) Sex) = > sex} / / = Map ("Male"-> List (("HomeWay", "Male"), ("Mr.Wang", "Male")), "Female"-> List (("XSDYM", "Femail")) val fixSizeGroup = data.grouped (2). ToList / = Map ("Male"-> List (("HomeWay", "Male"), ("XSDYM", "Femail")), "Female"-> List (("Mr.Wang", "Male"))

12.scan

Scan [B >: a, That] (z: B) (op: (B, B) ⇒ B) (implicit cbf: CanBuildFrom [List [A], B, That]): That

Starting from an initial value, from left to right, accumulate the op operation, which is difficult to explain, let's look at the example.

Val nums = List (1mem2) val result = nums.scan (10) (_ + _) / / List (10 minus 10 minus 1 minus 10 minus 2 page2) = List (10 min11 pencils 12 pence 13)

13.scanLeft,scanRight

ScanLeft: scanLeft [B, That] (z: B) (op: (B, A) ⇒ B) (implicit bf: CanBuildFrom [List [A], B, That]): That

ScanRight: scanRight [B, That] (z: B) (op: (a, B) ⇒ B) (implicit bf: CanBuildFrom [List [A], B, That]): That

ScanLeft: operate the scan function from left to right, scanRight: operate the scan function from right to left

Val nums = List val result = nums.scanLeft (2.0) (math.pow) / / List (2.0 pow (2.0), pow (pow (2.0), 2.0), pow (2.0), 2.0), 3.0) = List (2.0 (2.0) (math.pow) / List (2.0) / List (2.0) Pow (3.0), pow (2.0), pow (1.0) = List (1.0512.0)

14.take,takeRight,takeWhile

Take: takeRight (n: Int): List [A] extract the first n elements of the list takeRight: takeRight (n: Int): List [A] extract the last n elements of the list takeWhile: takeWhile (p: (A) ⇒ Boolean): List [A] extract the elements of the list from left to right until condition p does not hold

Val nums = List (1) nums.take (4) / / List (1) nums.takeRight (4) / / List (4) val headNums = nums.takeWhile (_ = = nums.head) / / List (1)

15.drop,dropRight,dropWhile

Drop: drop (n: Int): List [A] discard the first n elements, return the remaining elements dropRight: dropRight (n: Int): List [A] discard the last n elements, return the remaining elements dropWhile: dropWhile (p: (A) ⇒ Boolean): List [A] discard the elements from left to right until condition p does not hold

Val nums = List (1) nums.drop (4) / / List (4) val right = nums.dropRight (4) / / List (1) val tailNums = nums.dropWhile (_ = = nums.head) / / List (4)

16.span, splitAt, partition

Span: span (p: (a) ⇒ Boolean): (List [A], List [A]) apply condition p from left to right until condition p is not established, and the list is divided into two lists

SplitAt: splitAt (n: Int): (List [A], List [A]) divide the list into the first n, and the rest

Partition: partition (p: (a) ⇒ Boolean): (List [A], List [A]) divides the list into two parts, the first part is the elements that satisfy condition p, and the second part is the elements that do not meet condition p

Val nums = List (prefix,suffix) = nums.span (_ = = 1) / / prefix = List (1), suffix = List (prefix,suffix) = nums.splitAt (3) / / prefix = List (1), suffix = List (2) val (prefix,suffix) = nums.partition (_ = = 1) / / prefix = List (1), suffix = List (2)

17.padTo

PadTo (len: Int, elem: a): List [A]

Expand the list to the specified length, and if the length is not long enough, fill it with elem, otherwise nothing will be done.

Val nums = List (1) val padded = nums.padTo (6) / List (1)

18.combinations,permutations

Combinations: combinations (n: Int): Iterator [list [A]] combines n elements in the list and returns a non-repetitive combination list, resulting in an iterator.

Permutations: permutations: Iterator [list [A]] arranges the elements in the list and returns a list that is not repeated. The result is an iterator.

Val nums = List val combinations = nums.combinations (2). ToList / / List (List (1), List (1)) val permutations = nums.permutations.toList / / List (List, List, List)

19.zip, zipAll, zipWithIndex, unzip,unzip3

Zip: zip [B] (that: GenIterable [B]): List [(A, B)] zips with another list to form a pair with elements in the corresponding position, and the length of the returned list is the shorter of the two lists

ZipAll: zipAll [B] (that: collection.Iterable [B], thisElem: a, thatElem: B): List [(A, B)] operates with another list to form a pair with elements in the corresponding position. If the length of the list is inconsistent and its own list is short, use thisElem to fill it. If the other list is short, use thatElem to fill it.

ZipWithIndex:zipWithIndex: List [(A, Int)] zips list elements with their indexes to form a pair

Unzip: unzip [A1, A2] (implicit asPair: (a) ⇒ (A1, A2)): (List [A1], List [A2]) unzipper operation

Unzip3: unzipper operation of unzip3 [A1, A2, A3] (implicit asTriple: (a) ⇒ (A1, A2, A3)): (List [A1], List [A2], List [A3])

Val alphabet = List ("A", B "," C ") val nums = List (1Magne2) val zipped = alphabet zip nums / / List ((" A ", 1), (" B ", 2)) val zippedAll = alphabet.zipAll (nums," * ",-1) / / List ((" A ", 1), (" B ", 2), (" C ",-1)) val zippedIndex = alphabet.zipWithIndex / / List ((" A ", 0), (" B ", 1), (" C ") 3) val (list1,list2) = zipped.unzip / / list1 = List ("A", "B"), list2 = List (1pim2) val (L1, one,'1') = List ((1, "one",'1'), (2, "two",'2'), (3, "three",'3'). Unzip3 / / l1=List (1pr 2p3), l2=List ("one", "two", "three"), l3=List

20.slice

Slice (from: Int, until: Int): List [A] extracts the list of elements from location from to location until (excluding that location) from the list

Val nums = List (1, 2, 3, 4, 5) val sliced = nums.slice (2, 2, 4) / / List (3, 4)

21.sliding

Sliding (size: Int, step: Int): Iterator [list [A]] groups the list according to the fixed size size. The step is step,step, the default is 1, and the returned result is iterator.

Val nums = List. Val groupStep2 = nums.sliding. ToList / / List (List (1), List (2), List (3), List (4) val groupStep1 = nums.sliding (2). ToList / / List (List (1), List (1), List (2), List (2), List (3), List (3), List (4))

22.updated

Updated (index: Int, elem: a): List [A] updates an element in the list

Val nums = List val fixed = nums.updated (3Power4) / / List (1Power2pint 3jin4) this is the answer to the question on the analysis of Scala's powerful collective data operation examples. I hope the above content can be of some help to everyone. If you still have a lot of doubts unsolved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.