Shulou(Shulou.com)05/31 Report--

This article mainly introduces the spark mllib data dimensionality reduction how to achieve singular value decomposition, the article is very detailed, has a certain reference value, interested friends must read it!

The running code is as follows: package spark.DataDimensionReductionimport org.apache.spark.mllib.linalg.Vectorsimport org.apache.spark.mllib.linalg.distributed.RowMatriximport org.apache.spark. {SparkContext, SparkConf} / * * data dimensionality reduction * A matrix is stretched in one direction during the calculation. What needs to be concerned about is the amplitude and direction of the stretch. * singular value decomposition (SVD): a matrix is decomposed into a matrix multiplied by a direction vector * Created by eric on 16-7-24. * / object SVD {val conf = new SparkConf () / / create the environment variable .setMaster ("local") / / set the localization handler .setAppName ("SVD") / / set the name val sc = new SparkContext (conf) Def main (args: Array [String]) {val data = sc.textFile (". / src/main/spark/DataDimensionReduction/a.txt") .map (_ .split (") .map (_ .toDouble)) .map (line = > Vectors.dense (line)) val rm = new RowMatrix (data) / / read row matrix val SVD = rm.computeSVD (2 ComputeU = true) / / SVD calculation println (SVD) / / print SVD result matrix / / find the matrix of SVD decomposition println ("*") val u = SVD.U val s = SVD.s val v = SVD.V println (u, s) V)}} a.txt1 2 3 45 6 7 89 0 8 76 4 2 1 results are as follows

These are all the contents of the article "how to achieve singular value decomposition in data dimensionality reduction in spark mllib". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.