Shulou(Shulou.com)05/31 Report--

This article mainly introduces "how to use PageRank". In daily operation, I believe many people have doubts about how to use PageRank. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "how to use PageRank". Next, please follow the editor to study!

Package week7

Import org.apache.log4j. {Level, Logger}

Import org.apache.spark. {SparkContext, SparkConf}

Import org.apache.spark.graphx._

Import org.apache.spark.rdd.RDD

Object PageRank {

Def main (args: Array [String]) {

/ / block logs

Logger.getLogger ("org.apache.spark") .setLevel (Level.WARN)

Logger.getLogger ("org.eclipse.jetty.server") .setLevel (Level.OFF)

/ / set up the running environment

Val conf = new SparkConf () .setAppName ("PageRank") .setMaster ("local")

Val sc = new SparkContext (conf)

/ / read the data file

Val articles: RDD [String] = sc.textFile ("/ home/mmicky/IdeaProjects/data/graphx/graphx-wiki-vertices.txt")

Val links: RDD [String] = sc.textFile ("/ home/mmicky/IdeaProjects/data/graphx/graphx-wiki-edges.txt")

/ / load vertices and edges

Val vertices = articles.map {line = >

Val fields = line.split ('\ t')

(fields (0) .toLong, fields (1))

}

Val edges = links.map {line = >

Val fields = line.split ('\ t')

Edge (fields (0) .toLong, fields (1) .toLong, 0)

}

/ / cache operation

/ / val graph = Graph (vertices, edges, ") .resume (StorageLevel.MEMORY_ONLY_SER)

Val graph = Graph (vertices, edges, ") .persist ()

/ / graph.unpersistVertices (false)

/ / Test

Println ("* *")

Println ("get 5 triplet messages")

Println ("* *")

Graph.triplets.take (5) .foreach (println (_))

/ / cache () is used in the pageRank algorithm, so only MEMORY_ONLY can be used in the previous persist.

Println ("* *")

Println ("PageRank Computing, getting the most valuable data")

Println ("* *")

Val prGraph = graph.pageRank (0.001) .cache ()

Val titleAndPrGraph = graph.outerJoinVertices (prGraph.vertices) {

(v, title, rank) = > (rank.getOrElse (0.0), title)

}

TitleAndPrGraph.vertices.top (10) {

Ordering.by ((entry: (VertexId, (Double, String) = > entry._2._1)

} .foreach (t = > println (t.foreach 2 + ":" + t.foreach 2.room1))

Sc.stop ()

}

}

At this point, the study on "how to use PageRank" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.