Shulou(Shulou.com)06/01 Report--

How to implement a PageRank algorithm in Hadoop? aiming at this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

The input file format is as follows:

1 1.0 2 3 4 5 6 7 8

2 2.0 3 4 5 6 7 8

3 3.0 4 5 6 7 8

4 4.0 5 6 7 8

5 5.0 6 7 8

6 6.0 7 8

7 7.0 8

8 8.0 1 2 3 4 5 6 7

The code is as follows:

Package com.apache.hadoop.io;import java.io.IOException;import java.text.DecimalFormat;import java.text.NumberFormat;import java.util.StringTokenizer;import java.util.Iterator;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.Path;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Job;import org.apache.hadoop.mapreduce.Mapper;import org.apache.hadoop.mapreduce.Reducer Import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;import org.apache.hadoop.util.GenericOptionsParser;public class PageRank {public static class MyMapper extends Mapper {private Text id = new Text (); public void map (Object key, Text value, Context context) throws IOException, InterruptedException {String line = value.toString () / / determine whether it is an input file if (line.substring (0jue 1). Matches ("[0-9] {1}")) {boolean flag = false If (line.contains ("_")) {line = line.replace ("_", "); flag = true } / / A pair of input files are processed String [] values = line.split ("\ t"); Text t = new Text (values [0]) String [] vals = values [1] .split (""); / / Save url, and String url= "_" will be used in the next calculation; double pr = 0 Int I = 0; int num = 0; if (flag) {iTun2 Pr=Double.valueOf (vals [1]); num=vals.length-2 } else {iTun1; pr=Double.valueOf (vals [0]) Num=vals.length-1;} for (; I

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.