Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to write the MapReduce program of Hadoop with PHP". In the daily operation, I believe that many people have doubts about how to write the MapReduce program of Hadoop with PHP. The editor consulted all kinds of materials and sorted out a simple and easy-to-use method of operation. I hope it will be helpful to answer the doubt of "how to write Hadoop MapReduce program with PHP". Next, please follow the editor to study!

Hadoop flow

Although Hadoop is written in java, Hadoop provides a Hadoop stream, and Hadoop stream provides an API that allows users to write map and reduce functions in any language.

The key to Hadoop flow is that it uses the UNIX standard stream as the interface between the program and Hadoop. Therefore, as long as any program can read data from the standard input stream and write the data to the standard output stream, the map and reduce functions of MapReduce programs can be written in any language through the Hadoop stream.

For example, bin/hadoop jar contrib/streaming/hadoop-streaming-0.20.203.0.jar-mapper / usr/local/hadoop/mapper.php-reducer / usr/local/hadoop/reducer.php-input test/*-output out4

Package introduced by Hadoop stream: there is no hadoop-streaming.jar in the hadoop-streaming-0.20.203.0.jar,Hadoop root directory. Because streaming is a contrib, you need to find it under contrib. Take hadoop-0.20.2 as an example, it is here:

-input: indicates the path to the input hdfs file

-output: indicates the path to the output hdfs file

-mapper: indicates the map function

-reducer: indicates the reduce function

Mapper function

Mapper.php file, write the following code:

[php]

#! / usr/local/php/bin/php

The general meaning of this code is to find out the words in each line of text entered and use "

Hello 1

World 1 "

Output in this form.

It's basically no different from the previous PHP, right? there are two things that may make you feel a little strange:

PHP as an executable program

In the first line

[php]

#! / usr/local/php/bin/php

Tell linux to use the program #! / usr/local/php/bin/php as the interpreter for the following code. Anyone who has written linux shell should be familiar with this. The first line of every shell script goes like this: #! / bin/bash, #! / usr/bin/python

With this line, after saving the file, you can directly execute mapper.php as a command like cat grep like this:. / mapper.php

At this point, the study on "how to write the MapReduce program of Hadoop with PHP" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.