How to implement MapReduce engine

Today, I will talk to you about how to achieve the MapReduce engine, many people may not know much about it. In order to make you understand better, the editor has summarized the following content for you. I hope you can get something according to this article.

How to implement MapReduce engine

See below for the use example, only one name import is required.

#! / usr/bin/env python

Import mincemeat

Data = ["Humpty Dumpty sat on a wall"

"Humpty Dumpty had a great fall"

"All the King's horses and all the King's men"

"Couldn't put Humpty together again"

]

Def mapfn (k, v):

For w in v.split ():

Yield w, 1

Def reducefn (k, vs):

How to implement MapReduce engine

Result = 0

For v in vs:

Result + = v

Return result

S = mincemeat.Server ()

# The data source can be any dictionary-like object

S.datasource = dict (enumerate (data))

S.mapfn = mapfn

S.reducefn = reducefn

Results = s.run_server (password= "changeme")

Print results

Then put mincemeat.py and example.py in the same directory and execute example.py

Python example.py

At this point the program will suspend a daemon

Then open another terminal to run:

Python mincemeat.py-p changeme localhost

You will see that the daemon just printed out the MapReduce result and exited.

{'averse: 2,' on': 1, 'great': 1,' Humpty': 3, 'again': 1,' wall': 1, 'Dumpty': 2,' men': 1, 'had': 1,' all': 1, 'together': 1, "King's": 2,' horses': 1, 'All': 1, "Couldn't": 1,' fall': 1, 'and': 1,' the': 2 'put': 1,' sat': 1}

After reading the above, do you have any further understanding of how to implement the MapReduce engine? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.