MongoDB-Map&Reduce
Simulate big data search scene-he Xudong
Please use the script to insert the student form of the Map&Reduce example in this course into the 1000W section file. Field
Stay the same.
Use Map&Reduce to calculate 10
Db.users.find ()
{
"_ id"
:
ObjectId ("55ca9ae785b177a46da9494f")
"classid"
:
one,
"age"
:
thirty-seven,
"name"
:
"name0"
}
{
"_ id"
:
ObjectId ("55ca9ae785b177a46da94950")
"classid"
:
one,
"age"
:
twelve,
"name"
:
"name1"
}
{
"_ id"
:
ObjectId ("55ca9ae785b177a46da94951")
"classid"
:
one,
"age"
:
thirty-one,
"name"
:
"name2"
}
{
"_ id"
:
ObjectId ("55ca9ae785b177a46da94952")
"classid"
:
two,
"age"
:
twenty-seven,
"name"
:
"name3"
}
The script creates the simulation data:
For (var iTunes 1)
I mapf = function () {emit (this.classid, 1)}
Function () {emit (this.classid, 1)}
Reduce function
The parameters received by the Reduce function are similar to the Group effect and have been aggregated once according to the key.
Combine the sequence of key values returned by Map into {key, [value1,value2,value3,....,valuen]} and pass it to
Statistics of values by Reduce,Reduce function
> reducef=function (key, values) {
... Var count = 0
... Values.forEach (function (v) {count + = v;}); return count
...}
Function (key, values) {
Var count = 0
Values.forEach (function (v) {count + = v;}); return count
}
More control details of Options
> res = db.runCommand ({mapreduce: "users", map:mapf, reduce:reducef
Out: "users_res"
Finalize:ff
Query: {age: {$lt:10}}
...
});
{
"result"
:
"users_res"
"timeMillis"
:
6251
"counts"
:
{
"input"
:
333716
"emit"
:
333716
"reduce"
:
6676
"output"
:
two
}
"ok"
:
one
}
>
Results:
>
Db.users_res.find ()
{
"_ id"
:
one,
"value"
:
{
"classid"
:
one,
"count"
:
167142
}
}
{
"_ id"
:
two,
"value"
:
{
"classid"
:
two,
"count"
:
166574
}
}
There are 167142 students in Class 1 under the age of 10. There are 166574 students under 10 years old in Class 2.
Continue to count the number of students under the age of 20 in each class:
> res = db.runCommand ({mapreduce: "users", map:mapf, reduce:reducef
Out: "users_2res", finalize:ff, query: {age: {$lt:20}}
.})
{
"result": "users_2res"
"timeMillis": 23247
"counts": {
"input": 3666243
"emit": 3666243
"reduce": 73326
"output": 2
}
"ok": 1
}
> db.users_2res.find ()
{"_ id": 1, "value": {"classid": 1, "count": 1832306}}
{"_ id": 2, "value": {"classid": 2, "count": 1833937}}
There are 1832306 students in Class 1 under the age of 20. There are 1833937 students under 20 years old in Class 2.
