Shulou(Shulou.com)06/02 Report--

In this issue, Xiaobian will bring you about how to realize aggregation operation in MongoDB. The article is rich in content and analyzes and narrates from a professional perspective. After reading this article, I hope you can gain something.

MongoDB, although it has always been called NO SQL, document database, but in the past two years MONGODB has not been idle, MONGODB as a big data analysis data storage space is now used by many people.

Aggregation as MONGODB for the traditional database GROUP BY, even the embodiment of window functions in MONGODB, is more commonly used. Performance is not an issue when the data volume is small, but aggregation operations using MONGODB are tricky and careful when the data volume is large.

First of all, from the initial principle of aggregation, MONGODB aggregation is phased, which can be roughly separated from data extraction and data calculation.

In the following diagram, you can see the pitfalls you might encounter in an aggregation operation. In a simple aggregation,(The amount of data is 50 million, so in order to emerge the problem, it is directly aggregated, and the value is a random value, so you know), directly reporting an error in the operation, this error message is due to pipleline stage during the operation aggregation.(See your operation specifically, here is the group stage, the use has exceeded the MONGODB limit of 100MB), so in the case of a large amount of data, and no data amount filtering, or data filtering (here called $match), or a large amount of data, we should do aggregation, add a parameter, this parameter is supported in MONGODB 3.4.

The specific sentence is

In this case, you can use the physical disk environment to break through the limitations of aggregation.

Generally speaking, there are the following points to be paid attention to in doing aggregation (incomplete, some operations are not used at present, and these operations are not within the scope of summary)

1. Data that is no longer aggregated should be filtered in advance by $match

2. The fields displayed need to be controlled. Fields unrelated to aggregation should be advanced by $project.

3. If the aggregate contains sorting, the sorting must be below $match

4, when there are multiple $match operations, try to merge.

5. For $match or $sort operations, indexes should be established in case of large data volume.

6 If the operation contains $sort $limit, the $limit operation should be followed

Depending on the current situation, most aggregations can be optimized for commands in MONGODB 3.4 and above. So if you have a large amount of data to do aggregation operations, it is better to upgrade to version 3.4 and above.

The above is how to realize aggregation operation in MongoDB shared by Xiaobian. If there is a similar doubt, please refer to the above analysis for understanding. If you want to know more about it, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.