Shulou(Shulou.com)06/02 Report--

This article is to share with you about why MONGODB Wiredtiger is so fast, the editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article.

If someone says that I am proficient in ORACLE and MYSQL, then I believe it, but if someone says that I am particularly proficient in the principles and source code of MONGODB, then I really admire it. There are few books on this subject, so it is not easy to master the principles of MONGODB.

Why is it a wild guess, because if you are wrong, it is waiting for you to correct each other and improve each other's technical level.

First of all, take a look at the overall architecture diagram of a mongodb that may have been damaged, from which you can summarize the following list

1 mongodb supports row and column storage, which is currently used in most states

2 the format of the data stored in disk and cache should be different, and it needs to be compressed and decompressed.

(3) the data processing is to deal with uncompressed data.

(4) the system carries out multi-version control of MVCC through snapshot to reduce the generation of locks.

5. When a transaction is generated, the LOG log is dropped first, and the data is flushed to disk in the buffer.

Because most of the data is processed in memory and the mutex between locks and concurrency is minimized, the structure of MONGODB is simpler than that of other traditional databases, which is one of the reasons why the operation of MONGODB is faster than other databases. To mention in memory, mongodb does not use traditional file system offsets as pointers in its addressing design. If it is the presentation of the three layers of the B+ tree, the information of the non-leaf nodes will be in memory in memory.

So addressing is done in memory, which leads to higher performance, which is one reason why the memory given to MONGODB should not be too stingy, as mentioned in some words.

Summarize the architecture of in memory

1 the system will cache trees and optimize part of the page to access in memory

2 when the pointer traverses the B+ tree, it does not lock reads and writes

3 ensure that the original data is separated from the updated data

4 structured changes (such as page merge and splits) are handled in background threads

Of course, some databases are doing the same, the reason why wiredtiger will become faster is that another word no locking no lock manager, other traditional databases have the concept of lock, some databases are similar to sql server lock and lock upgrade concept, it is conceivable that the more complex things, its speed will not be as fast as where. MONGODB abandons the place, creates a snapshot for the data being read through MVCC and its own characteristics, and creates a new version of the data being written, so that read and write are separated, and the overall MONGODB is separated from the word lock through the concept of timestamp (see previous article). The speed will naturally be faster and stronger. Of course, due to its own design reasons, there is no need to think too much about rollback.

The reason why MONGODB is fast can also be attributed to block manager, the director of Block mananger, who maximizes the compression of data when it is written. this advantage is that current machines have a very powerful CPU, but in fact, most of the bottleneck lies in the disk, and because the compression LEVEL of the data becomes very high by transferring the pressure to CPU, the burden of reading and writing data will be reduced. It skillfully takes advantage of the current hardware. At the same time, compression can also be selected according to the current machine and the environment.

Snappy is the default compression method, LZ4 is more suitable for page layout, and zlib has a high compression ratio.

The effective way of checkpoint is also a factor that promotes the speed of mongodb wiredtiger processing.

As you can see from the image above, the modification of the page is not carried out in the original space, but the changed page is generated on the new page, pointing to the unchanged page from the newly generated page through the pointer. Write the new root address to the metadata. This process is much faster than the way you lock and then change the data. And most of the MONGODB processing methods are write, and read, a large number of UPDATE is rare.

With the MVCC mentioned above, there must be an old version of the reader in the system, so that there is no interference between reading and writing data, and the thing that makes this simple is that MONGODB does not have UNDO LOG REDO, which makes the system more complex and slower (see MYSQL ORALCE UNDO REDO).

So why is MONGODB WIREDTIGER so fast?

1 its own architecture design, because there is no transaction that requires a logical rollback, writing is atomic

2 use a lot of memory to deal with read and write data

(3) the compression of written data is very helpful to Icano.

4 MVCC and related design, lock-free design

(5) the special way of data update will not affect the performance of other databases such as page merge page split.

This is why MONGODB Wiredtiger is so fast. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.