Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

Does MONGODB support lsm and cost control DUMP ROCKSDB?

2025-01-28 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Internet Technology >

Share

Shulou(Shulou.com)06/02 Report--

Whether MONGODB supports lsm and cost control DUMP ROCKSDB in the end, many novices are not very clear about this. In order to help you solve this problem, the following editor will explain it in detail. People with this need can come and learn. I hope you can get something.

Recently, we have encountered two questions: whether the wriedtiger engine supports LSM tree, and why percona's mongodb Dump has ROCKSDB's database engine.

First of all, the difference between BTREE and LSM TREE needs to be clarified.

1 BTREE has the advantages of orderly data storage and fast speed of reading bounded data. Based on the traditional disk principle, the data can be located quickly by index.

2 the advantages of LSM TREE, larger capacity of data storage, the use of merge mechanism, adaptability to SSD disk changes, through BLOOM filtering to find data, the speed is not slow

Roughly how LSM TREE works

After reaching the threshold size of the memory tree entering the mongodb wiretiger lsm tree in memory, a new memory tree is created, and the old tree is synchronized to the disk. After writing to the disk, the tree is read-only and the LSM tree is merged on the disk.

WiredTiger creates an Bloom filter when merging, which is an additional file, each key contains a configurable number of bits (default is 8), the key is hashed by configurable number of times (default is 4), and the corresponding bits are set, and the Bloom filter is used to avoid reading from the block when the key does not exist.

The above information is actually found in the help page of wiredtiger's database engine.

Https://github.com/wiredtiger/wiredtiger/wiki/Btree-vs-LSM

There are relevant comparisons on wiredtiger's github.

In the text, finally summed up a few points

1 Btree data storage structure in most cases can meet the needs of writing and query, LSM TREE will be based on data insertion, regular background maintenance, (related to the principle).

2 if there is a need for a large number of write operations, LSM TREE is a better alternative to BTREE

3 the tests are only for simple data schemas

4 if the requirements of collections in the database are complex, you can choose a suitable mixed mode of lsm and btree in a DATABASE.

But in fact, in the operation, it is found that even in accordance with the way provided by wiredtiger, it is written to btree. The following figure shows. So far, although wiredtiger can create collecion early, choosing the data structure is LSM, but it is only a configuration test interface. It is not generated in this way. Collection is the LSM data storage structure. But it was tested in a state of doubt.

The following generates a collection of lsm tree structure and builds an index of lsm tree

The related collection and index we established are all on the suffix.

In fact, we are inserting 200000 data per connection through 2000 connections, and there is more data to be inserted in one minute as a measure. After many tests, the final result is that the performance of collection with lsm tree config is better than that of collection with btree, but the related advantages are slightly unstable.

Another question is why percona Mongodb starts from 3.6and directly dump the rocksdb database (caused by facebook's lsm tree database structure storage)

The answer given by percona

Among the service customers, most of them do not have to use lsm tree, and most of them use wiredtiger database engine.

2. From the development of MONGODB, the new functions in 3.6are all developed around wiredtiger database engine, and MMAPV1 database engine will be removed in 4.0.In the future, most of the new functions will focus on wiredtiger database engine, so if rocksdb database engine wants to achieve the same multi-document transactions on MONGODB, as well as other new functions, the code changes will be huge, so it is related to the related costs.

The database engine of 1 ROCKSDB is not the mainstream on MONGODB, and the customers who use it do not put forward the special requirements of large area writing mentioned above.

2 MONGODB inclines to wiredtiger database engine in the new version and the existing version, just as there are many database engines to choose from before MYSQL, but finally mentioning MYSQL and mentioning INNODB have the same meaning, mentioning MONGODB and wiredtiger database engine also have the same meaning.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 279

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Internet Technology

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report