Shulou(Shulou.com)05/31 Report--

This article mainly explains "what is the MyRocks of MySQL". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "what is the MyRocks of MySQL"?

A preface

In the last year or two, the development of database technology, especially MySQL, can be described as letting a hundred flowers blossom. TokuDB,MyRocks, MySQL 5.7 GA,MySQL 8.0 doc release its software is also under development, ALiSQL is open source. Among them, there are some functional improvements and some optimizations for the shortcomings of Innodb itself (mainly in terms of storage space). As practitioners of database technology, they are somewhat overwhelmed.

What is two MyRocks?

MyRocks is an open source database storage engine developed by FB based on levelDB (using LSM to organize data structures). It supports general MySQL read and write, lock mechanism, MVCC, transactions (currently only RR,RC), master-slave replication. It is currently in use in the user center of FB.

What problem does MyRocks solve?

This needs to start from the business model of FB, fb as the world's largest sns website, its huge amount of data, the server that needs to store data must be hundreds of thousands or millions of levels, these are money. (it also applies to companies like Alibaba and Tencent.) in order to cope with the future growth of storage space and save servers, fb developed this storage engine. Why doesn't Innodb meet the demand? Friends who are familiar with the Innodb storage engine know that Innodb is a data storage based on B+Tree. Its default block size is 16k and currently supports adjustable block_size. But there are still the following problems:

1 write magnification.

When B*tree wants to modify the data, it needs to rearrange the data under the new data, especially when the written data is arranged in a higher position, it requires a large number of shift operations to complete the write. And the minimum logical unit of InnoDB read-write access data is the data block, in the case of random write, modify N rows, may have to modify N data blocks.

MyRocks writes in the form of append only. According to LSM Tree's algorithm, random writes are converted into sequential writes, write operations only need to update memory, and the data in memory is brushed to disk in the form of block data, which is a sequential IO operation. In addition, periodic merging of disk files will also bring disk IO operations.

The specific implementation methods are as follows:

A when there is a write operation (or update operation), the buffer is written to the memory, and the key is kept in order through some data structure (such as skiplist).

B the normal implementation also appends the data to the disk Log file for recovery if necessary.

C the data in memory is scrubbed to the disk regularly or by a fixed size, and the update operation is only constantly written to memory, and the existing files on the disk are not updated.

D with more and more writes, more and more files are accumulated on disk, which are unwritable and orderly.

E regularly merge files (compaction) to eliminate redundant data and reduce the number of files.

2 disk space debris

Due to the split of B*tree, there is more free space in page, and the overall space utilization is not high. Although Innodb provides a way of compression, compression in block units can also be wasteful. For example, a total of 16k of data is compressed to 5k, but the storage disk still needs 8k of space.

3 RocksDB alignment cost is small: SST file (default 2MB) needs alignment, but it is much larger than 4k, while RocksDB_block_size (default 4k) does not need alignment, so it wastes less space.

Fourth, what are the restrictions?

1 MyRocks currently supports only two isolation levels, RC and RR.

2 the lock mechanism is not perfect and does not support gap lock

(3) at this stage, the mixed use of Innodb and Myrocks is unstable, Percona is doing the work of merging, and the Percona engineer at the ACMUG conference said that 80% of the scenarios have been tested.

4. There is no xa between binlog and RocksDB, so abnormal crash may lose data. Therefore, MyRocks generally turns on semi-sync.

5 the read performance is relatively weak compared with Inndb, MRR is not supported, and range query is relatively slow.

At this point, I believe you have a deeper understanding of "what is the MyRocks of MySQL". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.