Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "what is non-volatile memory technology in the database". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Q1: what is non-volatile memory technology?

Arulraj,Pavlo: it is a wide range of technologies, including phase change memory and memristors. It has low latency, write speed comparable to DRAM, persistence and large storage capacity comparable to SSD. On the market, Intel has produced the products of Aoteng DC NVM module based on 3D XPoint technology.

Q2: what are the potential changes in database management systems based on volatile memory and persistent memory?

Arulraj,Pavlo: existing database management systems are divided into two categories: 1) disk-oriented and 2) memory-oriented. Disk-oriented database management systems are based on the same assumptions as the first generation of relational databases in the 1970s, such as IBM's R system. Based on two-tier storage, volatile memory such as DRAM is used as cache, and slow, non-volatile block devices are used as persistent memory, such as SSD. These systems are based on the pessimistic assumption that transactions can access data that is not in memory and that it takes a long delay to access data on disk. Traditional technologies, such as heavy concurrency control mechanisms, are used to serve these limitations.

Recently, commercial products have greatly expanded the memory of stand-alone DRAM. However, the disk-oriented database system is not designed for scenarios where all data resides in memory. For NVM, many parts of disk-oriented database are redundant.

In contrast, the memory-oriented database architecture assumes that all data is in memory, so there is no need for slow, disk-oriented parts. Therefore, the memory-oriented database is better than the disk-oriented database. However, due to the vulnerability of DRAM, heavy components are still needed to recover after the system crashes. With the advent of NVM, both disk-oriented and memory-oriented database architectures will have a disruptive impact.

Q3: why can't existing database management systems make full use of the advantages of NVM technology?

Arulraj,Pavlo:NVM features include:

1) byte addressable: unlike other non-volatile storage (which only supports data transfer in blocks), NVM supports byte addressability.

2) High speed write load: the write speed can be one order of magnitude higher than that of SSD,NVM. More importantly, there is a small gap between random writing and sequential writing.

3) asymmetry between reading and writing. In some NVM technologies, writing takes longer than reading. In addition, overwriting a single memory unit can damage it.

The advantages of NVM are obvious, and it is important to make full use of them in database systems. We evaluated disk-oriented and memory-oriented databases on NVM, and their performance is similar. Current database management systems assume that memory is volatile, so their architecture needs to persist data to persistent devices. This shows that in order to make full use of the characteristics of NVM, it is necessary to reconstruct the database management system.

Q4: which traditional database system components are not necessary with NVM?

Arulraj,Pavlo: for NVM, several key components of the database system need to be redesigned: 1) logging and recovery protocols; 2) storage and buffer management; 3) indexed data structures

Give examples of logging and recovery protocols. A database system needs to ensure the integrity of data. The transaction of the update operation needs to persist its data to a persistent device such as SSD before returning the application to success. Such storage devices are slower than DRAM, especially on random writes, and only support transmissions in block units.

During a transaction, if you need to overwrite the contents of the database before committing, you must perform a random write to disk. Improve database performance by converting random writes to log sequential writes.

NVM subverts the design of the WAL protocol because it supports fast random writing. So we need to re-customize the new protocol for NVM. For example, write behind logging (WBL). WBL not only improves performance, but also enables crash restart recovery to be completed quickly. WBL tracks which parts of the database have changed, not how. With this logging method, the database can write the data directly without recording it to the log. Write to NVM by sorting to ensure transaction persistence and atomicity, make each transaction write less data, and improve the life cycle of NVM devices.

Q5: you have designed and developed a database system storage engine adapted to NVM. What are the key modules?

Arulraj,Pavlo: traditional database systems are based on a two-tier architecture: DRAM+SSD. These devices have their own hardware characteristics and constraints, and the traditional database system architecture is based on the design to reduce these effects. For example, depending on these devices, two tuple layouts are maintained. Because DRAM bytes are addressed and handle random reads and writes efficiently, tuples in memory can apply for the non-lined field. Tuples stored on SSD have only inlined fields to avoid random writes. In order to apportion the overhead of accessing persistent devices, these engines perform delayed operations through bulk writes and refreshes. However, in systems with NVM storage hierarchies, many of these technologies will no longer be necessary modules. We use the storage and recovery mechanism of traditional engines to take advantage of the characteristics of NVM.

For example, NVM-aware storage engine with in-place update. When a transaction inserts a tuple, it does not need to be copied into WAL for recovery, etc., the storage engine only needs to record a tuple's non-volatile pointer in WAL. This is very efficient because pointers and tuples are stored on NVM. So after the system is rebooted, the tuple can be accessed through the pointer without the need to play back the WAL. The index is also used as a non-volatile B+tree, which can be accessed immediately after the system is rebooted without rebuilding. Because the modification is persisted immediately when the transaction is committed, the transaction committed after the system restart is also persistent. Because the timing of memory controller flushing for cache lines is uncertain, modifications made by uncommitted transactions may also be persisted. So the storage engine needs to roll back these transactions through WAL. Because the recovery protocol does not include the redo processing flow, the NVM-aware engine has less recovery latency than the traditional storage engine.

Q6: what is the main point of this book?

Arulraj, Pavlo: this book introduces key algorithms and data structures for adapting to NVM, which not only improves performance and reduces operational consumption, but also simplifies development and crash recovery time. Our project started in 2013. We're not sure if NVM will hit the ground, but intel's arrogant series based on 3D Xpoint technology will commercialize it in 2019. We are excited about the impact of NVM on the next generation of database systems.

This is the end of the content of "what is non-volatile memory technology in the database". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.