Shulou(Shulou.com)06/01 Report--

Introducing PMDK into PostgreSQL

Persistent memory (PMEM) is fast, non-volatile and byte accessible, and can be accessed directly by CPU through load/store instructions. This product is now available from suppliers. Database management systems perform better on PMEM than HSS or SSD. With the help of PMDK (Persistent Memory Development Kit), the performance of the database can be further improved by modifying the database to a product that adapts to PMEM. The topic of this lecture focuses on how to modify Postgresql to adapt to PMEM, and how the modified effect is. Our first step will be to improve OLTP performance and checkpoint time around WAL logs and tables.

There are two ways to use PMEM. The first is the easiest way to access the disk directly through direct access to the file system (DAX,direct-access), that is, skipping the operating system's page cache. This method does not need to modify the PG. Another approach: PMDK contains lib libraries adapted to PMEM, which can bypass the kernel and go directly to PMEM mapping files and skip the CPU cache for in-memory copy. This method has better performance.

Use PMDK to modify PG, focusing on WAL and table segment files. We use the PMEM function provided by PMDK instead of the system call functions open, lseek, read, write, and fdatasync. Then compare the performance with the native PG running on the DAX file system. In the experiment, we use the flying wing DIMM (NVDIMM) as the PMEM. The results show that in terms of WAL, we can increase the TPS by 1.8 times in the INSERT scenario. We have made nearly 1200 lines of changes. For watches, we can reduce time by nearly 20% when we checkpoint.

On the other hand, we also explore other aspects, such as controlling the impact of NUMA, eliminating the overhead of SQL parsing, and extending PMEM-mapped 's fixed-size table files.

Original address

Https://www.pgcon.org/2018/schedule/events/1154.en.html

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.