Shulou(Shulou.com)06/02 Report--

In this issue, the editor will bring you an overview of SQLite Database System Design and Implemention Pager Module. The article is rich in content and analyzes and narrates it from a professional point of view. I hope you can get something after reading this article.

By reading this section, you should be able to explain the following questions:

What is 1.page cache? Why do you need it? Who is using it?

two。 General cache management techniques

The transaction process and rollback process adopted by 3.SQLite

This chapter discusses the pager module, which implements an abstract database page file system on native byte files, acts as the manager of fixed-size data pages, and defines how to retrieve data from database files.

The interface of these pages. It helps the Tree module speed up the retrieval of database pages by providing a cache on the memory level of the data file, that is, it manages the page cache. It is also a transaction manager, which implements the handling of ACID features through concurrent access control and failure rollback.

It makes concurrency control and rollback operations completely transparent to Tree and other higher modules. It still plays the role of lock and log manager. In fact, the pager module implements persistence in general database management systems.

Except for the in-memory database, the database is located in external storage such as disk and stored in the original file.

SQLite cannot efficiently access and control data on disk.

When SQLite needs data, it reads it from the database file into main memory, controls the data in memory, and writes the data back to the database file if necessary.

In general, the total size of the database file is much larger than the amount of memory available. Due to the limited size of main memory, only part of the memory is reserved for database files, which is very small for the entire database file, and this reserved memory space is often called database cache or data buffer; in SQLite terms, it is called page cache. This cache is located in the address space processed by the application, not in the operating system space. The operating system has its own data cache.

In SQLite, the page cache manager is called pager.

This module is aimed at lower-level ordinary native files in bytes, and converts them into randomly accessible high-level files in units of page, these page are fixed-size objects that are read from the native file system.

Files at different levels can have different page sizes.

Pager defines easy-to-use interfaces for reading these database files independent of the file system.

The tree module is directly located on the upper layer of the pager module, and it uses the interface provided by the pager module to access the database from beginning to end, never directly accessing any database files or log files. The tree module faces database files like a logical array of uniform-sized page, and accesses these page through their subscripts.

In SQLite, the database files you open in the United States or database links maintain a separate page cache.

When an application opens a database file, the pager module creates and initializes a new page cache for the file.

If the program opens the same database file two or more times, in the default processing mode, pager creates and initializes as many separate page caches as the database file.

SQLite supports an advanced feature that allows all links that open the same database to share the same page cache, which may be opened multiple times by the same or different database links.

In memory databases, no data points to external storage, but they also process and save data through pager. Therefore, the tree module uses the same interface to get different types of databases.

Pager is the lowest-level module in SQLite. It is the only module that accesses native data files and log files through the IO interface provided by the native operating system.

He reads and writes database files and log files directly.

He doesn't understand how data is organized and stored in the database. He also does not interact with the content in the database and does not modify the data content himself.

He simply ensures that no matter what information is stored in the database file, it can be obtained repeatedly without any conversion.

In a sense, pager is a passive entity.

Although it may modify some header information of the database file, such as the number of changes to the file.

He abstracts the operation of database files from a general random access byte file system to a random access page system.

He defines a set of interfaces that are easy to use and independent of the file system interface that can randomly access database file pages.

For each database file, moving page between the database file and memory is a basic function of pager as a cache manager.

This page movement is transparent to tree and higher-level modules.

Pager is a medium between native file systems and high-level modules.

His main purpose is to make database pages addressable in memory so that these modules can directly access the contents of the pages in memory.

He also located the location where the page was written back to the database file.

He created the abstract concept that database files are stored in memory as an array of pages, and that the tree and pager modules work together through a well-defined page access protocol.

In addition to cache management, pager also takes on many other functions that a typical database management system should have.

He provides the core services of a typical transaction processing system: transaction management, data management, log management and lock management.

As a transaction manager, he implements the ACID features of transactions by managing concurrent null modulation and rollback operations.

He is also responsible for atomic commit and transaction rollback.

As a data manager, he locates reading and writing database files through in-memory cache pages, and is responsible for file space management.

As a log manager, he implements logging in past log files.

As a lock manager, he ensures that the transaction already has the appropriate lock on the database file before getting a database page.

In essence, the pager module implements persistence of storage and atomicity of transactions.

The above is the summary of SQLite Database System Design and Implemention Pager Module shared by the editor. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.