Shulou(Shulou.com)05/31 Report--

This article focuses on "how to understand SQLite software architecture". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Now let the editor take you to learn "how to understand SQLite software architecture"!

SQLite is a very popular database and has entered the top ten in the database rankings. This is mainly because the database is very small and can support mainstream operating systems such as Linux, Windows, iOS, and Andriod.

SQLite is very simple and is an in-process dynamic library database. Its biggest feature is that it can support different languages to use, such as C, C++, Java and so on. At the same time, SQLite is also an open source database, that is, developers can modify the functional features of the data according to their own needs.

Although SQLite is very small, it is very rich in functions, as the saying goes, "although the sparrow is small, it has all the internal organs." SQLite has not only basic SQL features, but also indexes, triggers, views, and transactions.

Primary API of SQLite

SQLite provides two access interfaces, one through the sqlite command line tool, and the other through the dynamic library, the API function. Before learning the SQLite architecture, it is necessary to give a brief introduction to its API. In fact, the API of SQLite is very simple, which mainly consists of three functions, namely sqlite3_open, sqlite3_exec and sqlite3_close. Where sqlite3_exec is the function used to execute SQL statements.

In other words, sqlite3_exec is the key entry to the SQLite function, and we should use this function as a breakthrough point for our later analysis code. Other functions are relatively simple and less important.

The overall architecture of SQLite

First of all, let's introduce SQLIte from the overall architecture. Its architecture is shown in the figure, including the interface layer, SQL command processor and storage back-end.

The core is not the SQLite kernel. It includes three parts: interface layer, SQL command processor and virtual machine. The SQL command processor is responsible for preprocessing the user's SQL and finally generating code suitable for virtual machine execution.

Below it is the back-end part, which is equivalent to the storage engine. Let's briefly introduce the functions of each module.

(1) Interface

The use of the SQLIte library is achieved through function calls. To avoid conflicts with other libraries, SQLite functions are prefixed with sqlite3. The implementation of the interface part is in the files main.c,legacy.c and vdbeapi.c. Main.c contains its main interfaces, including sqlite3_open, sqlite3_config, sqlite3_close and so on. The final function in SQLite is not in main.c, but in legacy.c, which contains only the implementation of this interface.

(2) lexical analyzer

The lexical analyzer parses the SQL statement string and finally generates a sequence of words (token). And pass the generated word sequence to the parser for the next step. The specific implementation of this function is in the file tokenize.c, and the core entry function is sqlite3RunParser.

(3) parser

SQLite's parser is based on the Lemon implementation, which parses SQL statement strings into syntax trees. Lemon is a lexical analysis library similar to YACC/BISON. The source code for the library is in the tool directory.

(4) Code generator

The code generator is used to generate code that corresponds to SQL statements and can be executed in the virtual machine. The implementation of the code generator is more complex, including files such as build.c, delete.c, attach.c, expr.c, insert.c, pragma.c, select.c, auth.c and so on. As can be seen from the file name, many files here actually correspond to a SQL statement, such as delete,insert and select.

(5) Virtual machine

The specific implementation of SQL is done in a component called a virtual machine, which has been shown in the previous architecture diagram. The code executed by the virtual machine is generated by the previous code generator. The virtual machine is implemented in the files vdbe.h and vdbe.c.

(6) B-tree

The data of SQLite is organized and managed by B-tree. Each table or index has a corresponding B-tree. All B trees are stored in a database file. The B-tree is implemented in btree.c and btree.h files.

(7) Page caching

SQLite files are divided into equal parts, and B-tree uses this size as the granularity to manage the data. Page cache is the memory content corresponding to this granularity, through which access to data blocks such as reading and writing is realized. The implementation related to page caching is in files such as pager.c and pcache.c.

(8) operating system interface

SQLite is a cross-platform database, which requires a file system API compatible with Windows and Linux to store data. For convenience, SQLite implements an abstraction layer. In this way, for SQLite business logic, you only need to call the interface of the abstraction layer, regardless of the operating system.

(9) basic library

Contains a basic library that may be used by various modules, such as memory allocation, string processing, etc.

SQLite file format

Earlier, we briefly introduced the software architecture of SQLite and the basic functions of each component. Next, let's introduce the relevant functions of the database file.

In SQLite, a file hosts a database instance, which is called the main library file (main database file). In addition to the main library file, there may be other files, such as log files for transactions. This article mainly focuses on the main library files, and other files are introduced later.

(1) Page

The database file consists of multiple pages, each with a size of between 512 and 65536 bytes, and the size must be a power of 2. Pages are numbered with a starting value of 1 and a maximum number of 2 to the power of 31. The default size of a page is 4KB. This article takes the default size as an example.

Each page in the database has a specific purpose, including:

Lock byte page (Lock-byte page)

Remaining page

B tree page

Pointer mapping page

Payload overflow page

The first page of the database file is special. It contains the description information of the entire database file, which is called the database header information.

(2) Database header

The database header contains 100 bytes of content, with the offset, size and function of each member shown in the figure below.

We can create a database instance and then compare the contents of the file with the format of the database header. For example, the first member of the database header is a magic number, which is used to identify the file as the SQLite database file and version. You can find this information in the following figure, and you can see that the two match exactly (SQLite format 3).

In addition to the format of the database header above, each different page has a different layout.

At this point, I believe you have a deeper understanding of "how to understand SQLite software architecture". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.