PostgreSQL pg_rewind principle

I. background

In the common high-availability architecture, if the master fails and some data is not synchronized to the backup, the high-availability system will promote the slave as the main external service. When it is possible for the owner to join the cluster as a backup, it may fail to build a stream replication relationship. You can use pg_rewind tools to make the master and backup data consistent.

II. Pg_rewind principle

III. Related codes

1. The difference of each file (directory) is recorded in the structure file_entry_t, which is defined as typedef struct file_entry_t {char * path; file_type_t type; file_action_t action; / * for a regular file * / size_t oldsize; size_t newsize; Bool isrelfile; / * is it a relation data file? * / datapagemap_t pagemap. / * for a symlink * / char * link_target; struct file_entry_t * next;} file_entry_t;2, file type typedef enum {FILE_TYPE_REGULAR,// regular files FILE_TYPE_DIRECTORY,// directory FILE_TYPE_SYMLINK// soft connection} file_type_t 3. Actiontypedef enum {FILE_ACTION_CREATE, / * create a directory or soft link: create_target (entry) * / FILE_ACTION_COPY, / * copy the entire file or rewrite the existing file: fetch_file_range (entry- > path, 0, entry- > newsize) * / FILE_ACTION_COPY_TAIL, / * copy part of fetch_file_range from oldsize to newsize from source (entry- > path, entry- > oldsize, entry- > newsize) * / FILE_ACTION_NONE, / * No operation * / FILE_ACTION_TRUNCATE, / * cut target cluster files to 'newsize' size: truncate_target_file (entry- > path) Entry- > newsize) * / FILE_ACTION_REMOVE / * Delete local files / directories / soft links: remove_target (entry) * /} file_action_t 4. Interpretation of other variables isrefile indicates whether the file is a table data file. The path of the table data file must meet the following conditions: isRelDataFile (path): the file under the global/ directory, that is, the file under the base/ directory of the table file directory shared by the database, that is, the file under the pg_tblspc/&rnode.spcNode/TABLESPACE_VERSION_DIRECTORY/ directory under the table file directory of the default tablespace. That is, the files under the table file directory of other tablespace, in which PG_9.4_201403261 and version-related file names conform to the format pagemap (how to use it? ExtractPageInfo) stores a bitmap, and each bit stores whether each page in the corresponding destination cluster file has changed since the bifurcation point of the two clusters. 1 indicates change and 0 indicates no change. Oldsize represents the size of the file in the destination cluster, and newsize represents the size of the file in the source cluster. In pg_rewind, specify the processing action of the file by comparing the corresponding file size between the source cluster and the destination cluster or whether the file (directory) exists, for example: oldsize > newsize: action=FILE_ACTION_TRUNCATE oldsize < newsize: action=FILE_ACTION_COPY_TAIL if the file does not exist, except for the action=FILE_ACTION_COPY,PG_VERSION file, if the directory does not exist, then action=FILE_ACTION_CREATE if the file is redundant Then the above actions of action=FILE_ACTION_REMOVE are handled by the functions process_target_file and process_source_file. To only set FILE_ACTION_REMOVE5 in process_target_file and extract the wal log to get the change page: extractPageInfo: for (block_id = 0; block_id max_block_id; block_id++) {if (! XLogRecGetBlockTag (record, block_id, & rnode, & forknum, & blkno)) continue; / * We only care about the main fork What does others are copied in toto * / if (forknum! = MAIN_FORKNUM) / / MAIN_FORKNUM mean? Continue; process_block_change (forknum, rnode, blkno) } 6. The bitmappg_rewind tool in pagemap needs to open full_page_writes, and after opening full_page_writes, all the contents of the data page corresponding to the first modification of each data page after checkpoint will be written in the WAL log record, so pg_rewind can easily find the corresponding modified data page information according to the organizational structure of the WAL log, and set the bitmap of the corresponding file_entry_t to 1. XLogRecGetBlockTag: XLogReaderState.blocks [XLR _ MAX_BLOCK_ID + 1]