Shulou(Shulou.com)06/01 Report--

I. principle

When performing promote, we often see the result: generate a new XLOG file, named: timeline plus 1, segment file name is the previous segment file number. So when is the time to do this? Is there only one phenomenon, and will there be other phenomena? First, when the last XLOG log is SWITCH before performing the promote action, the segment file number will be added by 1. Next, we do a detailed analysis of its process, and understand its principle through gdb.

The function that does this action is exitArchiveRecovery. When the startup process exits, see the stack:

Breakpoint 1, exitArchiveRecovery (endTLI=1, endOfLog=23053968) at xlog.c:54755475 InArchiveRecovery = false; (gdb) bt#0 exitArchiveRecovery (endTLI=1, endOfLog=23053968) at xlog.c:5475#1 0x0815fc4b in StartupXLOG () at xlog.c:7460#2 0x083dbc56 in StartupProcessMain () at startup.c:207#3 0x08173cbf in AuxiliaryProcessMain (argc=2, argv=0xbfa8a2f4) at bootstrap.c:451#4 0x083dab93 in StartChildProcess (type=StartupProcess) at postmaster.c:5386#5 0x083d5d86 in PostmasterMain (argc=1, argv=0xa22e7e8) at postmaster.c:1369#6 0x0831c76c in main (argc=1, argv=0xa22e7e8) at main.c:228

After the specific code behavior is restored:

StartupXLOG- > read checkpoint- > restore-> exitArchiveRecovery: EndOfLog is the last location of the current playback log, and EndOfLogTLI is the timeline of the current playback log upon exit. When there is a standby.signal file in the data directory, that is, the machine is standby, ArchiveRecoveryRequested is TRUE.

The process of calling the exitArchiveRecovery function is as follows: first, calculate the log number of the segment file through endOfLog, that is, the last location of playback: endLogSegNo= (endOfLog-1) / (wal_segment_size); startLogSegNo= endOfLog / (wal_segment_size). If endLogSegNo is equal to startLogSegNo, the playback location is in the middle of the file. When XLogFileCopy is called, a new file is generated and the contents of the previous XLOG file are copied to the new file; the segment file number is the same, and the timeline is added by 1. If endLogSegNo is not equal to startLogSegNo, that is, the playback location is just at the end of the file size, or it happens to be the log of SWITCH, then call the XLogFileInit function to initialize the file:

XlogFileCopy function call: call the XLogFilePath function to get the source XLOG file name, OpenTransientFile to open the file, create and open a temporary XLOG file pg_wal/xlogtemp.pid, pid is the ID number of the startup process. Sizeof (buffer) is a page size of 8192 bytes, read one page of data from the source file and write it to the xlogtemp file at a time. If the last page of data is less than 8192 bytes, read as much as you can and write to the file. When the copy of the file is complete, perform a sync. Finally, InstallXLogFileSegment is called to rename the file.

InstallXLogFileSegment function: when XlogFileCopy is called, find_free is false, renaming the file to the file name of timeline plus 1 When XLogFileInit is called as TRUE, add 1 to the segment file number (note that it is not 1 here, because it happens to be at the end of the file, and the next segment file number is obtained, but it appears to be 1). The file renamed to timeline plus 1 will stat the file first, and the return value of the process is 2, that is, the file does not exist, so segno will not be added 1, directly skipping the steps in the virtual box. Enter the renaming process durable_like_or_rename.

Call the XLogFileInit function: first get the file name of the new file, that is, the timeline plus 1, and the segment file name is the original file name. In the process of calling the exitArchiveRecovery function, use_existent is TRUE, so the file will be opened in the view. Of course, the opening failed because the file does not exist. Then create and open a temporary file xlogtemp.pid, clear all the files, and finally sync. The InstallXLogFileSegment function is then called to rename. Finally, open the new file for later use.

2. GDB tracking-the lsn location is in the middle of the xlog file

1. Enter the first breakpoint, that is, the entry of the function

Breakpoint 1, exitArchiveRecovery (endTLI=1, endOfLog=23053968) at xlog.c:54755475 InArchiveRecovery = false; (gdb) bt#0 exitArchiveRecovery (endTLI=1, endOfLog=23053968) at xlog.c:5475#1 0x0815fc4b in StartupXLOG () at xlog.c:7460#2 0x083dbc56 in StartupProcessMain () at startup.c:207#3 0x08173cbf in AuxiliaryProcessMain (argc=2, argv=0xbfa8a2f4) at bootstrap.c:451#4 0x083dab93 in StartChildProcess (type=StartupProcess) at postmaster.c:5386#5 0x083d5d86 in PostmasterMain (argc=1, argv=0xa22e7e8) at postmaster.c:1369#6 0x0831c76c in main (argc=1, argv=0xa22e7e8) at main.c:228

2. Then the segment file number is calculated, and the two values are equal, that is, when promote is executed, the last location of lsn is in the middle of the file.

5507 if (endLogSegNo = = startLogSegNo) (gdb) 5517 XLogSegmentOffset (endOfLog, wal_segment_size)); (gdb) p endLogSegNo$1 = 1 (gdb) p startLogSegNo$2 = 1

3. Lsn is in the middle of the file. Call XlogFileCopy. Upto is the offset of lsn in the last file.

XLogFileCopy (destsegno=1, srcTLI=1, srcsegno=1, upto=6276752) at xlog.c:34053405 XLogFilePath (path, srcTLI, srcsegno, wal_segment_size); (gdb) p 23053968% (1601024024) $3 = 6276752

4. Open the original file 00000001 0000000000000001 and the temporary file xlogtemp.29683

3406 srcfd = OpenTransientFile (path, O_RDONLY | PG_BINARY) (gdb) p path$4 = "pg_wal/00000001",'0', "1",'\ 000', "\ b\ 221\ 250\ 277gmE\ b\ 004\ 005\ 000\ 000\ 001\ 030\ 221\ 250\ 277\ bnE\ b\ 004\ 000\ 000\ 001X\ 221\ 277 $\ 207E\ b\ 004\ 005\ 000\ 000\ 001", '000' " \ 004\ 246X\ 221\ 250\ 277\ 000\ 000\ 000\ 000\ 000\ 000\ 270\ 004\ 000\ 000\ 000\ 000\ 000\ n\ 000\ 000\ 000\ 270\ 004\ 246\ 221\ 250\ 277dd\ 025\ 000\ 005\ 352\ 320a\ b\ 341\ 000\ 016b\ 000\ 000 "(gdb) p tmppath$5 =" pg_wal/xlogtemp.29683 " '\ 000' (gdb) n3420 fd = OpenTransientFile (tmppath, O_RDWR | O_CREAT | O_EXCL | PG_BINARY)

5. Copy in a loop, 8192 bytes per page at a time

3429 for (nbytes = 0; nbytes)

< wal_segment_size; nbytes += sizeof(buffer))(gdb) p sizeof(buffer)$6 = 8192(gdb) n3433 nread = upto - nbytes;(gdb) 3439 if (nread < sizeof(buffer))(gdb) p nread$7 = 6276752(gdb) p 6276752/8192$8 = 766 6 、 InstallXLogFileSegment 函数重命名， path 为 000000020000000000000001 Breakpoint 2, InstallXLogFileSegment (segno=0xbfa86968, tmppath=0xbfa88974 "pg_wal/xlogtemp.29683", find_free=false, max_segno=0, use_lock=false) at xlog.c:35453545 XLogFilePath(path, ThisTimeLineID, *segno, wal_segment_size); (gdb) p path$10 = "pg_wal/00000002", '0' , "1\000 \212\022\251\322\"[\000\000\000\000\000\277 `\b\204\211\250\277\347e\250\277\005\000\000\000\003\000\000\000 NҬ\003\000\000\000\002\000\000\000\005\000\000\000;\000\000\000\000\000\000\000\030f\250\277\206\032`\b\000\000\000\000\350h\250\277\350h\250\277\350h\250\277\240.{\262d\000\000\000\363s\000\000\000\000\000\000\240\020[\264\003\000\000\000\000\000\000\000\003\000\000\000\240\362:\266\003\000\000\000\220\251t\267\003\000\000\000\240\324\032\270\003\000\000\000\220\213T\271\003\000\000\000 \361\003\272\003\000\000\000\020\250=\273\003\000\000\000 \323\343\273\322\"[29683\000\000\000\000\n", '\000' , "\001\000\000\000\373\333y\b\005\000\000\000\000\000\000\000\001\000\000\000\270\240\250\277\310h\250\277\225\017`\b\363s\000\000\000\000\000\000d", '\000' "\350"...(gdb) n 7 、将临时文件重命名为 000000020000000000000001 3579 if (durable_link_or_rename(tmppath, path, LOG) != 0)(gdb) p path$11 = "pg_wal/00000002", '0' , "1\000 \212\022\251\322\"[\000\000\000\000\000\277 `\b\204\211\250\277\347e\250\277\005\000\000\000\003\000\000\000 NҬ\003\000\000\000\002\000\000\000\005\000\000\000;\000\000\000\000\000\000\000\030f\250\277\206\032`\b\000\000\000\000\350h\250\277\350h\250\277\350h\250\277\240.{\262d\000\000\000\363s\000\000\000\000\000\000\240\020[\264\003\000\000\000\000\000\000\000\003\000\000\000\240\362:\266\003\000\000\000\220\251t\267\003\000\000\000\240\324\032\270\003\000\000\000\220\213T\271\003\000\000\000 \361\003\272\003\000\000\000\020\250=\273\003\000\000\000 \323\343\273\322\"[29683\000\000\000\000\n", '\000' , "\001\000\000\000\373\333y\b\005\000\000\000\000\000\000\000\001\000\000\000\270\240\250\277\310h\250\277\225\017`\b\363s\000\000\000\000\000\000d", '\000' "\350"...(gdb) n三、GDB跟踪-lsn位置在xlog文件尾或最后一个为SWITCH 1 、 lsn 位于文件尾，调用函数 XLogFileInit Breakpoint 3, exitArchiveRecovery (endTLI=3, endOfLog=50331648) at xlog.c:54755475 InArchiveRecovery = false;(gdb) n5480 UpdateMinRecoveryPoint(InvalidXLogRecPtr, true);(gdb) 5486 if (readFile >

= 0) (gdb) 5488 close (readFile); (gdb) 5489 readFile =-1; (gdb) 5498 XLByteToPrevSeg (endOfLog, endLogSegNo, wal_segment_size); (gdb) 5499 XLByteToSeg (endOfLog, startLogSegNo, wal_segment_size); (gdb) 5507 if (endLogSegNo = = startLogSegNo) (gdb) 5525 bool use_existent = true; (gdb) 5528 fd = XLogFileInit (startLogSegNo, & use_existent, true); (gdb)

2. Path is 000000040000000000000003. The file does not exist.

Breakpoint 2, XLogFileInit (logsegno=3, use_existent=0xbfbd366f, use_lock=true) at xlog.c:32163216 XLogFilePath (path, ThisTimeLineID, logsegno, wal_segment_size) (gdb) n3221 if (* use_existent) (gdb) p path$1 = "pg_wal/00000004",'0' "3\ 000\ 000\ 000\ 000\ 000\ 346! `\ bX6\ 275\ 277\ 060\ 000\ 000\ 000\ a\ 000\ 000\ 030\ 333\ 177\ t\ 364\ 257`\ 000\ 270\ 331\ 000\ 000\ 000\ 000\ 000\ 000\ 000\ 000h3\ 277:`\ b4\ 000\ 000\ 330\ 065\ 275\ 277\ 277\ 000\ 000\ 000h3\ 277\ 277L\ 274M\ 000\ 000\ 000\ 000\ 000?\ 000\ 000\ 000\ a\ 000\ 000\ b3\ 277\ 206\ 000\ 000\ 330\ 065\ 275\ 277\ 330\ 065\ 275\ 277\ 373\ 210\ tX\ 025w\ b\ 004\ 000\ 000\ 000\ 000\ 000 177\ 323a\ bX\ 000\ 000\ 000\ 000\ 000\ 000\ 000\ b " '000', "\ b\ 000\ 000\ 000\ 000\ 000\ 000\ 000\ 000\ 060", "recovering 00000\ 322\" [\ 000\ 000\ 270\ 004\ 246\ 277`\ b`6\ 277'\ 253a\ b ",'\ 000'... (gdb) n3223 fd = BasicOpenFile (path, O_RDWR | PG_BINARY | get_sync_bit (sync_method)) (gdb) 3224 if (fd < 0) (gdb) 3226 if (errno! = ENOENT) (gdb) p fd$2 =-1 (gdb) p errno$3 = 2

3. Create and open a temporary file, clear the zbuffer.data, and then clear the file page by page.

3243 snprintf (tmppath, MAXPGPATH, XLOGDIR "/ xlogtemp.%d", (int) getpid ()); (gdb) 3245 unlink (tmppath); (gdb) 3248 fd = BasicOpenFile (tmppath, O_RDWR | O_CREAT | O_EXCL | PG_BINARY); (gdb) 3249 if (fd < 0) (gdb) 3254 memset (zbuffer.data, 0, XLOG_BLCKSZ); (gdb) p fd$4 = 33269 for (nbytes = 0; nbytes < wal_segment_size; nbytes + = XLOG_BLCKSZ) (gdb) 3271 errno = 0 Gdb) 3272 if (write (fd, zbuffer.data, XLOG_BLCKSZ)! = XLOG_BLCKSZ)

4. Enter the InstallXLogFileSegment function, 000000040000000000000003 file stat failed, call durable_link_or_rename to rename.

InstallXLogFileSegment (segno=0xbfbd0de0, tmppath=0xbfbd2de8 "pg_wal/xlogtemp.31765", find_free=true, max_segno=45, use_lock=true) at xlog.c:35503550 if (use_lock) (gdb) n3551 LWLockAcquire (ControlFileLock, LW_EXCLUSIVE) (gdb) 3553 if (! find_free) (gdb) 3561 while (stat (path, & stat_buf) = = 0) (gdb) p path$5 = "pg_wal/00000004",'0' "3\ 000\ 212\ 022\ 251\ 322\" [\ 000\ 000\ 000\ 000\ 000\ 277 `\ b\ 370 -\ 275\ 277W\ n\ 277\ 005\ 000\ 000\ 003\ 003\ 000\ 000\ 003\ 000\ 000\ 000\ 000\ 000\ 000\ 000 \ 000\ 000\ 000\ 000\ 000\ 210\ n\ 275\ 277\ 206\ 032`\ b\ 000\ 000\ 000\ 000X\ r\ 275\ 277X\ r\ 275\ 277. {\ 262d\ 000\ 000\ 000\ 020 [\ 264\ 003\ 000\ 000\ 000\ 000\ 000\ 000\ 000\ 000\ 000\ 000\ 000\ 240\ 362:\ 266\ 003 \ 000\ 000\ 220\ 251t\ 267\ 003\ 000\ 000\ 240\ 324\ 032\ 270\ 003\ 000\ 000\ 220\ 213T\ 271\ 003\ 000\ 000\ 361\ 003\ 272\ 003\ 000\ 000\ 020\ 250 =\ 273\ 003\ 000\ 000\ 323\ 343\ 322\ "[31765\ 000\ 000\ 000\ n" '\ 000', "\ 001\ 000\ 000\ 000\ 000\ 373\ 333y\ b\ 005\ 000\ 000\ 000\ 000\ 000\ 000\ 003\ 000\ 000 (E\ 275\ 277\ 070\ r\ 275\ 225\ 017`\ b\ 025 | (gdb) n3579 if (durable_link_or_rename (tmppath, path, LOG)! = 0) (gdb) p errno$6 = 2

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.