Shulou(Shulou.com)05/31 Report--

This article mainly explains "what is the processing process of GetSnapshotData in PostgreSQL". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "what is the processing process of GetSnapshotData in PostgreSQL".

I. data structure

Global / static variables

/ * Currently registered Snapshots. Ordered in a heap by xmin, so that we can * quickly find the one with lowest xmin, to advance our MyPgXact- > xmin. * currently registered snapshots. * sort by xmin heap so that we can quickly find the smallest xmin, so we can set MyPgXact- > xmin. * / static int xmin_cmp (const pairingheap_node * a, const pairingheap_node * b, void * arg); static pairingheap RegisteredSnapshots = {& xmin_cmp, NULL, NULL}; / * first GetTransactionSnapshot call in a transaction? * / bool FirstSnapshotSet = false;/* * Remember the serializable transaction snapshot, if any. We cannot trust * FirstSnapshotSet in combination with IsolationUsesXactSnapshot (), because * GUC may be reset before us, changing the value of IsolationUsesXactSnapshot. * take a snapshot of the serializable transaction if it exists. * We cannot trust FirstSnapshotSet used in conjunction with IsolationUsesXactSnapshot (), * because GUC may reset before us to change the value of IsolationUsesXactSnapshot. * / static Snapshot FirstXactSnapshot = NULL;/* * CurrentSnapshot points to the only snapshot taken in transaction-snapshot * mode, and to the latest one taken in a read-committed transaction. * SecondarySnapshot is a snapshot that's always up-to-date as of the current * instant, even in transaction-snapshot mode. It should only be used for * special-purpose code (say, RI checking.) CatalogSnapshot points to an * MVCC snapshot intended to be used for catalog scans; we must invalidate it * whenever a system catalog change occurs. * CurrentSnapshot points to the only snapshot taken in transaction-snapshot mode / the latest snapshot taken in a read-committed transaction. * SecondarySnapshot is always the latest snapshot, even in transaction-snapshot mode. It should only be used for special purpose codes (for example, RI checks). * CatalogSnapshot points to the MVCC snapshot intended for catalog scanning; * whenever a system catalog change occurs, we must invalidate it immediately. * * These SnapshotData structs are static to simplify memory allocation * (see the hack in GetSnapshotData to avoid repeated malloc/free) * these SnapshotData structures are static to simplify memory allocation. * (you can look back at how the GetSnapshotData function avoids duplicate malloc/free) * / static SnapshotData CurrentSnapshotData = {HeapTupleSatisfiesMVCC}; static SnapshotData SecondarySnapshotData = {HeapTupleSatisfiesMVCC}; SnapshotData CatalogSnapshotData = {HeapTupleSatisfiesMVCC}; / * Pointers to valid snapshots * / / points to a valid snapshot static Snapshot CurrentSnapshot = NULL;static Snapshot SecondarySnapshot = NULL;static Snapshot CatalogSnapshot = NULL;static Snapshot HistoricSnapshot = NULL;/* * These are updated by GetSnapshotData. We initialize them this way * for the convenience of TransactionIdIsInProgress: even in bootstrap * mode, we don't want it to say that BootstrapTransactionId is in progress. * these variables are updated through the function GetSnapshotData. * to facilitate TransactionIdIsInProgress, initialize them in this way: * even in boot mode, we do not want to indicate that BootstrapTransactionId is in progress. * * RecentGlobalXmin and RecentGlobalDataXmin are initialized to * InvalidTransactionId, to ensure that no one tries to use a stale * value. Readers should ensure that it has been set to something else * before using it. * RecentGlobalXmin and RecentGlobalDataXmin are initialized to InvalidTransactionId, * to ensure that no one is trying to use outdated values. * before using it, the reading process should ensure that it has been set to a different value. * / TransactionId TransactionXmin = FirstNormalTransactionId;TransactionId RecentXmin = FirstNormalTransactionId;TransactionId RecentGlobalXmin = InvalidTransactionId;TransactionId RecentGlobalDataXmin = InvalidTransactionId;/* (table, ctid) = > (cmin, cmax) mapping during timetravel * / static HTAB * tuplecid_data = NULL

MyPgXact

Current transaction information.

/ * * Flags for PGXACT- > vacuumFlags * PGXACT- > vacuumFlags tag * * Note: If you modify these flags, you need to modify PROCARRAY_XXX flags * in src/include/storage/procarray.h. * Note: if you modify these tags, you need to update the PROCARRAY_XXX tags * * PROC_RESERVED may later be assigned for use in vacuumFlags, but its value is * used for PROCARRAY_SLOTS_XMIN in procarray.h, so GetOldestXmin won't be able * to match and ignore processes with this flag set in src/include/storage/procarray.h. * PROC_RESERVED may be assigned to vacuumFlags later, * but it is used to identify PROCARRAY_SLOTS_XMIN in procarray.h, * so GetOldestXmin cannot match and ignore processes that use this tag. * / / is auto vacuum worker?#define PROC_IS_AUTOVACUUM 0x01 / * is it an autovac worker? * / / running lazy vacuum#define PROC_IN_VACUUM 0x02 / * currently running lazy vacuum * / / running analyze#define PROC_IN_ANALYZE 0x04 / * currently running analyze * / can only set # define PROC_VACUUM_FOR_WRAPAROUND 0x08 / * set by autovac via auto vacuum Only * / / logical decoding is being performed outside the transaction # define PROC_IN_LOGICAL_DECODING 0x10 / * currently doing logical * decoding outside xact * / / keep the MASK used for procarray#define PROC_RESERVED 0x20 / * reserved for procarray * / / * flags reset at EOXact * / to reset the tag during EOXact # define PROC_VACUUM_STATE_MASK\ (PROC_IN_VACUUM | PROC_IN_ANALYZE | PROC_VACUUM_FOR_WRAPAROUND) / * * Prior to PostgreSQL 9.2 The fields below were stored as part of the * PGPROC. However, benchmarking revealed that packing these particular * members into a separate array as tightly as possible sped up GetSnapshotData * considerably on systems with many CPU cores, by reducing the number of * cache lines needing to be fetched. Thus, think very carefully before adding * anything else here. * / typedef struct PGXACT {/ / current top-level transaction ID (non-sub-transaction) / / for optimization purposes, read-only transactions do not assign transaction numbers (xid = 0) TransactionId xid; / * id of top-level transaction currently being * executed by this proc, if running and XID * is assigned Else InvalidTransactionId * / / when starting the transaction, the minimum transaction number currently executing XID, but not including LAZY VACUUM / / vacuum, cannot clear and delete the tuple TransactionId xmin of transaction number xid > = xmin. / * minimal running XID as it was when we were * starting our xact, excluding LAZY VACUUM: * vacuum must not remove tuples deleted by * xid > = xmin! * / vacuum related tags uint8 vacuumFlags; / * vacuum-related flags, see above * / bool overflowed Bool delayChkpt; / * true if this proc delays checkpoint start; * previously called InCommit * / uint8 nxids;} PGXACT;extern PGDLLIMPORT struct PGXACT * MyPgXact

Snapshot

SnapshotData structure pointer, the information that can be expressed by SnapshotData structure includes all possible snapshots.

There are several different types of snapshots:

1. Regular MVCC Snapshot

two。 MVCC snapshot during restore (in Hot-Standby mode)

3. Historical MVCC snapshots used during logical decoding

4. A snapshot passed as an argument to the HeapTupleSatisfiesDirty () function

5. A snapshot passed as an argument to the HeapTupleSatisfiesNonVacuumable () function

6. Snapshots for SatisfiesAny, Toast, and Self without member access

/ / SnapshotData structure pointer typedef struct SnapshotData * invalid snapshot # define InvalidSnapshot ((Snapshot) NULL) / * * We use SnapshotData structures to represent both "regular" (MVCC) * snapshots and "special" snapshots that have non-MVCC semantics. * The specific semantics of a snapshot are encoded by the "satisfies" * function. * We use SnapshotData structures to represent "regular" (MVCC) snapshots and "special" snapshots with non-MVCC semantics. * / / Test function typedef bool (* SnapshotSatisfiesFunc) (HeapTuple htup, Snapshot snapshot, Buffer buffer) / / the common ones are: / / HeapTupleSatisfiesMVCC: determine whether the tuple is valid for a snapshot version / / HeapTupleSatisfiesUpdate: determine whether the tuple can be updated (update the same tuple at the same time) / / HeapTupleSatisfiesDirty: determine whether the current tuple has dirty data / / HeapTupleSatisfiesSelf: determine whether the tuple is valid for its own information / / HeapTupleSatisfiesToast: determine whether the TOAST table / / HeapTupleSatisfiesVacuum: determine whether the tuple can be deleted by VACUUM / / HeapTupleSatisfiesAny: all tuples are visible / / HeapTupleSatisfiesHistoricMVCC: for CATALOG table / * * Struct representing all kind of possible snapshots. * the information that can be expressed by this structure includes all possible snapshots. * * There are several different kinds of snapshots: * * Normal MVCC snapshots * * MVCC snapshots taken during recovery (in Hot-Standby mode) * * Historic MVCC snapshots used during logical decoding * * snapshots passed to HeapTupleSatisfiesDirty () * * snapshots passed to HeapTupleSatisfiesNonVacuumable () * * snapshots used for SatisfiesAny, Toast, Self where no members are * accessed. * there are several different types of snapshots: * * regular MVCC snapshots * * MVCC snapshots during restore (in Hot-Standby mode) * * Historical MVCC snapshots used during logical decoding * * snapshots passed as arguments to the HeapTupleSatisfiesDirty () function * * Snapshots passed as arguments to the HeapTupleSatisfiesNonVacuumable () function * * for SatisfiesAny without member access, Snapshots of Toast and Self * * TODO: It's probably a good idea to split this struct using a NodeTag * similar to how parser and executor nodes are handled With one type for * each different kind of snapshot to avoid overloading the meaning of * individual fields. * TODO: using parser/executor nodes-like processing, using NodeTag to split structures is a good practice, * using OO (object-oriented inheritance) method. * / typedef struct SnapshotData {/ / the function SnapshotSatisfiesFunc satisfies; / * tuple test function * / / * The remaining fields are used only for MVCC snapshots, and are normally * just zeroes in special snapshots to test whether tuple is visible. (But xmin and xmax are used * specially by HeapTupleSatisfiesDirty, and xmin is used specially by * HeapTupleSatisfiesNonVacuumable.) * the remaining fields are for MVCC snapshots only, usually 0 in special snapshots. * (xmin and xmax can be used for HeapTupleSatisfiesDirty,xmin and HeapTupleSatisfiesNonVacuumable) * * An MVCC snapshot can never see the effects of XIDs > = xmax. It can see * the effects of all older XIDs except those listed in the snapshot. Xmin * is stored as an optimization to avoid needing to search the XID arrays * for most tuples. Transactions of * XIDs > = xmax are not visible to the snapshot (have no effect). * XIDs that is less than xmax but not in the snapshot list is visible to this snapshot. * recording xmin is for optimization purposes, avoiding searching XID arrays for most tuples. * / XID ∈ [2minmin) is visible TransactionId xmin; / * all XID

< xmin are visible to me */ //XID ∈ [xmax,∞)是不可见的 TransactionId xmax; /* all XID >

< 0) { /* * If not using "snapshot too old" feature, fill related fields with * dummy values that don't require any locking. * 如启用"snapshot too old"特性,使用虚拟值填充相关的字段,这里不需要锁. */ snapshot->

Lsn = InvalidXLogRecPtr; snapshot- > whenTaken = 0;} else {/ * Capture the current time and WAL stream location in case this * snapshot becomes old enough to need to fall back on the special * "old snapshot" logic. * capture the current time and Wall stream location in case the snapshot becomes old enough to use special "old snapshot" logic. * / snapshot- > lsn = GetXLogInsertRecPtr (); snapshot- > whenTaken = GetSnapshotCurrentTimestamp (); MaintainOldSnapshotTimeMapping (snapshot- > whenTaken, xmin);} / / return snapshot return snapshot;} 3. Tracking and analysis

Execute a simple query to trigger the snapshot logic.

16:35:08 (xdb@ [local]: 5432) testdb=# begin;BEGIN16:35:13 (xdb@ [local]: 5432) testdb=#* select 1

Start gdb and set breakpoint

(gdb) b GetSnapshotDataBreakpoint 1 at 0x89aef3: file procarray.c, line 1519. (gdb) cContinuing.Breakpoint 1, GetSnapshotData (snapshot=0xf9be60) at procarray.c:15191519 ProcArrayStruct * arrayP = procArray; (gdb)

Input parameter snapshot, which is essentially a global variable CurrentSnapshotData

(gdb) p * snapshot$1 = {satisfies = 0xa9310d, xmin = 2354, xmax = 2358, xip = 0x24c7e40, xcnt = 1, subxip = 0x251dfa0, subxcnt = 0, suboverflowed = false, takenDuringRecovery = false, copied = false, curcid = 0, speculativeToken = 0, active_count = 0, regd_count = 0, ph_node = {first_child = 0x0, next_sibling = 0x0, prev_or_parent = 0x0}, whenTaken = 0, lsn = 0}

View the information in shared memory (ShmemVariableCache).

NextXID = 2358, the next transaction to be assigned ID = 2358.

(gdb) p * ShmemVariableCache$2 = {nextOid = 42605, oidCount = 8183, nextXid = 2358, oldestXid = 561, xidVacLimit = 200000561, xidWarnLimit = 2136484208, xidStopLimit = 2146484208, xidWrapLimit = 2147484208, oldestXidDB = 16400, oldestCommitTsXid = 0, newestCommitTsXid = 0, latestCompletedXid = 2357, oldestClogXid = 561} (gdb)

Get the global process array procArray, assign-> arrayP.

Initialize the relevant variables.

(gdb) n1524 int count = 0; (gdb) n1525 int subcount = 0; (gdb) 1526 bool suboverflowed = false; (gdb) 1527 volatile TransactionId replication_slot_xmin = InvalidTransactionId; (gdb) 1528 volatile TransactionId replication_slot_catalog_xmin = InvalidTransactionId; (gdb) 1530 Assert (snapshot! = NULL); (gdb) 1543 if (snapshot- > xip = = NULL) (gdb)

View process array information and allPgXact [] array number (arrayP- > pgprocnos array).

AllPgXact definition: static PGXACT * allPgXact

(gdb) p * arrayP$3 = {numProcs = 5, maxProcs = 112, maxKnownAssignedXids = 7280, numKnownAssignedXids = 0, tailKnownAssignedXids = 0, headKnownAssignedXids = 0, known_assigned_xids_lck = 0'\ 000mm, lastOverflowedXid = 0, replication_slot_xmin = 0, replication_slot_catalog_xmin = 0 Pgprocnos = 0x7f8765d9a3a8} (gdb) p arrayP- > pgprocnos [0] $4 = 97 (gdb) p arrayP- > pgprocnos [1] $5 = 98 (gdb) p arrayP- > pgprocnos [2] $6 = 99 (gdb) p arrayP- > pgprocnos [3] $7 = 103 (gdb) p arrayP- > pgprocnos [4] $9 = 11111

Lock, acquire / modify relevant information

(gdb) 1568 LWLockAcquire (ProcArrayLock, LW_SHARED)

Calculate xmax

(gdb) n1571 xmax = ShmemVariableCache- > latestCompletedXid; (gdb) 1572 Assert (TransactionIdIsNormal (xmax)); (gdb) p xmax$10 = 2357 (gdb) n1573 TransactionIdAdvance (xmax); (gdb) 1576 globalxmin = xmin = xmax; (gdb) 1578 snapshot- > takenDuringRecovery = RecoveryInProgress (); (gdb) p xmax$11 = 2358

Judge whether it is in the recovery state, not the current recovery state, and enter the corresponding processing logic

(gdb) n1580 if (! snapshot- > takenDuringRecovery) (gdb) p snapshot- > takenDuringRecovery$13 = false (gdb) n1582 int * pgprocnos = arrayP- > pgprocnos; (gdb)

Get the number of processes and the PGXACT index array, ready to traverse

(gdb) n1590 numProcs = arrayP- > numProcs; (gdb) 1591 for (index = 0; index

< numProcs; index++)(gdb) (gdb) p *pgprocnos$14 = 97(gdb) p numProcs$15 = 5(gdb) 获取pgxact信息 (gdb) n1593 int pgprocno = pgprocnos[index];(gdb) 1594 volatile PGXACT *pgxact = &allPgXact[pgprocno];(gdb) 1601 if (pgxact->

VacuumFlags & PROC_IN_LOGICAL_DECODING) (gdb) 1605 if (pgxact- > vacuumFlags & PROC_IN_VACUUM) (gdb) 1609 xid = pgxact- > xmin; / * fetch just once * / (gdb) p * pgxact$16 = {xid = 0, xmin = 0, vacuumFlags = 0'\ 000mm, overflowed = false, delayChkpt = false, nxids = 0'\ 000'} (gdb)

Not a normal xid. Next pgxact.

(gdb) n1610 if (TransactionIdIsNormal (xid) & & (gdb) 1615 xid = pgxact- > xid; (gdb) 1623 if (! TransactionIdIsNormal (xid) (gdb) p xid$17 = 0 (gdb) n1625 continue; (gdb))

Next xid = 2355, normal transaction ID

(gdb) 1591 for (index = 0; index)

< numProcs; index++)(gdb) 1593 int pgprocno = pgprocnos[index];(gdb) 1594 volatile PGXACT *pgxact = &allPgXact[pgprocno];(gdb) 1601 if (pgxact->

VacuumFlags & PROC_IN_LOGICAL_DECODING) (gdb) p * pgxact$18 = {xid = 2355, xmin = 0, vacuumFlags = 0'\ 000mm, overflowed = false, delayChkpt = false, nxids = 0'\ 000'} (gdb)

Carry on the processing

(gdb) n1605 if (pgxact- > vacuumFlags & PROC_IN_VACUUM) (gdb) 1609 xid = pgxact- > xmin; / * fetch just once * / (gdb) 1610 if (TransactionIdIsNormal (xid) & & (gdb) 1615 xid = pgxact- > xid (gdb) 1623 if (! TransactionIdIsNormal (xid) (gdb) 1624 | |! NormalTransactionIdPrecedes (xid, xmax)) (gdb) 1631 if (NormalTransactionIdPrecedes (xid, xmin)) (gdb) p xid$19 = 2355 (gdb) p xmin$20 = 2358 (gdb) n1632 xmin = xid; (gdb) 1633 if (pgxact = = MyPgXact) (gdb)

This is the same xact that handles the next xact.

(gdb) 1633 if (pgxact = = MyPgXact) (gdb) p pgxact$21 = (volatile PGXACT *) 0x7f8765d9a218 (gdb) p MyPgXact$22 = (struct PGXACT *) 0x7f8765d9a218 (gdb) n1634 continue; (gdb)

Next is 2354.

(gdb) p * pgxact$23 = {xid = 2354, xmin = 0, vacuumFlags = 0'\ 000mm, overflowed = false, delayChkpt = false, nxids = 0'\ 000'} (gdb)

Xmin adjusted to 2354

1631 if (NormalTransactionIdPrecedes (xid, xmin)) (gdb) 1632 xmin = xid; (gdb) 1633 if (pgxact = = MyPgXact) (gdb) p xmin$24 = 2354 (gdb)

Write to xip_list

1637 snapshot- > xip [count++] = xid; (gdb) 1654 if (! suboverflowed) (gdb) (gdb) p count$25 = 1

Continue the loop to complete the traversal of 5 pgxact

1591 for (index = 0; index)

< numProcs; index++)(gdb) 1715 replication_slot_xmin = procArray->

Replication_slot_xmin; (gdb)

No replication information

(gdb) 1715 replication_slot_xmin = procArray- > replication_slot_xmin; (gdb) p procArray- > replication_slot_xmin$28 = 0 (gdb) n1716 replication_slot_catalog_xmin = procArray- > replication_slot_catalog_xmin; (gdb) 1718 if (! TransactionIdIsValid (MyPgXact- > xmin))

Adjust the transaction information for this process

(gdb) n1719 MyPgXact- > xmin = TransactionXmin = xmin; (gdb) p MyPgXact- > xmin$29 = 0 (gdb) n

Release lock

1721 LWLockRelease (ProcArrayLock); (gdb) 1728 if (TransactionIdPrecedes (xmin, globalxmin)) (gdb)

Adjust the global xmin

(gdb) p xmin$30 = 2354 (gdb) p globalxmin$31 = 2358 (gdb) n1729 globalxmin = xmin; (gdb)

Update additional information

(gdb) 1732 RecentGlobalXmin = globalxmin-vacuum_defer_cleanup_age; (gdb) p RecentGlobalXmin$32 = 2354 (gdb) p vacuum_defer_cleanup_age$33 = 0 (gdb) n1733 if (! TransactionIdIsNormal (RecentGlobalXmin)) (gdb) 1737 if (TransactionIdIsValid (replication_slot_xmin) & & (gdb) 1742 RecentGlobalDataXmin = RecentGlobalXmin; (gdb) p RecentGlobalXmin$34 = 2354 (gdb) n1748 if (TransactionIdIsNormal (replication_slot_catalog_xmin) & (gdb))

Populate snapshot domain field information

(gdb) 1752 RecentXmin = xmin; (gdb) 1754 snapshot- > xmin = xmin; (gdb) 1755 snapshot- > xmax = xmax; (gdb) 1756 snapshot- > xcnt = count; (gdb) 1757 snapshot- > subxcnt = subcount; (gdb) 1758 snapshot- > suboverflowed = suboverflowed; (gdb) 1760 snapshot- > curcid = GetCurrentCommandId (false); (gdb) 1766 snapshot- > active_count = 0; (gdb) 1767 snapshot- > regd_count = 0 (gdb) 1768 snapshot- > copied = false; (gdb) 1770 if (old_snapshot_threshold

< 0)(gdb) 1776 snapshot->

Lsn = InvalidXLogRecPtr; (gdb) 1777 snapshot- > whenTaken = 0; (gdb) 1791 return snapshot; (gdb)

Return to snapshot

(gdb) p snapshot$35 = (Snapshot) 0xf9be60 (gdb) p * snapshot$36 = {satisfies = 0xa9310d, xmin = 2354, xmax = 2358, xip = 0x24c7e40, xcnt = 1, subxip = 0x251dfa0, subxcnt = 0, suboverflowed = false, takenDuringRecovery = false, copied = false, curcid = 0, speculativeToken = 0, active_count = 0, regd_count = 0, ph_node = {first_child = 0x0, next_sibling = 0x0, prev_or_parent = 0x0}, whenTaken = 0, lsn = 0} (gdb)

Note: the snapshot- > satisfies function has been set to HeapTupleSatisfiesMVCC when initializing the global variable.

Thank you for reading, the above is the content of "what is the processing process of GetSnapshotData in PostgreSQL". After the study of this article, I believe you have a deeper understanding of what is the processing process of GetSnapshotData in PostgreSQL, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.