PostgreSQL Source Code interpretation (116)-MVCC#1 (take Snapshot # 1)

This section describes the main implementation logic for PostgreSQL to take a snapshot of a transaction, and the corresponding implementation function is GetTransactionSnapshot.

I. data structure

Global / static variables

/ * Currently registered Snapshots. Ordered in a heap by xmin, so that we can * quickly find the one with lowest xmin, to advance our MyPgXact- > xmin. * currently registered snapshots. * sort by xmin heap so that we can quickly find the smallest xmin, so we can set MyPgXact- > xmin. * / static int xmin_cmp (const pairingheap_node * a, const pairingheap_node * b, void * arg); static pairingheap RegisteredSnapshots = {& xmin_cmp, NULL, NULL}; / * first GetTransactionSnapshot call in a transaction? * / bool FirstSnapshotSet = false;/* * Remember the serializable transaction snapshot, if any. We cannot trust * FirstSnapshotSet in combination with IsolationUsesXactSnapshot (), because * GUC may be reset before us, changing the value of IsolationUsesXactSnapshot. * take a snapshot of the serializable transaction if it exists. * We cannot trust FirstSnapshotSet used in conjunction with IsolationUsesXactSnapshot (), * because GUC may reset before us to change the value of IsolationUsesXactSnapshot. * / static Snapshot FirstXactSnapshot = NULL;/* * CurrentSnapshot points to the only snapshot taken in transaction-snapshot * mode, and to the latest one taken in a read-committed transaction. * SecondarySnapshot is a snapshot that's always up-to-date as of the current * instant, even in transaction-snapshot mode. It should only be used for * special-purpose code (say, RI checking.) CatalogSnapshot points to an * MVCC snapshot intended to be used for catalog scans; we must invalidate it * whenever a system catalog change occurs. * CurrentSnapshot points to the only snapshot taken in transaction-snapshot mode / the latest snapshot taken in a read-committed transaction. * SecondarySnapshot is always the latest snapshot, even in transaction-snapshot mode. It should only be used for special purpose codes (for example, RI checks). * CatalogSnapshot points to the MVCC snapshot intended for catalog scanning; * whenever a system catalog change occurs, we must invalidate it immediately. * * These SnapshotData structs are static to simplify memory allocation * (see the hack in GetSnapshotData to avoid repeated malloc/free) * these SnapshotData structures are static to simplify memory allocation. * (you can look back at how the GetSnapshotData function avoids duplicate malloc/free) * / static SnapshotData CurrentSnapshotData = {HeapTupleSatisfiesMVCC}; static SnapshotData SecondarySnapshotData = {HeapTupleSatisfiesMVCC}; SnapshotData CatalogSnapshotData = {HeapTupleSatisfiesMVCC}; / * Pointers to valid snapshots * / / points to a valid snapshot static Snapshot CurrentSnapshot = NULL;static Snapshot SecondarySnapshot = NULL;static Snapshot CatalogSnapshot = NULL;static Snapshot HistoricSnapshot = NULL;/* * These are updated by GetSnapshotData. We initialize them this way * for the convenience of TransactionIdIsInProgress: even in bootstrap * mode, we don't want it to say that BootstrapTransactionId is in progress. * these variables are updated through the function GetSnapshotData. * to facilitate TransactionIdIsInProgress, initialize them in this way: * even in boot mode, we do not want to indicate that BootstrapTransactionId is in progress. * * RecentGlobalXmin and RecentGlobalDataXmin are initialized to * InvalidTransactionId, to ensure that no one tries to use a stale * value. Readers should ensure that it has been set to something else * before using it. * RecentGlobalXmin and RecentGlobalDataXmin are initialized to InvalidTransactionId, * to ensure that no one is trying to use outdated values. * before using it, the reading process should ensure that it has been set to a different value. * / TransactionId TransactionXmin = FirstNormalTransactionId;TransactionId RecentXmin = FirstNormalTransactionId;TransactionId RecentGlobalXmin = InvalidTransactionId;TransactionId RecentGlobalDataXmin = InvalidTransactionId;/* (table, ctid) = > (cmin, cmax) mapping during timetravel * / static HTAB * tuplecid_data = NULL

MyPgXact

Current transaction information.

/ * * Flags for PGXACT- > vacuumFlags * PGXACT- > vacuumFlags tag * * Note: If you modify these flags, you need to modify PROCARRAY_XXX flags * in src/include/storage/procarray.h. * Note: if you modify these tags, you need to update the PROCARRAY_XXX tags * * PROC_RESERVED may later be assigned for use in vacuumFlags, but its value is * used for PROCARRAY_SLOTS_XMIN in procarray.h, so GetOldestXmin won't be able * to match and ignore processes with this flag set in src/include/storage/procarray.h. * PROC_RESERVED may be assigned to vacuumFlags later, * but it is used to identify PROCARRAY_SLOTS_XMIN in procarray.h, * so GetOldestXmin cannot match and ignore processes that use this tag. * / / is auto vacuum worker?#define PROC_IS_AUTOVACUUM 0x01 / * is it an autovac worker? * / / running lazy vacuum#define PROC_IN_VACUUM 0x02 / * currently running lazy vacuum * / / running analyze#define PROC_IN_ANALYZE 0x04 / * currently running analyze * / can only set # define PROC_VACUUM_FOR_WRAPAROUND 0x08 / * set by autovac via auto vacuum Only * / / logical decoding is being performed outside the transaction # define PROC_IN_LOGICAL_DECODING 0x10 / * currently doing logical * decoding outside xact * / / keep the MASK used for procarray#define PROC_RESERVED 0x20 / * reserved for procarray * / / * flags reset at EOXact * / to reset the tag during EOXact # define PROC_VACUUM_STATE_MASK\ (PROC_IN_VACUUM | PROC_IN_ANALYZE | PROC_VACUUM_FOR_WRAPAROUND) / * * Prior to PostgreSQL 9.2 The fields below were stored as part of the * PGPROC. However, benchmarking revealed that packing these particular * members into a separate array as tightly as possible sped up GetSnapshotData * considerably on systems with many CPU cores, by reducing the number of * cache lines needing to be fetched. Thus, think very carefully before adding * anything else here. * / typedef struct PGXACT {/ / current top-level transaction ID (non-sub-transaction) / / for optimization purposes, read-only transactions do not assign transaction numbers (xid = 0) TransactionId xid; / * id of top-level transaction currently being * executed by this proc, if running and XID * is assigned Else InvalidTransactionId * / / when starting the transaction, the minimum transaction number currently executing XID, but not including LAZY VACUUM / / vacuum, cannot clear and delete the tuple TransactionId xmin of transaction number xid > = xmin. / * minimal running XID as it was when we were * starting our xact, excluding LAZY VACUUM: * vacuum must not remove tuples deleted by * xid > = xmin! * / vacuum related tags uint8 vacuumFlags; / * vacuum-related flags, see above * / bool overflowed Bool delayChkpt; / * true if this proc delays checkpoint start; * previously called InCommit * / uint8 nxids;} PGXACT;extern PGDLLIMPORT struct PGXACT * MyPgXact

Snapshot

SnapshotData structure pointer, the information that can be expressed by SnapshotData structure includes all possible snapshots.

There are several different types of snapshots:

1. Regular MVCC Snapshot

two。 MVCC snapshot during restore (in Hot-Standby mode)

3. Historical MVCC snapshots used during logical decoding

4. A snapshot passed as an argument to the HeapTupleSatisfiesDirty () function

5. A snapshot passed as an argument to the HeapTupleSatisfiesNonVacuumable () function

6. Snapshots for SatisfiesAny, Toast, and Self without member access

/ / SnapshotData structure pointer typedef struct SnapshotData * invalid snapshot # define InvalidSnapshot ((Snapshot) NULL) / * * We use SnapshotData structures to represent both "regular" (MVCC) * snapshots and "special" snapshots that have non-MVCC semantics. * The specific semantics of a snapshot are encoded by the "satisfies" * function. * We use SnapshotData structures to represent "regular" (MVCC) snapshots and "special" snapshots with non-MVCC semantics. * / / Test function typedef bool (* SnapshotSatisfiesFunc) (HeapTuple htup, Snapshot snapshot, Buffer buffer) / / the common ones are: / / HeapTupleSatisfiesMVCC: determine whether the tuple is valid for a snapshot version / / HeapTupleSatisfiesUpdate: determine whether the tuple can be updated (update the same tuple at the same time) / / HeapTupleSatisfiesDirty: determine whether the current tuple has dirty data / / HeapTupleSatisfiesSelf: determine whether the tuple is valid for its own information / / HeapTupleSatisfiesToast: determine whether the TOAST table / / HeapTupleSatisfiesVacuum: determine whether the tuple can be deleted by VACUUM / / HeapTupleSatisfiesAny: all tuples are visible / / HeapTupleSatisfiesHistoricMVCC: for CATALOG table / * * Struct representing all kind of possible snapshots. * the information that can be expressed by this structure includes all possible snapshots. * * There are several different kinds of snapshots: * * Normal MVCC snapshots * * MVCC snapshots taken during recovery (in Hot-Standby mode) * * Historic MVCC snapshots used during logical decoding * * snapshots passed to HeapTupleSatisfiesDirty () * * snapshots passed to HeapTupleSatisfiesNonVacuumable () * * snapshots used for SatisfiesAny, Toast, Self where no members are * accessed. * there are several different types of snapshots: * * regular MVCC snapshots * * MVCC snapshots during restore (in Hot-Standby mode) * * Historical MVCC snapshots used during logical decoding * * snapshots passed as arguments to the HeapTupleSatisfiesDirty () function * * Snapshots passed as arguments to the HeapTupleSatisfiesNonVacuumable () function * * for SatisfiesAny without member access, Snapshots of Toast and Self * * TODO: It's probably a good idea to split this struct using a NodeTag * similar to how parser and executor nodes are handled With one type for * each different kind of snapshot to avoid overloading the meaning of * individual fields. * TODO: using parser/executor nodes-like processing, using NodeTag to split structures is a good practice, * using OO (object-oriented inheritance) method. * / typedef struct SnapshotData {/ / the function SnapshotSatisfiesFunc satisfies; / * tuple test function * / / * The remaining fields are used only for MVCC snapshots, and are normally * just zeroes in special snapshots to test whether tuple is visible. (But xmin and xmax are used * specially by HeapTupleSatisfiesDirty, and xmin is used specially by * HeapTupleSatisfiesNonVacuumable.) * the remaining fields are for MVCC snapshots only, usually 0 in special snapshots. * (xmin and xmax can be used for HeapTupleSatisfiesDirty,xmin and HeapTupleSatisfiesNonVacuumable) * * An MVCC snapshot can never see the effects of XIDs > = xmax. It can see * the effects of all older XIDs except those listed in the snapshot. Xmin * is stored as an optimization to avoid needing to search the XID arrays * for most tuples. Transactions of * XIDs > = xmax are not visible to the snapshot (have no effect). * XIDs that is less than xmax but not in the snapshot list is visible to this snapshot. * recording xmin is for optimization purposes, avoiding searching XID arrays for most tuples. * / XID ∈ [2minmin) is visible TransactionId xmin; / * all XID

< xmin are visible to me */ //XID ∈ [xmax,∞)是不可见的 TransactionId xmax; /* all XID >

= xmax are invisible to me * / * * For normal MVCC snapshot this contains the all xact IDs that are in * progress, unless the snapshot was taken during recovery in which case * it's empty. For historic MVCC snapshots, the meaning is inverted, i.e. * it contains * committed* transactions between xmin and xmax. * for normal MVCC snapshots, xip stores all XIDs in progress, except for snapshots generated during restore (when the array is empty) * for historical MVCC snapshots, it means the opposite, that is, it contains * committed * transactions between xmin and xmax. * * note: all ids in xip [] satisfy xmin regd_count++; pairingheap_add (& RegisteredSnapshots, & FirstXactSnapshot- > ph_node);} else / / non-transaction-snapshot mode, get CurrentSnapshot = GetSnapshotData (& CurrentSnapshotData) directly; / / set the tag FirstSnapshotSet = true; return CurrentSnapshot } / / transaction-snapshot mode if (IsolationUsesXactSnapshot ()) return CurrentSnapshot; / * Don't allow catalog snapshot to be older than xact snapshot. * / / do not allow catalog snapshots than transaction snapshots old InvalidateCatalogSnapshot (); / / take snapshots CurrentSnapshot = GetSnapshotData (& CurrentSnapshotData); / / return return CurrentSnapshot;} 3. Tracking analysis

Execute a simple query to trigger the snapshot logic.

16:35:08 (xdb@ [local]: 5432) testdb=# begin;BEGIN16:35:13 (xdb@ [local]: 5432) testdb=#* select 1

Start gdb and set breakpoint

(gdb) b GetTransactionSnapshotBreakpoint 1 at 0xa9492e: file snapmgr.c, line 312. (gdb) cContinuing.Breakpoint 1, GetTransactionSnapshot () at snapmgr.c:312312 if (HistoricSnapshotActive ()) (gdb)

If logical decoding is performed, a history snapshot is returned (not in this case).

(gdb) n319 if (! FirstSnapshotSet) (gdb)

First call? Yes, enter the corresponding logic

319 if (! FirstSnapshotSet) (gdb) n325 InvalidateCatalogSnapshot (); (gdb) 327 Assert (pairingheap_is_empty (& RegisteredSnapshots)); (gdb) 328 Assert (FirstXactSnapshot = = NULL); (gdb) n330 if (IsInParallelMode ()) (gdb)

Non-transaction-snapshot mode, call GetSnapshotData directly to get

(gdb) 341if (IsolationUsesXactSnapshot ()) (gdb) 356 CurrentSnapshot = GetSnapshotData (& CurrentSnapshotData) (gdb) p CurrentSnapshotData$1 = {satisfies = 0xa9310d, xmin = 2342, xmax = 2350, xip = 0x14bee40, xcnt = 2, subxip = 0x1514fa0, subxcnt = 0, suboverflowed = false, takenDuringRecovery = false, copied = false, curcid = 0, speculativeToken = 0, active_count = 0, regd_count = 0, ph_node = {first_child = 0x0, next_sibling = 0x0, prev_or_parent = 0x0}, whenTaken = 0, lsn = 0} (gdb)

The function executes successfully. Check CurrentSnapshot.

Note: 2342 the process in which the transaction is located has been kill

(gdb) n358 (gdb) p CurrentSnapshot$2 = (Snapshot) 0xf9be60 (gdb) p * CurrentSnapshot$3 = {satisfies = 0xa9310d, xmin = 2350, xmax = 2350, xip = 0x14bee40, xcnt = 0, subxip = 0x1514fa0, subxcnt = 0, suboverflowed = false, takenDuringRecovery = false, copied = false, curcid = 0, speculativeToken = 0, active_count = 0, regd_count = 0, ph_node = {first_child = 0x0, next_sibling = 0x0, prev_or_parent = 0x0}, whenTaken = 0, lsn = 0} (lsn)

Successful execution

(gdb) n359 return CurrentSnapshot; (gdb) 371} (gdb) exec_simple_query (query_string=0x149aec8 "select 1;") at postgres.c:10591059 snapshot_set = true; (gdb)

View the global variable MyPgXact

(gdb) p MyPgXact$7 = (struct PGXACT *) 0x7f47103c01f4 (gdb) p * MyPgXact$8 = {xid = 0, xmin = 2350, vacuumFlags = 0'\ 000mm, overflowed = false, delayChkpt = false, nxids = 0'\ 000'} (gdb)

Note:

1.xid = 0, indicating that the transaction number is not assigned. For optimization reasons, PG assigns the transaction number only when it modifies the data.

The 2.txid_current () function assigns the transaction number; the txid_current_if_assigned () function does not.

DONE!

Remaining questions:

When is the information in the 1.CurrentSnapshotData global variable initialized / changed?

The implementation of the 2.GetSnapshotData function (described in the next section).

IV. Reference materials

PG Source Code