Shulou(Shulou.com)06/01 Report--

This section briefly introduces the processing flow of manual vacuum execution by PostgreSQL, and mainly analyzes the implementation logic of the ExecVacuum- > vacuum- > vacuum_rel- > heap_vacuum_rel- > lazy_scan_heap- > lazy_vacuum_index function, which cleans up index relation.

I. data structure

Macro definition

Vacuum and Analyze command options

/ *-- * Vacuum and Analyze Statements * Vacuum and Analyze command options * * Even though these are nominally two statements, it's convenient to use * just one node type for both. Note that at least one of VACOPT_VACUUM * and VACOPT_ANALYZE must be set in options. Although there are two different statements here, you only need to use a uniform Node type. * Note that at least VACOPT_VACUUM/VACOPT_ANALYZE is set in the options. *-/ typedef enum VacuumOption {VACOPT_VACUUM = 1 index; RELATION_CHECKS; CHECK_REL_PROCEDURE (ambulkdelete); / / the actual function pointed to by ambulkdelete is btbulkdelete return indexRelation- > rd_indam- > ambulkdelete (info, stats, callback, callback_state);}

Lazy_vacuum_index- > index_bulk_delete- > … Btbulkdelete

The rd_amroutine- > ambulkdelete of Index Relation is actually a btbulkdelete function

/ * Bulk deletion of all index entries pointing to a set of heap tuples. * The set of target tuples is specified via a callback routine that tells * whether any given heap tuple (identified by ItemPointer) is being deleted. * bulk delete index entries that point to the heap tuples collection. * the target tuple set is specified by the callback function to get which given tuples (defined by ItemPointer) will be deleted. * * Result: a palloc'd struct containing statistical info for VACUUM displays. * return result: statistics for VACUUM display * / IndexBulkDeleteResult * btbulkdelete (IndexVacuumInfo * info, IndexBulkDeleteResult * stats, IndexBulkDeleteCallback callback, void * callback_state) {/ / relation Relation rel = info- > index; BTCycleId cycleid / * allocate stats if first time through, else re-use existing struct * / / allocate memory if it is called for the first time, otherwise reuse the existing structure if (stats = = NULL) stats = (IndexBulkDeleteResult *) palloc0 (sizeof (IndexBulkDeleteResult)) / * Establish the vacuum cycle ID to use for this scan * / * The ENSURE stuff ensures we clean up shared memory on failure * / / establish a vacuum loop ID for this scan / / PG_ENSURE_ERROR_CLEANUP to ensure that the shared memory PG_ENSURE_ERROR_CLEANUP (_ bt_end_vacuum_callback, PointerGetDatum (rel)) is cleaned in the event of a failure; {TransactionId oldestBtpoXact / / transaction ID / / start vacuum cycleid = _ bt_start_vacuum (rel); / / point to BTree vacuum scan btvacuumscan (info, stats, callback, callback_state, cycleid, & oldestBtpoXact); / * * Update cleanup-related information in metapage. This information is * used only for cleanup but keeping them up to date can avoid * unnecessary cleanup even after bulkdelete. * update clean-up related information. * this information is used for cleaning, but keeping it up-to-date can avoid unnecessary cleanup. * / _ bt_update_meta_cleanup_info (info- > index, oldestBtpoXact, info- > num_heap_tuples);} PG_END_ENSURE_ERROR_CLEANUP (_ bt_end_vacuum_callback, PointerGetDatum (rel)); _ bt_end_vacuum (rel); / / return statistics return stats;} 3. Tracking analysis

Test script: delete data, execute vacuum

10:08:46 (xdb@ [local]: 5432) testdb=# delete from T1 where id

< 1200;DELETE 10011:26:03 (xdb@[local]:5432)testdb=# checkpoint;CHECKPOINT11:26:04 (xdb@[local]:5432)testdb=# 11:25:55 (xdb@[local]:5432)testdb=# vacuum t1; 启动gdb,设置断点 (gdb) b lazy_vacuum_indexBreakpoint 1 at 0x6bea40: file vacuumlazy.c, line 1689....Breakpoint 1, lazy_vacuum_index (indrel=0x7f7334825050, stats=0x2aaffb8, vacrelstats=0x2aaf958) at vacuumlazy.c:16891689 pg_rusage_init(&ru0);(gdb) 输入参数 (gdb) p *indrel$6 = {rd_node = {spcNode = 1663, dbNode = 16402, relNode = 50823}, rd_smgr = 0x0, rd_refcnt = 1, rd_backend = -1, rd_islocaltemp = false, rd_isnailed = false, rd_isvalid = true, rd_indexvalid = 0 '\000', rd_statvalid = false, rd_createSubid = 0, rd_newRelfilenodeSubid = 0, rd_rel = 0x7f733491ad20, rd_att = 0x7f733491a9b8, rd_id = 50823, rd_lockInfo = {lockRelId = {relId = 50823, dbId = 16402}}, rd_rules = 0x0, rd_rulescxt = 0x0, trigdesc = 0x0, rd_rsdesc = 0x0, rd_fkeylist = 0x0, rd_fkeyvalid = false, rd_partkeycxt = 0x0, rd_partkey = 0x0, rd_pdcxt = 0x0, rd_partdesc = 0x0, rd_partcheck = 0x0, rd_indexlist = 0x0, rd_oidindex = 0, rd_pkindex = 0, rd_replidindex = 0, rd_statlist = 0x0, rd_indexattr = 0x0, rd_projindexattr = 0x0, rd_keyattr = 0x0, rd_pkattr = 0x0, rd_idattr = 0x0, rd_projidx = 0x0, rd_pubactions = 0x0, rd_options = 0x0, rd_index = 0x7f733491a8d8, rd_indextuple = 0x7f733491a8a0, rd_amhandler = 330, rd_indexcxt = 0x2a05340, rd_amroutine = 0x2a05480, rd_opfamily = 0x2a05598, rd_opcintype = 0x2a055b8, rd_support = 0x2a055d8, rd_supportinfo = 0x2a05600, rd_indoption = 0x2a05738, rd_indexprs = 0x0, rd_indpred = 0x0, rd_exclops = 0x0, rd_exclprocs = 0x0, rd_exclstrats = 0x0, rd_amcache = 0x0, rd_indcollation = 0x2a05718, rd_fdwroutine = 0x0, rd_toastoid = 0, pgstat_info = 0x2a5e198}(gdb) p *indrel->

Rd_rel$9 = {relname = {data = "idx_t1_id",'\ 000'}, relnamespace = 2200, reltype = 0, reloftype = 0, relowner = 10, relam = 403, relfilenode = 50823, reltablespace = 0, relpages = 60, reltuples = 8901, relallvisible = 0, reltoastrelid = 0, relhasindex = false, relisshared = false, relpersistence = 112 'packs, relkind = 105i, relnatts = 1, relchecks = 0, relhasoids = false, relhasrules = false, relhastriggers = false, relhassubclass = false, relrowsecurity = false, relforcerowsecurity = false, relispopulated = relispopulated Relreplident = 110n, relispartition = false, relrewrite = 0, relfrozenxid = 0, relminmxid = 0} (gdb) p * stats$7 = (IndexBulkDeleteResult *) 0x0 (gdb) p * vacrelstats$8 = {hasindex = true, old_rel_pages = 124,124, scanned_pages = 59, pinskipped_pages = 0, frozenskipped_pages = 1, tupcount_pages = 59, old_live_tuples = 12686, new_rel_tuples = 14444, new_live_tuples = 14444, new_dead_tuples = 0, pages_removed = 0, tuples_deleted = 14444 Nonempty_pages = 124,100,100, max_dead_tuples = 36084, dead_tuples = 0x2ab8820, num_index_scans = 0, latestRemovedXid = 397076, lock_waiter_detected = false} (gdb)

Initialize the IndexVacuumInfo structure

(gdb) n1691 ivinfo.index = indrel; (gdb) 1692 ivinfo.analyze_only = false; (gdb) 1693 ivinfo.estimated_count = true; (gdb) 1694 ivinfo.message_level = elevel; (gdb) 1696 ivinfo.num_heap_tuples = vacrelstats- > old_live_tuples; (gdb) 1697 ivinfo.strategy = vac_strategy; (gdb)

Call index_bulk_delete to enter the function

1700 * stats= index_bulk_delete (& ivinfo, * stats, (gdb) stepindex_bulk_delete (info=0x7fff39c5d620, stats=0x0, callback=0x6bf507, callback_state=0x2aaf958) at indexam.c:748748 Relation indexRelation = info- > index; (gdb)

Input parameters

Info-> IndexVacuumInfo structure

Stats is NULL

The callback function is lazy_tid_reaped

The callback function state structure is callback_state

(gdb) p * info$10 = {index = 0x7f7334825050, analyze_only = false, estimated_count = true, message_level = 13, num_heap_tuples = 12686, strategy = 0x2a9d478} (gdb) (gdb) p * callback_stateAttempt to dereference a generic pointer. (gdb) p * info- > strategy$11 = {btype = BAS_VACUUM, ring_size = 32, current = 4, current_was_in_ring = false, buffers = 0x2a9d488} (gdb)

Call indexRelation- > rd_amroutine- > ambulkdelete, which actually points to btbulkdelete

(gdb) n750 RELATION_CHECKS; (gdb) 751 CHECK_REL_PROCEDURE (ambulkdelete) (gdb) 753 return indexRelation- > rd_amroutine- > ambulkdelete (info, stats, (gdb) p indexRelation- > rd_amroutine$12 = (struct IndexAmRoutine *) 0x2a05480 (gdb) p * indexRelation- > rd_amroutine$13 = {type = T_IndexAmRoutine, amstrategies = 5, amsupport = 3, amcanorder = true, amcanorderbyop = false, amcanbackward = true, amcanunique = true, amcanmulticol = true, amoptionalkey = true, amsearcharray = true, amsearchnulls = true, amstorage = false, false = amclusterable, amclusterable = amclusterable, true = true, true = ampredlocks, = 0, Aminsert = 0x507f11, ambulkdelete = 0x5096b6, amvacuumcleanup = 0x509845, amcanreturn = 0x50a21f, amcostestimate = 0x9c5356, amoptions = 0x511cd4, amproperty = 0x511cfe, amvalidate = 0x51522b, ambeginscan = 0x5082f7, amrescan = 0x508492, amgettuple = 0x507f90, amgetbitmap = 0x50819e, amendscan = 0x508838, ammarkpos = 0x508b28, amrestrpos = 0x508d20, amestimateparallelscan = 0x5090e6, aminitparallelscan = 0x5090f1, amparallelrescan = 0x50913f}

Enter btbulkdelete

(gdb) stepbtbulkdelete (info=0x7fff39c5d620, stats=0x0, callback=0x6bf507, callback_state=0x2aaf958) at nbtree.c:857857 Relation rel = info- > index; (gdb)

Input parameters see the above function input parameters, similar to

Get relation, initialize statistics

857 Relation rel = info- > index; (gdb) n861 if (stats = = NULL) (gdb) 862 stats = (IndexBulkDeleteResult *) palloc0 (sizeof (IndexBulkDeleteResult)); (gdb) 866 PG_ENSURE_ERROR_CLEANUP (_ bt_end_vacuum_callback, PointerGetDatum (rel)); (gdb)

Get cycleid

(gdb) n870 cycleid = _ bt_start_vacuum (rel); (gdb) 872 btvacuumscan (info, stats, callback, callback_state, cycleid, (gdb) p cycleid$14 = 1702 (gdb)

Call btvacuumscan to return statistics

(gdb) n880 _ bt_update_meta_cleanup_info (info- > index, oldestBtpoXact, (gdb) 883 PG_END_ENSURE_ERROR_CLEANUP (_ bt_end_vacuum_callback, PointerGetDatum (rel)); (gdb) 884 _ bt_end_vacuum (rel); (gdb) 886 return stats (gdb) p * stats$15 = {num_pages = 60, pages_removed = 0, estimated_count = false, num_index_tuples = 8801, tuples_removed = 8801, pages_deleted = 6, pages_free = 6} (gdb)

DONE!

Btvacuumscan will be introduced in the next section.

IV. Reference materials

PG Source Code

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.