Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "what is the role of RecordAndGetPageWithFreeSpace in PostgreSQL". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

I. data structure

FSMAddress

The internal FSM process works as a logical address scheme, and each level of the tree can be thought of as an independent address file.

/ * The internal FSM routines work on a logical addressing scheme. Each * level of the tree can be thought of as a separately addressable file. * the internal FSM process works on a logical address scheme. * each level of the tree can be thought of as a separate address file. * / typedef struct {/ / hierarchical int level; / * level * / / the page number within this hierarchy is int logpageno; / * page number within the level * /} FSMAddress;/* Address of the root page. * / / Root page address static const FSMAddress FSM_ROOT_ADDRESS = {FSM_ROOT_LEVEL, 0}

FSMPage

FSM page data structure. For details, please refer to src/backend/storage/freespace/README.

/ * Structure of a FSM page. See src/backend/storage/freespace/README for * details. * FSM page data structure. For details, please refer to src/backend/storage/freespace/README. * / typedef struct {/ * fsm_search_avail () tries to spread the load of multiple backends by * returning different pages to different backends in a round-robin * fashion. Fp_next_slot points to the next slot to be returned (assuming * there's enough space on it for the request). It's defined as an int, * because it's updated without an exclusive lock. Uint16 would be more * appropriate, but int is more likely to be atomically * fetchable/storable. The fsm_search_avail () function attempts to spread the load over background processes by returning different pages to different background processes in a loop. * this field is defined as an integer because it does not require an exclusive lock. * unit16 may be more appropriate, but integers seem to be more suitable for atomic extraction and storage. * / int fp_next_slot; / * fp_nodes contains the binary tree, stored in array. The first * NonLeafNodesPerPage elements are upper nodes, and the following * LeafNodesPerPage elements are leaf nodes. Unused nodes are zero. * fp_nodes stores the binary tree as an array. * the first NonLeafNodesPerPage element is the node of the previous layer, and the next LeafNodesPerPage element is the leaf node. * the unused node is 0. * / uint8 fp_ Nodes [flex _ ARRAY_MEMBER];} FSMPageData;typedef FSMPageData * FSMPage

FSMLocalMap

For small tables, there is no need to create a FSM to store spatial information, using local memory mapping information.

/ * Either already tried, or beyond the end of the relation * / / has been tried or has been at the end of the table # define FSM_LOCAL_NOT_AVAIL 0x00 * Available to try * / / can be used to try # define FSM_LOCAL_AVAIL 0x01max * * For small relations, we don't create FSM to save space, instead we use * local in-memory map of pages to try. To locate free space, we simply try * pages directly without knowing ahead of time how much free space they have. * for small tables, there is no need to create a FSM to store space information, using local memory mapping information. In order to locate free space, we don't need to know how much free space they have, but simply try page. * Note that this map is used to the find the block with required free space * for any given relation. We clear this map when we have found a block with * enough free space, when we extend the relation, or on transaction abort. * See src/backend/storage/freespace/README for further details. * Note that this map is used to search the request free space for a given table. * when a block/ with enough free space is found to extend relation/ in a transaction rollback, the map is cleared. * for more information, please see src/backend/storage/freespace/README. * / typedef struct {number of BlockNumber nblocks;// blocks uint8 map [heap _ FSM_CREATION_THRESHOLD]; / Array} FSMLocalMap;static FSMLocalMap fsm_local_map = {0, {FSM_LOCAL_NOT_AVAIL}}; # define FSM_LOCAL_MAP_EXISTS (fsm_local_map.nblocks > 0) II. Source code interpretation

RecordAndGetPageWithFreeSpace returns the block that meets the condition. The main logic is as follows:

1. Initialize related variables

two。 If there is a local map, use this file first and call fsm_local_search

3. If there is neither a local map nor a FSM, create a local map and then call fsm_local_search

4. Search using FSM

4.1 obtain the catalog corresponding to the available space of the original page in FSM

4.2 obtain the corresponding catalog in FSM according to the amount of space required

4.3 get the location of the heap block according to the original page (FSMAddress)

4.4 retrieve and get the target slot

4.5 if the target slot is legal, get the corresponding block, otherwise use fsm_search to search for the appropriate block

/ * RecordAndGetPageWithFreeSpace-update info about a page and try again. * RecordAndGetPageWithFreeSpace-update page info and try again. * * We provide this combo form to save some locking overhead, compared to * separate RecordPageWithFreeSpace + GetPageWithFreeSpace calls. There's * also some effort to return a page close to the old page; if there's a * page with enough free space on the same FSM page where the old one page * is located, it is preferred. * compared to individual RecordPageWithFreeSpace + GetPageWithFreeSpace calls, * We provide this combination to save some lock load. * some efforts are also stored here to return page that is close to the old page. * if there is a page with enough free space on the same FSM page as the old page, then this page will be selected. * * For very small heap relations that don't have a FSM, we update the local * map to indicate we have tried a page, and return the next page to try. * for very small heap tables, FSM is not required. Update the local map directly to indicate that the process needs to try to get a page and return the next page. * / BlockNumberRecordAndGetPageWithFreeSpace (Relation rel, BlockNumber oldPage, Size oldSpaceAvail, Size spaceNeeded) {int old_cat; int search_cat; FSMAddress addr;//FSM address uint16 slot;// slot number int search_slot; BlockNumber nblocks = InvalidBlockNumber; / * First try the local map, if it exists. * / / if there is a local map, use this file first. / / # define FSM_LOCAL_MAP_EXISTS (fsm_local_map.nblocks > 0) if (FSM_LOCAL_MAP_EXISTS) {Assert ((rel- > rd_rel- > relkind = = RELKIND_RELATION | | rel- > rd_rel- > relkind = = RELKIND_TOASTVALUE) & & fsm_local_ map.map [oldPage] = = FSM_LOCAL_AVAIL) / / set oldPage to not available fsm_local_ map. Map [oldPage] = FSM_LOCAL_NOT_AVAIL; / / search and return the result return fsm_local_search () } if (! fsm_allow_writes (rel, oldPage, InvalidBlockNumber, & nblocks)) {/ /-if FSM does not allow writing / * * If we have neither a local map nor a FSM, we probably just tried * the target block in the smgr relation entry and failed, so we'll * need to create the local map. * if there is no local map and no FSM, * then we just try the target block in smgr relation and fail, then we need to create a local map. * / / set the local map fsm_local_set (rel, nblocks); / / search the local map return fsm_local_search ();} / * Normal FSM logic follows * / /-use the logic / / oldSpaceAvail/32 of FSM, with a maximum of 255amp 254old_cat = fsm_space_avail_to_cat (oldSpaceAvail) / / (needed + FSM_CAT_STEP-1) / FSM_CAT_STEP / / # define FSM_CAT_STEP (BLCKSZ / FSM_CATEGORIES) / / # define FSM_CATEGORIES 256 search_cat = fsm_space_needed_to_cat (spaceNeeded); / * Get the location of the FSM byte representing the heap block * / / get the location of the corresponding heap block addr = fsm_get_location (oldPage, & slot) / / set the value in the given FSM page and slot, and return slot search_slot = fsm_set_and_search (rel, addr, slot, old_cat, search_cat); / * If fsm_set_and_search found a suitable new block, return that. * Otherwise, search as usual. * if fsm_set_and_search succeeds in finding a suitable block, it returns; otherwise, regular retrieval is performed. * / if (search_slot! =-1) return fsm_get_heap_blk (addr, search_slot); else return fsm_search (rel, search_cat);} / * * Search the local map for an available block to try, in descending order. * As such, there is no heuristic available to decide which order will be * better to try, but the probability of having space in the last block in the * map is higher because that is the most recent block added to the heap. * retrieve local map in reverse order to find available block. * in this case, there is no particularly good way to determine which sorting method is better, * but it is more likely that there is free space in the last block in the map, because this is the recently added block to the heap. * * This function is used when there is no FSM. * use this function if there is no FSM. * / static BlockNumberfsm_local_search (void) {BlockNumber target_block; / * Local map must be set by now. * / / now the local map must have Assert (FSM_LOCAL_MAP_EXISTS) set; / / Target block target_block = fsm_local_map.nblocks; do {/ / Loop target_block--;// starts with the last block if (fsm_local_ map.map [target _ block] = = FSM_LOCAL_AVAIL) return target_block / / if the last block is available, return} while (target_block > 0); / / target_block = = 0 / * * If we didn't find any available block to try in the local map, then * clear it. This prevents us from using the map again without setting it * first, which would otherwise lead to the same conclusion again and * again. * if no available block is found in the local map, clear the relevant information. * this prevents us from using the map without correctly setting the map, * this leads to the same repetitive conclusion (no block is available). * / FSMClearLocalMap (); / / return InvalidBlockNumber return InvalidBlockNumber;} / * * Initialize or update the local map of blocks to try, for when there is * no FSM. * if there is no FSM, initialize and update the local map * * When we initialize the map, the whole heap is potentially available to * try. Testing revealed that trying every block can cause a small * performance dip compared to when we use a FSM, so we try every other * block instead. * by the time we initialize the map, the entire heap may be available. * Test the table name, trying each block results in a slight performance degradation compared to using FSM, so try each block. * / static voidfsm_local_set (Relation rel, BlockNumber cur_nblocks) {BlockNumber blkno, cached_target_block; / * The local map must not be set already. * / / verify Assert (! FSM_LOCAL_MAP_EXISTS); / * * Starting at the current last block in the relation and working * backwards, mark alternating blocks as available. * reduce backwards at the beginning of the current last block of the relationship, marking updatable blocks available. * / blkno = cur_nblocks-1 Cache the number of blocks / the last block while (true) {/ / updated to be available fsm_local_ map.map.map[ blkno] = FSM_LOCAL_AVAIL; if (blkno > = 2) blkno-= 2; else break;} / * Cache the number of blocks. * / / number of cache blocks fsm_local_map.nblocks = cur_nblocks; / * Set the status of the cached target block to 'unavailable'. * / / set the cached target block state to unavailable cached_target_block = RelationGetTargetBlock (rel); if (cached_target_block! = InvalidBlockNumber & & cached_target_block)

< cur_nblocks) fsm_local_map.map[cached_target_block] = FSM_LOCAL_NOT_AVAIL;}/* * Return category corresponding x bytes of free space * 返回相应有x字节空间空间的目录 */static uint8fsm_space_avail_to_cat(Size avail){ int cat; //确保请求的小于块大小 Assert(avail < BLCKSZ); //如大于最大请求大小,返回255 //#define MaxFSMRequestSize MaxHeapTupleSize //#define MaxHeapTupleSize (BLCKSZ - MAXALIGN(SizeOfPageHeaderData + sizeof(ItemIdData))) if (avail >

= MaxFSMRequestSize) return 255; / / # define FSM_CAT_STEP (BLCKSZ / FSM_CATEGORIES) / / # define FSM_CATEGORIES 256 / / if the block size is 8K, FSM_CAT_STEP = 32 cat = avail / FSM_CAT_STEP; / * * The highest category, 255, is reserved for MaxFSMRequestSize bytes or * more. * the top-level directory, 255, is reserved for MaxFSMRequestSize or larger. * / if (cat > 254) cat = 254 rounds down / return 254return (uint8) cat;} / * * Which category does a page need to have, to accommodate x bytes of data? * While fsm_size_to_avail_cat () rounds down, this needs to round up. * which directory has the required page to meet the data of x bytes size. * because fsm_size_to_avail_cat () is rounded down, it needs to be rounded up here. * / static uint8fsm_space_needed_to_cat (Size needed) {int cat; / * Can't ask for more space than the highest category represents * / / cannot require the space size that the maximum directory may represent if (needed > MaxFSMRequestSize) elog (ERROR, "invalid FSM request size% zu", needed); if (needed = = 0) return 1; cat = (needed + FSM_CAT_STEP-1) / FSM_CAT_STEP If (cat > 255) cat = 255; return (uint8) cat;} / * * Return the FSM location corresponding to given heap block. * returns the FSM location of the given heap block. * / addr = fsm_get_location (oldPage, & slot); static FSMAddressfsm_get_location (BlockNumber heapblk, uint16 * slot) {FSMAddress addr; addr.level = FSM_BOTTOM_LEVEL / / # define SlotsPerFSMPage LeafNodesPerPage / / # define LeafNodesPerPage (NodesPerPage-NonLeafNodesPerPage) / / # define NodesPerPage (BLCKSZ-MAXALIGN (SizeOfPageHeaderData) -\ offsetof (FSMPageData, fp_nodes)) / / # define NonLeafNodesPerPage (BLCKSZ / 2-1) addr.logpageno = heapblk / SlotsPerFSMPage; * slot = heapblk% SlotsPerFSMPage; return addr;} 3. Tracking analysis

Test script

15:54:13 (xdb@ [local]: 5432) testdb=# insert into T1 values

Start gdb and set breakpoint

(gdb) b RecordAndGetPageWithFreeSpaceBreakpoint 1 at 0x8879e4: file freespace.c, line 152. (gdb) cContinuing.Breakpoint 1, RecordAndGetPageWithFreeSpace (rel=0x7fad0df13788, oldPage=1, oldSpaceAvail=16, spaceNeeded=32) at freespace.c:152152 int old_cat = fsm_space_avail_to_cat (oldSpaceAvail); (gdb)

Input parameters

(gdb) p * rel$5 = {rd_node = {spcNode = 1663, dbNode = 16402, relNode = 50820}, rd_smgr = 0x2084b00, rd_refcnt = 1, rd_backend =-1, rd_islocaltemp = false, rd_isnailed = false, rd_isvalid = true, rd_indexvalid = 1'\ 001, rd_statvalid = false, rd_createSubid = 0, rd_newRelfilenodeSubid = 0, rd_rel = 0x7fad0df139a0, rd_att = 0x7fad0df13ab8, rd_id = 50820, rd_lockInfo = {relId = 50820, dbId = 16402}} Rd_rules = 0x0, rd_rulescxt = 0x0, trigdesc = 0x0, rd_rsdesc = 0x0, rd_fkeylist = 0x0, rd_fkeyvalid = false, rd_partkeycxt = 0x0, rd_partkey = 0x0, rd_pdcxt = 0x0, rd_partdesc = 0x0, rd_partcheck = 0x0, rd_indexlist = 0x7fad0df12820, rd_oidindex = 0, rd_pkindex = 0, rd_replidindex = 0, rd_statlist = 0x0, rd_indexattr = 0x0, rd_projindexattr = 0x0, rd_keyattr = 0x0, rd_pkattr = rd_pkattr, 0x0 = 0x0 Rd_projidx = 0x0, rd_pubactions = 0x0, rd_options = 0x0, rd_index = 0x0, rd_indextuple = 0x0, rd_amhandler = 0, rd_indexcxt = 0x0, rd_amroutine = 0x0, rd_opfamily = 0x0, rd_opcintype = 0x0, rd_support = 0x0, rd_supportinfo = 0x0, rd_indoption = 0x0, rd_indexprs = 0x0, rd_indpred = 0x0, rd_exclops = 0x0, rd_exclprocs = 0x0, rd_exclstrats = 0x0, rd_amcache = rd_amcache, 0x0 = 0x0, 0x0 = rd_indcollation Rd_toastoid = 0, pgstat_info = 0x20785f0} (gdb)

1. Initialize related variables

two。 If there is a local map, use this file first and call fsm_local_search

3. If there is neither a local map nor a FSM, create a local map and then call fsm_local_search

4. Search using FSM

4.1 obtain the catalog-> 0 corresponding to the original page available space in FSM

4.2 obtain the corresponding catalog-> 1 in FSM according to the required space

(gdb) n153 int search_cat = fsm_space_needed_to_cat (spaceNeeded); (gdb) 159 addr = fsm_get_location (oldPage, & slot); (gdb) p old_cat$1 = 0 (gdb) p search_cat$2 = 1 (gdb)

4.3 get the location of the heap block according to the original page (FSMAddress)

(gdb) n161 search_slot = fsm_set_and_search (rel, addr, slot, old_cat, search_cat); (gdb) p addr$3 = {level = 0, logpageno = 0} (gdb)

4.4 retrieve and get the target slot

(gdb) n167 if (search_slot! =-1) (gdb) p search_slot$4 = 4 (gdb)

4.5 if the target slot is legal, get the corresponding block, otherwise use fsm_search to search for the appropriate block

(gdb) n168 return fsm_get_heap_blk (addr, search_slot); (gdb) 171} (gdb) RelationGetBufferForTuple (relation=0x7fad0df13788, len=32, otherBuffer=0, options=0, bistate=0x0, vmbuffer=0x7ffe1b797dcc, vmbuffer_other=0x0) at hio.c:397397 while (targetBlock! = InvalidBlockNumber) (gdb) p targetBlock$6 = 4 (gdb) the content of "what is RecordAndGetPageWithFreeSpace in PostgreSQL" is introduced here, thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.