Shulou(Shulou.com)06/01 Report--

Oracle RAC Learning-- RAC Ping and RAC Cache Fusion in OPS

1. RAC Ping in OPS

2. Cache Fusion in RAC

What is Cache Fusion?

Cache Fusion transfers blocks between the SGA of each node in the cluster through the Internet to avoid the inefficient implementation of pushing the block to disk and then re-reading it into the cache of other instances (the implementation of OPS). When a block is read into the cache of an instance in the RAC environment, the block is assigned a lock resource (unlike row-level locks) to ensure that other instances know that the block is being used. Then, if another instance requests a copy of the block and the block is already in the cache of the previous instance, the block is passed directly to the SGA of the other instance over the Internet. If the block in memory has been changed, but the change has not been committed, a copy of the CR will be passed. This means that whenever possible, blocks can be moved between instance caches without having to write back to disk, thus avoiding the additional Icano cost of synchronizing multi-instance caches. Obviously, the data cached by different instances can be different, that is, before an instance accesses a particular block, and it never accesses that block, it either cache fusion from another instance or reads it from disk.

There are still some questions to think about:

1. When the block is not read by all instances, but when the first instance is read, how is the lock added and what lock is added? If there is another instance to read this block at this time, almost at the same time, how does Oracle arbitrate, how do you get one of them to read and the other to get it from the former's cache through cache?

2. If a block has been read into another instance, how can this instance determine its existence?

3. If one instance changes the data block, will the change be passed to other instances, or will the other instances know and update the status?

4. If an instance wants to swap out a block, and other instances also have the cache of this block, modified and unmodified, modified by this instance and modified by other instances, what to do? Truncate a table, drop a table. How is it different from a single instance?

5. How should applications be designed so that RAC really works, rather than introducing competition that weakens the system?

6. The realization of RAC unlocking. Locks are resources reserved in the SGA of each instance and are typically used to control access to database blocks. Each instance usually retains or controls a certain number of locks related to the block range. When an instance requests a block, the block must acquire a lock, and the lock must come from the instance that currently controls the locks. That is, locks are distributed over different instances. To get a specific lock, you have to get it from a different instance. However, from this process, these locks are not fixed on an instance, but will be adjusted to the most frequently used instances according to the lock request frequency, thus improving efficiency.

For some of the previous problems, you can learn from other concepts, which are global caching services and global queuing services.

Global caching Service (GCS):

Global caching involves data blocks. The global cache service is responsible for maintaining cache consistency in the global cache, ensuring that an instance can obtain a global lock resource when it wants to modify a block at any time, thus avoiding the possibility that another instance will modify the block at the same time. The modified instance will have the current version of the block (both committed and uncommitted) as well as the post p_w_picpath of the block. If another instance also requests the block, the global caching service is responsible for tracking the instance that owns the block, what version of the block is owned, and what mode the block is in. The LMS process is a key component of the global caching service.

Global queuing Service (GES):

Global Enqueue Service (GES) tracks the status of all Oracle enqueuing mechanism. It is mainly responsible for maintaining the consistency between dictionary cache and library cache. Dictionary cache is the cache of data dictionary information stored in the SGA of an instance for high-speed access. Because the dictionary information is stored in memory, changes made to the dictionary on one node, such as DDL, must be immediately propagated to the dictionary cache on all nodes. GES is responsible for handling the above situations and eliminating differences between instances. For the same reason, in order to analyze the SQL statements that affect these objects, the library cache locks on the objects in the database are removed. These locks must be maintained between instances, and the global queuing service must ensure that deadlocks do not occur between multiple instances requesting access to the same object. The LMON, LCK, and LMD processes work together to implement the global queuing service. GES is an important service to regulate other resources between nodes in the RAC environment in addition to the maintenance and management of the data block itself (completed by GCS).

View GCS and GES:

SQL > set linesize 1000

SQL > select * from gv$sysstat where name like 'gcs%'

INST_ID STATISTIC# NAME CLASS VALUE STAT_ID--1 44 gcs messages sent 32 5981 27654518042 44 gcs messages sent 32 3632 2765451804

SQL > select * from gv$sysstat where name like 'ges%'

INST_ID STATISTIC# NAME CLASS VALUE STAT_ID- 145 ges messages sent 32 3760 11454254332 45 ges messages sent 32 4447 1145425433

Here you can see the number of gcs and ges messages sent.

(if you do not use DBCA to create the database, then you want SYSDBA permission to run the CATCLUST.SQL script to create RAC-related views and tables)

Configuration requirements for InterConnect in RAC:

How to configure the interconnect Internet to ensure efficient operation?

1. Hardware: gigabit network

2. Parameter setting

Net.core.rmem_max 's largest TCP data reception buffer

Cache Fusion

Provides scalability for transmission, transfers block p_w_picpath between instances, tracks the current location and status of resources, and stores resource information in the directory structure of the sga of each instance

GRD:Global Resoure Directory

GES and GCS together maintains Global Resource Directory (GRD). GRD is like an in-memory database which contains details about all the blocks that are present in cache. GRD know what is the location of latest version of block, what is the mode of block, what is the role of block (Mode and role will be discussed shortly) etc. When ever a user ask for any data block GCS gets all the information from GRD. GRD is a distributed resource, meaning that each instance maintain some part of GRD. This distributed nature of GRD is a key to fault tolerance of RAC. GRD is stored in SGA.

Typically GRD contains following and more information

(1) Data Block Address-This is the address of data block being modified

(2) Location of most current version of data block

(3) Modes of data block

(4) Roles of data block

(5) SCN number of data block

7) Image of data block-Could be current p_w_picpath or past p_w_picpath.

Global Resoure Directory is managed by Global Cache Service

Record the mode of the resource, the role of the resource, the status of the block in the instance, publish the master of the resource at each active node, and republish the master if necessary (such as startup and shutdown of the instance)

Global Cache Service MODE and ROLE:

1. Resource mode (mode)

Null (default)

Share (s) (query)

Exclusive (x) (you can change the content of block, other examples are null mode)

2. Resource role (role)

Local:

The initial trial mode of the resource request for the first time; only one instance can have the dirty copy of this block

Global:

When a Block changes to dirty in multiple instances, Local becomes Global Block that can only be written to disk by Global Cache Service.

Past Images

Past Image concept was introduced in Oracle 9i to maintain data integrity. In an Oracle database, a typical block is not written to disk immediately after it is dirtied. This is to reduce excessive IO. When the same dirty block is requested by some other instance for write of read purpose, an p_w_picpath of the block is created in owning instance and then the block is shifted to requesting instance. This p_w_picpath copy of the block is called Past Image (PI). In the event of failure Oracle can reconstruct the block by reading PIs. It is also possible to have more then 1 PI of the block, depending on how many times the block was requested in dirty stage.

A past p_w_picpath of the block is different then CR (Consistent read) p_w_picpath. Past p_w_picpath is required to create CR by applying undo data.

"Juggling" Data with Multiple Past Images

(1) Multiple Past Image versions of a data block may be kept by different instances

(2) Upon a checkpoint, only the current p_w_picpath is written to disk; Past Images are discarded

(3) In the event of a failure, current version of block can be reconstructed from PIs

(4) Since PIs are kept in memory, they aid in avoiding frequent disk writes

(5) This avoids "disk pinging" experienced with 8i OPS due to frequent writes to disk

(6) Data is "juggled" in memory, without touching down on the disk

Transmission of Cache Fusion Block

For example, there are four nodes A, B, C and D.

1. Read with no transfer

If the C node needs to read a Block to the shared disk file, it sends a request to the Global Cache Service, and the request is directed to the node DMague D is the Master of the Block (each resource has a Master). GCS authorizes the resource to Share Mode and Local Role, records his status in the directory (the directory is on node D), and then tells C Magi C to change the resource from Null to Share. C starts with Block O, and now C has this Block to read from disk files in Share mode.

2. Read to write transfer

B also wants this Block, and not only read it, but also change its content. B sends a request to the GCS of D (the Mater of this Block), and GCS sends a request to C, asking C to give the Block to BMague C and the Block to Bmai B after receiving it, tell GCS that B can modify the block now.

3. Write to write transfer

A sends a request to the GCS of node D, and GCS tells node B to give up his Exclusive lock and pass the current Image to A. if the request is not completed, it will be placed in the queue of GCS. B passes this Block to A, and at this time, write Log, force Log Flush, and change the pattern to Null. Send it to An and tell it that the Exclusive resource is available. A receives the Image of this Block, notifies GCS and tells it that the Status of Block is Exclusive. At this time, B cannot operate on the Block, although it also has the Copy of the Block in its Buffer Cache.

4. Write to read transfer

To read the Block, C first sends a request to D (Master), and GCS asks A to transmit it to C Master A to receive the request to finish its work, probably before A writes Log and Log Flush sends the Block. A will lower its Exclusive lock to Share mode. C takes out the SCN of the Block received from An and builds a resource Assumption information to update the Global Resource Directory for GCS.

You can turn off Cache Fusion by setting the parameter gc_files_to_locks. This is like the OPS of 8i, other nodes must wait for other nodes to submit and write back to the data file if they want to access the data quickly.

Remastering of Cache Resoure:

Cache Resoure no longer needs to continue on one node. Master,Dynamic Remastering can move it to different nodes. GCS and GES use dynamic Remastering: redistribute resources after a new instance is added to the Active Set, and redistribute resources after an instance leaves the Active Set.

Sample Cache Fusion diagram:

Read/Read Cache Fusion-GCS Processing

Write/Write Cache Fusion-GCS Processing

Blocks to Disk-GCS Processing

Online Instance Recovery Steps

The steps are as follows:

(1) Instance Failure detected by Cluster Manager and GCS

(2) Reconfiguration of GES resources (enqueues); global resource directory is frozen

(3) Reconfiguration of GCS resources; involves redistribution among surviving instances

(4) One of the surviving instances becomes the "recovering instance"

(5) SMON process of recovering instance starts first pass of redo log read of the failed instance's redo log thread

(6) SMON finds BWR (block written records) in the redo and removes them as their PI is already written to disk

(7) SMON prepares recovery set of the blocks modified by the failed instance but not written to disk

(8) Entries in the recovery list are sorted by first dirty SCN

(9) SMON informs each block's master node to take ownership of the block for recovery

(10) Second pass of log read begins.

(11) Redo is applied to the data files.

(12) Global Resource Directory is unfrozen

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.