Shulou(Shulou.com)06/01 Report--

Links to related articles:

Asm Instance Parameter Best Practice

Why is there no ASMLIB on RHEL 6?

How to view TRACE files with the file name beginning with "+ asm" on Unix

The influence of asm_power_limit on IO

Recovery measures for 11.2 RAC loss of OCR and ASM Diskgroup where Votedisk is located

10g ASM lost disk log

Uncover the secret of 11gR2 RAC ASM startup

Modify the ASM DISK Path disk path in 11gR2 RAC

Using UDEV on Linux 6 to resolve RAC ASM storage device name issues

Script: find the Spfile parameter file in ASM

How to diagnose ASMLIB faults

Script: collecting ASM diagnostic information

Comparation between ASM note [ID 373242.1] and note [ID 452924.1]

Why ASMLIB and why not?

ASM file metadata operation wait event

Several questions about oracle 11g ASM

Using UDEV Service to solve RAC ASM Storage device name

Discover Your Missed ASM Disks

Oracle Internal View X$KFFXP

Fixed X$ Tables in ASM

Understand the MAP files generated by the AMDU tool

Use the AMDU tool to extract data files from DISKGROUP that cannot be MOUNT

Overview of Oracle automatic storage management

Automatic Storage Management (ASM) is a feature of Oracle Database that provides database administrators with a simple storage management interface that is consistent on all servers and storage platforms. As a vertically integrated file system and volume manager created specifically for Oracle database files, ASM provides the performance of direct asynchronous Icano and the manageability of the file system. ASM provides features that save DBA time, as well as the flexibility to manage dynamic database environments, and increases efficiency. The main advantages of ASM are:

Simplify and automate storage management, improve storage utilization and agility, provide predictable performance, availability, and scalability

Overview of Oracle Cloud File System

Oracle Cloud File System (CloudFS) simplifies storage management, provisioning automation, and storage consolidation of common files like never before. CloudFS is a storage cloud infrastructure that provides resource pooling, network accessibility, rapid scaling, and rapid provisioning-all key requirements for cloud computing environments. The product includes:

Oracle ASM Dynamic Volume Manager (ADVM)

ADVM provides a common volume management service and a standard device driver interface for system administrators to manage across different platforms. ACFS and third-party file systems can use ASM dynamic volumes to create and manage file systems that take advantage of all the features of ASM. As a result, ADVM volumes can be easily resized to meet the storage needs of the file system without downtime.

Oracle ASM Cluster File System (ACFS)

A general-purpose POSIX, X/OPEN, and Windows compatible file system designed for single-node and single-cluster configurations. Use commands, ASM asmcmd, and Oracle Enterprise Manager that come with the operating system to manage ACFS. ACFS supports advanced data services such as point-in-time replication snapshots, file system replication and labels, and file system security and encryption.

Automatic Storage Management is a database storage automatic solution first put forward by Oracle in version 10g (compared with other RDBMS), and has been further sublimated in version 11g. ASM provides a simple and effective storage management interface for database management, which implements cross-server and storage platforms. ASM is the integration of file system filesystem and volume manager volume management software, specially designed for Oracle database file locks; ASM provides high-performance asynchronous Async IO on the basis of ensuring that it is easy to manage as a file system. The introduction of ASM not only improves the scalable capacity of the database, but also saves the time of DBA, making it more agile and efficient to manage a mobile database environment.

ASM appeared to manage file storage for RDBMS

Note that ASM will not replace RDBMS to implement IO reading and writing, and many misunderstand that RDBMS sends IO request to ASM,ASM to do the real IO operation, which is wrong. The real IO is implemented by the RDBMS process, just like a bare device without ASM, so ASM is not the middle layer of IO. There is no so-called IO bottleneck because of ASM. For ASM, LUN DISK can be either a bare device or a block device (after 10.2.0.2) the file types suitable for storage in ASM include: data file datafile, control file controlfile, redo log redolog, archive log archivelog, flashback log flashback log, spfile, RMAN backup and block tracking file, datapump files starting from 11gR2 ASM introduces the ACFS feature to store files of any type. But ACFS does not support storing data files.

Basic concepts of ASM:

The minimum storage unit of ASM is a "allocation unit" (AU), usually 1MB. The core of 4MBASM recommended on Exadata is that the storage file is divided into multiple file slices, which is called "extent". Before 11g, the size of extent is always one AU,11g, followed by an extent that can be 1 or 8 or 64 AUASM using file extent map maintenance file extent location ASM maintains its metadata in the header header of LUN DISK Instead of a data dictionary, RDBMS DB also caches file extent map in shared pool, which is used when server process deals with IO because ASM instance uses an instance/crash recovery similar to the principle of ordinary RDBMS, so ASM instance can always recover after a crash.

ASM storage is presented in the concept of diskgroups:

Diskgroup DG is visible to RDBMS instances, such as a DATA DG, which for RDBMS is a storage point represented by'+ DATA', on which you can create a tablespace, such as create tablespace ONASM datafile'+ DATA' size 10m. Below the Diskgroup is one or more failure group (FG) FG defined as a set of DiskDisk, where it can be a bare physical volume, disk partition, LUN representing a disk array, or disk in multiple FG of LVM or NAS devices should not have the same single point of failure, otherwise the redundancy of the ASM is invalid.

High availability provided by ASM:

ASM provides data mirroring to recover from disk failure. Users can choose three redundant images: EXTERNAL, NORMAL, and HIGH. EXTERNAL, that is, ASM itself does not mirror, but depends on the underlying storage array to achieve mirroring. Any write error under External will cause Disk Group to be forced to dismount. In this mode, all ASM DISK must be healthy, otherwise Disk Group will not be able to MOUNTNORMAL, that is, ASM will create an additional copy of each extent for redundancy; by default, all files will be mirrored so that each file extent has 2 copies. If the write error occurs on 2 Disk and the 2 Disk is partners, it will cause the disk Disk Group to be forced dismount. If the failed disk is not partners, it will not cause data loss and unavailability. HIGH, or ASM, creates two additional copies of each extent for higher redundancy. The failure of two Disk that are partners each other will not cause data loss, and of course no more partners Disk can fail. Data mirroring depends on failure group and extent partnering implementations. ASM can tolerate the loss of all disks in a failure group with NORMAL or HIGH redundancy.

Use of Failure Group mirrors

The image of ASM is not like that of RAID 1. The image of ASM is based on the granularity of file extent. Extent is distributed among multiple disks. PartnerPartner disk is stored on one or more separate failure group. ASM automatically selects partner and limits its number to less than 10 disks. If the disk fails, ASM updates its extent map so that future read operations point to the remaining healthy partner in 11g, if a disk is in offline state. Changes to the file are tracked so that when the disk is reproduced the online is reapplied, as long as the time of the offline does not exceed the time specified by the DISK_REPAIR_TIME (the default is 3.6h). This often occurs in storage controller failures or similar short-term disk failures: this tracking of file changes is based on a bitmap of a changed file extent that tells ASM which extents needs to be copied from the healthy partner and where to the disk that needs to be repaired. This feature is called that fast mirror resync has no fast mirror resync feature in 10g, and if disk has offline, it will be automatically drop out. There is no cycle that allows repair. For disk that can no longer online, it must be dropped by drop. A new disk is selected by the ASM and copied through the rebalancing operation, which is done automatically in the background.

Rebalance Rebalancing

Rebalancing is the process of moving files extent between disks to achieve IO load balancing on diskgroup. Rebalancing occurs asynchronously in the background and can be monitored. In a cluster environment, the rebalancing of a diskgroup can only occur on one ASM instance and cannot be processed simultaneously by multiple nodes in the cluster to speed up when disk is added or removed. ASM automatically starts data rebalancing in the background. The speed and strength of data rebalancing can be controlled by the asm_power_limit parameter. The asm_power_limit parameter defaults to 1, which ranges from 0 to 11 (0-1024 from 11.2.0.2). This parameter controls the number of rebalancing background processes. Level 0 indicates no rebalancing. IO performance (mainly throughput and response time) may be affected during rebalancing, depending on the capacity of the storage itself and the strength of rebalancing. The default asm_powner_limit=1 will not cause excessive impact.

Performance aspect

ASM maximizes available IO bandwidth by striping file extent distribution in DG. There are two available striping widths: coarse rough striping size to 1 AU,fine fine striping to 128K even though fine fine striping still uses normal size file extent, but striping is cyclically distributed in smaller slices over multiple extent ASM by default does not allow RDBMS to read the backup mirror copy extent Even so, rest assured that IO is still balanced. By default, RDBMS always reads the main primary extent. From 11.1 onwards, you can let the local node read the extent in a failure group first through the PREFERRED_READ_FAILURE_GROUP parameter setting. This feature is mainly designed for extended distance RAC and is not recommended in regular ASM.

Other knowledge

It is not only RAC that can use ASM, but also where a single node can benefit from ASM. An ASM instance instance on a node can serve multiple RDBMS DB instances. ASM in a RAC environment must also be clustered so that the metadata can be updated in a coordinated manner. Starting from 11.2, ASM is separated from RDBMS HOME and installed under GRID HOME with clusterware.

Disk Group:

Disk Group "disk group" is the logical concept object of ASM management, and a Disk Group consists of multiple ASM disk. Each Disk Group is a subdescription, just like a standard file system. All metadata about the usage of the Diskgroup space is fully contained in this disk group. If ASM can find all the DISK that belongs to the ASM diskgroup, he doesn't need any additional metadata.

File space is allocated from Disk Group. Any ASM file is always fully contained in a separate Disk Group. However, a Disk Group may contain files that belong to multiple databases, and files from a single database can be stored in multiple different Disk Group. In most real-world deployments, you don't create too many Disk Groups, typically 3-4.

Disk Group provides three different redundancy redundancy, as detailed above.

ASM Disk

An ASM Disk is the basic persistent storage that makes up Disk Group. When an ASM Disk is added to the Disk Group, it uses either the ASM Disk Name specified by the administrator or the Disk Name automatically assigned by the system. This is different from the "stage name" given by OS to access the device. In a Cluster cluster, the same Disk may display different Device Name device names on different nodes, such as / dev/sdc on Node1, corresponding to / dev/sdd on Node2. ASM Disk must be accessible on all instances that use the Disk Group using the direct disk Imando O.

In fact, for RDBMS Oracle, accessing ASM disk is no different from accessing ordinary files, unless you use ASMLIB (ASMLIB is not required for ASM, again!) . Normally, ASM Disk is the partition of LUN visible on OS, and this partition covers the space of all disks that are not reserved by the operating system. Most operating systems need to keep the first block of LUN as the partition table (partition table); since ASM always writes the first block of ASM Disk, make sure that ASM does not overwrite the partition table (partition table) on the previous block, for example, do not mark the first few cylinders to partition when partitioning on Solaris. The LUN can be a simple physical JBOD or a virtual LUN managed by an advanced storage array. It can be either a directly connected device or a SAN. ASM Disk can be anything accessed by a development system call, except the local file system. Even the files on NFS can be used as an ASM Disk, which makes it easy for users who like NAS to use ASM. Of course, I would rather use ISCSI than NFS.

Note that although you can use a normal logical Volume Manager LVM-managed logical volume as an ASM Disk, this is not a recommended combination unless you can't think of a better way. Even if you must use it, be careful not to mirror and stripe at the LVM level.

ASM distributes any file evenly over all Disk of Disk Group in AU size. Each ASM Disk is maintained to maintain the same usage ratio. This ensures that the IO load of all Disk in the same Disk Group is basically the same. Due to the load balancing of ASM on disks in one Disk Group, dividing different areas of the same physical disk into two ASM Disk does not affect performance, while dividing two different partitions on the same physical disk into two different Disk Group is effective.

When ASM Disk Group enables redundancy, a single ASM Disk is only a unit of failure. A write failure for the ASM Disk will automatically drop the Disk from the Disk Group drop at 10g, provided that the loss of the Disk is allowed.

Allocation Unit

Each ASM Disk is divided into a number of AU allocation units (the size of a single AU is between 1MB and 64MB, note that it is always MB to the power of 2). And AU allocation unit is also the basic allocation unit of Disk Group. The free space on an ASM Disk is always an integral multiple of AU. There is a table in the header of each ASM Disk, and each record in the table represents an AU on that ASM Disk. The extent pointer (pointer) of the file gives the ASM Disk Number disk number and AU number, which describes the physical location of the extent. Since all space operations are in AU units, there is no concept or problem of so-called ASM fragments.

An AU (1M~64M) is small enough that a file always contains many AU so that it can be distributed across many disks without causing hotspots. An AU is large enough to be accessed in an IO operation for more throughput and to provide efficient sequential access. The time spent accessing an AU will be spent more on the disk transfer rate than on finding the AU. The rebalancing of Disk Group is also done on a case-by-case basis for each AU.

Understand the role of the ASM background process:

GMON: ASM Diskgroup monitoring process

ASMB: ASM background network process

RBAL: ASM reblance master process rebalances the main process

ARBx: the background process for reblance slave process to actually implement reblance

MARK: the conductor process of AU resync AU resynchronization

Understand the role of the ASM foreground process:

The client of ASM (mainly RDBMS DB and CRSD) will generate a foreground process when connecting to the ASM instance, and the name of the process the day before yesterday is usually oracle+ASM__ (for example: oracle+ASM_DBW0_DB1).

Foreground: oracle+ASM1_ocr, a foreground process unique to OCR

V$ and X$ views related to ASM

View name X$ Base Table name describes V$ASM_DISKGROUPX$KFGRP implementation disk Discovery disk discovery and lists disk groups V$ASM_DISKGROUP_STATX$KFGRP_STAT displays disk group status V$ASM_DISKX$KFDSK, X$KFKID implements disk Discovery disk discovery and lists disks and usage Metrics Information for these disks Varians ASM Discovery Discovery STATX disks STATMaging Xlists KFKIDs lists disks and their usage metrics information V$ASM_FILEX$KFFIL lists ASM files also includes metadata information V$ASM_ALIASX$KFALS lists aliases for ASM Files and directories V$ASM_TEMPLATEX$KFTMTA list available templates and their properties V$ASM_CLIENTX$KFNCL list DB instances linked to ASM V$ASM_OPERATIONX$KFGMG list rebalancing rebalance operation N/AX$KFKLIB available ASMLIB paths N/AX$KFDPARTNER list Disk-partners relationships N/AX$KFFXP all ASM files extent mapN/AX$KFDAT all ASM Disk extent list N/AX$KFBH describes ASM cacheN/AX$KFCCEASM block's linked list V$ASM_ATTRIBUTE (new in 11g) X$KFENV (new in 11g) Asm attribute The X$ base table also shows some hidden attributes V$ASM_DISK_IOSTAT (new in 11g) X$KFNSDSKIOST (new in 11g) I Zero statistics N/AX$KFDFS (new in 11g)

N/AX$KFDDD (new in 11g)

N/AX$KFGBRB (new in 11g)

N/AX$KFMDGRP (new in 11g)

N/AX$KFCLLE (new in 11g)

N/AX$KFVOL (new in 11g)

N/AX$KFVOLSTAT (new in 11g)

N/AX$KFVOFS (new in 11g)

N/AX$KFVOFSV (new in 11g)

X$KFFXP contains the mapping between files, extent, and AU. From this X$ view, you can track the striping and mirroring of the extent of a given file. Note that the load for primary au and mirror au reads is balanced, while write operations require writing both to disk. Here is the meaning of the X$KFFXP view column

X$KFFXP Column NameDescriptionADDRx$ table address/identifierINDXrow unique identifierINST_IDinstance number (RAC) NUMBER_KFFXPASM file number. Join with v$asm_file and v$asm_aliasCOMPOUND_KFFXPFile identifier. Join with compound_index in v$asm_fileINCARN_KFFXPFile incarnation id. Join with incarnation in v$asm_filePXN_KFFXPProgressive file extent numberXNUM_KFFXPASM file extent number (mirrored extent pairs have the same extent value) GROUP_KFFXPASM disk group number. Join with v$asm_disk and v$asm_diskgroupDISK_KFFXPDisk number where the extent is allocated. Join with v$asm_diskAU_KFFXPRelative position of the allocation unit from the beginning of the disk. The allocation unit size

(1 MB) in voluasmically diskgroupLXNFXP0-> primary extent,-> mirror extent, 2-> 2nd mirror copy (high redundancy and metadata) FLAGS_KFFXPN.K.CHK_KFFXPN.K.

The X$ view of X$KFDAT contains all the details of allocation unit AU, whether it's FREE or USED.

X$KFDAT Column NameDescriptionADDRx$ table address/identifierINDXrow unique identifierINST_IDinstance number (RAC) GROUP_KFDATdiskgroup number, join with v$asm_diskgroupNUMBER_KFDATdisk number, join with v$asm_diskCOMPOUND_KFDATdisk compund_index, join with v$asm_diskAUNUM_KFDATDisk allocation unit (relative position from the beginning of the disk), join with

X$kffxp.au_kffxpV_KFDATV=this Allocation Unit is used; F=AU is freeFNUM_KFDATfile number, join with v$asm_fileI_KFDATN/KXNUM_KFDATProgressive file extent number join with x$kffxp.pxn_kffxpRAW_KFDATraw format encoding of the disk,and file extent information

The X$ view of X$KFDPARTNER contains the mapping relationship of disk-partner (1murn). In a given ASM Diskgroup, if two Disk have mirrored copies of the same extent, two disk are regarded as partners. Therefore, partners must belong to different failgroup under the same diskgroup.

X$KFDPARTNER Column NameDescriptionADDRx$ table address/identifierINDXrow unique identifierINST_IDinstance number (RAC) GRPdiskgroup number, join with v$asm_diskgroupDISKdisk number, join with v$asm_diskCOMPOUNDdisk identifier. Join with compound_index in v$asm_diskNUMBER_KFDPARTNERpartner disk number, I.E. Disk-to-partner (1Mel N) relationshipMIRROR_KFDPARNERif=1 in a healthy normal redundancy configPARITY_KFDPARNERif=1 in a healthy normal redundancy configACTIVE_KFDPARNERif=1 in a healthy normal redundancy config

The necessary skills to study ASM

1) find the mirror mirror extent of ASM, which in this case is the spfile of ASM

[grid@localhost] $sqlplus / as sysasmSQL*Plus: Release 11.2.0.3.0 Production on Wed Feb 13 11:13:39 2013Copyright (c) 1982, 2011, Oracle. All rights reserved.Connected to:Oracle Database 11g Enterprise Edition Release 11.2.0.3.0-64bit ProductionWith the Automatic Storage Management optionINSTANCE_NAME-+ASMSQL > SQL > show parameter spfileNAME TYPE VALUE -spfile string + SYSTEMDG/asm/asmparameterfile / registry.253.805993079select GROUP_KFFXP DISK_KFFXP, AU_KFFXP from x$kffxp where number_kffxp = (select file_number from v$asm_alias where name = 'REGISTRY.253.805993079') GROUP_KFFXP DISK_KFFXP AU_KFFXP- 3 2 38 3 1 39 30 0 44 can also locate select GROUP_KFDAT and NUMBER_KFDAT in this way. AUNUM_KFDAT from x$kfdat where fnum_kfdat = (select file_number from v$asm_alias where name = 'REGISTRY.253.805993079') GROUP_KFDAT NUMBER_KFDAT AUNUM_KFDAT- 30 44 3 1 39 3 2 38 SQL = > find the path corresponding to the DISK SQL > select path DISK_NUMBER from v$asm_disk where GROUP_NUMBER=3 and disk_number in (0meme1pyr2) PATH DISK_NUMBER--/ dev/asm-diski 2/dev/asm-diskh 1/dev/asm-diskg 0SQL > create pfile='/home/grid/pfile' from spfile File created.SQL > Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0-64bit ProductionWith the Automatic Storage Management option [grid@localhost ~] $cat pfile + ASM.asm_diskgroups='EXTDG' 'NORDG'#Manual Mount*.asm_diskstring='/dev/asm*'*.asm_power_limit=1*.diagnostic_dest='/g01/app/grid'*.instance_type='asm'*.large_pool_size=12M*.local_listener='LISTENER_+ASM'*.remote_login_passwordfile='EXCLUSIVE' reads the AU [grid@localhost ~] $dd if=/dev/asm-diski of=/tmp/spfile.dmp skip=38 bs=1024k count=11+0 records in1+0 records out1048576 bytes (1.0 MB) copied, 0.00328614 seconds through dd 319 MB/s [grid@localhost ~] $strings / tmp/spfile.dmp + ASM.asm_diskgroups='EXTDG' 'NORDG'#Manual Mount*.asm_diskstring='/dev/asm*'*.asm_power_limit=1*.diagnostic_dest='/g01/app/grid'*.instance_type='asm'*.large_pool_size=12M*.local_listener='LISTENER_+ASM'*.remote_login_passwordfile='EXCLUSIVE' [grid@localhost ~] $dd if=/dev/asm-diskh of=/tmp/spfile1.dmp skip=39 bs=1024k count=1 1' 0 records in1+0 records out1048576 bytes (1.0 MB) copied, 0.0325114 seconds 32.3 MB/s [grid@localhost ~] $strings / tmp/spfile1.dmp + ASM.asm_diskgroups='EXTDG' 'NORDG'#Manual Mount*.asm_diskstring='/dev/asm*'*.asm_power_limit=1*.diagnostic_dest='/g01/app/grid'*.instance_type='asm'*.large_pool_size=12M*.local_listener='LISTENER_+ASM'*.remote_login_passwordfile='EXCLUSIVE' [grid@localhost ~] $dd if=/dev/asm-diskg of=/tmp/spfile2.dmp skip=44 bs=1024k count=11+0 records in1+0 records out1048576 bytes (1.0 MB) copied, 0.0298287 seconds 35.2 MB/s [grid@localhost ~] $strings / tmp/spfile2.dmp + ASM.asm_diskgroups='EXTDG','NORDG'#Manual Mount*.asm_diskstring='/dev/asm*'*.asm_power_limit=1*.diagnostic_dest='/g01/app/grid'*.instance_type='asm'*.large_pool_size=12M*.local_listener='LISTENER_+ASM'*.remote_login_passwordfile='EXCLUSIVE'

2) display the mapping relationship between asm disk failure group and disk partners:

1 * select DISK_NUMBER,FAILGROUP Path from v$asm_disk where group_number=3SQL > / DISK_NUMBER FAILGROUP PATH--3 SYSTEMDG_0003 / dev/asm-diskj 2 SYSTEMDG_0002 / dev/asm-diski 1 SYSTEMDG_0001 / dev/asm-diskh 0 SYSTEMDG_0000 / dev/asm-diskgSQL > select disk NUMBER_KFDPARTNER,DISKFGNUM from X$KFDPARTNER where grp=3 DISK NUMBER_KFDPARTNER DISKFGNUM- 0 1 1 0 2 1 0 3 1 1 0 2 1 2 2 1 3 2 2 0 3 2 1 3 2 3 3 3 0 4 3 1 4 3 2 412 rows selected.

ASM FAQ, FAQ:

What are the basic particles for Q:ASM to do rebalance and mirror?

A: the basic particle of ASM for mirror image is the extent of file. By default, an extent equals an AU,11g, and then an extent can be 1 or 8 or 64 AU.

The basic particle that ASM does rebalance rebalancing is also extent, although rebalancing is done on a case-by-case basis for each AU.

What is the relationship between Q:ASMLIB and ASM?

A:ASMLIB is a variety of kernel support libraries (kernel support library) based on Linux module and specifically designed for Oracle Automatic Storage Management features.

To put it simply, ASMLIB is a package under Linux, which does not belong to Oracle ASM kernel. Device name binding can be achieved through ASMLIb to facilitate the use of ASM, but there are many services on Linux that can bind device name and facilitate the use of ASM, such as udev, mpath, etc.

Therefore, ASMLIB is not a necessary component of ASM; most of the Chinese articles in China do not have a clear description of the concept, resulting in the misunderstanding that ASMLIB=ASM or ASM must use ASMLIB.

The shortcomings of ASMLIB can be found in my work "Why ASMLIB and why not?"

Q: is ASM raid 10 or raid 01?

A:ASM 's mirror is based on file extent, not disk or block like raid. So ASM is neither Raid 10 nor Raid 01. If you insist on similarities, because ASM is mirrored first by mirror and then striped by stripe, it is more like Raid 10 in this feature. Note, however, that ASM is neither RAID 10 nor RAID 01, repeated a thousand times.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.