Shulou(Shulou.com)05/31 Report--

This article mainly explains the "Oracle ASM Rebalance implementation process is how", the content of the explanation is simple and clear, easy to learn and understand, the following please follow the editor's ideas slowly in-depth, together to study and learn "Oracle ASM Rebalance implementation process is how" it!

When will the rebalance of the disk group be completed? This does not have a specific value, but ASM itself has provided you with an estimate (GV$ASM_OPERATION.EST_MINUTES), want to know the exact time rebalance completed, although can not give an exact time, but you can view some rebalance operation details, let you know whether the current rebalance is in progress, which stage, and whether this stage needs your attention.

Understand rebalance

The rebalance operation itself consists of three stages-planning, extents relocation and compacting. In terms of the total time required by rebalance, the planning phase requires very little time, you usually do not have to pay attention to this stage, the second stage extent relocation will generally take up most of the time of the rebalance phase, which is also the stage that we most need to pay attention to. Finally, we will talk about what the third stage compacting is doing.

First of all, you need to understand why you need to do rebalance, if you add a new disk to increase the free space of the disk group, or to adjust the disk space, such as resizing or delete the disk, you may not pay too much attention to when the rebalance will be completed. However, if a disk in the disk group is damaged, at this time you have enough reason to pay attention to the progress of rebalance, if your disk group is normal redundant, at this time in case you damage the partner disk of the disk is also damaged, then your entire disk group will be dismount, all databases running on this disk group will crash, and you may also lose data. In this case, you really need to know when the rebalance is complete, in fact, you need to know when the second phase extent relocation is completed, and once it is done, the redundancy of the entire disk group is complete (the third stage is not important for redundancy, which will be discussed later).

Extents relocation

To take a closer look at the extents relocation phase, I deleted a disk on the disk group with the default parallelism:

SQL > show parameter powerNAME TYPE VALUE -asm_power_limit integer 114select group_number 4715 SQL > select group_number Disk_number,name,state,path,header_status from v$asm_disk where group_number=5 GROUP_NUMBER DISK_NUMBER NAME STATE PATH HEADER_STATUS-- -50 TESTDG_0000 NORMAL / dev/raw/raw7 MEMBER 5 2 TESTDG_0002 NORMAL / dev/raw/raw13 MEMBER 5 1 TESTDG_0001 NORMAL / dev/raw/raw12 MEMBER 5 3 TESTDG_0003 NORMAL / dev/raw/raw14 MEMBER14:48:38 SQL > alter diskgroup testdg drop disk TESTDG_0000 Diskgroup altered.

The ESTMINUTES field in the following view GV$ASMOPERATION shows the estimated time in minutes, and the estimated time given here is 9 minutes.

14:49:04 SQL > select inst_id, operation, state, power, sofar, est_work, est_rate, est_minutes from gv$asm_operation where group_number=5 INST_ID OPERATION STATE POWER SOFAR EST_WORK EST_RATE EST_MINUTES-- -1 REBAL RUN 1 4 4748 475 9

After about 1 minute, the value of EST_MINUTES changes to 0 minutes:

14:50:22 SQL > select inst_id, operation, state, power, sofar, est_work, est_rate, est_minutes from gv$asm_operation where group_number=5 INST_ID OPERATION STATE POWER SOFAR EST_WORK EST_RATE EST_MINUTES-- -1 REBAL RUN 1 3030 4748 2429 0

Sometimes the value of EST_MINUTES may not give you much evidence, and we can also see that the value of SOFAR (the number of UA moved so far) has been increasing, well, yes, this is a good observation indicator. ASM's alert log also shows the operation of deleting the disk, and the ID,ASM of the OS ARB0 process uses it to do all the rebalance work. More importantly, there is no error output throughout the process:

SQL > alter diskgroup testdg drop disk TESTDG_0000 NOTE: GroupBlock outside rolling migration privileged regionNOTE: requesting all-instance membership refresh for group=5Tue Jan 10 14:49:01 2017GMON updating for reconfiguration, group 5 at 222 for pid 42, osid 6197NOTE: group 5 PST updated.Tue Jan 10 14:49:01 2017NOTE: membership refresh pending for group 5/0x97f863e8 (TESTDG) GMON querying group 5 at 223 for pid 18 Osid 5012SUCCESS: refreshed membership for 5/0x97f863e8 (TESTDG) NOTE: starting rebalance of group 5/0x97f863e8 (TESTDG) at power 1Starting background process ARB0SUCCESS: alter diskgroup testdg drop disk TESTDG_0000Tue Jan 10 14:49:04 2017ARB0 started with pid=39 OS id=25416 NOTE: assigning ARB0 to group 5/0x97f863e8 (TESTDG) with 1 parallel I/Ocellip.ora not found.NOTE: F1X0 copy 1 relocating from 0:2 to 2:2 for diskgroup 5 (TESTDG) NOTE: F1X0 copy 3 relocating from 2:2 to 3pur2599 for diskgroup 5 (TESTDG) Tue Jan 10 14:49:13 2017NOTE: Attempting voting file refresh on diskgroup TESTDGNOTE: Refresh completed on diskgroup TESTDG. No voting file found.Tue Jan 10 14:51:05 2017NOTE: stopping process ARB0SUCCESS: rebalance completed for group 5/0x97f863e8 (TESTDG) Tue Jan 10 14:51:07 2017NOTE: GroupBlock outside rolling migration privileged regionNOTE: requesting all-instance membership refresh for group=5Tue Jan 10 14:51:10 2017GMON updating for reconfiguration, group 5 at 224 for pid 39 Osid 25633NOTE: group 5 PST updated.SUCCESS: grp 5 disk TESTDG_0000 emptiedNOTE: erasing header on grp 5 disk TESTDG_0000NOTE: process _ x000_+asm1 (25633) initiating offline of disk 0.3915944675 (TESTDG_0000) with mask 0x7e in group 5NOTE: initiating PST update: grp = 5, dsk = 0/0xe96892e3, mask = 0x6a, op = clearGMON updating disk modes for group 5 at 225 for pid 39 Osid 25633NOTE: group TESTDG: updated PST location: disk 0001 (PST copy 0) NOTE: group TESTDG: updated PST location: disk 0002 (PST copy 1) NOTE: group TESTDG: updated PST location: disk 0003 (PST copy 2) NOTE: PST update grp = 5 completed successfully NOTE: initiating PST update: grp = 5, dsk = 0/0xe96892e3, mask = 0x7e, op = clearGMON updating disk modes for group 5 at 226 for pid 39, osid 25633NOTE: cache closing disk 0 of grp 5: TESTDG_0000NOTE: PST update grp = 5 completed successfully GMON updating for reconfiguration, group 5 at 227 for pid 39 Osid 25633NOTE: cache closing disk 0 of grp 5: (not open) TESTDG_0000NOTE: group 5 PST updated.NOTE: membership refresh pending for group 5/0x97f863e8 (TESTDG) GMON querying group 5 at 228 for pid 18, osid 5012GMON querying group 5 at 229 for pid 18, osid 5012NOTE: Disk TESTDG_0000 in mode 0x0 marked for de-assignmentSUCCESS: refreshed membership for 5/0x97f863e8 (TESTDG) Tue Jan 10 14:51:16 2017NOTE: Attempting voting file refresh on diskgroup TESTDGNOTE: Refresh completed on diskgroup TESTDG. No voting file found.

So ASM estimated that it would take nine minutes to complete the rebalance, but it only took two minutes, so it's important to know what the rebalance is doing before you know when the rebalance will be done. Note that the estimated time is dynamic and may increase or decrease, depending on your system load and your rebalance power setting, for a very large disk group, rebalance may take you hours or even days.

The trace file of the ARB0 process also shows that the extent of which ASM file is currently being reassigned, and through this trace file, we can know that ARB0 is really doing its job and is not lazy.

[grid@jyrac1 trace] $tail-f + ASM1_arb0_25416.trc*** 2017-01-10 14:49:20.160ARB0 relocating file + TESTDG.256.932913341 (120entries) * * 2017-01-10 14:49:24.081ARB0 relocating file + TESTDG.256.932913341 (120entries) * * 2017-01-10 14:49:28.290ARB0 relocating file + TESTDG.256.932913341 (120entries) * * 2017-01-10 14:49:32.108ARB0 relocating file + TESTDG.256.932913341 (120entries) ) * 2017-01-10 14:49:35.419ARB0 relocating file + TESTDG.256.932913341 (120entries) * 2017-01-10 14:49:38.921ARB0 relocating file + TESTDG.256.932913341 (120entries) * * 2017-01-10 14:49:43.613ARB0 relocating file + TESTDG.256.932913341 (120entries) * January-10 14:49:47.523ARB0 relocating file + TESTDG.256.932913341 (120entries) * * January-10 14:49:51.073ARB0 Relocating file + TESTDG.256.932913341 (120 entries) * * 2017-01-10 14:49:54.545ARB0 relocating file + TESTDG.256.932913341 (120 entries) * * 2017-01-10 14:49:58.538ARB0 relocating file + TESTDG.256.932913341 (120 entries) * * 2017-01-10 14:50:02.944ARB0 relocating file + TESTDG.256.932913341 (120 entries) * * 2017-01-10 14:50:06.428ARB0 relocating file + TESTDG.256.932913341 (120 entries) * * 2017- 01-10 14:50:10.035ARB0 relocating file + TESTDG.256.932913341 (120entries) * * 2017-01-10 14:50:13.507ARB0 relocating file + TESTDG.256.932913341 (120entries) * * 2017-01-10 14:50:17.526ARB0 relocating file + TESTDG.256.932913341 (120entries) * * January-10 14:50:21.692ARB0 relocating file + TESTDG.256.932913341 (120entries) * * January-10 14:50:25.649ARB0 relocating file + TESTDG.256. 932913341 (2017-01-10 14:50:29.360ARB0 relocating file) * * 2017-01-10 14:50:29.360ARB0 relocating file + TESTDG.256.932913341 (120entries) * * 2017-01-10 14:50:33.233ARB0 relocating file + TESTDG.256.932913341 (120entries) * * 2017-01-10 14:50:37.287ARB0 relocating file + TESTDG.256.932913341 (120entries) * * January-10 14:50:40.843ARB0 relocating file + TESTDG.256.932913341 (120entries) * * January-10, 2017 14:50 : 44.356ARB0 relocating file + TESTDG.256.932913341 (120entries) * * 2017-01-10 14:50:48.158ARB0 relocating file + TESTDG.256.932913341 (120entries) * * 2017-01-10 14:50:51.854ARB0 relocating file + TESTDG.256.932913341 (120entries) * * January-10 14:50:55.568ARB0 relocating file + TESTDG.256.932913341 (120entries) * * January-10 14:50:59.439ARB0 relocating file + TESTDG.256.932913341 (120entries) * * * 2017-01-10 14:51:02.877ARB0 relocating file + TESTDG.256.932913341 (50 entries)

Note that there may be many trace files for arb0 in the trace directory, so we need to know that the OS of arb0 is the process number and which arb0 is actually doing the work of rebalance. This information will be displayed in the alert file when the ASM instance performs the rebalance operation. We can also track the ARB0 process with the operating system command pstack to see exactly what it is doing, which shows us that ASM is reassigning extent (the key function kfgbRebalExecute-kfdaExecute-kffRelocate in the stack):

[root@jyrac1 ~] # pstack 25416 00x0000003aa88005f4 in? () from / usr/lib64/libaio.so.1#1 0x0000000002bb9b11 in skgfrliopo () # 2 0x0000000002bb9909 in skgfospo () # 3 0x00000000086c595f in skgfrwat () # 4 0x00000000085a4f79 in ksfdwtio () # 5 0x000000000220b2a3 in ksfdwat_internal () # 6 0x0000000003ee7f33 in kfk_reap_ufs_async_io () # 7 0x0000000003ee7e7b in kfk_reap_ios_from_subsys () # 8 0x0000000000aea0ac in kfk_reap_ios () # 9 0x0000000003ee749e in kfk_io1 () # 10 0x0000000003ee7044 in KfkRequest () # 11 0x0000000003eed84a in kfk_transitIO () # 12 0x0000000003e40e7a in kffRelocateWait () # 13 0x0000000003e67d12 in kffRelocate () # 14 0x0000000003ddd3fb in kfdaExecute () # 15 0x0000000003ec075b in kfgbRebalExecute () # 16 0x0000000003ead530 in kfgbDriver () # 17 0x00000000021b37df in ksbabs () # 18 0x0000000003ec4768 in kfgbRun () # 19 0x00000000021b8553 in ksbrdp () # 20 0x00000000023deff7 in opirip () # 21 0x00000000016898bd in opidrv () # 22 0x0000000001c6357f in sou2o () # 23 0x00000000008523ca in opimai_real () # 24 0x0000000001c6989d in ssthrdmain () # 25 0x00000000008522c1 in main ()

Compacting

In the following example, let's take a look at the compacting phase of rebalance. I add back the disk deleted above and set the power of rebalance to 2:

17:26:48 SQL > alter diskgroup testdg add disk'/ dev/raw/raw7' rebalance power 2 * Diskgroup altered.

The estimated time for rebalance given by ASM is 6 minutes:

16:07:13 SQL > select INST_ID, OPERATION, STATE, POWER, SOFAR, EST_WORK, EST_RATE, EST_MINUTES from GV$ASM_OPERATION where GROUP_NUMBER=1 INST_ID OPERA STAT POWER SOFAR EST_WORK EST_RATE EST_MINUTES- 1 REBAL RUN 10 489 53851 7920 6

After about 10 seconds, the value of EST_MINUTES changes to 0. 5%.

16:07:23 SQL > / INST_ID OPERA STAT POWER SOFAR EST_WORK EST_RATE EST_MINUTES--1 REBAL RUN 10 92407 97874 8716 0

At this point, we observed in the alert log of ASM:

SQL > alter diskgroup testdg add disk'/ dev/raw/raw7' rebalance power 2NOTE: GroupBlock outside rolling migration privileged regionNOTE: Assigning number (5 initializing header on grp 0) to disk (/ dev/raw/raw7) NOTE: requesting all-instance membership refresh for group=5NOTE: initializing header on grp 5 disk TESTDG_0000NOTE: requesting all-instance disk validation for group=5Tue Jan 10 16:07:12 2017NOTE: skipping rediscovery for group 5/0x97f863e8 (TESTDG) on local instance.NOTE: requesting all-instance disk validation for group=5NOTE: skipping rediscovery for group 5/0x97f863e8 (TESTDG) on local instance.Tue Jan 10 16:07:12 2017GMON updating for reconfiguration Group 5 at 230 for pid 42, osid 6197NOTE: group 5 PST updated.NOTE: initiating PST update: grp = 5GMON updating group 5 at 231 for pid 42, osid 6197NOTE: PST update grp = 5 completed successfully NOTE: membership refresh pending for group 5/0x97f863e8 (TESTDG) GMON querying group 5 at 232 for pid 18, osid 5012NOTE: cache opening disk 0 of grp 5: TESTDG_0000 path:/dev/raw/raw7GMON querying group 5 at 233 for pid 18 Osid 5012SUCCESS: refreshed membership for 5/0x97f863e8 (TESTDG) NOTE: starting rebalance of group 5/0x97f863e8 (TESTDG) at power 1SUCCESS: alter diskgroup testdg add disk'/ dev/raw/raw7'Starting background process ARB0Tue Jan 10 16:07:14 2017ARB0 started with pid=27, OS id=982 NOTE: assigning ARB0 to group 5/0x97f863e8 (TESTDG) with 1 parallel I/Ocellip.ora not found.Tue Jan 10 16:07:23 2017NOTE: Attempting voting file refresh on diskgroup TESTDG

The above output means that ASM has completed the second phase of rebalance and started the third phase of compacting. If I am right, you can see the kfdCompact () function through the pstack tool. The following output shows that this is indeed the case:

# pstack 982'0 0x0000003957ccb6ef in poll () from / lib64/libc.so.6...#9 0x0000000003d711e0 in kfk_reap_oss_async_io () # 10 0x0000000003d70c17 in kfk_reap_ios_from_subsys () # 11 0x0000000000aea50e in kfk_reap_ios () # 12 0x0000000003d702ae in kfk_io1 () # 13 0x0000000003d6fe54 in kfkRequest () # 14 0x0000000003d76540 in kfk_transitIO () # 15 0x0000000003cd482b in kffRelocateWait () # 16 0x0000000003cfa190 in kffRelocate () # 17 0x0000000003c7ba16 in kfdaExecute () # 18 0x0000000003c4b737 in kfdCompact () # 19 0x0000000003c4c6d0 in KfdExecute () # 20 0x0000000003d4bf0e in kfgbRebalExecute () # 21 0x0000000003d39627 in kfgbDriver () # 22 0x00000000020e8d23 in ksbabs () # 23 0x0000000003d4faae in kfgbRun () # 24 0x00000000020ed95d in ksbrdp () # 25 0x0000000002322343 in opirip () # 26 0x0000000001618571 in opidrv () # 27 0x0000000001c13be7 in sou2o () # 28 0x000000000083ceba in opimai_real () # 29 0x0000000001c19b58 in ssthrdmain () # 30 0x000000000083cda1 in main ()

Looking at the trace file of ARB0 through the tail command, we found that relocating is in progress, and only one item at a time is relocating. (this is another important clue that is going on to the compacting phase):

$tail-f + ASM1_arb0_25416.trcARB0 relocating file + DATA1.321.788357323 (1 entries) ARB0 relocating file + DATA1.321.788357323 (1 entries) ARB0 relocating file + DATA1.321.788357323 (1 entries).

During the compacting process, the EST_MINUTES field of the V$ASM_OPERATION view is displayed as 0 (also an important clue):

16:08:56 SQL > / INST_ID OPERA STAT POWER SOFAR EST_WORK EST_RATE EST_MINUTES--2 REBAL RUN 10 98271 98305 7919 0

The REBALST_KFGMG field of the solid table X$KFGMG displays as 2, indicating that you are in compacting.

16:09:12 SQL > select NUMBER_KFGMG, OP_KFGMG, ACTUAL_KFGMG, REBALST_KFGMG from X-ray KFGMG OP_KFGMG ACTUAL_KFGMG REBALST_KFGMG- 1 1 10 2

Once the compacting phase is complete, stopping process ARB0 and rebalance completed are displayed in ASM's alert log:

Tue Jan 10 16:10:19 2017NOTE: stopping process ARB0SUCCESS: rebalance completed for group 5/0x97f863e8 (TESTDG)

Once the extents relocation is complete, all the data has met the redundancy requirements, and there is no longer the worry that the partern disk of the failed disk will fail again and cause a serious failure.

Changing the power

Rebalance's power can be changed dynamically during the disk group rebalance process, and if you think the default level of the disk group is too low, you can easily add it. But how much does it increase to? This needs to be determined by the IO load and IO throughput of your system. In general, you can first try to increase to a conservative value, such as 5, after ten minutes to see if there is an improvement, and whether it affects other business on the use of IO, if your IO performance is very strong, then you can continue to increase the value of power, but in my experience, it is rare to see the setting of power more than 30 can be greatly improved. The key point of testing is that you need to test under the normal load of your production system. Different business pressures and different storage systems may make a big difference in rebalance time.

Thank you for your reading, the above is the content of "what is the implementation process of Oracle ASM Rebalance". After the study of this article, I believe you have a deeper understanding of how the implementation process of Oracle ASM Rebalance is, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.