Shulou(Shulou.com)06/02 Report--

Overview of storage failure 1. Fault environment

Two groups of raid5 arrays are composed of 4 SAS hard disks with 600G capacity, and the two arrays are divided into LUN to form a LVM structure and formatted as an EXT3 file system.

2. Fault analysis

A hard disk is unexpectedly offline, and the hot spare is online to replace the offline hard disk. However, when the hot spare is online, another hard disk goes offline, which leads to the failure of hot spare synchronization, the crash of one of the two groups of raid arrays, the incomplete LVM structure and the failure of the file system. Through the detection of two offline hard drives, it is found that the first offline hard disk can not be identified, and it is preliminarily inferred that it is a hardware fault, which needs to be opened and repaired, and the other hard disk can be identified.

II. Overview of the solution

According to the results of previous fault analysis, the following solutions are summarized:

1. Repair the faulty hard disk and back up the faulty hard disk using MRT professional data recovery software.

2. Use professional data recovery software winhex to make a full backup of the rest of raid member disk and another group of raid member disk.

3. Analyze the data of each hard disk and reorganize the RAID array according to the analyzed structure.

4. Analyze the reassembled array, find the LVM information, and reorganize the LVM volume.

5. Parse the EXT3 file system on the reorganized LVM volume, recover and export all the data.

Third, implement the solution 1. Repair the faulty disk

Perform an open disk repair operation on the failed hard drive. After opening, it is found that the wear and tear of the hard disk is so serious that it can not be repaired, so we can only deal with the missing disk of the array.

2. Hard disk backup

Use the professional data recovery software winhex to make a full backup of the remaining member disks of the failed raid array, and a full backup of all member disks of another good raid array, as follows:

3. Reassemble the raid array

Carefully analyze the underlying data of the hard disk, analyze the structure of the EXT3 file system, analyze the configuration information of the two groups of raid arrays, such as disk order, stripe size, check direction, etc., and reassemble two groups of raid arrays using professional data recovery software winhex. According to the analysis, the block size of the two groups of raid array is 64K, and the check direction is synchronization. When reassembling the fault raid, attention should be paid to the missing disk.

4. Reorganize the LVM structure

After reassembling the two groups of raid arrays, we analyze the underlying data in the two groups of raid, find the LVM structure information, analyze the LVM structure, export the LUN in the two groups of raid as PV (LVM physical volumes), and then use the professional data recovery software UFS E xplorer to reassemble the two PV to regenerate the LVM logical volumes.

5. Restore data

After the LVM reorganization, the EXT3 file system in the LV (logical volume) is parsed, recovered, and exported. The following is the data replied:

IV. Data verification

For the recovered data, some compressed files are selected for verification, and some files are found to be corrupted. Comparing the parsing results with the recovery results, it is found that some files are damaged and cannot be recovered. After analysis, it is concluded that the file damage is related to the existence of bad paths in some hard drives in the two groups of raid. The following are some of the bad hard drives in the two groups of raid:

R aid 1: 2# 67 bad source sectors encountered.

4# 13 bad source sectors encountered.

R aid 2: 2# 37 bad source sectors encountered.

V. Restoration conclusion

Due to the serious damage to the faulty hard disk, the hardware can not be repaired, and some hard drives have bad channels, which may lead to defects in the raid structure and damage to some files, but most of the files have been verified and recovered successfully, only a small number of files have been lost or damaged, and the data recovery has been completed successfully.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.