Shulou(Shulou.com)06/03 Report--

Data recovery failure description:

The original storage is a Linux RAID6 composed of 12 2T hard drives, and the file systems are all EXT3. There are 3 LUN on this storage, each of which is the size of 6TB. One day, after the RAID fails, the maintenance personnel assign RAID to the failed storage in order to rescue the data, and initialize it.

After a long time of initialization, the maintenance staff sensed that the situation was different and forced to stop initialization, but the initialization has reached more than 50%. There is irreversible damage to the data section.

Data recovery failure analysis:

The cause of the failure is only the failure of RAID. During the subsequent rescue of the data, the maintenance personnel used 11 hard drives to redistribute the RAID5 and initialized it for a long time, which caused irreversible damage to the original data. It was later proved that only the third LUN could recover the data using the ordinary RAID6 method, but the third LUN did not have the important data that customers wanted, and the important data was mainly concentrated in the first LUN.

As the fault of this case is extremely complex, our company has changed hands among domestic data recovery companies for many times when we received repairs from customers, including a number of well-known data recovery companies, which have not yet been solved.

Data recovery process:

The recovery process is divided into four steps:

1. Analyze the RAID and disk organization structure of the original 12 disk RAID6.

2. Analyze the organizational structure of RAID and disk when redistributing raid 5.

3. Judge the recoverability, and how to realize the algorithm of the recovery program.

Recovery and repair.

The structure of the original RAID6 was quickly analyzed, but it took a whole day to analyze the structure of redistributed RAID5 because of the overlap of a large amount of information between the underlying RAID6 and RAID5.

The first and second steps have been completed. After analysis, the data destroyed by initialization can be restored by other methods, and the recovery algorithm can be worked out. It takes one day to write the program and correct the program algorithm. The program mirrors the first and second LUN of the original data in 12 disks to the storage of the two 7TB respectively.

It has been verified that the second LUN data is completely normal, but the most important first LUN has about the destruction of 10MB data, which before 10MB data is very fatal, the root directory of EXT3 and the I nodes of the first block group are all in the front 10MB, and then the recovery results of UFS Explorer and R-Studio, which are commonly used software for data recovery, are not ideal, which may be the reason for large storage.

In this case, we have to repair the damaged EXT3 file system, write a program to find the EXT3 orphan directory, find three directories under the root directory, rebuild the root directory and I node, and open it with the file system parser, but in order to ensure some permissions and attributes of the original data, LINUX can be mounted normally in LINUX. Then on the partition of a single disk where LINUX copies the files to the formatted EXT3 with the cp command. In this way, when customers use the data, they no longer need any other settings, and after direct cp, the file directory structure and properties are exactly the same.

Data recovery conclusion:

It took 6 days to recover the data successfully.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.