Shulou(Shulou.com)06/01 Report--

This article will explain in detail the example analysis of Linux array RAID for you. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

1. Detailed explanation of RAID 2. Introduction of mdadm tools 3. Basic process of creating a RAID 4. Management of disk arrays 5. RAID optimization

RAID details:

Description: RAID: (Redundant Array of indenpensive Disk) redundant array of independent disks: a disk array is an array of multiple disks, which is used as a single disk. It stores data in different disks in a striping manner. When accessing data, the relevant disks in the array act together to greatly reduce the access time of data and have better space utilization. Disk arrays use different technologies, called RAID level, and different level aims at different systems and applications to solve the problem of data security. Simply put, RAID combines multiple hard drives into a single logical sector, so the operating system only treats it as a single hard disk. Generally, high-performance disk arrays are achieved in the form of hardware. Disk access control and disk arrays are further combined on a controller (RAID controler) or control card to solve the four major requirements for disk input and output systems for different users: (1) increase access speed, (2) fault tolerance, that is, security (3) effective use of disk utilization. (4) balance the performance differences of CPU, memory and disk as far as possible to improve the overall performance of the host. Available features:

1. Redundancy (fault tolerance)

two。 Performance improvement RAID classification: 1. Hardware RAID: implemented with RAId interface card; the kernel needs to support its driver, and this kind of device is displayed as a SCSI device, code name / dev/sd*2. Software RAID: implemented with MD (multiple devices) module in the kernel. This kind of device is expressed under / etc/ as: md*; uses mdadm tools to manage soft RAID; in the current version of RH 5 (although raid can be simulated by software, the function and performance of disk control is not as good as that implemented by hardware. It is best to use several common RAID types of hardware raid in production environment:

The figure shows:

1. RAID-0: striping (stripe mode) requires at least two disks, and the size of the RAID partition should be the same (full play and advantages). While data is scattered on different disks, it can achieve concurrency when reading and writing, so its read and write performance is the best. But without fault tolerance, damage to any disk will damage all data.

2. RAID-1:mirroring (mirror volume) requires at least two hard disks, and the raid size is equal to the smallest capacity of the two raid partitions (it is best to divide the partition size into the same), which can increase the backup capacity of the hot spare; the data is redundant, and it is written to both hard disks at the same time to achieve data backup; but the write performance is relatively reduced, but the data can be read concurrently, almost similar to the reading efficiency of raid-0.

3. RAID-5: three or more hard disks are required, and hot spares can be provided for fault recovery. Parity is highly reliable, and the data will be completely damaged only when two hard disks are damaged at the same time. When only one hard disk is damaged, the system will reconstruct the data according to the stored parity bits and provide temporary services. If there is a hot spare, the system will automatically rebuild the data on the faulty disk on the hot spare.

Storage method: to put it simply, the first disk segment of the disk array is the check value, the second disk to the next disk and then back to the first disk is the data, and then the segment of the second disk is the check value. the segment from the third disk to the second disk is data, and so on until the data is played. In this way, the cyclic separation and storage of data and check values can achieve a certain function of fault reconstruction, but the control of raid-5 is more complex and a large number of check codes are calculated, which may cause additional computational burden to the system (for soft raid, hardware has its own data processing capability).

Note: fault tolerance in RAID means that even if the disk fails, the data can remain intact, allowing the system to access the correct data, while the SCSI disk array can swap the disk at work and automatically reconstruct the data of the failed disk.

Hot backup (hot spare or hot standby driver): in order to enhance the function of fault tolerance and enable the system to quickly rebuild data in the event of disk failure, in order to maintain the performance of the system, the general disk array system can use the function of hot backup (hot spare or hot standby driver). The so-called hot backup is to designate one of the disks as the backup disk when establishing the disk array system (configure). The disk does not operate normally. However, if a disk in the array fails, the disk array replaces the failed disk with backup disk, and automatically rebuilds the data of the failed disk (rebuild) on the backup disk. Because the response is fast and the cache memory reduces disk access, the data reconstruction can be completed quickly and has little impact on the performance of the system.

Do not use multiple areas on the same hard disk to do RAID at any time, which will not only not improve the performance of the system, but will greatly reduce the system capacity of the whole system.

After analyzing the above commonly used RAID types, we can see that RAID-0 can mainly improve disk performance, and RAID-1 can mainly achieve backup fault tolerance (RAID-5 is not used here because of its complex control mechanism), so we can consider combining RAID-0 and RAID-1 in the actual production environment to achieve high performance and high fault tolerance of disk storage system.

The two raid combination methods shown in the following figure are resolved:

For one: the bottom layer uses raid0 with two hard drives to achieve high concurrency, and then assembles two raid0 groups to achieve redundancy; any hard disk in any raid0 group in the lower layer will invalidate the reorganization, but one group outside the two can still provide all the data

For two: the bottom layer uses raid-1 to achieve digital redundancy, and the upper layer uses raid-2 to achieve high concurrency. The failure of any hard disk in this structure does not have a destructive impact on the overall data work of this group, so it feels that this scheme is more excellent and has some applications in actual production (the specific configuration process is attached at the end of the article)

Introduction to the mdadm tool:

Description: mdadm (multiple devices admin) is a standard soft raid management tool under linux, is a modeling tool (in different modes); the program works in the memory user program area, providing users with RAID interfaces to operate kernel modules to achieve various functions; RedHat has built this tool The official latest version should be 3.2. if you need the latest version, you can download the source code package from the official website or http://www.kernel.org/pub/linux/utils/raid/mdadm/ for compilation and installation (the official website has been unable to open o (╯□╰) o).

Experimental environment: RedHhat5.4; mdadm version is v2.6.9

There may be slight changes in different version options, please be careful to compare your own version of the man document when using it.

Basic syntax:

# mdadm [mode] [options]

Currently supported modes: LINEAR (linear mode), RAID0 (striping stripe mode), RAID1 (mirroring), RAID-4, RAID-5, RAID-6, RAID-10, MULTIPATH and FAULTY

LINEAR: linear mode, which is not the standard mode of raid, its main function is to combine several small hard drives into one large hard disk; array storage is full of one hard disk at a time to use the next hard disk; for the upper layer, it is a large hard disk.

7 modes: Assemble: Assembly mode: join a previously defined array; an array that can be made worthwhile or removed from other hosts

Build: create: create an array without super blocks Create: create a new array, each device has a super block

Follow or Monitor: monitors the status of RAID. Generally, it is only used for redundant modes such as RAID-1/4/5/6/10.

Grow: (Grow or shrink) change the capacity of the RAID or the number of devices in the array; contraction generally refers to data contraction or reconstruction

Manage: manage arrays (such as adding a space disk and deleting a failed disk)

Incremental Assembly: add a device to an appropriate array.

Misc: allows individual operations on a device in the array (such as erasing superblocks or stopping the array)

Auto-detect: this mode does not work on specific devices or arrays, but requires any automatically detected arrays to be started in the Linux kernel.

OPTIONS:

Select an option for a mode: (Options for selecting a mode)

-A,-- assemble: join and open a previously defined array

-B,-- build: create an array without super blocks (Build a legacy array without superblocks.)

-C,-- create: create a new array

-F,-- follow,-- monitor: select Monitor mode-

G,-- grow: change the size or shape of the active array-I

-- incremental: add a separate device to the appropriate array and may start the array

-- auto-detect: request the kernel to start any automatically detected array

Options that are not specific to a pattern: (Options that are not mode-specific)

-c,-- config=: specifies the configuration file, default is / etc/mdadm.conf

-s,-- scan: scan the configuration file or / proc/mdstat for missing information. Default profile: / etc/mdadm.conf

-h,-- help: help information, which is displayed when used in the above option

-v,-- verbose: displays details. Generally, it can only be used with-- detile or-- examine to display intermediate information.

-b,-- brief: fewer details. For-- detail and-- examine options

-- help-options: show more detailed help

-V,-- version: version information

Quiet mode; plus this option allows mdadm not to display purely informational information unless it is an important report

Options used when create build or grow:

-n,-- raid-devices=: specifies the number of device active in the array, excluding spare disks, which can only be modified by-- grow

-x,-- spare-devices=: specifies the number of redundant device of the initial array, i.e. the number of spare device

-c,-- chunk=: Specify chunk size of kibibytes. The default is 64. Chunk-size is an important parameter that determines the amount of data written to each disk in the array at a time.

(Chunk:, it can be understood as the size of each data segment when raid divides and stores data (usually the size of a number such as 32, 64, 128, etc.); it is very important to choose a reasonable chunk size, if the chunk is too large, the band space on one disk can satisfy most of the RAID O operations, so that the reading and writing of data is limited to one hard disk, which cannot give full play to the advantage of RAID concurrency. If the chunk setting is too small, any very small I _ sign O instruction may cause a large number of read and write operations, which can not give full play to the concurrency performance, occupy too much controller bus bandwidth, and affect the overall performance of the array. Therefore, when creating a band, we should reasonably choose the size of the band according to the needs of practical application. )

-z,-- the total amount of space size=: fetches from each device after building the RAID1/4/5/6; but the size must be a multiple of chunk, and you need to leave at least the size of 128KB for RAID's superblock at the end of each device.

-- rounding=: Specify rounding factor for linear array (= = chunk size)

-l,-- level=: sets several times the raid level.raid--

-create: available: linear, raid0, 0, stripe, raid1,1, mirror, raid4, 4, raid5, 5, raid6, 6, multipath, mp.

-- build: available: linear, raid0, 0, stripe

.-p,-- layout=: sets the parity rules of raid5 and raid10, and controls the fault mode; where the parity of RAID-5 can be set to: eft-asymmetric, left-symmetric, right-asymmetric, right-symmetric, la, ra, ls, rs. The default is left-symmetric

-parity: similar to-layout=

-- assume-clean: currently available only for-- build option

-R,-- run: when a portion of an array appears in another array or file system, mdadm acknowledges the array. This option will not be confirmed.

-f,-- force: usually mdadm does not allow an array to be created with only one device, and a device is used as the missing drive when creating a raid5. This option is just the opposite

-NMAE: set the name of the array

Administrative mode options (For Manage mode):

-a,-- add: add the listed devices to a working array; when the array is in a degraded state (failure state), you add a device that will be used as a backup device and start data reconstruction on that standby device. -r,-- remove: removes the listed device from the array and the device cannot be active (redundant disk or failed disk)

-set-faulty: Mark the listed device as faulty, and then remove the device. (it can be used as a test method for fault recovery)-- Modem: ditto

Monitoring mode options (For Monitor mode):

-m,-- mail: set a mail address to send a message to the mail when it is alarmed; this address can be written to the conf file and is effective on the boot array

-p,-- program,-- alert: runs a specified program when an event is detected

-y,-- syslog: set all events to be logged in syslog

-t,-- test: generates an test warning message for each array found at startup; this information is passed to the mail or alarm program; (to test whether the alarm message is received correctly)

MISC mode option: Usage: mdadm options. Devices...

-Q,-- query: look at a device and determine whether it is a md device or part of a md array

-D,-- detail: print the details of one or more md device

-E,-- examine: the basic process of printing the contents of md superblock on device to create a soft RAID:

(take three partition simulations to create a raid5 as an example, and the How-To of level0/1 is no longer written. If you understand the principle, the configuration is really simple.)

1. Generate raid composition: the array component unit in linux is a partition, and the partition can be the whole device or one of multiple device partitions. After the fdisk partition, you need to change the partition flag to Linux raid auto type.

# after the partition is as follows:

Device Boot Start End Blocks Id System

/ dev/sdb1 1 609 4891761 fd Linux raid autodetect

/ dev/sdc1 1 609 4891761 fd Linux raid autodetect

/ dev/sdd1 1 609 4891761 fd Linux raid autodetect

two。 Establish a disk array

# mdadm-C / dev/md0-a yes-l 5-n 3 / dev/sd {brecronomy d} 1

Mdadm: array / dev/md0 started.

-C: create an array followed by the array name

-l: specify the level of the array

-n: specifies the number of active devices in the array

3. View array status

[root@bogon ~] # cat / proc/mdstat

Personalities: [raid6] [raid5] [raid4]

Md0: active raid5 sdd1 [2] sdc1 [1] sdb1 [0]

9783296 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

Unused devices:

Check the status of all running RAID arrays through the cat / proc/mdstat information. In the first line, the MD device name md0,active and inactive options indicate whether the array can read / write, followed by the RAID level raid5 of the array, followed by the block device belonging to the array, the number in square brackets [] indicates the sequence number of the device in the array, (S) indicates that it is a hot spare, and (F) indicates that the disk is in faulty status. The next line is the size of the array, expressed in the number of blocks, followed by the size of chunk-size, and then the type of layout. Different RAID levels have different types of layout. [3 UUU] indicates that the array has 3 disks and all 3 disks are operational, while [2can3] and [_ UU] indicate that 2 of the 3 disks in the array are operational, and the disk in the position corresponding to the underscore is faulty.

View the details of the array:

[root@bogon] # mdadm-- detail / dev/md0

/ dev/md0:

Version: 0.90

Creation Time: Tue Mar 15 08:17:52 2011

Raid Level: raid5

Array Size: 9783296 (9.33 GiB 10.02 GB)

Used Dev Size: 4891648 (4.67 GiB 5.01 GB)

Raid Devices: 3

Total Devices: 3

Preferred Minor: 0

Persistence: Superblock is persistent

Update Time: Tue Mar 15 08:20:25 2011

State: clean

Active Devices: 3

Working Devices: 3

Failed Devices: 0

Spare Devices: 0

Layout: left-symmetric verification rule

Chunk Size: 64K

UUID: e0d929d1:69d7aacd:5ffcdf9b:c1aaf02d

Events: 0.2

Number Major Minor RaidDevice State

0 8 17 0 active sync / dev/sdb1

1 8 33 1 active sync / dev/sdc1

2 8 49 2 active sync / dev/sdd1

4. Mdadm.conf configuration: mdadm.conf is the default configuration file of the software, and its main function is to facilitate tracking the configuration of soft RAID, especially to configure monitoring and event reporting options. Write the details of the composition of each array, which is used to assemble the array according to this file next time, otherwise the members of the array need to be manually defined when the array is turned on. Of course, it is recommended to create this file to prevent unexpected situations. For detailed configuration and examples, please see man document # man mdadm.conf.

[root@bogon ~] # echo "DEVICE / dev/sdb1 / dev/sdc1 / dev/sdd1" > > / etc/mdadm.conf

[root@bogon ~] # mdadm-Ds > > / etc/mdadm.conf

[root@bogon ~] # echo "MAILADDR mospiral@gmail.com" > > / etc/mdadm.conf

MAILADDR specifies the email address of the monitoring system when there is a problem.

# the format is as follows:

DEVICE / dev/sdb1 / dev/sdc1 / dev/sdd1

ARRAY / dev/md0 level=raid5 num-devices=3 metadata=0.90 UUID=e0d929d1:69d7aacd:5ffcdf9b:c1aaf02d

MAILADDR mospiral@gmail.com

# DEVICE line indicates that when the array is turned on according to the configuration file, look up the super fast information of those devices; if there is no such line

Search for super fast information on all device partitions in mtab; so you don't have to write it, but just write it and add spare later.

When you use a device, you need to modify the career change information at the same time.

# ARRAY line indicates basic information such as raid name, level uuid, etc.

# you can add monitoring alarm information in specified monitor status, such as MAILADDR and PROGRAM.

Disk array management: various disk management tasks can be performed in manage mode

Add a space disk to raid-5:

[root@bogon] # mdadm-a / dev/md0 / dev/sda5

Mdadm: added / dev/sda5

View the status at this time:

[root@bogon ~] # cat / proc/mdstat

Personalities: [raid6] [raid5] [raid4]

Md0: active raid5 sda5 [3] (S) sdd1 [2] sdc1 [1] sdb1 [0]

9783296 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

Unused devices:

Simulate a hard drive failure:

[root@bogon] # mdadm-f / dev/md0 / dev/sdd1

Mdadm: set / dev/sdd1 faulty in / dev/md0

# check the status at this time and find that the hot spare just now has replaced the location of the failed disk

# and the progress bar shows the process of data reconstruction:

[root@bogon ~] # cat / proc/mdstat

Personalities: [raid6] [raid5] [raid4]

Md0: active raid5 sda5 [3] sdd1 [4] (F) sdc1 [1] sdb1 [0]

9783296 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]

[>] Recovery = 1.3% (66912amp 4891648) finish=6.0min speed=13382K/sec

Unused devices:

Hot remove the failed hard drive:

[root@bogon] # mdadm-r / dev/md0 / dev/sdd1

Mdadm: hot removed / dev/sdd1

[root@bogon ~] # cat / proc/mdstat

Personalities: [raid6] [raid5] [raid4]

Md0: active raid5 sda5 [3] sdc1 [1] sdb1 [0]

9783296 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]

[>.] Recovery = 16.1% (792136 + 4891648) finish=4.6min speed=14828K/sec

Unused devices:

For redundant raid, data can generally be rebuilt within a period of time when a single disk fails, but when the amount of data is very large, the reconstruction will be very slow, and the system pressure in the reconstruction process is relatively high, so it is necessary to pay more attention to the system load to prevent errors in the reconstruction process; in the thermal removal of a failed disk, it is also necessary to replace a new hard disk as soon as possible and add a Spare disk. After fault repair and reconstruction, the configuration file needs to be regenerated to prevent it from being opened according to the original definition mode the next time it is opened.

Stop RAID:

# specify to stop an array

[root@bogon] # mdadm-S / dev/md0

# stop all arrays defined in the configuration file

[root@bogon ~] # mdadm-Ss

Mdadm: stopped / dev/md0

#-s-scan to query the configuration file or query all arrays in the mdstat when there is no configuration file

Enable RAID:

[root@bogon ~] # mdadm-As

Mdadm: / dev/md0 has been started with 2 drives (out of 3).

#-s-scan: open all arrays according to the configuration file, which has been restored due to failure

# but I cannot load the newly added hard disk on the array because the configuration file has not been rebuilt; (need to be stopped and re-specified manually)

[root@bogon ~] # cat / proc/mdstat

Personalities: [raid6] [raid5] [raid4]

Md0: active raid5 sdb1 [0] sdc1 [1]

9783296 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]

Unused devices:

If there is no configuration file at this time, you need to specify the device name manually:

[root@bogon] # mdadm-A / dev/md0 / dev/sdb1 / dev/sdc1 / dev/sda5

Mdadm: / dev/md0 has been started with 3 drives.

# Note: after a raid device is added to the md array, the md information will be written to the superblock of the device partition

# during manual assembly, the mdadm tool automatically verifies whether the array configuration is legal and takes appropriate actions

If you take over a new raid, do not have a configuration file, or forget the exact composition of the device, you need to check whether it is a raid device and other information one by one, and then assemble the array according to the information:

[root@bogon] # mdadm-E / dev/sdb1

/ dev/sdb1:

Magic: a92b4efc

Version: 0.90.00

UUID: e0d929d1:69d7aacd:5ffcdf9b:c1aaf02d

Creation Time: Tue Mar 15 08:17:52 2011

Raid Level: raid5

Used Dev Size: 4891648 (4.67 GiB 5.01 GB)

Array Size: 9783296 (9.33 GiB 10.02 GB)

Raid Devices: 3

Total Devices: 3

Preferred Minor: 0

Update Time: Tue Mar 15 09:25:10 2011

State: clean

Active Devices: 3

Working Devices: 3

Failed Devices: 0

Spare Devices: 0

Checksum: b0cd088f-correct

Events: 8

Layout: left-symmetric

Chunk Size: 64K

Number Major Minor RaidDevice State

This 0 8 17 0 active sync / dev/sdb1

0 0 8 17 0 active sync / dev/sdb1

1 1 8 33 1 active sync / dev/sdc1

2 2 8 5 2 active sync / dev/sda5

# what is displayed here is the md information contained in the partition superblock; if there is no configuration file, you can assemble the md according to this information

Delete the array:

If you need to completely erase this array:

[root@bogon ~] # umount / dev/md0

Mdadm-Ss / dev/md0

[root@bogon] # mdadm-- zero-superblock / dev/sd {brecronomy d} 1

#-when zero-superblock adds this option, it will determine whether the array is a package or not

# contains a valid array super fast, if any, erase the array information in the super block.

[root@bogon ~] # rm / etc/mdadm.conf

RAID Optimization:

(1) set stride value

The stride is the software RAID device's chunk-size in filesystem blocks.For example,with an ext3 filesystem that will have an 4KB block size on a RAID device with a chunk-size of 64KB, the stride should be set to 16: (the translation is very complicated, so the original text of the textbook is pasted. )

Mk2fs-j-b 4096-E stride=16 / dev/md0

# when setting, you need to extend it with the-E option

A well-set stride value can reduce the burden on block calculation when writing data in later use, thus improving RAID performance.

Attached: the method of RAID 1-0 two-tier architecture:

First create two underlying RAID-1

[root@bogon] # mdadm-C / dev/md0-a yes-l 1-n 2 / dev/sd [bc] 1

Mdadm: array / dev/md0 started.

[root@bogon] # mdadm-C / dev/md1-a yes-l 1-n 2 / dev/sd [bc] 2

Mdadm: array / dev/md1 started.

Implement the upper RAID-0 with two RAID-1:

[root@bogon] # mdadm-C / dev/md2-a yes-l 0-n 2 / dev/md [01]

Mdadm: array / dev/md2 started.

View the array status:

[root@bogon ~] # cat / proc/mdstat

Personalities: [raid6] [raid5] [raid4] [raid0] [raid1]

Md2: active raid0 md0 [0] md1 [1]

9783232 blocks 64k chunks

Md1: active raid1 sdb2 [0] sdc2 [1]

4891712 blocks [2/2] [UU]

Md0: active raid1 sdb1 [0] sdc1 [1]

4891648 blocks [2/2] [UU]

Unused devices:

Create a profile:

[root@bogon ~] # mdadm-Ds > / etc/mdadm.conf

Stop and turn on the array:

[root@bogon ~] # mdadm-Ss

Mdadm: stopped / dev/md2

Mdadm: stopped / dev/md1

Mdadm: stopped / dev/md0

[root@bogon ~] # mdadm-As

Mdadm: / dev/md0 has been started with 2 drives.

Mdadm: / dev/md1 has been started with 2 drives.

Mdadm: / dev/md2 has been started with 2 drives.

# # during the above shutdown and opening process, the system can only identify the level, and close the upper layer first.

# # enable the upper layer to prevent conflicts

This is the end of this article on "sample Analysis of Linux Array RAID". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.