Shulou(Shulou.com)06/03 Report--

I. introduction to disk management

Format of the hard drive:

The hard disk is divided into two formats: MBR and GPTMBR: on the track 0 sector of the hard disk, the sector size is 512 bytes and is specially used to store MBR boot information. MBR is divided into three parts: 446bytes: storage boot loader, boot loader 64 bytes: storage partition table, how many partitions are there, where each partition starts and ends Every 16 bytes represents a partition, so there can only be a maximum of four primary partitions of 2 bytes on a hard disk in MBR format: a valid identification of the previous information, and an end flag. Expressed as 55aa with four hexadecimal numbers, it means that the first 510 bytes are valid, that is, the partition is valid, and if these two values are null, then even if the partition is established, it is not valid. Because there are only 64 bytes to store partition information, the hard disk in MBR format can be divided into up to four primary partitions or three primary partitions plus one extended partition, which in turn can be divided into multiple logical partitions. It should be noted that MBR supports a maximum of 2T partitions, and more than 2T MBR cannot be recognized, so GPT is required to implement GPT: compatible with MBR, a maximum of 128primary partitions can be created, because there are enough primary partitions to no longer need to expand partitions. at the same time, GPT also breaks MBR's concept of up to 2T partitions.

Common tools for disk management

MBR Management tool: fdisk

GPT Management tool: gdisk

Advanced management tool: parted

2.fdisk: create disk partition management tool, MBR format file system management tool, poor disk compatibility for GPT format

Fdisk-l: view partition information fdisk / dev/device_name: managing disk partitions fdisk creating partitions opens an interactive window There are many subcommands m: check which commands are available p: display the partition information of the current disk n: create a new partition p: primary partition e: for extended partition d: delete partition w: save partition information and exit Q: do not save partition information Exit t: modify the partition's system ID l: list all systems on the system ID L: list all supported partition types or hexadecimal ID represented by the partition type

3.gdisk: create disk partition management tool, GPT format file system management tool, and fdisk operation is basically the same

Gdisk-l: view partition information gdisk / dev/device_namegdisk create a partition will open an interactive window Many of these commands?: display the list of supported commands p: show the partition information of the current disk n: create a new partition d: delete partition w: save partition information and exit Q: do not save partition information exit l: list all system ID on the system, that is, all partition types

4.parted: integrated management

Usage: parted [option]. [device [command [parameter].].] Parted / dev/sdb mklabel gpt | msdos: specify the type of partition parted / dev/sdb print: print the current partition information parted / dev/sdb mkpart primary 1200 (default M): partition primary (primary partition) extended (extended partition), logical (logical partition) parted / dev/sdb rm 1: delete the partition, this deletion is based on the partition number parted-l lists the partition information

Normally, the hard disk we add directly when the system is powered on will not be recognized by the system. Use the following command to let the system reread the hard disk information.

Echo'--- > / sys/class/scsi_host/host0/scan: triggers the hardware to re-identify the hard disk

For the partition that has just been created, the system will not recognize it. It needs to be identified manually, that is, we need to synchronize the partition table.

Synchronize the partition table to see if the kernel has recognized the new partition cat / proc/partations centos6 informs the kernel to reread the hard disk partition table new partition with partx-a / dev/DEVICE: option to add partition That is to add a partition to this device to make the added partition take effect kpartx-a / dev/DEVICE-f: force (force) delete partition use partx-d-nr Mmuri N (partition serial number) / dev/DEVICE: delete partition CentOS 5L7: use partprobe: synchronize partition information Partprobe the hard disk information with the partition information in memory [/ dev/DEVICE]: to synchronize without adding devices is to refresh the partition information of all devices by default. Add devices to refresh only the partition information of specified devices Note: the hard disk partition used now must be a continuous space in the hard disk.

5. Common file system classification

Commonly used Linux file system: ext2 (Extended file system): it is suitable for those situations where the partition capacity is not too large and updates are not frequent. For example, / boot partition ext3: is an improved version of ext2, which supports logging and can help the system recover from anomalies caused by abnormal shutdown. It is commonly used as the general file system ext4: the latest version of the ext file system. Provides a number of new features, including nanosecond timestamps, creation and use of jumbo files (16TB), file systems with maximum 1EB, and speed gains (for use with gpt) xfs:SGI File systems supporting maximum 8EB: btrfs (Oracle), reiserfs, jfs (AIX), swap CD: iso9660Windows:FAT32, exFAT,NTFSUnix: FFS (fast), UFS (unix), JFS2 Network File system: NFS, CIFS Cluster File system: GFS2, OCFS2 (oracle) distributed File system: fastdfs,ceph, moosefs, mogilefs, glusterfs, Lustre

6. Tools for creating file systems:

Mkfs command the first way: directly use the file system suffix commands mkfs.btrfs, mkfs.ext2, mkfs.ext4, mkfs.minix, mkfs.vfat, mkfs.cramfs, mkfs.ext3, mkfs.fat, mkfs.msdos, mkfs.xfs the second way:-t: specify the file system type-L: set the volume label, equivalent to comments for the swap file system swap, there are specific tools But to change the system ID of the disk to 8emkswap / dev/device_nameblkid: view the attribute information of the specified device-U: find the corresponding device according to the UUID-L: find the corresponding device according to the volume label tune2fs: adjust the relevant properties of the file system-l: view the information in the super block-L: set or modify the volume label-m: adjust Integral reservation percentage-r: specify the number of reserved blocks-o: set the default hanging option-c #: self-test after the specified mount times reaches # times 0 or-1 to turn off this feature-I #: self-test after how many days each mount, 0 or-1 to turn off this feature dumpe2fs: display file system attribute information-h: display only information in super blocks File system Detection: fsck: check and repair the Linux file system Files become incomplete after repair-t: specify file system type-a: automatic repair-r: interactive repair e2fsck: dedicated to repair ext series file systems-y: custom reply to problems yes-f: forced check-p: automatic repair

7. Super block

The function is to store the size of the file system, how many are empty and filled, as well as their respective total and other information. if you use a partition to access data, the first one to access is the super block. so if the super block is broken, then the disk basically can not save the super block to occupy the first physical block, which is the control block of the file system. Super blocks are used to store metadata, that is, file systems that belong to a block from several to several, such as size, number of free blocks, index tables of free blocks, number of free iNode nodes, free iNode index tables, and so on. In order to prevent super block damage, you need to back up the super block so that it can be repaired if it is damaged.

8. Mounting and unmounting of file system

Mount: associate the extra file system with an existing directory in the root file system, thus unmounting this directory as an entry for other file access: the process of disassociating this association: mount: mount tool format: mount device mount point the root file system gives temporary mount points by default: / mnt, / opt, / media can also be mounted to other directories But the mount directory must exist beforehand and preferably empty, otherwise the files under the directory will be hidden Commonly used mount mount options:-t: specify file system type-r: read-only mount, directory contents can only be read-write mount, directory contents can be read-writable, modifiable-L label: specify device by volume label, mount-L label mount point, rarely use-U UUID: specify device as UUID, usually written in / etc/fstab file Commonly used-a: automatically mount all devices in / etc/fstab file, scan automatically, mount those that are not mounted, ignore those that have been mounted-n: do not update / etc/mtab file when mounting-B: bind to a directory-o option:mount-o option is rich and colorful, customize many available mount options Remount: re-mount, add additional mount options without unmounting ro: read-only mount rw: read-write mount async: asynchronous Imax O, data write operations are completed in memory first, and then synchronized to the disk as a whole sync: synchronous Imao Atime/noatime: whether to modify the access time of the file, which can be turned off when pursuing character, because modifying the access time will also have write disk operation dirname/nodirname: whether to modify the access time of the directory auto/noauto: whether the device supports automatic mount Mount-an option dev/nodev: whether to support the use of device files on this device exec/noexec: whether to allow the execution of binary programs on this device suid/nosuid: whether to support files on secondary devices with special permissions user/nouser: whether to support ordinary users to hang on this device Acl: whether ACL is supported on this device Centos7 supports it by default. Previous versions do not support mounting with multiple options at the same time. Use comma separation to mount without option default mount option: rw,suid,dev,exec,auto,nouser,async swap partition mount: swapon: enable swap partition-a: make / etc/fstab Effective for all switching devices marked with 'sw'-s: displays all swap partition information on the system-p #: priority can be specified swapoff: disable one swap partition-a: disable all free: view memory and swap usage information-m: display in MB Show-g: display the unmount command in GB to view the mount findmnt MOUNT_POINT | device: confirm whether it is the mount point to view the processes accessing the specified file system lsof MOUNT_POINT fuser-v MOUNT_POINT terminates all processes accessing the specified file system Fuser-km MOUNT_POINT: close all directories that use this process and uninstall umount DEVICE umount MOUNT_POINT

9. File system Space View tool: df,du

Df: list the overall disk usage of the file system-a: list all file systems, including system-specific / proc file systems;-k: display each file system in the capacity of KBytes;-m: display each file system in the capacity of MBytes;-h: display itself in GBytes, MBytes, KBytes and other formats that are easy for people to read -H: replace the filesystem 1024K carry way with MIMO 1000K;-T: show the file system type, together with the filesystem name of the partition (such as ext3) also listed;-I: do not use the hard disk capacity, and the number of inode to display du: check disk space usage-a: list all files and directories capacity, because the default statistics is only the number of files under the directory. -h: shown in the capacity format that is easy for people to read (Ghip M);-s: list the total, but not the capacity of each individual directory;-S: does not include the total under the subdirectory, which is slightly different from-s. -k: display in KBytes list capacity;-m: list capacity in MBytes; dd: copy a file with a block of specified size and perform the specified conversion at the same time. Parameter: if= file name: input file name, default is standard input. That is, specify the source file. Of= file name: output file name, default to standard output. That is, specify the destination file. Bs=bytes: set the read / output block size to bytes bytes at the same time. Count=blocks: only blocks blocks are copied, with the block size equal to the number of bytes specified by ibs. Commonly used are the above four examples: dd if=/dev/zero of=/root/file1 bs=1 count=1G, will generate a 1G file dd command can test the disk IO rate, the above example can be seen in ibs=bytes: read bytes bytes at a time, that is, specify a block size of bytes bytes. Obs=bytes: output bytes bytes at a time, specifying a block size of bytes bytes. Cbs=bytes: convert bytes bytes at a time, that is, specify the conversion buffer size. Skip=blocks: skip blocks blocks from the beginning of the input file before you start copying. Seek=blocks: skip blocks blocks from the beginning of the output file before you start copying. Conv=conversion: converts the file with the specified parameters. Ascii: convert ebcdic to ascii ebcdic: convert ascii to ebcdic ibm: convert ascii to alternate ebcdic block: convert each line to cbs in length, and fill unblock with blanks in the deficiency: make each line cbs in length Fill in lcase with blanks: convert uppercase characters to lowercase characters ucase: convert lowercase characters to uppercase characters swab: swap each pair of bytes entered noerror: do not stop in case of error notrunc: not cut Short output file sync: populate each input block to ibs bytes The deficiency is filled with NUL characters.

10. Realize automatic mount on boot

The above is temporarily mounted, but the restart will fail. If you want to boot and mount automatically, you need to write to the file. In fact, if Linux wants to save the configuration permanently, it is written as a file / etc/fstab: the entries recorded in the file correspond to different devices, and the file will be read after the system starts. The format of the device file recorded in the automatic mount file: each behavior is the mount information of a device. The information in one line is: / etc/fstab each line defines a file system to be mounted 1, device to mount or pseudo file system device file LABEL:LABEL= "" UUID:UUID= "" pseudo file system name: proc, sysfs 2, mount point 3, file system type: ext4,xfs,nfs None 4. Mounting options: defaults Acl,bind 5, dump frequency: 0: no backup 1: dump every day 2: dump every other day 6, the order of file systems checked by fsck: the allowed number is 0,1, and 20: no self test 1: self test first Generally speaking, only rootfs is used 2: non-rootfs use

Second, the introduction of soft raid

1. What is raid?

RAID is the abbreviation of "Redundant Array of Independent Disk", which means independent redundant disk array in Chinese. Redundant disk array technology was born in 1987 and was proposed by the University of California, Berkeley. The simple explanation is to combine N hard drives through RAID Controller (sub-Hardware,Software) into a virtual single large-capacity hard disk to use. The adoption of RAID brings great benefits to the storage system (or the built-in storage of the server), among which improving the transmission rate and providing fault tolerance are the biggest advantages.

RAID disk Array (Redundant Array of Independent Disks) is characterized by faster reading speed and fault-tolerant Fault Tolerant on N hard disks at the same time, so RAID mainly solves the storage speed problem of accessing data (Storage) rather than backup problem (Backup Solution). To put it simply, RAID is a combination of multiple independent hard drives (physical hard drives) in different ways to form a hard disk group (logical hard disk), thus providing higher storage performance and data backup technology than a single hard disk. RAID can be divided into different levels according to the different combinations of disk displays. Different technologies used in disk arrays for different applications are called RAID level, and each level represents a different technology. At present, the accepted standard in the industry is RAID 0~RAID 5.

Features of 2.raid:

The storage capacity is expanded, which can be composed of multiple hard disks with huge storage space.

Reduced cost per unit capacity the price per megabyte of the largest hard disk on the market is much higher than that of universal hard drives, so the unit price of an array consisting of multiple popular hard drives is much lower.

Improving the storage speed the improvement of the speed of a single hard disk is limited by the technical conditions of each period, so it is often difficult to go further, while using RAID, multiple hard disks can share the read or write operation of data at the same time, so the overall speed is increased exponentially.

Reliable RAID system can use two sets of hard drives to complete mirror storage synchronously, which is the most important security measure for network servers.

One of the key functions of fault-tolerant RAID controller is fault-tolerant processing. If there is an error in a single hard disk in the fault-tolerant array, it will not affect the continued use of the whole, and the advanced RAID controller also has the function of saving data.

For IDE RAID, there is currently another feature that supports ATA/66/100.

How to implement 3.raid:

External disk arrays: provide adaptation through expansion cards

Built-in RAID: the motherboard integrates RAID controller, which is configured in BIOS before installing OS

Software RAID: implemented through OS

The level of 4.raid

Raid level: raid-0: distributed storage data read and write performance improved fault-free ability minimum number of disks 2 minus 2 + disk utilization 100% raid-1: mirror storage data read performance improved, There is a slight decline in write performance with a minimum of 2 disks with redundancy. 2 N free space 50% raid-5: improved read and write performance with redundancy Allow a maximum of one disk to damage and minimize the number of disks 3 and 3 + free space: NMUI 1 raid-6: read and write performance improvement and fault tolerance Allow a maximum of two disks to damage the minimum number of disks 4 raid-1 4 + available space: NMel 2 raid-1 0 = = raid-0 1: do 1 first do 0 or do 0 first improve read and write performance with fault tolerance. Each set of mirrors can only be broken by a maximum of one piece, with a minimum of 4 disks. 4 + Multi-implementation software radio mdadm tool for 100% free space raid-10: provides management for software raid-V: version information-v: details at creation-Q: silent mode creation option Item:-C-l: specify raid level-n: number of devices-a:yes | no Whether to automatically create a device file-c: specify chunk size-x: specify the number of free disks management option:-- add-r: remove a device-f: simulate a device failure

5. Create a soft raid

Requirements: create a RAID1 device with 1G of free space, the file system is ext4, have a free disk, and boot can automatically mount to the / backup directory

It takes two disks to create raid1. Choose sdb sdc for exercise here, and use sdd for free disk.

First use the fdisk tool to partition the hard disk, and change id to raid.

W Save partition information sdc and sdd use the same operation to create partitions because they are organized and used together. We format the raid after we have created it. We do not need a single disk format. Create a raid and view it.

Write to the file system

Configuration / etc/fstab file to realize automatic mount on boot

6. Clear soft raid

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.