Shulou(Shulou.com)06/01 Report--

Linux disk Management Advanced

Dd command

SWAP and Mobile Media

Disk quota

RAID

Dd command dd-convert and copy a fileCopy a file, converting and formatting according to the operands.

The basic usage of the dd command:

Ddif=/PATH/FROM/SRC of=/PATH/TO/DESTbs=#:block size, copy unit size count=#: copy how many bsskip=blocks ignore blocks ibs size blocks seek=blocks ignore blocks obs size blocks from the beginning

Common options:

Conv=conversion [, conversion...] Converts the file with the specified parameters. Lcase converts uppercase characters to lowercase characters ucase converts lowercase characters to uppercase characters every pair of bytes swapped input noerror error does not stop notrunc does not truncate the output file sync fills each input block to ibs bytes, the deficiency is filled with NUL characters

Use the dd command to manipulate the disk, preferably with block device files

Examples of use:

Create two files:

[root@centos7 ~] # echo {1... 9} | tr-d [: space:] > F1 [root@centos7 ~] # echo {a.. z} | tr-d [: space:] > f2 [root@centos7 ~] # cat f1123456789 [root@centos7 ~] # [root@centos7 ~] # cat f2abcdefghijklmnopqrstuvwxyz [root@centos7 ~] # [root@centos7 ~] # [root@centos7 ~] # dd if=f1 of=f2 bs=1 count=3 3room0 records in3+0 records out3 bytes (3B) copied, 0.00136469 s 2.2 kB/s [root@centos7 ~] # cat f2123 [root@centos7 ~] # # copied 3 bytes from F1 to cover f2 [root @ centos7] # dd if=f1 of=f2 bs=1 count=3 skip=3 seek=33+0 records in3+0 records out3 bytes (3B) copied, 0.000396399 s 7.6 kB/s [root@centos7 ~] # cat f2abc456 [root@centos7 ~] # # skipped 3 bytes from F1 and copied the subsequent 3 bytes inserted after the 3 bytes of f2 [root@centos7 ~] # dd if=f1 of=f2 bs=1 count=3 skip=3 seek=3 conv=notrunc # does not truncate the content after f2 3: 0 records in3+0 records out3 bytes (3 B) copied, 0.00277328 s, 1.1 kB/s [root@centos7 ~] # cat f2abc456ghijklmnopqrstuvwxyz [root@centos7 ~] #

Backup MBR:

[root@centos7] # dd if=/dev/sda of=/testdir/sda.mbr bs=512 count=11+0 records in1+0 records out512 bytes (512B) copied, 0.000380182 s, 1.3 MB/s [root@centos7] #

Disk copy:

[root@centos7 ~] # dd if=/dev/sda of=/dev/sdb

Destroy the bootloader in MBR:

[root@centos7 ~] # dd if=/dev/zero of=/dev/sda bs=64 count=1 seek=4461+0 records in1+0 records out64 bytes (64B) copied, 0.00180836 s, 35.4 kB/s [root@centos7 ~] #

Destroy the partition table in MBR:

[root@centos7 ~] # dd if=/dev/zero of=/dev/sda bs=1 count=64 skip=446 seek=44664+0 records in64+0 records out64 bytes (64B) copied, 0.00589307 s, 10.9 kB/s [root@centos7 ~] #

It can be seen that the partition of sda has been destroyed!

Repair the sda partition table:

[root@centos7 ~] # dd if=/testdir/sda.mbr of=/dev/sda bs=512 count=1 1: 0 records in1+0 records out512 bytes (512B) copied, 0.000348826 s, 1.5 MB/s [root@centos7] # partprobe # synchronous partition table [root@centos7 ~] #

Sda was successfully repaired!

Backup:

Dd if=/dev/sdx of=/dev/sdy backs up the local / dev/sdx whole to / dev/sdydd if=/dev/sdx of=/path/to/p_w_picpath / dev/sdx full data to the p_w_picpath file dd if=/dev/sdx in the specified path | gzip > / path/to/p_w_picpath.gz backup / dev/sdx full data, and use the gzip tool to compress Save to the specified path [root@centos7 ~] # dd if=/dev/sdb1 | gzip > / testdir/p_w_picpath.gz409600+0 records in409600+0 records out209715200 bytes (210MB) copied, 9.02236 s, 23.2MB/s [root@centos7 ~] # ll / testdir/p_w_picpath.gz-rw-r--r--. 1 root root 139129150 Aug 29 08:16 / testdir/p_w_ picpath.gz [root @ centos7 ~] #

Restore:

Dd if=/path/to/p_w_picpath of=/dev/sdx restores backup files to specified disk gzip-dc / path/to/p_w_picpath.gz | dd of=/dev/sdx restores compressed backup files to specified disk [root@centos7 ~] # gzip-cd / testdir/p_w_picpath.gz | dd of=/dev/sdc409600+0 records in409600+0 records out209715200 bytes (210MB) copied, 6.16743 s 34.0 MB/s [root@centos7 ~] # copy memory materials to hard disk dd if=/dev/mem of=/root/mem.bin bs=1024 copy data in memory to mem.bin file in root directory copy iso image dd if=/dev/cdrom of=/root/cd.iso copy CD data to root folder And save as cd.iso file to destroy disk data dd if=/dev/urandom of=/dev/sda1 fills the hard disk with random data, which can be used to destroy data in some necessary situations. "after this operation, / dev/sda1 cannot be mounted, and the create and copy operations cannot be performed."

Get the most appropriate block size: by comparing the command execution time shown in the dd instruction output, the optimal block size size of the system can be determined.

Test the read / write speed of the hard disk: the read / write speed of the test hard disk can be calculated from the execution time of the output of the last two commands

[root@centos7 ~] # dd if=/root/1Gb.file bs=64k | dd of=/dev/null [root@centos7 ~] # dd if=/dev/zero of=/root/1Gb.file bs=1024 count=1000000

Repair disk

[root@centos7 ~] # dd if=/dev/sda of=/dev/sda

When the hard disk is not in use for a long time (for example, 1 or 2 years), degaussing points will occur on the disk. It is difficult for the head to read these areas and may result in an Icano error. When this situation affects the first sector of the hard disk, it may cause the hard disk to be scrapped. The above command has the potential to bring the data back to life. And the process is safe and efficient.

System quota

SWAP

Swap partition is a supplement to the system RAM (bare device). The basic commands involved are: mkswap, swapon, swapoff, whose priority ranges from 0 to 32767. If the user does not specify the priority of the swap, the kernel will assign a priority to the swap, starting from-1, with each additional priority decreasing by one; the priority specified by the user (positive) is always higher than that specified by the core by default (negative). The swap partition can be an entire hard disk, a partition (to specify the partition type), or create a file as a swap partition.

Mkswap-set up a Linux swap areaswapon, swapoff-enable/disable devices and files for paging and swapping

Basic settings include:

Create a swap partition or file

Write a special signature using mkswap

Add the appropriate entry in the / etc/fstab file

Activate swap space using swapon-a

That is, swap partition-> format-> mount-- > activate

Create a swap partition:

[root@centos7 ~] # dd if=/dev/zero of=/swapfile bs=1M count=10241024+0 records in1024+0 records out1073741824 bytes (1.1 GB) copied, 1.75694 s, 611 MB/s [root@centos7 ~] # ls

Format the swap partition:

[root@centos7 ~] # mkswap / dev/sdc1Setting up swapspace version 1, size = 10485756 KiBno label, UUID=a0ac8c08-23b9-4672-bee3-449e949cea8c [root@centos7 ~] # mkswap / swapfile Setting up swapspace version 1, size = 1048572 KiBno label, UUID=4659cff5-5718-4119-a70e-6283ca443509 [root@centos7] #

Write / etc/fstab

[root@centos7 /] # swapon-a # activate swap type in / etc/fstab [root@centos7 /] # swapon-sFilename Type Size Used Priority/dev/sdc1 partition 10485756 0 10/swapfile file 1048572 0-1

So, of course, you can imagine that the mkfs command should "make" a file into a normal partition and mount it:

[root@centos7 ~] # dd if=/dev/zero of=/testdir/partfile bs=1M count=11+0 records in1+0 records out1048576 bytes (1.0 MB) copied, 0.00416736 s 252 MB/s [root@centos7 ~] # mkfs.ext4 / testdir/partfile # mkfs.xfs No [root@centos7 ~] # blkid / testdir/partfile/testdir/partfile: UUID= "e925d5a7-6def-430c-99e0-c4dc966caadf" TYPE= "ext4" [root@centos7 ~] # [root@centos7 ~] # mkdir / testdir/partfile [root@centos7 ~] # mount-a [root@centos7 ~] # mount | grep / testdir//testdir/partfile on / mnt/partfile type ext4 (rw,relatime Seclabel) [root@centos7 ~] # [root@centos7 ~] # cd / mnt/partfile/ [root@centos7 / mnt/partfile] # lslost+found [root@centos7 / mnt/partfile] #

Moving medium

All mobile media must be mounted in order to be recognized by the kernel, and mounting means making the foreign file system look as if it is part of the home directory tree and must comply with: before access, the media must be mounted; when removed, the media must be unmounted.

By default, non-root users can only mount certain devices (CD, DVD, USB, and so on), and the mount point is usually under / media or / mnt.

How to use the CD:

The eject command uninstalls or ejects the disk, and the following commands are available for burning CDs:

Wodim-v-eject centos.iso

The following commands are available to create IOS files:

Cp/dev/cdrom / root/centos7.isomkisofs-r-o / root/etc.iso / etc

Manual mounting

Mount / dev/cdrom/mnt/

So how do I mount the iso file in the / etc/fstab file? How to mount ordinary files to a directory?

Disk quota

The so-called disk quota means that superusers allocate a certain amount of disk space to ordinary users to prevent ordinary users from occupying a lot of disk space; disk quotas are implemented in the kernel and enabled on a file system basis. Different policies can be made for different groups or users, and restrictions can be made according to blocks or nodes. In addition, disk quota limits are soft limit (soft limit) and hard limit (hatd limit), soft limit is to give a warning after reaching a certain disk quota, while hard limit is to restrict what you do very strongly, in addition to creating empty files and empty directories (in advance, your number of nodes has not reached the limit, that is, there is still free inode.

Common commands for disk quotas:

Usrquota: enable user quotas grpquota: enable group quotas noquota: do not enable quotas quotacheck, quotaon, quotaoff, repquota

There are five main steps to achieve disk quotas:

1. Enable the configuration mount option vim / etc/fstab usrquota,grpquota2. Create a quota database mount-o remount / dev/sdb1quotacheck-cug / homesetenforce 03. Enable database quotaon / home4. Configure quota entry edquota liansiredquota-p liansir shitousetquota tom 100000 150000 00 / home5. Test quota liansirrepquota / home

Hard disk quota / home directory

In Linux, disk quotas are for partitions, not hard disks, that is, disk quotas are based on the file system. In the following experiment, we will have a hard disk quota on the / home directory, which requires home to partition independently, but now home is not an independent partition with the root, if you want to achieve it, you need to do a disk quota on the root, but this is unreasonable. You need to separate the home into a partition.

So, one of the things we need to do at this point is to migrate / home to five separate file systems.

Partitioning, formatting, and mounting must take one step:

[root@centos7 ~] # mkfs.ext4 / dev/sdb3-L / home # Let its volume be labeled / home [root@centos7 ~] # blkid / dev/sdb3/dev/sdb3: LABEL= "/ home" UUID= "e18fa7a2-d436-4bd5-bfdb-453d0af53fb6" TYPE= "ext4" [root@centos7 ~] # [root@centos7 ~] # mkdir / mnt/home [root@centos7 ~] # mount / dev/sdb3/ mnt/home # New device must be mounted before it can be used

Next, there is a problem, that is, migrate the data under / home to the new partition / dev/sdb3, copy or move, but be careful not to change the permissions!

Mount to the / home directory, and pay attention to the data migration under the / home directory; therefore, instead of mounting to / home at one time, / dev/sdb3 is first mounted to another empty directory, and then mounted to / home after the data migration is completed. Finally, cancel the previous mount. Note: only the last mount will be displayed on centos 7.

This is the end of the preparation for disk quotas! There is no such thing as data migration if there is a hard disk quota for an ordinary empty directory.

The following official disk quotas:

Initialization: write partition mount options in / etc/fstab configuration file: usrquota, grpquota; initialize database: checkquota, note that you may need to remount before initializing the disk quota database

[root@centos7 ~] # mount-o remount / home [root@centos7 ~] # quotacheck-cug / home

[root@centos7 / home] # file aquota.group aquota.user # these two files are binary files for the database, not text files aquota.group: dataaquota.user: data [root@centos7 / home] #

Note: the permission is not allowed on centos 6, and SELinux,setenforce 0. 0 needs to be closed.

Enable the database:

[root@centos7 ~] # quotaon / home [root@centos7 ~] # quotaon-p / homegroup quotaon / home (/ dev/sdb3) is onuser quotaon / home (/ dev/sdb3) is on [root@centos7 ~] #

Configure disk quotas for users

[root@centos7 ~] # edquota liansir

Modify directly in place:

Switch to the home directory of an ordinary user and test

The test was successful!

User survey: view disk quotas for specific users

[root@centos7 ~] # quota liansir Disk quotas for user liansir (uid 1000): Filesystem blocks quota limit grace files quota limit grace / dev/sdb3 102400 * 80000 102400 6days 13300

Overview of quotas: view disk quotas for specified partitions

[root@centos7 ~] # repquota / home*** Report for user quotas on device / dev/sdb3Block grace time: 7days Inode grace time: 7days Block limits File limitsUser used soft hard grace used soft hard grace----root-- 20 00 200 liansir +-102400 80000 102400 6days 133 00 hacker-52 00 13 00 xue-32 00 800 [root@centos7] #

We can see that liansir users do have disk quotas, and now if we want to set the disk quotas of hacker users to the same as liansir, we are just a command at this time! Edquata-p

[root@centos7 ~] # edquota-p liansir hacker [root@centos7 ~] #

Of course, we can also set the user's disk quota directly on the command line:

[root@centos7 ~] # setquota xue 100000 200000 00 / home # Note to specify the partition [root@centos7 ~] #

RAID

RAID (Redundant Arrays of Independent Disks), that is, independent redundant disk array, its important technology is parity check (Parity Check). When any one of the hard disks in the array of hard disks fails, the data can still be read out, and when the data is reconstructed, the data will be calculated and re-placed into the new hard disk.

There are three styles of disk array, one is external disk array cabinet, the other is internal disk array card, and the third is to use software to simulate. External disk array cabinets are most commonly used on large servers with Hot Swap characteristics; internal disk array cards are relatively cheap and require higher installation technology, and hardware arrays can provide online expansion, dynamic modification of array levels, automatic data recovery, drive roaming, hyperbuffering and other functions The way of software simulation is to configure multiple hard disks on the connected ordinary SCSI card into logical disks to form an array through the disk management function provided by the operating system itself. Among them, the first two are called hard RAID, and the software simulation is called soft RAID.

RAID has levels. This article takes soft RAID as an example, and the RAID levels can be combined as needed. The main command when configuring RAID is mdadm, and md modules (multi devices) are provided in the kernel, and RAID devices can be named / dev/md0, / dev/md1, / dev/md2, / dev/md3, and so on.

Mdadm

Mdadm-manage MD devices aka Linux Software RAIDmdadm [mode] [options] MODES mdadm has several major modes of operation:-A,-- assemble Assemble a pre-existing array. -B.-- build Build a legacy array without superblocks. -C.-- create Create a new array. F,-- follow,-- monitor Select Monitor mode. -G.-- grow Change the size or shape of an active array. That is: mode: create:-C Assembly:-A Monitoring:-F Management:-f,-r,-a

Let's take a look at the main levels of RAID:

RAID 0: stripe volume, strip

The simplest way to implement RAID 0 is to create a large volume set by cascading N blocks of the same hard disk in the form of hardware through an intelligent disk controller or using disk drivers in the operating system. Only need more than 2 hard drives, no redundancy or error repair ability, the realization cost is the lowest, the biggest disadvantage is that any hard disk fails, the whole system will be destroyed, and the reliability is only 1xN of a single hard disk. Therefore, RAID 0 is generally used only in situations where data security is not required.

RAID 1: mirror Volume, mirror

RAID 1 is called disk mirroring, which mirrors data from one disk to another, meaning that when the data is written to one disk, an image file is generated on another idle disk. It has good disk redundancy, but the disk utilization is 50%. Raid 1 is often used in situations where critical and important data is saved.

RAID 5: stripe machine with parity

The parity code of RAID 5 exists on all disks, where p0 represents the parity value of band 0, and the other is the same. For RAID 5, most data transfers operate only on one disk and can be operated in parallel. There is a "write loss" in RAID 5, that is, each write operation produces four actual read / write operations, including two reads of old data and parity information, and two writes of new data and parity information.

RAID 01

RAID 01 is a combination of RAID0 and RAID1. Except for data distributed on multiple disks, each disk has its own physical mirror disk, which provides full redundancy, allows one disk failure without affecting data availability, and has fast read / write capabilities. RAID 01 needs to establish a band set of at least 4 hard drives in the disk mirror.

RAID 10

RAID 10 is a band structure plus a mirror structure, which can complement each other to achieve the purpose of both high efficiency and high speed. It is mainly used in databases with small data capacity but requiring speed and error control.

Note: the RAID level picture is from the network.

After introducing the main levels of RAID, let's take a look at mdadm specific mode commands.

-C: create mode-n #: use # block devices to create this RAID;-l #: indicate the level of RAID to be created;-a {yes | no}: automatically create the device file for the target RAID device;-c CHUNK_SIZE: indicate the block size;-x #: indicate the number of free disks;-D: display the details of the raid Mdadm-D / dev/md# management mode:-f: Mark to specify disk as damaged-a: add disk-r: remove disk to observe md status: cat / proc/mdstat stop md device: mdstat-S DVICE

OK, now let's implement a RAID 5, 4 raid disks, 1 spare disk, a total of 5 disks.

We prepared four hard drives and created a partition with a size of 10G:

Sdb 8:16 0 20G 0 disk └─ sdb1 8:17 0 10G 0 part sdd 8:48 0 20G 0 disk └─ sdd1 8:49 0 10G 0 part sdc 8:32 020G 0 disk └─ sdc1 8:33 010G 0 part sde 8:64 020G 0 disk └─ sde1 8:65 010G 0 part sdf 8:80 020G 0 disk └─ sdf1 8:81 010G 0 part [root@centos6 ~] #

Create RAID: / dev/md0

[root@centos6 ~] # mdadm-C / dev/md0-l 5-n 4-x 1 / dev/sd {brecinct drecinct cRey f} 1mdadm: Defaulting to version 1.2 metadatamdadm: array / dev/md0 started. [root@centos6 ~] #

View:

[root@centos6 ~] # ll / dev/md0brw-rw----. 1 root disk 9,0 Sep 2 20:17 / dev/md0 [root@centos6 ~] # cat / proc/mdstatPersonalities: [raid6] [raid5] [raid4] md0: active raid5 sde1 [5] sdf1 [4] (S) sdc1 [2] sdd1 [1] sdb1 [0] 31446528 blocks super 1.2 level 5,512k chunk, algorithm 2 [4gam4] [UUUU] unused devices: [root@centos6 ~] #

Format RAID

[root@centos6 ~] # mkfs.ext4 / dev/md0

View:

[root@centos6 ~] # blkid / dev/md0/dev/md0: UUID= "0ea747b9-c98e-47fd-b08d-fd346ecbd654" TYPE= "ext4" [root@centos6 ~] #

Mounting

[root@centos6 ~] # mkdir / mnt/md0 [root@centos6 ~] # mount-a

View:

[root@centos6 ~] # mount | grep / dev/md0/dev/md0 on / mnt/md0 type ext4 (rw) [root@centos6 ~] #

test

[root@centos6 ~] # cd / mnt/md0 [root@centos6 / mnt/md0] # touch F1 # creatable file [root@centos6 / mnt/md0] # [root@centos6 ~] # mdadm / dev/md0-f / dev/sdb1 # corrupt / dev/sd1mdadm: set / dev/sdb1 faulty in / dev/md0 [root@centos6 ~] # [root@centos6 ~] # mdadm / dev/md0-r / dev/sdb1 # Heat removal mdadm: hot removed / dev / sdb1 from / dev/md0 [root@centos6 ~] #

At this point, let's restart the system!

After rebooting the system, we found that RAID failed! But this has nothing to do with damage / dev/sdb1

Repeat the above steps again at this time!

We must back up before restarting the system!

[root@centos6 ~] # mdadm-Ds / dev/md0 > / etc/mdadm.conf # otherwise RAID becomes invalid after restarting the system

Then restart the system:

View RAID:

[root@centos6] # mdadm-D / dev/md0

Test again:

[root@centos6 ~] # mdadm / dev/md0-f / dev/sdb1mdadm: set / dev/sdb1 faulty in / dev/md0 [root@centos6 ~] #

[root@centos6] # mdadm / dev/md0-r / dev/sdb1

At this point the spare disk has been removed

Create some files on the raid disk, and then we remove a RAID hard drive.

[root@centos6 / mnt/md0] # dd if=/dev/zero of=/mnt/md0/f2 bs=100M count=11+0 records in1+0 records out104857600 bytes (105MB) copied, 0.254593 s, 412 MB/s [root@centos6 / mnt/md0] # lltotal 102416 1 root root 0 Sep 2 21:56 F1 RW Murray RW Murray. 1 root root 104857600 Sep 2 21:56 f2drwxmuri. 2 root root 16384 Sep 2 21:28 lost+found [root@centos6 / mnt/md0] # [root@centos6 ~] # mdadm / dev/md0-f / dev/sdf1mdadm: set / dev/sdf1 faulty in / dev/md0 [root@centos6 ~] # mdadm / dev/md0-r / dev/sdf1mdadm: hot removed / dev/sdf1 from / dev/md0 [root@centos6 ~] # [root@centos6 ~] # cd / mnt/md0 # check whether the data is normal [root@centos6 / mnt/md0] # lsf1 f2 lost+found [root@centos6 / mnt/md0] #

The test was successful!

RAID management: add new partitions and extend new members (expand on the number of hard drives)

[root@centos6 ~] # mdadm / dev/md0-a / dev/sdg1 # add a new partition to RAID mdadm: added / dev/sdg1 [root@centos6 ~] #

[root@centos6] # mdadm-G / dev/md0-n 6-a / dev/sdh2 # extension mdadm: added / dev/sdh2

Note that when doing RAID, it is best to have the same partition size of each hard disk, because the smallest one will be the standard, and the extra space will be wasted; in addition, no one uses soft RAID in the production environment, and soft RAID is only used for testing and understanding RAID!

This paper mainly introduces the general usage of dd command, how to establish swap partition SWAP, the basic characteristics of common RAID and the configuration of RAID 5. 2016.9.3

Stop the war

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.