Shulou(Shulou.com)06/01 Report--

This article mainly introduces "the process analysis of Linux system recovery caused by NAS storage failure". In daily operation, I believe that many people have doubts about the process analysis of Linux system recovery caused by NAS storage failure. Xiaobian consulted all kinds of data and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts of "process analysis of Linux system recovery caused by NAS storage failure". Next, please follow the editor to study!

I. description of fault phenomena

The kernel of the NAS operating system is Linux, and the built-in storage has 16 hard disks, which are divided into two groups. Each group cannot start the RAID5,Linux operating system normally, it stops when the service starts to cups, the button ctrl+c is forced to disconnect and there is no response, it is normal to check the status of the hard disk, and there is no alarm or warning.

Second, the way to judge the problem.

Through the above phenomena, first of all, it is concluded that the NAS hardware should be fine, and the NAS storage disk should also be normal. Now the Linux cannot be started, which should be due to the problems of the Linux system itself. Therefore, we should first start with the Linux system for troubleshooting.

III. Problem handling process

1. The first process

The NAS system itself is a Linux kernel loaded with a file system management software, which can manage and operate system disks, system services, file systems, etc. Under normal circumstances, a NAS system based on Linux kernel should be booted to init3 or init5 mode. Because NAS only uses Linux kernel module and several simple services, it is determined that the Linux system under NAS must be booted to init3 mode. So now it is impossible to start to the multi-user character interface, why not let Linux directly enter the single-user (init 1) mode, because the single-user mode only enables a few services necessary for the system, while the cpus service is at the application level and will certainly not be started in "init 1" mode, so it avoids the problem that cups cannot be started, so the following work is to enter the Linux single-user mode.

Many Linux distributions can enter single-user mode through relevant settings in the boot interface of startup. By viewing the startup process of NAS, we can basically judge that this Linux system is very similar to RHEL/Centos distribution. Therefore, try to enter single-user mode through RHEL/Centos.

It is very simple for RHEL/Centos to enter single-user mode, that is, under the boot welcome interface of the system, press e, then edit the correct kernel boot option, add the "single" option at the end, and finally directly press "b" to enter the single user.

Next, restart NAS, then the hardware self-test, and then start Linux. I have been waiting for the welcome interface of the NAS to start, but the welcome interface has not come out, so I directly enter the kernel image, load the kernel phase, without the kernel boot interface, how to enter a single user, after simple thinking, or decide to press the keyboard "e" key directly after the hardware test is finished. NAS entered the kernel boot interface, through a simple observation, the release of the second kernel option is to boot, so move the keyboard up and down keys, select the kernel, and then press the key "e" to enter the kernel boot editing interface. At the end of this line, type "single", then press the enter key to return to the previous interface, and then press the key "b" to start single user boot, after a minute. The system successfully entered the shell command line under a single user.

After entering single-user mode, there are many things you can do. The first thing to do is to self-start and shut down the cups service in multi-user mode. Execute the command as follows:

Chkconfig-levle 35 cups off

After successful execution, restart the system into multi-user mode to see if the system can start normally.

2. The second treatment process

After booting and shutting down the cups service, restart NAS and find that the problem is still the same. NAS still starts to stop the cups service. Is the above command not executed successfully? It is obvious that the cups service has been disabled, but why is it still started? So, continue to restart NAS and go back into single-user mode to see what the problem is.

After entering a single user, execute the chkconfig command again, and you can still succeed. If there is a problem with the cups service, first look at the configuration file and execute the following command:

Vi / etc/cups/cupsd.conf

A problem is found here. When vi opens cupsd.conf, it prompts "write file in swap". How can it be said that the file exists in virtual memory? after consideration, there is only one possibility. The Linux system partition of NAS device should not be mounted correctly, resulting in that when entering a single user, all files are stored in virtual memory. It is very easy to verify, just execute the "df" command to check it, as shown below:

You can see here that the system partition of Linux is not mounted. Check the status of the disk partition through "fdisk-l", and the output is as follows:

As you can see from the output, the system disk of NAS is / dev/sda, which is only divided into two system partitions: / dev/sda1 and / dev/sda2, while the data disk is completed by RAID5, and the device identities on the system are / dev/sdb1 and / dev/sdc1, respectively. Since a single user does not mount any NAS disks by default, try to mount the NAS system disk manually and execute the following command:

[root@NASserver ~] # mount / dev/sda2 / mnt

[root@NASserver ~] # mount / dev/sda1 / opt

/ mnt and / opt here are directories that can be mounted at will, or you can mount them to other empty directories. After the mount is completed, enter these directories to see what the contents are, as shown in the figure below:

From the view of these two contents, it is preliminarily concluded that / dev/sda2 partition should be the root partition of Linux, and / dev/sda1 should be / boot partition. Now that the partition has been mounted, execute the df command again to see the mount, as shown in the following figure:

So far, a problem has been found. / dev/sda2 disk partition has no available disk space, and this partition happens to be the root partition of the NAS system, and the root partition has no space, so there must be something wrong with the system startup.

Let's turn to the previous case. Because the system cups service writes the startup log to the root partition at startup, and the root partition cannot write the log because there is no space, the result is that the cups service cannot be started, which explains why the NAS system stops every time it starts to the cups service.

Fourth, solve the problem

Since the NAS system has only root partition and / boot partition, all the related logs generated by the system will be stored in the root partition. Now that the root partition is full, the first thing you can clean up is the system-related log files in the / var directory. Usually the directory you can clean up is / var/log. Execute the following command to check the disk space occupied by the / var/log log directory:

[root@NASserver ~] # du-sh / var/log

50.1G / var/log

Through the command output, it is found that the / var/log directory occupies only 70% of the root partition space. Cleaning the log files in this directory can release most of the root partition space. After cleaning and restarting the NAS system, it is found that the system cups service can start normally and the NAS service will start normally.

At this point, the study of "process analysis of Linux system recovery caused by NAS storage failure" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.