Shulou(Shulou.com)06/01 Report--

Editor to share with you what is the reason why the file has been deleted but the space is not released in Linux. I hope you will gain something after reading this article. Let's discuss it together.

1. Error phenomenon

The monitoring system of the operation and maintenance staff sends a notice, reporting that a server space is full, log in to the server to check, and the root partition really has no space, as shown in figure 1.

Figure 1 View server disk space

First of all, I will explain some deletion policies of the server. Because Linux does not have the Recycle Bin feature, all files to be deleted on the online server will be moved to the system / tmp directory first, and then the data in the / tmp directory will be cleared periodically. There is nothing wrong with this strategy itself, but it is found that there is no separate / tmp partition in the system partition of this server, so the data under / tmp actually takes up the space of the root partition. Now that you've found the problem, delete some of the larger data files in the / tmp directory and check the three largest data files under / tmp, as shown in figure 2.

Figure 2 View the first three largest data files under / tmp

Through the command output, it is found that there is a 66GB-sized file access_log in the / tmp directory. This file should be an access log file generated by Apache. Judging from the log size, the Apache log file has not been cleaned for a long time. It is basically determined that the root space caused by this file is full. After confirming that this file can be deleted, perform the following deletion operation:

[root@localhost ~] # rm / tmp/access_log

Then check to see if the system root partition space is freed, as shown in figure 3.

Figure 3 to see if disk space is freed

As you can see from the output, the root partition space is still not freed. What's going on?

2. Solution

Generally speaking, space will not be released after deleting a file, but there are exceptions, such as a file is locked by a process, or a process has been writing data to the file. To understand this problem, you need to know the storage mechanism and storage structure of files under Linux.

The storage of a file in the file system is divided into two parts: the data part and the pointer part, the pointer is located in the meta-data of the file system, after the data is deleted, the pointer is removed from the meta-data, and the data part is stored on disk. After the pointer corresponding to the data is removed from the meta-data, the space occupied by the data part of the file can be overwritten and written to the new content. The reason why the space has not been released after the deletion of the access_log file is that the httpd process has been writing to the file. Although the access_log file has been deleted, the corresponding pointer part of the file has not been removed from the meta-data because of the process lock. Since the pointer is not deleted, the system kernel believes that the file has not been deleted, so it is not surprising that the query space is not released through the df command.

3. Troubleshooting

Now that you have an idea to solve the problem, let's see if any process has been writing data to the access_log file. Here you need to use the lsof command under Linux, through which you can get a list of deleted files that are still occupied by the application. The command execution is shown in figure 4.

Figure 4 View the list of deleted files locked by the application

As you can see from the output, the / tmp/access_log file is locked by the process httpd, and the httpd process keeps writing log data to this file. As you can see from column 7, the size of this log file is about 70GB, while the total size of the system root partition is only 100GB, so this file is the culprit that causes the system root partition to run out of space. The "deleted" status of the last column indicates that the log file has been deleted, but the space has not been freed because the process is still writing data to the file.

4. Solve the problem

At this point, the problem is basically clear, there are many ways to solve this kind of problem, the easiest way is to shut down or restart the httpd process, of course, you can restart the operating system, but these are not the best methods. To deal with the constant logging of files in this process, the best way to free the disk space occupied by files is to empty the file online, which can be done by using the following command:

[root@localhost ~] # echo "" > / tmp/acess.log

This method not only frees disk space immediately, but also ensures that the process continues to write logs to files. This method is often used to clean up log files generated by Apache, Tomcat, Nginx and other Web services online.

After reading this article, I believe you have a certain understanding of "what is the reason why the files in Linux have been deleted but the space is not released". If you want to know more about it, welcome to follow the industry information channel, thank you for your reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.