Shulou(Shulou.com)06/03 Report--

preface

This article comes from some of the work records of the author during his work on Suning Technology Group Cloud Platform. Go to public first, hoping to help peers with similar problems.

1. Description of the problem

The/var/log/sa directory is part of the sysstat installation package. The target machine is missing this directory. When the scheduled task initiated by crond executes/usr/lib64/sa/sa1 and fails to access the directory, an error will be generated and placed in/var/spool/postfix/maindrop/. After a long time, the directory space of/var is full, resulting in a large number of postdrop process blocking accumulations, occupying system resources.

symptom 2

Another symptom later discovered is that root presumably has some mechanism for consuming these mails when logging in./var/spool/postfix/maildrop/If there is already a file pile, it will cause the CPU and memory consumption of login bash to be very large.

2. Scene reproduction

1)Testing with a virtual machine, the cron file is located in/etc/cron.d/

/usr/lib64/sa/sa1 is executed every 10 minutes

In order to reproduce as soon as possible, adjust to 1 minute to execute once

2)Delete the/var/log/sa folder

rm -Rf /var/log/sa

3)Use dd command to occupy var space to close to 100%

4)Because there is no hard disk space available, a large number of sendmail processes and postdrop processes are blocked

When the sa1 file is not found, the sendmail process sends a message and logs it to the/var/mail/root file.

When there is not enough disk space, sendmail is blocked due to the inability to write, and the postdrop process is called to drop the message to the/var/spool/postfix/maildrop/folder:

But lack of disk space also blocks postdrop, so the intuitive expression is to produce a lot of postdrop and sendmail hanging there.

5)Clear a part of the file under var directory, create free disk space, sendmail and postdrop process will immediately unblock, postdrop process will all end.

3. Solution

1)Clear some files under var directory to free disk space

2)Recover sa folder, avoid constant sendmail and postdrop

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.