Shulou(Shulou.com)06/01 Report--

Editor to share with you what examples of the use of Shell in linux, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Shell script is a powerful tool to help programmers and system administrators complete time-consuming and tedious work, and it is an effective way to interact with computers and manage files and system operations. With just a few lines of code, you can get the computer close to doing what you want.

Test file consistency under the directory specified by two servers #! / bin/bash # detect file consistency under the directory specified by two servers # # # by comparing the MD5 values of files on the two servers To achieve the purpose of checking consistency, dir=/data/web b_ip=192.168.88.10 # traverses all the files in the specified directory as parameters of the md5sum command, and then gets the MD5 value of all files. And write to the specified file find $dir-type f | xargs md5sum > / tmp/md5_a.txt ssh $b_ip "find $dir-type f | xargs md5sum > / tmp/md5_b.txt" scp $b_ip:/tmp/md5_b.txt / tmp # compare the file name as a traversal object one by one to for fin `awk'{print 2} / tmp/md5_a.txt' `do # with a machine as the standard Output the result if grep-qw "$f" / tmp/md5_b.txt then md5_a= `grep-w "$f" / tmp/md5_a.txt directly when the file in the traversal object does not exist on the b machine | awk'{print 1} '`md5_b= `grep-w "$f" / tmp/md5_b.txt | awk' {print 1}'`# when the file exists If the md5 value is inconsistent, the output file changes the result if [$md5_a! = $md5_b] then echo "$f changed." Fi else echo "$f deleted." Fi done regularly empties the contents of the file and records the file size #! / bin/bash # execute the script (task schedule) every hour, when the time is 0: 00 or 12:00 Empty all the files in the target directory, but do not delete the files. At other times, you will only count the size of each file, one file per line, and output it to a file named between hours and dates. You need to consider the files # logfile=/tmp/ `date +% Hcopyright% F`.log n = `date +% H` if [$n-eq 00] | | [$n-eq 12] then # through the for loop under the target directory Use the find command as the traversal condition to traverse all the files in the target directory and do the corresponding operations for i in `find / data/log/-type f` do true > $i done else for i in `find / data/log/-type f` do du-sh $I > > $logfile done fi to detect the network card traffic. And record it in the log according to the prescribed format #! / bin/bash # detect the network card traffic The log format is as follows: # 2019-08-12 20:40 # ens33 input: 1234bps # ens33 output: 1235bps # 3 while: log # set the language to English and ensure that the output is in English Otherwise, bug LANG=en logfile=/tmp/ `date +% d`.log # will redirect the output of the following command to the logfile log exec > > $logfile date + "% F% Hazard% M" # sar command counts the traffic in kb/s and the log format is bps Therefore, to * 1000 million 8 sar-n DEV 1 59 | grep Average | grep ens33 | awk'{print $2, "\ t", "input:", "\ t", $5 million 1000 million 8, "bps", "\ n", $2, "\ t", "output:", "\ t", $6 million 1000 8, "bps"} 'echo "#" # because it takes 59 seconds to execute the sar command Therefore, there is no need for sleep done to calculate the number of digits in each line of the document, and to calculate the total number of digits in the entire document #! / bin/bash # calculate the number of digits in each line of the document And calculate the total number of numbers for the entire document # use awk to output only the number of document lines (intercept the first paragraph) n = `wc-l a.txt | awk'{print $1} '`space may exist in each line of the document Therefore, when the lines of for i in `seq 1$ n` do # output cannot be traversed directly with document contents, you need to use double quotation marks line= `sed-n "$I" p a.txt` # wc-L option. Count the length of the longest line n = `echo $line | sed slots / [^ 0-9] / / g | wc-L` echo $sum=$ [$sum+$n_n] done echo "sum:$sum" Kill all scripts #! / bin/bash # # some scripts have been added to cron There are situations where the script has not yet been run and there are new tasks to be executed, # leads to an increase in system load, so you can write scripts to filter out processes that affect the load and kill them all at once. # ps aux | grep specifies the process name | grep-v grep | awk'{print $2}'| xargs kill-9 downloads the file #! / bin/bash if [$#-ne 1] from the FTP server Then echo "Usage: $0 filename" fi dir=$ (dirname $1) file=$ (basename $1) ftp-n-v / dev/null # set 60s delay to prevent script bug from causing infinite restart of php-fpm service sleep 60 fi sleep 10 done assigns the results to variables respectively

Application scenario: you want to assign the execution result or position parameter to the variable for later use.

Method 1:

For i in $(echo "4 5 6"); do eval a$i=$i done echo $A4 $A5 $a6

Method 2: split the position parameter 192.168.1.1 {1jin2} into each variable

Num=0 for i in $(eval echo $*); do # eval decomposes {1Yue2} into 12 let num+=1 eval node$ {num} = "$I" done echo $node1 $node2 $node3 # bash a.sh 192.168.1.1 {1pai2} 192.168.1.11 192.168.1.12

Method 3:

Arr= (4 56) INDEX1=$ (echo ${arr [0]}) INDEX2=$ (echo ${arr [1]}) INDEX3=$ (echo ${arr [2]}) modify file names in batches

Example:

# touch article_ {1..3} .html # ls article_1.html article_2.html article_3.html

Objective: to change article to bbs

Method 1:

For file in $(ls * html); do mv $file bbs_$ {file#*_} # mv $file $(echo $file | sed-r's bbs. * (_. *) / bbs\ 1Universe') # mv $file $(echo $file | echo bbs_$ (cut-d _-f2) done

Method 2:

For file in $(find. -maxdepth 1-name "* html"); do mv $file bbs_$ {file#*_} done

Method 3:

# rename article bbs * .html deletes the lines containing letters in the first five lines of a document, and deletes all letters contained in lines 6 to 10

1) prepare the test file, named 2.txt

Line 1 1234567 does not contain the letters line 2 56789BBBBBB line 3 67890CCCCCCCC line 4 line 78asdfDDDDDDDDD line 5 line 123456EEEEEEEE line 6 line 1234567ASDF line 7 line 56789ASDF line 8 line 67890ASDF line 9 line 78asdfADSF line 10 123456AAAA line 11 line 67890ASDF line 12 78asdfADSF line 13 123456AAAA

2) the script is as follows:

#! / bin/bash # delete the lines containing letters in the first five lines of a document Delete all letters contained in lines 6 to 10 at the same time # sed-n '1Magazine 5p 2.txt | sed' / [a-zA-Z] / d sed-n '610p 2.txt | sed slots / [a-zA-Z] / /' g sed-n'11 $'p 2.txt # the final result is just printed on the screen If you want to change the file directly, you can write the output to a temporary file, replace 2.txt, or use the-I option to count the total size of files ending in .html in the current directory

Method 1:

# find. -name "* .html"-exec du-k {}\; | awk'{sum+=$1} END {print sum}'

Method 2:

For size in $(ls-l *. Html | awk'{print $5}'); do sum=$ (($sum+$size)) done echo $sum scan host port status #! / bin/bash HOST=$1 PORT= "22 25 80 8080" for PORT in $PORT; do if echo & > / dev/null > / dev/tcp/$HOST/$PORT Then echo "$PORT open" else echo "$PORT close" fi done uses shell to print words with letters less than 6 in the sample statement # sample statement: # Bash also interprets a number of multi-character options. #! / bin/bash # words with less than 6 letters in the shell print example statement # # # # for s in Bash also interprets a number of multi-character options. Don = `echo $s | wc-c`if [$n-lt 6] then echo $s fi done enter a number to run the corresponding command #! / bin/bash # enter a number to run the corresponding command # # echo "* cmd menu* 1-date 2-ls 3-who 4-pwd 0-exit" while: do # capture the user typed value read-p "please input number:" nn1 = `echo $n | sed slots / [0-9] / /'g` # empty input detection if [- Z "$n"] then continue fi # non-numeric input detection if [- n "$N1"] then exit 0 fi break done case $n in 1) date ; 2) ls;; 3) who;; 4) pwd;; 0) break;; # prompt for entering digits other than 1-4 *) echo "please input number is [1-4]" esacExpect implements SSH without interactive execution commands

Expect is a tool for automated interactive applications, such as telnet,ftp,passwd.

You need to install the expect package first.

Method 1:EOF standard output as expect standard input

#! / bin/bash USER=root PASS=123.com IP=192.168.1.120 expect > / tmp/userpassword done monitors the number of processes in httpd Deal with #! / bin/bash # according to the monitoring situation # requirements: # 1. Monitor the number of httpd processes every 10 seconds. If the number of processes is greater than or equal to 500, automatically restart the Apache service and check whether the service is restarted successfully # 2. If it is not successful, it needs to be started again. If the restart is still unsuccessful for 5 times, send an alarm email to the administrator and exit detection # 3. If the startup is successful, the number of httpd processes will be detected again after waiting for 1 minute. If the number of processes is normal, the detection will return to normal (once every 10 seconds), otherwise the restart will be abandoned and an alarm email will be sent to the administrator. And exit detection # # # counter function check_service () {jack0 for i in `seq 1 5` do # Command / usr/local/apache2/bin/apachectl restart 2 > / var/log/httpderr.log # determine whether the service has been restarted successfully if [$?-eq 0] then break else fi $[$jacu1] fi # determine whether the service has attempted to restart if 5 times [$j- Eq 5] then mail.py exit fi done} while: don = `pgrep-l httpd | wc-l` # determine whether the number of httpd service processes exceeds 500if [$n-gt 500] then / usr/local/apache2/bin/apachectl restart if [$?-ne 0] then check_service else sleep 60n2 = `pgrep-l httpd | wc-l` # determine whether the number of if [$N2-gt 500] then mail.py exit fi # still exceeds 500 if after restart Sleep 10 done batch modify server user password every 10s

Linux host SSH connection information: old password

# cat old_pass.txt 192.168.18.217 root 123456 22 192.168.18.218 root 123456 22

Content format: IP User Password PortSSH

Remote password modification script: new passwords are randomly generated

Https://www.linuxprobe.com/books

#! / bin/bash OLD_INFO=old_pass.txt NEW_INFO=new_pass.txt for IP in $(awk'/ ^ [^ #] / {print $1}'$OLD_INFO) Do USER=$ (awk-v I=$IP 'Illustrated blocks 1 {print $2}' $OLD_INFO) PASS=$ (awk-v I=$IP 'Illustrated blocks 1 {print $3}' $OLD_INFO) PORT=$ (awk-v I=$IP 'Illustrated packages 1 {print $4}' $OLD_INFO) NEW_PASS=$ (mkpasswd-l8) # Random password echo "$IP $USER $NEW_PASS $PORT" > > $NEW_INFO expect-c "spawn ssh-p$PORT $USER@$IP set timeout 2 expect { \ "(yes/no)\" {send\ "yes\ r\" Exp_continue}\ "password:\" {send\ "$PASS\ r\"; exp_continue}\ "$USER@*\" {send\ "echo\'$NEW_PASS\'| passwd-- stdin $USER\ r exit\ r\"; exp_continue}} "done

Generate a new password file:

192.168.18.217 root n8wX3mU% 22 192.168.18.218 root c87bot ZnnL 22iptables automatically blocks frequent IP visits to the website

Scenario: malicious access, security precautions

1) block IP accessing more than 200 per minute

Method 1: according to the access log (Nginx as an example)

#! / bin/bash DATE=$ (date +% d/%b/%Y:%H:%M) ABNORMAL_IP=$ (tail-n5000 access.log | grep $DATE | awk'{a [$1] + +} END {for (i in a) if (a [I] > 100) print I}') # first tail prevents the file from being too large and reads slowly, and the number can adjust the maximum number of visits per minute. Awk cannot filter the log directly because it contains special characters. For IP in $ABNORMAL_IP; do if [$(iptables-vnL | grep-c "$IP")-eq 0]; then iptables-I INPUT-s $IP-j DROP fi done

Method 2: connections established through TCP

#! / bin/bash ABNORMAL_IP=$ (netstat-an | awk'$4mm Univ 80 $/ & & $6~/ESTABLISHED/ {gsub (/: [0-9] + /, ", $5); {a [$5] + +}} END {for (i in a) if (a [I] > 100) print I}') # gsub removes the colons and ports in the fifth column (client IP) for IP in $ABNORMAL_IP; do if [$(iptables-vnL | grep-c" $IP ")-eq 0] Then iptables-I INPUT-s $IP-j DROP fi done

2) block IP where SSH attempts to log in more than 10 times per minute

Method 1: get the login status through lastb:

#! / bin/bash DATE=$ (date + "% a% b% e% HGV% M") # week, month, day, hour, day, month, day, month, day, day, day Then iptables-I INPUT-s $IP-j DROP fi done

Method 2: obtain login status through log

#! / bin/bash DATE=$ (date + "% b% d% H") ABNORMAL_IP= "$(tail-n10000 / var/log/auth.log | grep" $DATE "| awk'/ Failed/ {a [(NF-3)] + +} END {for (i in a) if (a [I] > 5) print I}')" for IP in $ABNORMAL_IP; do if [$(iptables-vnL | grep-c "$IP")-eq 0] Then iptables-An INPUT-s $IP-j DROP echo "$(date +"% F% T ")-iptables-An INPUT-s $IP-j DROP" > > ~ / ssh-login-limit.log fi done according to the web access log, block IP with abnormal requests, such as IP, return to normal after half an hour. Unblock #! / bin/bash # according to the web access log, block IP with abnormal requests, such as IP, return to normal after half an hour Then unblock # logfile=/data/log/access.log # displays the hour and minute D1 = `date-d "- 1 minute one minute ago +% H% M`d2 = `date +% M` ipt=/sbin/iptables ips=/tmp/ips.txt block () {# filter out all the logs from one minute ago and extract the IP and count the number of visits grep'$d1pura'$logfile | awk'{print $1}'| sort-n | uniq-c | sort-n > $ips # use the for loop to traverse the IP with more than 100 times and block for i in `awk'$1 > 100 {print $2}'$ips` do $ipt-I INPUT -p tcp-- dport 80-s $I-j REJECT echo "`date +% Flysee% T` $I" > > / tmp/badip.log done} unblock () {# unblock the IP generated by blocking the number of pkts less than 10 and unblock the for an in `$ ipt-nvL INPUT-- line-numbers | grep '0.0.0.0Uniq0' | awk' $2

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.