How to realize controllable multithreaded shell script
This article will explain in detail how to achieve a controllable multithreaded shell script. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.
Let's start with a scene description:
One day, in the goose factory received this task, the need in the Linux server, thousands of IP for a Ping test, as long as the ping can reach the IP. If a single IP goes to ping for testing, although the task can be done, thousands of IP are fine, but what if there are more?
In view of the simplicity of this case, we immediately abandoned the previously used pipeline scheme and adopted the idea of breaking it one by one.
Simple ideas:
According to the "strategic thinking" of task cutting, I first save these thousands of IP into an iplist file, then write a split function to divide this file into multiple temporary IP lists, and finally, use multithreading to traverse these temporary IP files to achieve multithreading in disguise.
Specific code:
#! / bin/sh# text segmentation function: split text $1 by number of copies $2 SplitFile () {linenum= `wc-l $1 | awk'{print $1} '`if [[$linenum-le $2]] then echo "The lines of this file is less then $2, Are you kidding me..." Exit fi Split= `expr $linenum / $2` Num1=1 FileNum=1 test-d SplitFile | | mkdir-p SplitFile rm-rf SplitFile/* while [$Num1-lt $linenum] do Num2= `expr $Num1 + $Split`expr-n "${Num1}, ${Num2} p" $1 > SplitFile/$1-$FileNum Num1= `expr $Num2 + 1`expr $FileNum + 1`expr + 1`expr} # Parameter 1 indicates how many parts are divided, that is, how many threads are opened. The default 10 FILE=$ {2:-iplist} # parameters 2 represent the divided object. Default iplist file # split file SplitFile $FILE $SPLIT_NUM # Loop through temporary IP file for iplist in $(ls. / SplitFile/*) do # Loop ping Test ip (drop background) cat $iplist in temporary IP file | while read ip do ping-c 4-w 4$ ip > / dev/null & & echo $ip | tee-ai okip.log # IP reachable IP is written to the log done & # add the & symbol after the while loop and let the nested loop execute done in the background
After saving the code as ping.sh, the procedure for executing sh ping.sh iplist 100 is as follows:
First cut the iplist into 100 pieces and store it in the SplitFile folder
Then, read the split files through the for loop and use the while loop to ping the ip in the background.
Because while is lost in the background, the for loop executes 100 while at a time, which is equivalent to starting 100 threads, so the speed is naturally different.
Among them, the number of cutting copies is the number of multithreads you want to open. Obviously, although the idea of task segmentation is not as high-end as the pipeline scheme, it is easier to understand and more versatile. Suitable for entry-level simple multithreaded tasks.
On "how to achieve controllable multithreaded shell script" this article is shared here, I hope the above content can be of some help to you, so that you can learn more knowledge, if you think the article is good, please share it out for more people to see.
