Shulou(Shulou.com)06/03 Report--

This article will give you a detailed explanation on how to optimize the efficiency of Shell scripts. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

First, let's talk about the limitations of Shell scripting language.

As an interpretive scripting language, it is inherently flawed in efficiency. Although the other commands it calls may be efficient.

Shell scripts are executed sequentially, not in parallel. To a large extent, this wastes the system resources that may be used.

Every time Shell executes a command, it creates a new process, which is a waste of system resources if the script writer is not aware of it and writes the script improperly.

Second, we try our best to improve the efficiency of scripts through our experienced coding on the limitations of Shell scripting language.

1. For example, if I want to do a loop to process data, I may simply process the data, which makes it easier for people to think of loops in Shell like this:

The code is as follows:

Sum=0

For ((iSuppli / dev/null)

But you don't want to know how efficient it is:

Real 0m12.296s

User 0m12.033s

Sys 0m0.262s

Believe me, I don't want to watch the cursor flash for another 12 seconds. But if this is done:

Awk'{print $9}'

/ dev/null

The efficiency of this sentence is as follows:

Real 0m3.691s

User 0m5.219s

Sys 0m0.630s

Real 0m3.660s

User 0m5.169s

Sys 0m0.618s

Real 0m3.660s

User 0m5.150s

Sys 0m0.612s

The average time is about 3.6 seconds, which is about 4 times the difference in efficiency, although it is not a hundred times as different as the previous one, but it is enough to turn four hours into one hour. I think you understand the gap.

In fact, you can try to speculate about other situations in this regular example, because every time the regular is run, you need to start a string match, and the default delimiter is quickly distinguished by field. So after we know some data rules, we can try to greatly shorten the string in which we are going to do complex regular matching, which will have a very significant efficiency improvement according to your reducing the size of the data. the above is still a relatively simple regular matching situation verified, with only a single character "\", you can imagine if the regular expression is like this:

[s]? html |\ .php |\ .xml |\ / $/ & & ($9 million / 200 | $9 million / $100 / ^ 103\ .108 | ^ 224\ .215 | ^ 127\ .0 | ^ 122\ .110\ .5 /

I think you can imagine the great significance when a target matching string is reduced from 500 characters to 50 characters!

Ps: for more detailed canonical optimization, please see a blog post after this date.

3. Talk about the redirection and plumbing of shell. I will not give another example of this entry, just to talk about my personal understanding.

It is well known that a prominent efficiency bottleneck in many programs or languages is that IO,Shell is no exception (personally speaking). So it is recommended that you use as little redirection as possible to do input and output operations or create temporary files for subsequent use, of course, if you have to do so, I'm just talking about a process as much as possible.

We can use the pipeline provided by Shell to transfer data between commands. If you carry out continuous filtering commands for data, try to put the commands that filter more at one time first, do you understand this reason? Reduce the scale of data transmission.

Finally, I would like to say that even the pipeline should be used as little as possible. Although the pipeline is several orders of magnitude faster than the normal co-directional IO, it also requires additional resources, so design your code to reduce this overhead. For example, the sort | uniq command can be implemented using sort-u.

4. Let's talk about the sequential execution of Shell scripts. The optimization of this block depends on whether your system load has reached its limit, and if your system even reaches a high line for sequential execution of commands, there is no need for parallel modification of Shell scripts. The following is an example. If you want to imitate this optimization, please make sure your system still has load space. For example, there is a program like this:

Supportdatacommand1

Supportdatacommand2

Supportdatacommand3

Supportdatacommand4

Supportdatacommand5

Supportdatacommand6

Need13datacommand

Need24datacommand

Need56datacommand

The main idea is that there are six commands that provide data in front, followed by three commands that need data, the first command that needs data needs data 13, the second command needs 24, and the third command needs 56. But normally Shell executes these commands sequentially, from supportdatacommand1 to need56datacommand. Does this process look painful to you, too? Obviously can do this piece better, the egg pain program can be modified like this:

The code is as follows:

Supportdatacommand1 &

Supportdatacommand2 &

Supportdatacommand3 &

Supportdatacommand4 &

Supportdatacommand5 &

Supportdatacommand6 &

# 2012-02-22 ps: there is a problem with the loop here to determine whether the background command has been executed. The pidnum will not get a value of 0 if it is reduced to 1 in the # loop. For the specific solution, please see the appendix, because there are explanations, so you will not add or modify it in this #.

While true

Do

Sleep 10s

Pidnum= `jobs-p | wc-l`

If [$pidnum-le 0]

Then

Echo "run over"

Break

Fi

Done

Need13datacommand &

Need24datacommand &

Need56datacommand &

Wait

...

It can be similar to the transformation above. After this transformation, the feeling of egg pain will be much relieved. But it still doesn't feel very pleasant, so well, we can have a little more fun (I mean the program. ), which can be something like this

The code is as follows:

For ((iSuppli)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.