Shulou(Shulou.com)06/02 Report--

This article mainly explains "what are the skills of writing reliable Bash scripts". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn what are the skills of writing reliable Bash scripts.

When writing a script, add the following sentence at the beginning (after Shebang), or an abbreviated version of it, to avoid a lot of problems and, more importantly, expose many hidden problems:

Set-xeuo pipefail

The following describes the role of each parameter and how to handle some exceptions:

-x: print out the command after the variable is expanded before executing each command.

This is very useful for debug scripts and when outputting Log. Scripts that are officially run can also be left out.

-e: exit immediately when a command fails (the return code is non-zero).

One of the biggest differences between bash and other scripting languages is that you should continue to run the next command when you encounter an exception. This will encounter unexpected problems in many cases. Adding-e causes bash to exit immediately if a command fails.

If you do need to ignore the return codes of individual commands sometimes, you can use | | true. Such as:

Some_cmd | | true # even if some_cmd fails, it will continue to run some_cmd | | RET=$? # or you can collect the return code of some_cmd for later logic judgment.

However, in the case of multiple commands strung together in the pipe, only the last command fails to exit. If you want any of the commands in the pipeline to fail and exit, use the-o pipefail mentioned later.

Adding-e may sometimes be inconvenient and easy to quit. However, you should adhere to the so-called fail-fast principle, which is to stop running normally when there is an exception, rather than continuing to try to run a process that may be defective. If there is a command that explicitly ignores exceptions, you can explicitly ignore them using the methods mentioned above, such as | | true, etc.

-u: if you try to use an undefined variable, exit immediately.

If you use an undefined variable in bash, the default is to expand into an empty string. Sometimes this behavior can lead to problems, such as:

Rm-rf $MYDIR/data

If the MYDIR variable is not assigned for some reason, this command becomes rm-rf / data. This is more funny. This can be avoided by using-u.

But sometimes after-u has been set, some places still want to expand the undefined variable to an empty string, you can write:

${SOME_VAR:-} # bash variable expansion syntax, please refer to: https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html

-o pipefail: whenever one subcommand in the pipe fails, the entire pipe command fails.

If pipefail is used in conjunction with-e, you can exit the script if a subcommand in the pipe fails.

1. Prevent overlapping operation

In some scenarios, we usually don't want multiple instances of a script running at the same time. For example, when you run a script periodically with crontab, sometimes you don't want the next round to start before the last round is finished. This can be solved with the flock command. Flock ensures exclusive operation through file locks, and another advantage is that when a process exits, the file lock is automatically released without additional processing.

Usage 1: assuming your entry script is myscript.sh, you can create a new script and run it through flock:

# flock-- wait timeout-e lock file-c "commands to be executed" # for example: flock-- wait 5-e "lock_myscript"-c "bash myscript.sh"

Usage 2: you can also use flock in the original script. You can open the file as a file descriptor and lock it using flock (flock accepts file descriptor parameters).

Exec 123lock_myscript # opens lock_myscript as file descriptor 123flock-- wait 5123 | | {echo 'cannot get lock, exit'; exit 1;}

two。 Kill all child processes on unexpected exit

Our script usually starts many child scripts and processes, and when the parent script exits unexpectedly, the child process does not quit, but continues to run. If the script is run periodically, some unexpected problems can occur.

One method found on stackoverflow is to use the trap command to kill the entire process group when the script exits. Add the following code to the beginning of the script, and the actual measurement works:

Trap "trap-SIGTERM & & kill-$" SIGINT SIGTERM EXIT

But not if the parent process is killed with SIGKILL (kill-9). Because when SIGKILL, the process does not have a chance to run any code.

3. Timeout limits the running time

Sometimes you need to set a timeout for the command. At this point, you can use the timeout command, which is simple:

Timeout 600s some_command arg1 arg2

When the command ends within the timeout period, the return code is 0, otherwise a non-zero return code is returned.

Timeout defaults to sending TERM signals when it times out, or you can have it send other signals with the-s parameter.

4. When continuous piping, consider using tee to drop the intermediate results in order to find the problem.

Sometimes we use a situation where many commands are piped together. For example, cmd1 | cmd2 | cmd3 |... This makes the problem difficult to troubleshoot because we can't see the intermediate data.

If you change it to this format:

Cmd1 > out1.dat cat out1 | cmd2 > out2.dat cat out2 | cmd3 > out3.dat

The performance is not very good, because cmd1, cmd2, and cmd3 run serially, so you can use the tee command:

Cmd1 | tee out1.dat | cmd2 | tee out2.dat | cmd3 > out3.dat Thank you for reading, these are the contents of "what are the skills of writing reliable Bash scripts". After the study of this article, I believe you have a deeper understanding of the skills of writing reliable Bash scripts, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.