Shulou(Shulou.com)06/03 Report--

This article mainly introduces the example analysis of pipeline and execution sequence in Shell, which is very detailed and has certain reference value. Friends who are interested must read it!

1. Basic concepts

A, I FD O redirection is usually related to FD, and the number of FD of shell is usually 10, that is, zero nine.

B. There are 3 commonly used FD, which are 0 (stdin, standard input), 1 (stdout, standard output) and 2 (stderr, standard error output). By default, they are related to keyboard, monitor and monitor.

C, use

< 来改变读进的数据信道(stdin)，使之从指定的档案读进； d、用 >

To change the outgoing data channel (stdout, stderr) to output to the specified file

E, 0 is

< 的默认值，因此 < 与 0 与 1>

It's the same.

F. In IO redirection, the pipeline between stdout and stderr will be ready before the data is read from stdin.

G, pipe "|" (pipe line): the stdout of the previous command receives the stdin of the next command

The h and tee commands are to copy a copy of the stdout to the file without affecting the original Imax O.

I. Bash (ksh) the process of executing a command: analyze command-variable evaluation-command substitution (``and $())-redirect-wildcard expansion-determine path-execute command

J, () put command group into sub-shell to execute, also known as nested sub-shell, which has one very important feature: inheriting the Standard input, output, and error plus any other open file descriptors of the parent shell.

K, exec commands: commonly used to replace the current shell and restart a shell, in other words, there is no promoter shell. Any existing environment will be cleared when using this command. When exec operates on file descriptors, and only then, exec will not overwrite your current shell environment.

2. Basic IO

Cmd > file redirects stdout to the file file

Cmd > > file redirects stdout to the file file (append)

Cmd 1 > fiel redirects stdout to the file file

Cmd > file 2 > & 1 redirect stdout and stderr to the file file together

Cmd 2 > file redirects stderr to the file file

Cmd 2 > > file redirects stderr to file file (append)

Cmd > > file 2 > & 1 redirect stderr and stderr to the file file together

Cmd

< file >

The file2 cmd command takes file file as stdin and file2 file as stdout

Cat file opens file in read and write mode

Cmd

< file cmd 命令以 file 文件作为 stdin cmd &n 使用系统调用 dup (2) 复制文件描述符 n 并把结果用作标准输出 file 运行一个命令并把错误输出(文件描述符 2)定向到 file。 ... 2>

& 1 runs a command and merges its standard output and output. (strictly speaking, file descriptor 2 is created by copying file descriptor 1, but the effect is usually to merge two streams.)

2 > & 1 description: 2 > & 1, that is, FD2=FD1, this does not mean that the value of FD2 is equal to the value of FD1, because > is to change the data channel sent, popular to say: add stderr to stdout.

However, when using a form such as cmd 1 > & 3, the principle is the same, but it is often different from 2 > & 1 and 1 > & 2, which are usually used for merging.

Note: there is a difference between cmd n > & n and exec n > & n for normal cmd commands.

Exec 0 exec 1 > outfilename # Open the file outfilename as stdout

Exec 2 > errfilename # Open the file errfilename as stderr

Exec 0 close-# close FD1

Exec 5 > &-# close FD5

Q:

What are the consequences of shutting down FD0, FD1, FD2?

What is the difference between restoring FD0, FD1, FD2 and closing FD0, FD1, FD2? What are the codes?

After opening the FD3~FD9, after we have finished using it, do you think we should shut them down or restore them?

Exec 6 > & 22 > ver # FD2 (originally sent to monitor) is directed to the file ver

Command > > dev/null & # discard FD1 (stdout)

Exec 2 > & 6 # restore FD2

4. Simple example (where yes.txt exists, but no.txt does not exist)

A, stdout and stderr are all sent to egrep through the pipeline:

(ls yes.txt 2 > & 1 * ls no.txt 2 > & 1) 2 > & 1 | egrep\ * > file

(ls yes.txt;ls no.txt) 2 > & 1 | egrep\ * > file

# # #

This example is to let everyone understand the command execution order and pipeline "|"

Before the command is executed, the redirection is processed, and the stdout of the nested sub-shell is connected to the stdin of the egrep command.

Nested sub-shell, the two commands in () can be thought of as one command. Its stdout (FD1) passes "|" as the stdin of egrep, and when 2 > & 1, the initial stdout and stderr are sent to the pipeline "|".

# # #

B. Nothing is sent to egrep through the pipeline, all to monitor.

(ls yes.txt 2 > & 1 * ls no.txt 2 > & 1) > & 2 | egrep\ * > file

Although FD2 is transferred to FD1 in (), outside (), > & 2 is encountered, and as a result, all are sent to monitor.

5. Middle-level examples (you file exists, but no and wu files do not exist)

R2007 brother's: http://bbs.chinaunix.net/forum/viewtopic.php?t=221848&show_type=new&sid=cf30398c911e0d2b16313c6922123f67

Condition: stderr is sent to egrep via pipeline, and the correct message is still sent to monitor (unchanged)

Exec 4 > & 1; (ls you no 2 > & 11 > & 44 > &; ls wu 2 > & 11 > & 44 > & -) | egrep\ * > file;exec 4 > &-

Or

Exec 4 > & 1; (ls you no;ls wu) 2 > & 11 > & 4 > & & | egrep\ * > file;exec 4 > &-

Brother r2007 has detailed instructions in his post, if two conditions are added:

(1) cmd1 and cmd2 are required to run in parallel.

(2) assign the return value of cmd1 to the variable ss.

Is as follows:

Exec 3 > & 1 * * exec 4 > & 1

Ss=$ (ls you no 2 > & 1 1 > & 3 > &; echo $? > & 4) | egrep\ * > file) 4 > & 1)

Exec 3 > &; exec 4 > &-

Description:

Exec 3 > & 1 * 4 > & 1

# establishing FD3 is used to restore the FD1 in the following ls statement (sub-shell) to normal FD1, that is, output to monitor. You can think of FD3 as a hard disk backup of the original FD1 (that is, output to monitor)

# create a FD4, which will be used to save the return value of ls (echo $?). You can think of FD4 as storing the calculation "echo $?" when you take the exam. Draft paper

(ls you no 2 > & 1 1 > & 3 3 > &; echo $? > & 4)

# remember the sub-shell and pipes mentioned earlier. This command first inherits FD0, FD1, FD2, FD3, FD4, which is located in front of the pipe, so connect the sub-shell's own FD1 to the pipe "|" before running the command.

But our condition is that stderr is piped to egrep,stdout and still output to monitor.

So through 2 > & 1, first send the pipe of the FD1 of the child shell to the FD2, so the stderr in the child shell is sent to the pipe "|"

Then through 1 > & 3, the previous "hard disk backup" is restored to the FD1 of the sub-shell, so the FD1 in the sub-shell is sent to monitor.

Then close 3 through 3 > & -.

Then run echo $?, whose output value should have been sent to the pipeline, through > & 4, send the output to the "draft paper" FD4, leaving it for backup.

((ls you no 2 > & 1 1 > & 3 3 > &; echo $? > & 4) | egrep\ * > file)

So stderr piped it to egrep, stdout sent it to monitor, but then there was FD4, where did it go?

$(ls you no 2 > & 1 1 > & 3 > &; echo $? > & 4) | egrep\ * > file) 4 > & 1)

The last 4 > & 1 is to redirect FD4 to FD1. But because its output is in $(), its value is assigned to the variable ss.

The last line closes FD3 and FD4.

6. High-level examples

Lightspeed moderator's Big: Shell Classical problem [Imax O redirection] (http://bbs.chinaunix.net/forum/viewtopic.php?t=452079&show_type=new)

[Q] for commands cmd1, cmd2, cmd3, cmd4. How to use one-way pipes to perform the following functions:

1. All commands are executed in parallel

2. Cmd1 and cmd2 do not need stdin

3. Stdout of cmd1 and cmd2 is directed to stdin of cmd3

4. Stderr of cmd1 and cmd2 is directed to stdin of cmd4

5. Stdout of cmd3 is directed to file a, and stderr is directed to the screen

6. Stdout of cmd4 is directed to file b, and stderr is directed to the screen

7. The return code of cmd1 is assigned to the variable s

8. Temporary files cannot be used

Solution:

Exec 3 > & 1; exec 4 > & 1

My_value=$ (cmd1 1 > & 3; echo $? > & 4) | cmd2) 3 > & 1 | cmd3 > a 2 > & 3) 2 > & 1 | cmd4 > b) 4 > & 1)

Exec 3 > &; exec 4 > &-

Explanation:

Exec 3 > & 1; exec 4 > & 1

Establish FD3, restore its FD1 use to cmd1 and restore its FD2 use to cmd3

Set up FD4 and save "echo $?" Output value of "draft paper"

The first pair of parentheses to the first pipe: (cmd1 1 > & 3; echo $? > & 4) |

Cmd1 itself does not have stdin, its stdout is originally to be sent to the first pipeline, because of the function of 1 > & 3, its stdout is sent to FD3;, and the function of > & 4 is actually to send the return code after cmd1 operation to FD4. The stderr of cmd1 waits for the next step by default. In the end, nothing was sent to the pipe

Second pair of parentheses to the second pipe: (cmd1 1 > & 3; echo $? > & 4) | cmd2) 3 > & 1 |

The stdout of cmd2 is the default

The second pair of parentheses shows a command that sends all its stdout to the second pipe "|". At the same time, due to the function of 3 > & 1, the original stdout of cmd1 is sent to FD3 and merged with the stdout of cmd2, so the stdout of cmd1 and cmd2 are sent to the second pipe "|". However, the stderr of cmd1 and cmd2 still waits for the next step by default.

The third pair of parentheses to the third pipe: ((cmd1 1 > & 3; echo $? > & 4) | cmd2) 3 > & 1 | cmd3 > a 2 > & 3) 2 > & 1 |

Cmd3 > a 2 > & after 3:cmd3 receives and processes the stdin from the pipeline, its stdout is sent to file a, and its stderr is sent to FD3. Because FD3 inherits FD1, its stderr is actually sent to monitor.

Treat the third parenthesis as a command whose stdout is sent to the pipe "|", and because 2 > & 1, stderr is also sent to the pipe. However, since the stdout of cmd1 and cmd2 has been sent to cmd3 for processing, and the stdout output of cmd3 to the file AMagincmd3 stderr is also sent to monitor, in fact, only the stderr of cmd1 and cmd2 is sent to the pipeline.

After cmd4 > b:cmd4 receives and processes the stdin from the pipeline, its stdout is directed to the file bjournal stderr and output to monitor by default.

The fourth pair of parentheses: (cmd1 1 > & 3; echo $? > & 4) | cmd2) 3 > & 1 | cmd3 > a 2 > & 3) 2 > & 1 | cmd4 > b) 4 > & 1

The FD1 and FD2 of all the commands in the four pairs of parentheses have been processed, but there is still "echo $? > & 4". The function of "4 > & 1" is to "merge FD4 to stdout", but since the stdout of other commands has been processed, it is actually only $? The value of.

Is assigned to the variable my_value

+

I try to answer the following questions. If there are any mistakes, please correct them!

7. In an interactive (Interactive) shell, exec is used to redirect.

1)。 Stdin, can stderr be directed to the file?

In interactive shell, you can direct stdin to a file. Execute: exec 0 result: every line in the in file is executed automatically, and at the end, another exit command is executed, resulting in exiting (or returning to the normal shell).

For example, the content of in file: $more in

Date

Read lsp

Echo hahha

Echo "this is $lsp"

Execute the command at the prompt: $exec 0$ date

Tue Jan 18 18:29:07 HKT 2005

The "" of the original sentence "echo hahha" below $read lsp # has been read into the variable lsp

$echo "this is $lsp"

This is echo hahha

$exit

B. in interactive shell, you can direct stderr to a file. Execute: exec 2 > err

The result is that the command prompt PS is masked, and so are the commands entered. However, the result of the command execution will be echoed to the screen if it is stdout, but not to the screen if it is stderr. Among them, the command prompt, command, stderr will be saved to the file err. Such as:

$exec 2 > err

Err in out # execute ls command

Tue Jan 18 18:55:58 HKT 2005 # executes the date command, followed by "ls nofile". The file nofile does not exist.

$# execute exit command

Now let's look at the err file:

$more err

[lsp@ii lsp] $ls

[lsp@ii lsp] $date

[lsp@ii lsp] $ls nofile

Ls: nofile: No such file or directory

[lsp@ii lsp] $exit

Exit

C. In interactive shell, you can direct stdout to a file. This is what we often use. I won't say anything. Is to direct the wrong output to the file. The correct output is not affected.

2)。 Stdin, can Stderr be shut down? What's the result?

In interactive shell, if you close stdin, such as: exec 0directed -, the state is the same as stderr directed to the file, the only difference is that it is not saved.

In interactive shell, if you close stdoutr, such as: exec 1 > & -, whenever you execute a command with stdout or stderr content sent to monitor, such as ls or date, an error will be reported: "ls: write error: Bad file descriptor". Others such as cd, mkdir, etc. Such commands are not affected.

3)。 If stdin, stdout, stderr are redirected or closed, but the original FD is not saved, can it be restored to the default state?

* * if stdin is closed because it will cause an exit, it certainly cannot be restored.

* if one of stdout and stderr is redirected or closed, it can be restored because they are sent to monitor by default (but I don't know if there will be any other effects). For example, restore redirected or closed stdout:exec 1 > & 2, restore redirected or closed stderr:exec 2 > & 1.

* if both stdout and stderr are disabled and the original FD is not saved, you can restore it with: exec 1 > / dev/tty.

+

Refer to the reply of brother r2007 below! I would like to thank you.

+

8. Why is cmd > a 2 > a different from cmd > a 2 > & 1?

Cmd > a 2 > a: both stdout and stderr are sent directly to file a, and file an is opened twice, causing stdout and stderr to overwrite each other.

Cmd > a 2 > & 1: stdout is sent directly to file a, and stderr is sent to file an after inheriting the pipe of FD1. The a file is opened only once, and FD1 opens it.

The differences between them are:

Cmd > a 2 > an is equivalent to using FD1 and FD2, two pipes that compete with each other to use file a.

Cmd > a 2 > & 1 uses only one pipe FD1, but already includes stdout and stderr.

In terms of IO efficiency, the efficiency of cmd > a 2 > & 1 is higher.

The above is all the contents of the article "sample Analysis of pipes and execution order in Shell". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.