Shulou(Shulou.com)06/01 Report--

This article mainly introduces the Linux Shell multi-process concurrency and concurrency control example analysis, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let Xiaobian take you to understand.

1. Basic knowledge preparation

1.1. Linux background process

Unix is a multitasking system that allows multiple users to run multiple programs at the same time. Metacharacters of shell & provides a way to run programs that do not require keyboard input in the background. After entering the command, followed by the & character, the command is sent to the linux background for execution, and the terminal can continue to enter the next command.

For example:

Sh a.sh & sh b.sh & sh c.sh &

These three commands are sent to the linux background for execution at the same time, to this extent, the three commands are considered to be executed concurrently.

1.2. Linux file descriptor

Undefined

The file descriptor (abbreviated fd) is formally a non-negative integer. In fact, it is an index value that points to the record table of files opened by the kernel for each process maintained by that process. When a program opens an existing file or creates a new file, the kernel returns a file descriptor to the process. Each unix process has three standard file descriptors that correspond to three different streams:

The file descriptor name is 0Standard Input1Standard Output2Standard Error

Each file descriptor corresponds to an open file, and different file descriptors can also correspond to the same open file; the same file can be opened by different processes or multiple times by the same process.

In / proc/PID/fd, enumerate the file descriptors owned by the process PID, such as

#! / bin/bashsource / etc/profile;# $indicates the PIDPID=$# of the current process to view the file descriptor of the current process points to ll / proc/$PID/fdecho "-" Echo# file descriptor 1 binds to file tempfd1 ([- e. / tempfd1] | | touch. / tempfd1) & & exec 1./tempfd1# to view the current process's file descriptor pointing to ll / proc/$PID/fdecho "-"; echo; [ouyangyewei@localhost learn_linux] $sh learn_redirect.sh total 0lrwxink. 1 ouyangyewei ouyangyewei 64 Jan 4 22:17 0-> / dev/pts/0lrwx-. 1 ouyangyewei ouyangyewei 64 Jan 4 22:17 1-> / dev/pts/0lrwx-. 1 ouyangyewei ouyangyewei 64 Jan 4 22:17 2-> / dev/pts/0lr-x-. 1 ouyangyewei ouyangyewei 64 Jan 4 22:17 255-> / home/ouyangyewei/workspace/learn_linux/learn_redirect.sh- [ouyangyewei@localhost learn_linux] $cat tempfd1 total 0lrwx Mel. 1 ouyangyewei ouyangyewei 64 Jan 4 22:17 0-> / dev/pts/0lrwx-. 1 ouyangyewei ouyangyewei 64 Jan 4 22:17 1-> / home/ouyangyewei/workspace/learn_linux/tempfd1lrwx-. 1 ouyangyewei ouyangyewei 64 Jan 4 22:17 2-> / dev/pts/0lr-x-. 1 ouyangyewei ouyangyewei 64 Jan 4 22:17 255-> / home/ouyangyewei/workspace/learn_linux/learn_redirect.sh-

In line 12 of the above example, the file descriptor 1 is bound to the file tempfile. After that, the file descriptor 1 points to the tempfile file, and the standard output is redirected to the file tempfile.

1.3. Linux pipeline

In Unix or Unix-like operating systems, a pipe is a collection of processes linked by standard input and output, so the output of each process will be directly used as input to the next process.

There are two types of linux pipes:

Anonymous pipe named pipe

A characteristic of the pipeline is that if there is no data in the pipeline, the operation of fetching the pipeline data will be held up until the data is entered into the pipeline and then read out; similarly, if the operation of writing to the pipe does not read the operation of the pipe, this action will be held up.

1.3.1. Anonymous pipeline

On the command line of Unix or Unix-like operating systems, anonymous pipes use the vertical line in ASCII as anonymous pipe characters. There are two normal, anonymous, open file descriptors on both sides of the anonymous pipe: a read-only side and a write-only side, which makes it impossible for other processes to connect to the anonymous pipe.

For example:

Cat file | less

To execute the above instructions, Shell creates two processes to execute cat and less, respectively. The following figure shows how these two processes use pipes:

It is worth noting that both processes are connected to the pipe, so that the write process cat connects its standard output (file descriptor fd 1) to the write side of the pipe, and the reader process less connects its standard input (file descriptor fd 0) to the read end of the pipe. In fact, these two processes are not aware of the existence of pipes; they just read and write data from standard file descriptors. Shell must complete the relevant work.

1.3.2. Named pipe (FIFO,First In First Out)

Named pipes are also called FIFO. Semantically speaking, FIFO is actually similar to anonymous pipes, but it is worth noting:

In the file system, FIFO has a name and exists in the form of device custom files; any process can share data through FIFO; unless there are read and write processes on both sides of the FIFO, the data flow of FIFO will be blocked; anonymous pipes are automatically created by shell and exist in the kernel; while FIFO is created by programs (such as the mkfifo command) and exists in the file system Anonymous pipes are an one-way byte stream, while FIFO is a two-way byte stream

For example, you can use FIFO to implement single-server, multi-client applications:

With the above knowledge preparation, you can now begin to talk about how to control the number of concurrent processes at a time when multiple processes are concurrent in linux.

2. Linux multi-process concurrency control

Recently, Xiao A needs to produce KPI data reports for the whole year of 2015. Now that Xiao A has written the production script, the production script can only produce KPI data for one day at a time. If it takes 5 minutes to run a production script, then:

* if it is executed in a circular sequence, it will take time: 5 * 1825 minutes, which is about 6 days.

* if it is put into the linux backend and executed concurrently, the system can not bear the 365 background tasks.

Since you can't put 365 tasks in the background of linux at a time, can you automatically put N tasks in the background and execute them concurrently? Of course you can.

#! / bin/bashsource / etc/profile #-tempfifo=$.fifo # $represents the PIDbegin_date=$1 # start time end_date=$2 # end time if [$#-eq 2] then if ["$begin_date"\ > "$end_date"] then echo "Error! $begin_date of the current execution file Is greater than $end_date "exit 1 Fielse echo "Error! Not enough params. " Echo "Sample: sh loop_kpi 2015-12-01 2015-12-07" exit 2 exec #-- trap "exec 1000 >; exec 1000

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.