Shulou(Shulou.com)06/02 Report--

This article mainly talks about "what is the pipeline of Linux". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "what is the pipeline of Linux"?

Pipe is an inter-process communication mechanism inherited by Linux from Unix, and it is an important communication mechanism in the early days of Unix. The idea is to create a shared file in memory so that both sides of the communication can use the shared file to transmit information. Because this method has the characteristic of one-way transmission of data, the shared file that is used as the delivery message is called "pipeline".

In the specific implementation of the pipe, there are "anonymous pipe" and "named pipe" depending on whether the file used for communication has a name or not.

The difference between pipes and shared memory

At first glance, it doesn't feel very different between pipes and shared memory. Here's a look at the difference between the two:

Pipes need to copy data four times in kernel and user space: copy data into kernel from user space buf-> kernel copy data into memory-> memory to kernel-> kernel to user space buf. Shared memory copies data only twice: user space to memory-> memory to user space.

The pipeline is implemented with a circular queue, and the continuous transmission of data can be unlimited in size. The data size of each transfer in shared memory is fixed.

Shared memory can randomly access any location of the mapped file, and pipes can only read and write sequentially.

The pipeline can independently complete the data transmission and notification mechanism, and the shared memory needs to use other means of communication for message delivery.

In other words, the biggest difference between the two is that the shared memory area is the fastest available IPC form. Once such a memory area is mapped to the address space of the process that shares it, the transfer of data between these processes will no longer be done by performing any system calls into the kernel to transfer each other's data, saving time.

Anonymous pipeline

Anonymous pipes are a way to communicate between processes that have a common ancestor.

As mentioned earlier when introducing the creation of the process, the child process created by the parent process will assign some resources of the parent process, including files. If the parent process creates a file before creating the child process, the descriptor of the file will be shared by the parent process in subsequent child processes. That is, the parent and child processes can communicate through this file. If both sides of the communication can only read and the other can only write, then this file is a channel that can only send messages in one direction, as shown in the following figure:

The process can create a pipe by calling the function pipe (). The prototype of the function pipe () is as follows:

Int pipe (int fildes [2])

The prototype of the system call sys_pipe () corresponding to the function pipe () is as follows:

Asmlinkage int sys_pipe (unsigned long _ _ user * fildes)

In essence, the function of the pipe () function is to create a memory file, but unlike the function that creates a normal file, the function pipe () returns the two file descriptors fildes [0] and fildes [1] of this file for the process in the parameter fildes. Among them, fildes [0] is a file descriptor with "read-only" attribute, and fildes [1] is a file descriptor with "write-only" attribute, that is, processes can only read files through fildes [0], and can only write files through fildes [1].

In this way, the file is like a pipe that can only flow in one way, one end is used to input data, and the other end is used to output data, so it is called pipe. Because this kind of file does not have a file name, can not be opened by non-parent processes, and can only be used for communication between relatives, so the communication pipeline formed by such unnamed files is called "anonymous pipeline".

Obviously, if the parent process creates such a file only for communication, then it is only interested in the memory space occupied by the file, so there is no need to create an official file, just create a temporary file that exists only in memory. From this point of view, anonymous pipes have something in common with shared memory, except that anonymous pipes communicate one-way, and this communication can only be done between relative processes.

To support anonymous pipes, a special file system is installed by the kernel function kernel_mount () during kernel initialization, in which all temporary files are created.

Because the anonymous pipe is a file, it also has an I node, which is structured as follows:

Struct inode

{

...

Struct file_operations * iObjecfop; / / set of file manipulation functions

Struct pipe_inode_info * iPipex; / / pipe file pointer

...

}

You can see that in the structure of the I node, there is a pointer i_pipe of type pipe_inode_info. In a normal file, the value of this pointer is NULL, while in the pipe file, the pointer only wants a pipe_inode_info called the pipe node information structure, to indicate that this is a pipe file. The structure of pipe_inode_info is as follows:

Struct pipe_inode_info {

Wait_queue_head_t wait; / / waiting process queue

Unsigned int nrbufs, curbuf

Struct page * tmp_page

Unsigned int readers

Unsigned int writers

Unsigned int waiting_writers

Unsigned int ringing countermeasure; / / access the pipeline's process counter read-only

Unsigned int process counter; / / access pipeline process counters in write-only mode

Struct fasync_struct * fasync_readers

Struct fasync_struct * fasync_writers

Struct inode * inode

Struct pipe_buffer bugs [pipe _ BUFFERS]; / / buffer array

}

The domain bufs in the structure is the memory buffer that makes up the pipe. The buffer is described by the structure pipe_buffer:

Struct pipe_buffer {

Structure of struct page * page; / / buffer pages

Unsigned int offset, len

Operation function set pointer of const struct pipe_buf_operations * ops; / / buffer

Unsigned int flags

Unsigned long private

}

As you can see from the above data structure, a pipe is essentially a memory buffer that is managed as a file.

When you create an I node of a pipe, the domain i_fop in the structure inode is assigned rdwr_pipefifo_fops, that is, the pipe file itself is both readable and writable. Rdwr_pipefifo_fops is defined in the file linux/fs/pipe.c as follows:

Const struct file_operations rdwr_pipefifo_fops = {

.llcards = no_llseek

.read = do_sync_read

.aio _ read = pipe_read

.write = do_sync_write

.aio _ write = pipe_write

.poll = pipe_poll

.unloaded _ ioctl = pipe_ioctl

.open = pipe_rdwr_open

.release = pipe_rdwr_release

.fasync = pipe_rdwr_fasync

}

The i_fop in the open file descriptor fildes [0] and fildes [1] created for the process is given the read-only function operation set read_pipefifo_fops and the write-only function operation set write_pipefifo_fops, respectively.

The sets of operation functions read_pipefifo_fops and write_pipefifo_fops are defined in the file linux/fs/pipe.c as follows:

Const struct file_operations read_pipefifo_fops = {

.llcards = no_llseek

.read = do_sync_read

.aio _ read = pipe_read

.write = bad_pipe_w

.poll = pipe_poll

.unloaded _ ioctl = pipe_ioctl

.open = pipe_read_open

.release = pipe_read_release

.fasync = pipe_read_fasync

}

Const struct file_operations write_pipefifo_fops = {

.llcards = no_llseek

.read = bad_pipe_r

.write = do_sync_write

.aio _ write = pipe_write

.poll = pipe_poll

.unloaded _ ioctl = pipe_ioctl

.open = pipe_write_open

.release = pipe_write_release

.fasync = pipe_write_fasync

}

The relationship between the process of creating an anonymous pipe and the pipe is shown in the following figure:

When a process calls the function pipe () to create a pipe, the pipe is connected as follows:

As you can see from the figure, because the entrances and exits of the pipe are all in the same process, this kind of pipe is of little use. But when this process creates a new process, the situation becomes very different.

If the parent process creates a pipe and then creates a child process, because the child process inherits the file resources of the parent process, the connection of the pipe in the parent-child process becomes the same as in the following figure:

After determining the transmission direction of the pipe, close the (close ()) file descriptor fildes [0] in the parent process, and close the (close ()) file descriptor fildes [1] in the child process, so that the connection of the pipe becomes an one-way transport pipe in the following cases:

It is also conceivable that a communication pipeline can be implemented between two sibling processes by closing the file descriptor.

After creating the pipeline, how do you use the pipeline to communicate with the data?

The pipeline uses the read () and write () functions to use byte flow with fluidity. When reading data, each piece of data is read, the read data will be cleared in the pipeline.

When reading the pipe, if the pipe is empty, it is blocked until the write at the other end of the pipe writes the data to the pipe. If the write segment is closed, 0 is returned.

When writing the pipe, if the pipe is full, it will be blocked until the read at the other end of the pipe takes away the data in the pipe.

With the close () function, when creating a pipe, the writer needs to close the fildes [0] descriptor and the reader needs to close the fildes [1] descriptor. Before a process shuts down, each process needs to close the descriptors that are not closed.

Anonymous pipes have the following characteristics:

Because this pipeline has no other synchronization measures, in order not to create confusion, it can only be half-duplex, that is, data can only flow in one direction. If two parties are required to transfer data to each other, two pipes need to be established.

Communication can only be made between related processes, such as parent-child processes or sibling processes.

An anonymous pipe is a special file that exists only in memory for processes at both ends of the pipe.

What is written by a process to the pipe is read by the process at the other end of the pipe. The written content is added to the end of the pipeline buffer each time, and the data is read from the header of the buffer each time.

The anonymous pipeline has two main limitations: one is that the capacity of the pipeline cannot be very large because it is built in memory, and the other is that the pipeline transmits an unformatted byte stream, which requires that the two implementations of the pipeline must agree on the format of the transmitted data.

Example: use anonymous pipes to communicate between parent and child processes.

# include

# include

# include

# include

# define MAX_LINE 80

Int main ()

{

Int testPipe [2], ret

Char buf [Max _ LINE + 1]

Const char * testbuf = "data sent by the main program"

If (pipe (testbuf) = = 0) {

If (fork () = = 0) {

Ret = read (testPipe [0], buf, MAX_LINE)

Buf [ret] = 0

Printf ("data read by subroutine:% s", buf)

Close (testPipe [0])

} else {

Ret = write (testPipe [1], testbuf, strlen (testbuf))

Ret = wait (NULL)

Close (testPipe [1])

}

}

Return 0

}

Named pipe

Because the anonymous pipeline has no name, it can only communicate between some related processes, which greatly limits its application.

Named pipe is a communication mechanism implemented on the actual file system. Because it is a real and independent file that has no "blood relationship" with the process, it can communicate between any process. Because named pipes do not support file positioning operations such as lseek (), they strictly follow the first-in, first-out principle to transfer data, that is, reading to the pipe always returns data from the beginning, and writing to it always adds the data to the end, so this kind of pipe is also called FIFO file.

Similarly, because the synchronization between communication processes needs to be ensured by the pipe itself, the named pipe is also a file that can only be accessed in one direction, and the data transfer mode is FIFO.

That is, a named pipe provides a pathname associated with it, exists in the file system as a FIFO file, generates a physical file in the file system, and other processes can communicate with each other through the pipe as long as they access the file path. Open the pipe file as read-only on the read data side and write-only on the write data side.

The difference between FIFO files and ordinary files:

Ordinary files cannot be managed by byte streaming, and access to shared resources between multiple processes can cause unexpected problems.

FIFO files are managed by byte stream, follow the principle of first-in, first-out, and do not involve access to shared resources.

The operation flow is: mkfifo-> open-> read (write)-> close-> unlink.

At this point, I believe you have a deeper understanding of "what is the pipeline of Linux". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.