Shulou(Shulou.com)06/03 Report--

This article mainly introduces "the method of Java NIO multiplexing and the detailed explanation of the principle of Linux epoll realization". In the daily operation, I believe that many people have doubts about the method of Java NIO multiplexing and the detailed explanation of the principle of Linux epoll realization. Xiaobian consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts of "the method of Java NIO multiplexing and the detailed explanation of Linux epoll realization principle". Next, please follow the editor to study!

Why do I have to use Icano Multiplexing?

When we need to constantly read data from a descriptor called r_fd and write the read data to a descriptor called w_fd, we can use the loop blocking iCandle O:

While ((n = read (r_fd, buf, BUF_SIZE)) > 0) if (write (w_fd, buf, n)! = n) err_sys ("write error")

But what if you want to read data from two places? At this point, you can no longer use the read function that blocks the program. Because there may be no time to process the r_fd2 while blocking the r_fd1 data, the data of the r_fd2 that has arrived may be lost.

In this case, you need to use non-blocking Ibank O.

Just make a mark, mark the file descriptor as non-blocking, and then use the read function on it later: if it doesn't have data to read, the function will immediately return and set the value of errorno to 35, so we know it has no data to read, and then we can immediately use read; on other descriptors if it has data to read, we read its data. After calling read on all the descriptors to be read, we can wait a long time (for example, a few seconds) before calling read from the first file descriptor. This kind of cycle is called polling.

In this way, the program is not blocked because a descriptor read is waiting for data for a long time, as it did with blocking Ihand O.

The disadvantage of polling is that too much CPU time is wasted. Most of the time we have no data to read, but we still use the system call read, which switches from the user mode to the kernel state when using the system call. In most cases, we call read and get stuck in kernel state. The kernel finds that the descriptor is not ready, then switches back to user mode and only gets EAGAIN (errorno is set to 35), doing useless work. When there are so many descriptors, each switching process is a huge waste.

Therefore, Imax O multiplexing is required. Ipaw O multiplexing waits for the readable and writable state of multiple descriptors by using a single system function.

To achieve this, what we need to do is to create a list of descriptors and what events we care about (readable or writable or exceptions), and call a system function. This function will not return until at least one event associated with the descriptor occurs in the descriptor list.

Select, poll, epoll are such system functions.

Select

We can use the select function in all POSIX-compatible systems for I-do-O-multiplexing. The messages we need to pass to the kernel through the parameters of the select function are:

* which descriptors do we care about * what events do we care about * how long do we want to wait?

When select returns, the kernel tells us:

* number of readable descriptors * which descriptors occurred and which events # include int select (int maxfdp1, fd_set* readfds, fd_set* writefds, fd_set* exceptfds, struct timeval* timeout); / / return value: the number of descriptors that are ready. 0 for timeout and-1 for error

Maxfdp1 means "max file descriptor plus 1", which means adding 1 to the largest of all the file descriptors you want to monitor. (it actually determines the number of times the kernel traverses file descriptors. For example, if you monitor file descriptors 5 and 20 and set maxfdp1 to 21, the kernel checks from descriptor 0 to 20 each time. )

The middle three parameters are a collection of file descriptors you want to monitor. You can think of the fd_set type as a 1024-bit binary number, which means that select can only monitor file descriptors less than 1024 (1024 is the value set by the FD_SETSIZE macro in Linux's sys/select.h). After the select returns, we use FD_ISSET to determine whether the descriptor representing the bit is in a ready state.

The last argument is the length of time to wait for the timeout: when this length of time is reached but no descriptor is available, the function returns 0.

Use a code snippet to show the use of select:

/ / this example monitors the readable status of file descriptors 3, 4, and the writable status of 4, 5 / initializes two fd_set and timeval fd_set read_set, write_set; FD_ZERO (read_set); FD_ZERO (write_set); timeval t; t.tv_sec = 5; / / timeout is 5 seconds t.tv_usec = 0 / / add 0 microseconds / / set two fd_set int fd1 = 3; int fd2 = 4; int fd3 = 5; int maxfdp1 = 5 + 1; FD_SET (fd1, & read_set); FD_SET (fd2, & read_set); FD_SET (fd2, & write_set); FD_SET (fd3, & write_set) / / prepare standby fd_set fd_set r_temp = read_set; fd_set w_temp = write_set; while (true) {/ / reset fd_set read_set = rroomtemp; write_set = wroomtemp; / / use select int n = select (maxfdp1, & read_set, & write_set, NULL, & t) every time you put it into select / / the select function above will block until / / 3, 4 is readable and 4, 5 is writable. At least one of these four events occurs / / or the waiting time reaches 5 seconds, and returns 0 for (int item0; i0; iFue +) {if (FD_ISSET (I, & read_set)) {NFue- If (i==fd1) prinf ("descriptor 3 readable"); if (i==fd2) prinf ("descriptor 4 readable");} if (FD_ISSET (I, & write_set)) {nmuri- If (i==fd2) prinf ("descriptor 3 writable"); if (i==fd3) prinf ("descriptor 4 writable") }} / / the above printf statement is replaced by the corresponding read or write function and / / the corresponding descriptor can be read or written immediately without waiting}

As you can see, the disadvantages of select are:

By default, the file descriptor that can be monitored cannot be greater than 1024, which also means that the total number of monitored files does not exceed 1024. Even if you change the kernel FD_SETSIZE value because the descriptor you need to monitor is greater than 1024, because select linearly scans the entire fd_set every time, the larger the collection, the slower the speed, so the performance will be poor.

When the select function returns, you can only see the number of descriptors that have been prepared. As to which descriptor is ready, it needs to be checked by FD_ISSET in a loop. When there are many unprepared descriptors and few are prepared, the efficiency is relatively low.

Each time the select function executes, it copies the three fd_set passed in the argument from user space to kernel space. It's not cost-effective to copy them all every time the descriptors you want to monitor in fd_set don't change much. Similarly, when there are many unprepared descriptors and few are prepared each time, select is called frequently, and data replication between users / kernels becomes a big overhead.

Another problem that bothers me about the way the code is written is that I have to reset three fd_set before calling select. The fd_set type is only a 1024-bit binary number (actually an array of several long variables in the structure; for example, if long is 64 bit on a 64-bit machine, then there is an array of 16 long variables in fd_set). The 1 and 0 of a bit represent the state of a file descriptor, but the meaning of the state of 1 select 0 before and after calling select is not the same.

Let's first talk about the effects of several functions on fd_set operations: FD_ZERO sets all bits of fd_set to 0; FD_SET sets a bit to 1; and FD_ISSET determines whether a bit is 1 or not.

Before calling select: we initialize all the fd_set with FD_ZERO, and then set the bit of the representative descriptor we care about to 1 with FD_SET. At this point, we can use FD_ISSET to determine whether this bit is set by us, which means whether the descriptor we want to monitor is set to the monitored state.

When calling select: the kernel judges the bits in the fd_set and records all the bits with a value of 1 in each fd_set, and then sets the fd_set to 0; when a corresponding event occurs on a descriptor, the bit representing the descriptor in the corresponding fd_set is set to 1.

After select returns: we also use FD_ISSET to determine whether each bit we care about is 0 or 1, which means whether this bit has occurred or not.

So, we have to reset all the fd_set that has been changed by the kernel before the next call to select.

Select performs poorly when monitoring a large number of descriptors, especially when more descriptors are not ready. According to Unix Advanced programming, programs that use select usually use only 3 to 10 descriptors.

Poll

Poll and select are similar, but given different interfaces.

# include int poll (struct pollfd fdarray [], nfds_t nfds, int timeout); / / return value: the number of descriptors that are ready. 0 for timeout and-1 for error

Fdarray is an array of pollfd. The pollfd structure looks like this:

Struct pollfd {int fd; / / File descriptor short events; / / events I expect short revents; / / events that actually occur: events that I expect to happen; or exceptions}

Nfds is the length of the fdarray, that is, the number of pollfd.

Timeout represents the number of milliseconds to wait for a timeout.

Compared to select, poll has these advantages: because poll uses int fd to represent file descriptors in pollfd rather than fd_set used in select, there is no limit that must be less than 1024 and there is no quantity limit; because poll uses events to express expected events and modify revents to represent events that occur, there is no need to reset descriptors and expected events before each call like select.

In addition, poll and select are almost the same. After the poll returns, you need to traverse the fdarray to check whether the expected events have occurred in the revents in each pollfd; each time you call poll, copy the fdarray to the kernel space. Poll has the same performance problem when there are too many descriptors and fewer are prepared at a time.

Epoll

Epoll made his debut in Linux 2.5.44. Unlike select and poll, it provides three system functions instead of one.

Epoll_create is used to create an epoll descriptor: # include int epoll_create (int size); / / return value: epoll descriptor

Size is used to tell the kernel the number of file descriptors you want to monitor, but instead of limiting the maximum number of descriptors that can be monitored, it is a suggestion for the initial allocation of space for the kernel. The system then allocates a space in the kernel to hold the event table and returns an epoll descriptor to manipulate the event table.

Epoll_ctl is used to add / delete / modify the event table in the kernel: int epoll_ctl (int epfd, int op, int fd, struct epoll_event * event); / / return value: 0 for success,-1 for failure

Epfd is the epoll descriptor.

Op is the type of operation (add / delete / modify).

Fd is the file descriptor you want to monitor.

Event is a pointer to an epoll_event structure. Epoll_event is defined like this:

Typedef union epoll_data {void * ptr; int fd; uint32_t U32; uint64_t U64;} epoll_data_t;struct epoll_event {uint32_t events; / / event epoll_data_t data; / / user data variable}

In this structure, in addition to the expected event, there is a data, a union, which is used to conveniently locate the file descriptor after we get the return value of the third function below.

Epoll_wait is used to wait for the event int epoll_wait (int epfd, struct epoll_event * result_events, int maxevents, int timeout); / / return value: the number of descriptors ready. 0 for timeout and-1 for error

Epfd is the epoll descriptor.

Result_events is a pointer to the epoll_event structure that will point to the epoll_event associated with all prepared event descriptors (associated when epoll_ctl was called in the previous step). The following example will let you know the meaning of this parameter.

Maxevents is the maximum number of events returned, that is, the maximum number of times you can traverse through the result_events pointer.

Timeout is the number of milliseconds to wait for a timeout.

Use a code snippet to show the use of epoll:

/ / this example monitors the readable status of file descriptors 3, 4, and the writable status of 4, 5

/ * create the epoll descriptor through epoll_create * / int epfd = epoll_create (4); int fd1 = 3 / struct epoll_event ev1;ev1.events = 4 / int fd3 = 5 * register four events through epoll_ctl * / struct epoll_event ev1;ev1.events = EPOLLIN; / / expect its readable event to occur ev1.data = fd1 / / We usually set data to fd to facilitate later viewing of epoll_ctl (epfd, EPOLL_CTL_ADD, fd1, & ev1); / / add to the event table struct epoll_event ev2;ev2.events = EPOLLIN;ev2.data = fd2;epoll_ctl (epfd, EPOLL_CTL_ADD, fd2, & ev2); struct epoll_event ev3;ev3.events = EPOLLOUT; / / expect its writable event to occur ev3.data = fd2 Epoll_ctl (epfd, EPOLL_CTL_ADD, fd2, & ev3); struct epoll_event ev4;ev4.events = EPOLLOUT;ev4.data = fd3;epoll_ctl (epfd, EPOLL_CTL_ADD, fd3, & ev4); / * waiting for events through epoll_wait * / # DEFINE MAXEVENTS 4struct epoll_event result_events [MAXEVENTS]; while (true) {int n = epoll_wait (epfd, & result_events, MAXEVENTS, 5000); for (int iTuno; I)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.