Shulou(Shulou.com)05/31 Report--

This article mainly explains "what are the problems related to the Icano model". The content of the explanation in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn what are the problems related to the Icano model.

1. The problem of Imax O model

Recently, through the study of the ucore operating system, let me open the black box of the operating system kernel, combined with the previous knowledge to answer some of the questions that have perplexed me for a long time.

1. Why can redis handle tens of thousands of concurrent requests in a single worker thread?

two。 What is Icano multiplexing? Why do servers such as redis, nginx, nodeJS and netty, which are famous for their high performance, take advantage of Icano multiplexing technology at the underlying layer?

3. Why is the non-blocking Iripple O so popular that it replaces the traditional blocking Imax O in many scenarios?

4. Is non-blocking IBG O really a silver bullet? Why are there so many servers working under the traditional blocking IO model, even in Internet companies that pursue high performance, even if they serve a large number of users?

5. What is a cooperative journey? Why is the Go language so popular?

In this blog, I will introduce the principles of different levels and different Icano models, and try to give my answers to the above questions. If you are more or less confused about the above questions, I hope this blog can help you.

The Icano model is closely related to hardware and operating system kernel, and the blog will cover concepts related to operating system and hardware, such as protected mode, interrupt, privilege level, process / thread, context switching, system call and so on. Because the knowledge in the computer is organized in a hierarchical way, it may affect the understanding of the overall content if you don't know much about these relatively low-level concepts. You can refer to my blog about operating system and hardware learning: x86 compilation learning, operating system learning (continuously updated).

Second, the hardware Icano model.

The function of the software is always built on the hardware, and the Imax O in the computer is essentially the one-way or two-way transmission of data between CPU/ memory and peripherals (network card, disk, etc.).

Reading data from peripherals to CPU/ memory is called Input input, and writing data from CPU/ memory to peripherals is called Output output.

In order to understand the different Icano models at the software level, we must first have a basic understanding of the hardware Icano model on which it is based. There are three hardware models: program-controlled, interrupt-driven, and DMA-driven.

Program control Icano:

In the program control model, CPU continuously polls whether the peripherals are ready through instructions, and reads / writes data repeatedly when the hardware is ready.

From the point of view of CPU, the program-controlled CPU O model is synchronous and blocking (synchronization means that the operation is still under the control of program instructions and is dominated by CPU; blocking means that CPU must continuously poll the completion state and cannot execute other instructions after initiating Icano).

The advantages of program control Icano are:

The hardware structure is simple, and it is also easy to write the corresponding program.

The disadvantages of program control Icano:

It consumes a lot of CPU, and the continuous rotation training makes the precious CPU resources wasted needlessly in the process of waiting for the completion of CPU O, resulting in low CPU utilization.

Interrupt-driven Icano:

In order to solve the problem of low utilization of CPU resources by the program-controlled Iamp O model, the designers of computer hardware make CPU have the function of dealing with interrupts.

In the interrupt-driven Icando model, after CPU initiates the external Icano request, it directly executes other instructions. When the hardware has finished processing the Imax O request, it notifies the CPU asynchronously by interrupting. After receiving the interrupt notification of read completion, CPU is responsible for writing data to memory from the peripheral buffer; after receiving the interrupt notification of write completion, CPU needs to write out the subsequent data in memory and give it to the peripheral for processing.

From the point of view of CPU, the interrupt-driven CPU O model is synchronous and non-blocking (synchronization means that the operation is still under the control of program instructions and is dominated by CPU; non-blocking means that the Icano does not stop waiting after it is initiated, but can execute other instructions).

The advantages of interrupt-driven Icano:

Because the CPU O is always relatively time-consuming, it is better than the constant rotation training under the program-controlled Ipaw O model. CPU can be freed to execute other commands in the process of waiting for the hardware iCandle O to be completed, which greatly improves the CPU utilization of Icano intensive programs.

The disadvantages of interrupt-driven Icano:

Limited by the size of the hardware buffer, the amount of data that can be processed by the hardware Istroke O is relatively limited. CPU needs to be interrupted repeatedly in processing a big data's Iamp O request, and there is some overhead in dealing with read and write interrupt events.

Use DMA's iDUBO:

In order to solve the problem that CPU needs to deal with interrupts repeatedly because of the large amount of data transmission in interrupt-driven IMab O model, computer hardware designers propose Imax O (DMA Direct Memory Access Direct memory access) based on DMA mode. DMA is also a kind of processor chip, which can access memory and peripherals just like CPU, but DMA chip is designed to deal with Ibank O data transmission, so its cost is lower than CPU.

In the IWeiO model using DMA, CPU interacts with the DMA chip to specify the block size that needs to be read / write and the destination memory address where the data needs to be read / written, and then processes other instructions asynchronously. The DMA interacts with the peripheral hardware, and a large amount of data requires DMA to interact with the peripheral repeatedly. When the DMA completes the Icano of the whole data block (completely reading the data into memory or writing the data of a certain memory block to the peripheral), it initiates a DMA interrupt to notify CPU.

From the point of view of CPU, the model of using DMA is asynchronous and non-blocking. (asynchronous means that the whole operation of DMA is not dominated by CPU, but by the interaction between the DMA chip and the peripheral; non-blocking means that the CPU does not stop waiting after initiating Icano, but can execute other instructions).

The advantages of using DMA:

Compared with the peripheral hardware interrupt notification, for a complete big data memory and the peripherals between the I _ gamma _ O _ CPU only need to process one interrupt. The utilization efficiency of CPU is relatively the highest.

The disadvantages of using DMA are:

1. The introduction of DMA chip makes the hardware structure more complex and the cost higher.

two。 Due to the introduction of DMA chip, DMA and CPU operate on memory concurrently. In CPU with cache, the problem of inconsistency between cache and memory is introduced.

Generally speaking, since the invention of DMA technology, it has become the standard hardware of most general-purpose computers because it greatly reduces the performance loss of CPU. With the development of technology, there is a more advanced channel iUniver O mode, which is equivalent to concurrent DMA, which allows concurrent processing of iUnix O operations involving multiple different memory areas and peripheral hardware.

Third, the operating system Istroke O model

After introducing the hardware Imax O model, the following is the focus of this blog: the operating system Imax O model.

The operating system shields us from many differences in hardware peripherals and provides friendly and unified services for application developers. In order to prevent the application from destroying the operating system kernel, CPU provides a protected mode mechanism, so that the application can not directly access the peripherals managed by the operating system, but must access the peripherals indirectly through the system calls provided by the kernel. The discussion on the operating system Imax O model is aimed at the system call model in which the application interacts with the kernel.

'There are several kinds of Imax O models provided by the operating system kernel: synchronous blocking Imax O, synchronous non-blocking Imax O, synchronous non-blocking Imax O multiplexing, and asynchronous non-blocking Iamp O (signal-driven Imax O is less used, so we won't expand it here).

Synchronous blocking I BIO O (Blocking I ram O O)

We have already known that the efficient hardware-level IMagano model is asynchronous for CPU, but application developers always want to return synchronously after executing the IUnip O system call and execute the subsequent logic linearly (for example, when the system call for disk reading returns, the read data can be accessed directly in the next line of code). However, this is contrary to the hardware-level time-consuming and asynchronous Imax O model (program-controlled Imax O is too wasteful of CPU), so the operating system kernel provides synchronous, blocking Imax O-based system calls (BIO) to solve this problem.

For example, when a thread reads disk through a BIO-based system call, the kernel puts the current thread into a blocking state and gives CPU resources to other concurrent ready threads in order to make more efficient use of CPU. When the DMA finishes reading and the asynchronous Ithumb O interrupt arrives, the kernel finds the previously blocked corresponding thread and wakes it up into a ready state. When this ready thread is selected by the kernel CPU scheduler to get the CPU again, the read disk data can be obtained from the corresponding buffer structure, and the synchronized execution flow of the program can be executed smoothly. (it feels as if the thread is stuck there, it takes a while to execute the next line, and the specified buffer already has the required data.)

The following pseudo-code example refers to the design of linux, which abstracts different peripherals into files and accesses them uniformly through the file descriptor (file descriptor).

BIO pseudo code example:

/ / create a TCP socket and bind port 8888 for service listening

Listenfd = serverSocket (8888, "tcp")

While (true) {

/ / accept synchronous blocking call

Newfd = accept (listenfd)

/ / read will block, so use thread asynchronous processing to avoid blocking accpet (thread pool is generally used)

New thread (()-> {

/ / synchronous blocking to read data

Xxx = read (newfd)

... Dosomething

/ / close the connection

Close (newfd)

});

}

Advantages of the BIO model:

Due to the characteristics of synchronization and blocking, the BIO's Icano model shields the essentially asynchronous hardware interaction at the bottom, so that programmers can write linear program logic that is easy to understand.

Disadvantages of the BIO model:

1. The synchronization and blocking characteristics of BIO are easy to use, but there are also some performance defects. Although the thread is blocked and does not consume CPU during the time BIO waits for IAccord O to complete, the kernel also has some overhead in maintaining a system-level thread (maintaining thread control block, kernel thread stack space, etc.).

two。 The context switching CPU of different threads is expensive when scheduling. In today's Internet era of a large number of users and high concurrency, it has increasingly become the bottleneck of web server performance. Thread context switching itself requires saving and restoring the site, emptying the CPU instruction pipeline and invalidating the cache massively. For a web server, if the BIO model is used, the server will need at least 1:1 to maintain the same number of system-level threads (kernel threads). Due to the continuous concurrent network data exchange, different threads are scheduled repeatedly by the kernel due to the completion events of the network Ithumb O.

In the context of the famous C10K problem, a server needs to maintain 1W concurrent tcp connections and 1W system-level threads at the same time. Quantitative change causes qualitative change, and the context switching caused by 1W system-level thread scheduling is completely different from the scheduling overhead of 100 system-level threads, which will exhaust CPU resources, make the whole system stuck and collapse.

Schematic diagram of BIO interaction process:

Synchronous non-blocking I NIO O (NonBlocking I sign O)

The BIO model is simple and easy to use, but its feature of blocking kernel threads makes it no longer suitable for web servers that need to handle a large number of concurrent network connection scenarios (above 1K). For this reason, the operating system kernel provides a non-blocking Imax O system call, namely NIO (NonBlocking-IO).

In view of the defects of the BIO model, the system call of the NIO model does not block the current calling thread. However, due to the time-consuming nature of NIO O, the result of processing can not be obtained immediately. The system call of IWeiO will return a specific identity when it is not completed, which means that the corresponding event is not completed yet. Therefore, the application is required to call repeatedly at a certain frequency to get the latest IO status.

NIO pseudo code example:

/ / create a TCP socket and bind port 8888 for service listening

Listenfd = serverSocket (8888, "tcp")

ClientFdSet = empty_set ()

While (true) {/ / enable event listening cycle

/ / accept synchronizes non-blocking calls to determine whether a new connection has been received

Newfd = acceptNonBlock (listenfd)

If (newfd! = EMPTY) {

/ / add a new connection to the listening connection collection if there is a new connection

ClientFdSet.add (newfd)

}

/ / apply for a buffer of 1024 bytes

Buffer = new buffer (1024)

For (clientfd in clientFdSet) {

/ / non-blocking read read

Num = readNonBlock (clientfd,buffer)

If (num > 0) {

/ / data exists in the read buffer

Data = buffer

... Dosomething

If (needClose (data)) {

/ / remove the currently monitored connection when the connection is closed

ClientFdSet.remove (clientfd)

}

}

... Dosomething

/ / clear buffer

Buffer.clear ()

}

}

Advantages of the NIO model:

Because of its non-blocking property, NIO makes it possible for a thread to handle multiple concurrent network Imax O connections. In the context of the C10K problem, it is theoretically possible to handle the 1W concurrent connections through a single thread (for multicore CPU, you can create multiple threads to share the load in each CPU core to improve performance).

Disadvantages of the NIO model:

NIO overcomes the shortcomings of BIO under the condition of high concurrency, but the original NIO system call still has some performance problems. In the pseudocode example above, the status query for I _ pico corresponding to each file descriptor must be completed through a NIO system call.

Because the operating system kernel makes use of the protected mode mechanism provided by CPU, the kernel runs at the high privilege level, while the user program runs at the low privilege level with limited execution and access. One of the advantages of this design is that the application can not directly access the hardware, but must be indirectly accessed by the system calls provided by the operating system (network card, disk and even power, etc.). When executing a system call, the application thread needs to be trapped in the kernel through the system call (that is, raising the application's current privilege level CPL so that it can access the protected hardware) and reverting to a low privilege level when the system call returns. This process is implemented through interrupts on the hardware.

The efficiency of system calls through interrupts is much lower than that of local function calls in the application, so it is inefficient to iterate through system calls to the I / O ready state of each file descriptor in the original NIO mode.

Schematic diagram of NIO interaction process:

Synchronous I Multiplexing O Multiplexing

In order to solve the defect of N system calls in one event loop in the system calls of the above NIO model. On the basis of NIO system call, the operating system kernel provides the system call of Imax O multiplexing model.

One of the optimizations of iUnip O multiplexing relative to the NIO model is that it allows multiple file descriptors to be passed in one system call for a batch of Imax O status queries. It only needs to make one system call in an event loop to get the Imax O state of the transferred file descriptor set, which reduces the unnecessary system call overhead in the original NIO model.

The multiplexing Ihamo model can be roughly divided into three implementations (although the final implementation is slightly different in different operating systems, but the principle is similar. The sample code takes the linux kernel as an example): select, poll, epoll.

Introduction of select Multiplexer

The select Icano multiplexer allows applications to pass a collection of file descriptors that need to listen for changes in events, listen for their read / write, accept connections, and so on.

The select system call itself is synchronous and blocked, and when there is no ready Icano event in the passed set of file descriptors, the thread executing the select system call will enter the blocking state until at least one file descriptor corresponding to the I O event is ready to wake up the thread blocked by select (you can specify a timeout to force wake up and return). After waking up, the thread that gets the CPU can traverse the set of file descriptors passed in after the select system call returns, processing the file descriptor of the Ibank O event.

Select pseudo code example:

/ / create a TCP socket and bind port 8888 for service listening

Listenfd = serverSocket (8888, "tcp")

FdNum = 1

ClientFdSet = empty_set ()

ClientFdSet.add (listenfd)

While (true) {/ / enable event listening cycle

/ / man 2 select (view linux system documentation)

/ / int select (int nfds, fd_set * readfds, fd_set * writefds, fd_set * exceptfds, struct timeval * timeout)

/ / Parameter nfds: the total number of file descriptors in readfds, writefds and exceptfds to be monitored + 1

/ / Parameter readfds/writefds/exceptfds: a collection of file descriptors that need to listen for read, write and exception events

/ / the parameter timeout:select is synchronously blocked. When no Icano events are ready within the timeout time, the calling thread wakes up and returns (ret=0).

/ / timeout indicates permanent blocking if null

/ / return value ret:

/ / 1. Returns an integer greater than 0, representing a total of ret activated in the incoming readfds/writefds/exceptfds (requires the application to traverse itself)

/ / 2. Returns 0, and no Iripple O event is ready before blocking timeout

/ / 3. Return-1 with an error

ListenReadFd = clientFdSet

/ / select multiplexing, passing in the full set of connections that need to listen for events at one time (timeout 1s)

Result = select (fdNum+1,listenReadFd,null,null,timeval ("1s"))

If (result > 0) {

/ / if the server listens for a read event in the connection

If (IN_SET (listenfd,listenReadFd)) {

/ / receive and establish a connection

NewClientFd = accept (listenfd)

/ / join the client connection collection

ClientFdSet.add (newClientFd); fdNum++

}

/ / traverse the entire set of client connections to be monitored

For (clientFd: clientFdSet) {

/ / if there is a read event in the current client connection

If (IN_SET (clientFd,listenReadFd)) {

/ / blocking reading data

Data = read (clientfd)

... Dosomething

If (needClose (data)) {

/ / remove the currently monitored connection when the connection is closed

ClientFdSet.remove (clientfd); fdNum--

}

}

}

}

}

Advantages of select:

1. Select multiplexing avoids the unnecessary system calls in the original NIO model to query the status of Istroke O for many times, aggregates it into a set, listens in batches and returns the result set.

2. The implementation of select is relatively simple. Mainstream operating systems such as windows and linux all implement select system calls, and have good cross-platform compatibility.

Disadvantages of select:

1. In the event loop, each select system call needs to pass the full set of file descriptors that need to be listened to from the user state, and when select returns, it also needs to fully traverse the state of the previously passed file descriptor set.

two。 For performance reasons, the kernel sets the maximum number of file descriptor collection elements that select listens to (usually 1024, which can be specified at kernel startup), so that the number of connections that select can listen to at a time is limited.

3. Performance considerations aside, from the point of view of interface design, select mixes the parameters of the system call with the return value (the return value overrides the parameters), making it more difficult for users to understand.

A schematic diagram of Icano multiplexing interaction:

Introduction of poll Multiplexer

The use of poll I _ peg O multiplexer is more or less the same as that of select. It also passes in the specified set of file descriptors and specifies that the kernel listens to the set of Imax O events on the corresponding file descriptors, but some optimizations are made based on select in the implementation details.

Like select, poll system calls are synchronous and blocked when no ready events occur (you can specify a timeout to force wake up and return), and when returning to determine whether there is a ready event, you also need to traverse the entire set of returned file descriptors.

Poll pseudo code example:

/ *

/ / man 2 poll (view linux system documentation)

/ / different from select, the parameter events and the return value revents are separated.

Struct pollfd {

File descriptor corresponding to int fd; / / file descriptor

Events that short events; / / requested events needs to listen to

Events ready when short revents; / / returned events returns

}

/ / Parameter fds, the collection of poolfd arrays to listen to

/ / Parameter nfds, which is the number of elements to be listened to in the fds array

/ / Parameter timeout, the timeout for blocking (passing in-1 means permanent blocking)

Int poll (struct pollfd * fds, nfds_t nfds, int timeout)

/ / events/revents is represented by a bitmap

/ / revents & POLLIN = = 1 there is a ready read event

/ / revents & POLLOUT = = 1 there is a ready write event

/ / revents & POLLHUP = = 1 there is a peer disconnection or communication completion event

, /

/ / create a TCP socket and bind port 8888 for service listening

Listenfd = serverSocket (8888, "tcp")

MAX_LISTEN_SIZE = 100

Struct pollfd fds[MAX _ LISTEN_SIZE]

/ / set the server to listen for sockets (listen for read events)

Fds [0] .fd = listenfd

Fds [0] .events = POLLIN

Fds [0] .revents = 0

/ / the number of client connections starts with 0

Int clientCount = 0

While (true) {

/ / poll synchronous blocking call (timeout-1 means permanent blocking until there is a listening ready event)

Int ret = poll (fds, clientCount + 1,-1)

For (int I = 0; I

< clientCount + 1; i++){ if(fds[i].fd == listenfd && fds[i].revents & POLLIN){ // 服务器监听套接字读事件就绪，建立新连接 clientCount++； fds[clientCount].fd = conn; fds[clientCount].events = POLLIN | POLLRDHUP ; fds[clientCount].revents = 0; }else if(fds[i].revents & POLLIN){ // 其他链接可读,进行读取 read(fds[i].fd); ... doSomething }else if(fds[i].revents & POLLRDHUP){ // 监听到客户端连接断开，移除该连接 fds[i] = fds[clientCount]; i--; clientCount--; // 关闭该连接 close(fd); } } } poll的优点： 　　1. poll解决了select系统调用受限于内核配置参数的限制问题，可以同时监听更多文件描述符的I/O状态(但不能超过内核限制当前进程所能拥有的最大文件描述符数目限制)。 　　2. 优化了接口设计，将参数与返回值的进行了分离。 poll的缺点： 　　1. poll优化了select，但在处理大量闲置连接时，即使真正产生I/O就绪事件的活跃文件描述符数量很少，依然免不了线性的遍历整个监听的文件描述符集合。每次调用时，需要全量的将整个感兴趣的文件描述符集合从用户态复制到内核态。 　　2. 由于select/poll都需要全量的传递参数以及遍历返回值，因此其时间复杂度为O(n)，即处理的开销随着并发连接数n的增加而增加，而无论并发连接本身活跃与否。但一般情况下即使并发连接数很多，大量连接都产生I/O就绪事件的情况并不多，更多的情况是1W的并发连接，可能只有几百个是处于活跃状态的，这种情况下select/poll的性能并不理想，还存在优化的空间。 epoll多路复用器： 　　epoll是linux系统中独有的，针对select/poll上述缺点进行改进的高性能I/O多路复用器。 　　针对poll系统调用介绍中的第一个缺点：在每次事件循环时都需要从用户态全量传递整个需要监听的文件描述符集合。 　　epoll在内核中分配内存空间用于缓存被监听的文件描述符集合。通过创建epoll的系统调用（epoll_create），在内核中维护了一个epoll结构，而在应用程序中只需要保留epoll结构的句柄就可对其进行访问(也是一个文件描述符)。可以动态的在epoll结构的内核空间中增加/删除/更新所要监听的文件描述符以及不同的监听事件（epoll_ctl），而不必每次都全量的传递需要监听的文件描述符集合。 　　针对select/poll的第二个缺点：在系统调用返回后通过修改所监听文件描述符结构的状态，来标识文件描述符对应的I/O事件是否就绪。每次系统调用返回时，都需要全量的遍历整个监听文件描述符集合，而无论是否真的完成了I/O。 　　epoll监听事件的系统调用完成后，只会将真正活跃的、完成了I/O事件的文件描述符返回，避免了全量的遍历。在并发的连接数很大，但闲置连接占比很高时，epoll的性能大大优于select/poll这两种I/O多路复用器。epoll的时间复杂度为O(m)，即处理的开销不随着并发连接n的增加而增加，而是仅仅和监控的活跃连接m相关；在某些情况下n远大于m，epoll的时间复杂度甚至可以认为近似的达到了O(1)。 　　通过epoll_wait系统调用，监听参数中传入对应epoll结构中关联的所有文件描述符的对应I/O状态。epoll_wait本身是同步、阻塞的(可以指定超时时间强制唤醒并返回)，当epoll_wait同步返回时，会返回处于活跃状态的完成I/O事件的文件描述符集合，避免了select/poll中的无效遍历。同时epoll使用了mmap机制，将内核中的维护的就绪文件描述符集合所在空间映射到了用户态，令应用程序与epoll的内核共享这一区域的内存，避免了epoll返回就绪文件描述符集合时的一次内存复制。 epoll伪代码示例： /** epoll比较复杂，使用时大致依赖三个系统调用 (man 7 epoll) 1. epoll_create 创建一个epoll结构,返回对应epoll的文件描述符 (man 2 epoll_create) int epoll_create(); 2. epoll_ctl 控制某一epoll结构(epfd)，向其增加/删除/更新(op)某一其它连接(fd)，监控其I/O事件(event) (man 2 epoll_ctl) op有三种合法值：EPOLL_CTL_ADD代表新增、EPOLL_CTL_MOD代表更新、EPOLL_CTL_DEL代表删除 int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event); 3. epoll_wait 令某一epoll同步阻塞的开始监听(epfd)，感兴趣的I/O事件(events),所监听fd的最大个数(maxevents)，指定阻塞超时时间(timeout) (man 2 epoll_wait) int epoll_wait(int epfd, struct epoll_event *events,int maxevents, int timeout); */ // 创建TCP套接字并绑定端口8888，进行服务监听 listenfd = serverSocket(8888,"tcp"); // 创建一个epoll结构 epollfd = epoll_create(); ev = new epoll_event(); ev.events = EPOLLIN; // 读事件 ev.data.fd = listenfd; // 通过epoll监听服务器端口读事件(新连接建立请求) epoll_ctl(epollfd,EPOLL_CTL_ADD,listenfd,ev)； // 最大监听1000个连接 MAX_EVENTS = 1000; listenEvents = new event[MAX_EVENTS]; while(true){ // 同步阻塞监听事件 // 最多返回MAX_EVENTS个事件响应结果 // (超时时间1000ms，标识在超时时间内没有任何事件就绪则当前线程被唤醒，返回值nfd将为0) nfds = epoll_wait(epollfd, listenEvents, MAX_EVENTS, 1 * 1000); for(n = 0; n < nfds; ++n){ if(events[n].data.fd == listenfd){ // 当发现服务器监听套接字存在可读事件，建立新的套接字连接 clientfd = accept(listenfd); ev.events = EPOLLIN | EPOLLET; ev.data.fd = clientfd; // 新建立的套接字连接也加入当前epoll的监听(监听读(EPOLLIN)/写(EPOLLET)事件) epoll_ctl(epollfd,EPOLL_CTL_ADD,clientfd,ev); } else{ // 否则是其它连接的I/O事件就绪,进行对应的操作 ... do_something } } } epoll的优点： 　　epoll是目前性能最好的I/O多路复用器之一，具有I/O多路复用优点的情况下很好的解决了select/poll的缺陷。目前linux平台中，像nginx、redis、netty等高性能服务器都是首选epoll作为基础来实现网络I/O功能的。 epoll的缺点： 　　1. 常规情况下闲置连接占比很大，epoll的性能表现的很好。但是也有少部分场景中，绝大多数连接都是活跃的，那么其性能与select/poll这种基于位图、数组等简单结构的I/O多路复用器相比，就不那么有优势了。因为select/poll被诟病的一点就是通常情况下进行了无谓的全量检查，而当活跃连接数占比一直超过90%甚至更高时，就不再是浪费了;相反的，由于epoll内部结构比较复杂，在这种情况下其性能比select/poll还要低一点。 　　2. epoll是linux操作系统下独有的，使得基于epoll实现的应用程序的跨平台兼容性受到了一定影响。 异步非阻塞I/O(Asynchronous I/O AIO) 　　windows和linux都支持了select系统调用，但linux内核在之后又实现了epoll这一更高性能的I/O多路复用器来改进select。 　　windows没有模仿linux，而是提供了被称为IOCP(Input/Output Completion Port 输入输出完成端口)的功能解决select性能的问题。IOCP采用异步非阻塞IO(AIO)的模型，其与epoll同步非阻塞IO的最大区别在于，epoll调用完成后，仅仅返回了就绪的文件描述符集合；而IOCP则在内核中自动的完成了epoll中原本应该由应用程序主动发起的I/O操作。 　　举个例子，当监听到就绪事件开始读取某一网络连接的请求报文时，epoll依然需要通过程序主动的发起读取请求，将数据从内核中读入用户空间。而windows下的IOCP则是通过注册回调事件的方式工作，由内核自动的将数据放入指定的用户空间，当处理完毕后会调度激活注册的回调事件，被唤醒的线程能直接访问到所需要的数据。 　　这也是为什么BIO/NIO/IO多路复用被称为同步I/O，而IOCP被称为异步I/O的原因。 　　同步I/O与异步I/O的主要区别就在于站在应用程序的视角看，真正读取/写入数据时是否是由应用程序主导的。如果需要用户程序主动发起最终的I/O请求就被称为同步I/O；而如果是内核自动完成I/O后通知用户程序，则被称为异步I/O。(可以类比在前面硬件I/O模型中，站在CPU视角的同步、异步I/O模型，只不过这里CPU变成了应用程序，而外设/DMA变成了操作系统内核) AIO的优点： 　　AIO作为异步I/O，由内核自动的完成了底层一整套的I/O操作，应用程序在事件回调通知中能直接获取到所需数据。内核中可以实现非常高效的调度、通知框架。拥有前面NIO高性能的优点，又简化了应用程序的开发。 AIO的缺点： 　　由内核全盘控制的全自动I/O虽然能够做到足够高效，但是在一些特定场景下性能并不一定能超过由应用程序主导的，经过深度优化的代码。像epoll在支持了和select/poll一样的水平触发I/O的同时，还支持了更加细致的边缘触发I/O，允许用户自主的决定当I/O就绪时，是否需要立即处理或是缓存起来等待稍后再处理。(就像java等支持自动内存垃圾回收的语言，即使其垃圾收集器经过持续的优化，在大多数情况下性能都很不错，但却依然无法达到和经过开发人员反复调优，手动回收内存的C、C++等语言实现的程序一样的性能) 　　（截图自《Unix网络编程 卷1》） 操作系统I/O模型小结 　　1. 同步I/O包括了同步阻塞I/O和同步非阻塞I/O，而异步I/O中由于异步阻塞I/O模型没有太大价值，因此提到异步I/O(AIO)时，默认指的就是异步非阻塞I/O。 　　 　　2. 在I/O多路复用器的工作中，当监听到对应文件描述符I/O事件就绪时，后续进行的读/写操作既可以是阻塞的，也可以是非阻塞的。如果是都以阻塞的方式进行读/写，虽然实现简单，但如果某一文件描述符需要读写的数据量很大时将耗时较多，可能会导致事件循环中的其它事件得不到及时处理。因此截图中的阻塞读写数据部分并不准确，需要辩证的看待。 四、非阻塞I/O是银弹吗？ 　　计算机技术的发展看似日新月异，但本质上有两类目标指引着其前进。一是尽可能的增强、压榨硬件的性能，提高机器效率；二是尽可能的通过持续的抽象、封装简化软件复杂度，提高程序员的开发效率。计算机软件的发展方向必须至少需要满足其中一种目标。 　　从上面关于操作系统内核I/O模型的发展中可以看到，最初被广泛使用的是易理解、开发简单的BIO模型；但由于互联网时代的到来，web服务器系统面临着C10K问题，需要能支持海量的并发客户端连接，因此出现了包括NIO、I/O多路复用、AIO等技术，利用一个内核线程管理成百上千的并发连接，来解决BIO模型中一个内核线程对应一个网络连接的工作模式中，由于处理大量连接导致内核线程上下文频繁切换，造成CPU资源耗尽的问题。上述的第一条原则指引着内核I/O模型的发展，使得web服务器能够获得更大的连接服务吞吐量，提高了机器效率。 　　但非阻塞I/O真的是完美无缺的吗？ 　　有着非阻塞I/O模型开发经验的程序员都知道，正是由于一个内核线程管理着成百上千个客户端连接，因此在整个线程的执行流中不能出现耗时、阻塞的操作(比如同步阻塞的数据库查询、rpc接口调用等)。如果这种操作不可避免，则需要单独使用另外的线程异步的处理，而不能阻塞当前的整个事件循环，否则将会导致其它连接的请求得不到及时的处理，造成饥饿。 　　对于多数互联网分布式架构下处理业务逻辑的应用程序服务器来说，在一个网络请求服务中，可能需要频繁的访问数据库或者通过网络远程调用其它服务的接口。如果使用的是基于NIO模型进行工作的话，则要求rpc库以及数据库、中间件等连接的库是支持异步非阻塞的。如果由于同步阻塞库的存在，在每次接受连接进行服务时依然被迫通过另外的线程处理以避免阻塞，则NIO服务器的性能将退化到和使用传统的BIO模型一样的地步。 　　所幸的是随着非阻塞I/O的逐渐流行，上述问题得到了很大的改善，越来越的框架/库都提供了异步非阻塞的api接口。 非阻塞I/O带来的新问题 　　异步非阻塞库改变了同步阻塞库下程序员习以为常的，线性的思维方式，在编码时被迫的以事件驱动的方式思考。逻辑上连贯的业务代码为了适应异步非阻塞的库程序，被迫分隔成多个独立片段嵌套在各个不同层次的回调函数中。对于复杂的业务而言，很容易出现嵌套为一层层的回调函数调用链，形成臭名昭著的callback hell(回调地狱)。 　　最早被callback hell折磨的可能是客户端程序的开发人员，因为客户端程序需要时刻监听着用户操作事件的产生，通常以基于事件驱动的方式组织异步处理代码。 callback hell伪代码示例： // 由于互相之间有前后的数据依赖，按照顺序异步的调用A、B、C、D A.dosomething((res)->

{

Data = res.xxx

B.dosomething (data, (res)-> {

Data = res.xxx

C.dosomething (data, (res)-> {

Data = res.xxx

D.dosomething (data, (res)-> {

/ / . The more complex and deep the dependent synchronization business is, it is like a bottomless pit.

})

})

})

})

The use of asynchronous non-blocking library separates the coherent structure of the code, making the program difficult to understand and debug, which is unbearable in web application server programs with complex and obscure business logic. This is the main reason why a large part of web servers are still developed using the traditional synchronous blocking BIO model. A large number of concurrent connections are distributed and clustered, but NIO is only widely used in Imax O-intensive middleware programs such as API gateways and message queues with relatively simple business (not really, business server clusters can add machines, so it is also important to ensure development efficiency).

So is there no way to have the performance advantage of non-blocking Iripple O to support massive concurrency and high throughput, and to make programmers think and write programs synchronously in order to improve development efficiency?

Solutions exist, of course, and the relevant technologies are still evolving. The above second principle of the development of computer technology guides the development of these technologies in order to simplify the complexity of code and improve the efficiency of programmers.

1. Optimize syntax and language libraries to simplify the difficulty of asynchronous programming

In the field of functional programming, there have been many obscure "cool techs" (CPS transformations, monad, etc.) that simplify callback hell and make it possible to write code that is essentially executed asynchronously in an almost synchronous way. For example, EcmaScript introduces promise and async/await into EcmaScript6 and EcmaScript7 respectively to solve this problem.

two。 Support for user-level threads (collaborators) at the language level

As mentioned earlier, the biggest advantage of the traditional working mode based on the BIO model is that it can write code synchronously, can be blocked when it is a time-consuming operation that needs to wait, and is easy to use. However, the 1:1 maintenance kernel thread is unable to deal with massive connections due to frequent kernel thread context switching, which gives birth to non-blocking I _ mano.

Because of the increased code complexity caused by the above non-blocking Istroke O, computer scientists have come up with another thread implementation, which was put forward in the operating system concept a long time ago, but has not been widely used: user-level thread.

As the name implies, a user-level thread is a thread implemented at the user level, which is imperceptible to the operating system kernel. User-level threads are similar to well-known kernel-level threads in many ways, having their own independent execution flow and sharing memory space with other threads in the process.

One of the biggest differences between user-level threads and kernel-level threads is that user-level threads cannot be preemptively scheduled based on interrupts because the operating system is not aware of them. In order to make different user-level threads in the same process work together, we must carefully write execution logic to actively transfer CPU to each other, otherwise it will cause one user-level thread to occupy CPU continuously and make other user-level threads in a hungry state, so user-level threads are also called cooperative programs, that is, cooperative threads.

In any case, user-level threads are based on at least one kernel thread / process, and multiple user-level threads can be mounted in a kernel thread / process and managed by the kernel.

In order to achieve the effect of synchronous blocking and transfer the execution flow of the program to another cooperative program, the cooperative program can choose to actively give up the CPU when it encounters time-consuming operations such as Ihammer O. Because multiple coprograms can reuse a kernel thread, the overhead of each coprogram is very small compared with kernel-level threads, and because there is no need to fall into the kernel during context switching, its switching efficiency is also much higher than that of kernel threads (the overhead is similar to a function call).

The recently popular Go language is highly respected for its support for language-level collaboration. Programmers can use some cooperation mechanisms provided at the language level to write efficient web server programs (such as adding keywords to control protocol synchronization in statements). By adding the corresponding co-program scheduling instruction to the compiled final code, it is taken over by the co-program scheduler to control the synchronization of the co-program to actively give up the CPU when the time-consuming Icano operation occurs, and can be dispatched back to execute after processing. Through the support of the cooperative program at the language level, Go reduces the difficulty of writing correct and coordinated cooperative program code.

If the high-performance web server written by Go runs in the linux operating system of multi-core CPU, it will generally create m kernel threads and n coprograms (m is proportional to the number of CPU cores, n is much greater than m and proportional to the number of concurrent connections). Each kernel thread at the bottom can still use epoll IO multiplexer to handle concurrent network connections and transfer the task of business logic processing to the user-mode gorountine. Each co-program can be scheduled back and forth in a different kernel thread (CPU core) to achieve maximum CPU throughput.

Using the co-program, programmers can write time-consuming Icano code for synchronous blocking without having to worry about the performance problems in the BIO model under high concurrency. It can be said that collaborative programming takes into account both program development efficiency and machine execution efficiency, so more and more languages also provide cooperative programming mechanism at the language level or in the function library.

3. Achieve user transparent collaboration

In the operating system platform-independent language (such as java), the virtual machine, as the middle layer between the application and the operating system kernel, can optimize the application in all aspects, so that programmers can easily write efficient code.

In an answer from Zhihu, Daniel mentioned that his team once led a transparent cooperation process in java when Alibaba was working. But it doesn't seem to be in line with official standards, so it's not open to the public.

If we can provide efficient and transparent cooperation mechanism in the virtual machine, so that the server program based on BIO multithreading can automatically support massive concurrency without modification, it is really too strong Orz.

Thank you for your reading, the above is the content of "what are the problems related to the Icano model". After the study of this article, I believe you have a deeper understanding of the problems related to the Icano model, and the specific use still needs to be verified by practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.