Shulou(Shulou.com)06/02 Report--

This article mainly introduces "the difference between NIO and BIO, the operation principle of NIO and the concurrent use scenario". In the daily operation, I believe that many people have doubts about the difference between NIO and BIO, the operation principle of NIO and the concurrent use scenario. The editor consulted all kinds of materials and sorted out simple and useful operation methods. I hope it will be helpful to answer the questions of "the difference between NIO and BIO, how NIO works and what the concurrency scenario is!" Next, please follow the editor to study!

NIO (Non-blocking Non-blocking O) is a synchronous non-blocking Imax O model, and it is also the basis of Imax O multiplexing. It has been more and more applied to large-scale application servers, and has become an effective way to solve the problems of high concurrency and large number of connections.

So what is the nature of NIO? How does it combine with the event model to liberate threads and improve system throughput?

This paper will start with the problems faced by the traditional blocking IMab O and thread pool models, and then compare several common Imax O models, and analyze step by step how NIO uses the event model to deal with Imax O and solve thread pool bottlenecks to deal with massive connections, including writing server / client programs in an event-oriented way. Finally, it extends to some advanced topics, such as the comparison of Reactor and Proactor models, the awakening of Selector, the choice of Buffer and so on.

Note: the code in this article is pseudo-code, mainly to indicate that it cannot be used in a production environment.

Analysis of traditional BIO Model

Let's first recall the classic programming model of the traditional server-side synchronous blocking Imax O processing (that is, BIO,Blocking Imax O):

{ExecutorService executor = Excutors.newFixedThreadPollExecutor; / / thread pool ServerSocket serverSocket = new ServerSocket (); serverSocket.bind (8088); while (! Thread.currentThread.isInturrupted ()) {/ / main thread dead loop waiting for a new connection to arrive Socket socket = serverSocket.accept (); executor.submit (new ConnectIOnHandler (socket)); / / create a new thread for a new connection} class ConnectIOnHandler extends Thread {private Socket socket; public ConnectIOnHandler (Socket socket) {this.socket = socket } public void run () {while (! Thread.currentThread.isInturrupted () & &! socket.isClosed ()) {endless loop processing read and write event String someThing = socket.read (). / / read data if {. / / process data socket.write () / / write data}}

This is a classic per-connection per-thread model, the main reason for the use of multithreading, the main reason is that socket.accept (), socket.read (), socket.write () three main functions are synchronous blocking, when a connection in the processing of IBG O, the system is blocked, if it is a single thread, then it is bound to hang there; but CPU is released, open multithreading, you can let CPU to deal with more things.

In fact, this is the essence of all the use of multithreading:

Using multiple cores.

CPU resources can be used with multithreading when the system is blocked by CPU O, but the CPU is idle.

Today's multithreads generally use thread pools, which can make the cost of thread creation and recovery relatively low. When the number of active connections is not very high (less than 1000), this model is quite good, which allows each connection to focus on its own Icano and the programming model is simple, and there is no need to think too much about system overload, current limitation and other problems. The thread pool itself is a natural funnel that buffers connections or requests that the system cannot handle.

However, the most essential problem with this model is that it is heavily dependent on threads. However, threads are very "expensive" resources, mainly as follows:

The cost of creating and destroying threads is very high. In operating systems like Linux, threads are essentially a process. Creating and destroying are heavyweight system functions.

The thread itself takes up a large amount of memory, such as the thread stack of Java, which generally allocates at least the space of 512K~1M. If the number of threads in the system exceeds one thousand, I am afraid that half of the memory of the whole JVM will be eaten.

The switching cost of threads is very high. When a thread switch occurs in the operating system, you need to retain the context of the thread and then execute the system call. If the number of threads is too high, the thread switching time may even be longer than the thread execution time. The performance of this time is that the system load is on the high side and the CPU sy utilization is particularly high (more than 20%), causing the system to almost fall into an unavailable state.

It is easy to cause jagged system load. Because the system load is the number of active threads or CPU cores, once the number of threads is high but the external network environment is not very stable, it is easy to cause a large number of request results to return at the same time, activate a large number of blocking threads and make the system load too much.

Therefore, when faced with 100, 000 or even millions of connections, the traditional BIO model is powerless. With the rise of mobile applications and the prevalence of a variety of online games, million-level long connections are becoming more and more common. At this time, it is necessary to need a more efficient Imax O processing model.

How does NIO work?

For many people who are new to NIO, the first thing they see is Java's relatively obscure API, such as Channel,Selector,Socket, and then a block of hundreds of lines of code to demonstrate NIO's server Demo. Do you have a big head in an instant?

Let's ignore this, put aside the phenomenon and look at the essence, and first analyze how NIO works.

1. Comparison of common Ipaw O models

All systems are divided into two phases: waiting to be ready and operating. For example, the read function is divided into waiting for the system to be readable and the real reading; similarly, the write function is divided into waiting for the network card to write and the real write.

It is important to note that blocking waiting for ready does not use CPU and is "waiting for nothing"; while blocking for real read and write operations uses CPU and is really "working", and the process is very fast, belongs to memory copy, and the bandwidth is usually above the 1GB/s level, which can be understood as basically time-consuming.

The following figure shows a comparison of several common Istroke O models:

Take socket.read () as an example:

In the traditional BIO socket.read (), if there is no data in the TCP RecvBuffer, the function will block until the data is received and the read data is returned.

For NIO, if TCP RecvBuffer has data, it reads the data from the network card into memory and returns it to the user; otherwise, it returns 0 directly and never blocks.

The latest AIO (Async Ithumb O) takes it a step further: not only is waiting ready non-blocking, but even the process of data from the network card to memory is asynchronous.

In other words, BIO users are most concerned with "I want to read", NIO users are most concerned with "I can read", and in the AIO model users need to focus more on "finished".

An important feature of NIO is that the main read, write, register, and receive functions of socket are non-blocking during the wait-and-ready phase, and the real Imax O operation is synchronously blocked (consuming CPU but with very high performance).

two。 How to use NIO synchronous non-blocking feature in conjunction with event model

Let's take a specific look at how to use the event model for single-threaded processing of all Iripple O requests:

There are several major events in NIO:

Read ready

Ready to write

A new connection is coming.

We first need to register the corresponding processor when these events come. Then tell the event selector at the right time: I am interested in this event. For write operations, you are interested in write events when you cannot write out; for read operations, you are interested in write events when the connection is completed and the system has no way to hold newly read data; for accept, it is usually when the server starts up; and for connect, it is usually when connect fails to reconnect or call connect asynchronously.

Second, selecting ready events with an endless loop executes system calls (select before Linux 2.6, IOCP after poll,2.6) and blocks waiting for new events to arrive. When a new event arrives, the tag bit is registered on the selector to indicate the arrival of a read, write, or connection.

Note that select is blocked, whether through operating system notification (epoll) or constant polling (select,poll), this function is blocked. So you can safely call this function in a while (true) without worrying about CPU idling.

So our program looks like this:

Interface ChannelHandler {void channelReadable (Channel channel); void channelWritable (Channel channel);} class Channel {Socket socket; Event event;// read, write or connect} / / I O thread main loop: class IoThread extends Thread {public void run () {Channel channel; while (channel=Selector.select ()) {/ / Select ready events and corresponding connection if (channel.event==accept) {registerNewChannelHandler (channel) / / if it is a new connection, register a new read-write processor} if (channel.event==write) {getChannelHandler (channel) .channelWritable (channel); / / if it is writable, execute the write event} if (channel.event==read) {getChannelHandler (channel) .channelReadable (channel); / / if it is readable, execute the corresponding event handler} for all channel of Map handlerMap;//}

This program is brief and the simplest Reactor mode: register all interested event handlers, single-thread poll select ready events, and execute event handlers.

3. Optimize thread model

From the above example, we can probably sum up how NIO solves thread bottlenecks and handles massive connections:

NIO has changed from blocking reading and writing (occupying threads) to a single-thread polling event to find a network descriptor that can read and write. Except that the polling of events is blocked (there is nothing to do that must be blocked), the rest of the Icano operations are pure CPU operations, so there is no need to start multithreading.

And due to the saving of threads, the problems caused by thread switching are also solved when the number of connections is large, which makes it possible to deal with massive connections.

Single-threaded processing of IWeiO is indeed very efficient, there is no thread switching, just desperately reading, writing, selecting events. But today's servers are generally multi-core processors, if we can use multi-core to carry out Ihop O, there is no doubt that the efficiency will be greatly improved.

A careful analysis of the threads we need mainly includes the following:

Event dispatcher, single-threaded selection of ready events.

I CPU O processors, including connect, read, write, etc., this pure CPU operation, generally open the CPU core threads.

Business thread, after processing IWeiO, the business will generally have its own business logic, and some will have other blocking IWeiO, such as DB operation, RPC and so on. As long as there is blocking, a separate thread is required.

Java's Selector has a fatal limitation for Linux systems: select of the same channel cannot be called concurrently. Therefore, if you have multiple I / O threads, you must ensure that a socket can only belong to one IoThread, and an IoThread can manage multiple socket.

In addition, the processing of connections and the processing of reading and writing can usually be separated, so that the registration and reading and writing of a large number of connections can be distributed. Although read () and write () are more efficient and non-blocking functions, they still take up CPU and are powerless in the face of higher concurrency.

The magic of NIO on the client side

From the above analysis, we can see that NIO does have its own opportunity to use its talents on the server side in terms of liberating threads, optimizing IPUBO and handling massive connections.

What are the usage scenarios for 1.NIO?

In the common client-side BIO+ connection pool model, n connections can be established, and then when one connection is occupied by Imax O, other connections can be used to improve performance.

However, the multithreaded model faces the same problem as the server: if you expect to increase the number of connections to improve performance, the number of connections is limited by the number of threads, threads are expensive, and can not build many threads, then the performance will encounter a bottleneck.

Redis requested per connection order

For Redis, because the server is globally serial, it can ensure that all requests and returns for the same connection are in the same order. In this way, you can use single thread + queue to buffer the request data. Then pipeline sends it and returns future, and when the channel is readable, just fetch the future from the queue and done () is fine.

The pseudo code is as follows:

Class RedisClient Implements ChannelHandler {private BlockingQueue CmdQueue; private EventLoop eventLoop; private Channel channel; class Cmd {String cmd; Future result;} public Future get (String key) {Cmd cmd= new Cmd (key); queue.offer (cmd); eventLoop.submit (new Runnable () {List list = new ArrayList (); queue.drainTo (list); if (channel.isWritable ()) {channel.writeAndFlush (list);}}) } public void ChannelReadFinish (Channel channel,Buffer Buffer) {List result = handleBuffer (); / / process data / / take the future from cmdQueue and set the value, future.done ();} public void ChannelWritable (Channel channel) {channel.flush ();}}

In this way, we can make full use of pipeline to improve the ability of Imax O and obtain asynchronous processing capability at the same time.

3. HttpClient with multiple connections and short connections

Similar to race-to-crawl projects, you often need to establish numerous HTTP short connections, then crawl them, and then destroy them. When you need to crawl thousands of website threads on a single machine and are restricted, how can you ensure performance?

Why not try NIO to connect, write, and read with a single thread? If the connection, read, and write operating system does not have the ability to handle it, simply register an event and wait for the next loop.

How do you store different requests / responses? Since http is a stateless protocol without a version, and there is no way to use queues, it seems that there are not many ways. A more stupid approach is to directly store a reference to socket as the key of map for different socket.

4. Common RPC frameworks, such as Thrift,Dubbo

This framework generally maintains the request protocol and request number, and can maintain a map with the request number as key and the resulting result as future, combined with NIO+ persistent connection to obtain very good performance.

NIO Advanced Topics

1.Proactor and Reactor

In general, the Istroke O reuse mechanism requires an event dispatcher (event dispatcher). The role of the event distributor, that is, those read and write event sources are distributed to the handlers of read and write events, just like the courier shouting downstairs: who whose express has arrived, come and get it! At the beginning, the developer needs to register the event of interest with the dispatcher and provide the corresponding event handler, or callback function; the event dispatcher will distribute the requested event to these handler or callback functions when appropriate.

The two modes that involve event dispatchers are called Reactor and Proactor. The Reactor mode is based on synchronous iCandle O, while the Proactor mode is associated with asynchronous Imab O. In Reactor mode, the event dispatcher waits for an event or the state of an application or operation to occur (such as a file descriptor readable or socket readable), and the event dispatcher passes the event to a pre-registered event handler or callback function, which does the actual read and write operation.

In Proactor mode, the event handler (or initiated by the event dispatcher) directly initiates an asynchronous read and write operation (equivalent to a request), while the actual work is done by the operating system. When initiating, the parameters you need to provide include the cache area used to store the read data, the size of the read data or the cache area used to store outgoing data, and the callback function after the request. The event dispatcher is aware of the request, it silently waits for the request to complete, and then forwards the completion event to the appropriate event handler or callback. For example, an event handler delivers an asynchronous IO operation (called overlapped technology) on Windows, and IO Complete events such as event dispatchers are completed. The typical implementation of this asynchronous pattern is based on the underlying asynchronous API of the operating system, so we can call it "system-level" or "real" asynchronous, because the specific read and write is done by the operating system.

The choice of 2.Buffer

For NIO, the use of caching can use DirectByteBuffer and HeapByteBuffer. If you use DirectByteBuffer, you can generally reduce a copy from system space to user space. However, Buffer creation and destruction are more expensive and less maintainable, and memory pools are often used to improve performance.

If the amount of data is relatively small in the case of small and medium-sized applications, you can consider using heapBuffer; instead of using directBuffer.

Problems in NIO

Use NIO! = high performance, when the number of connections

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.