Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "how to understand the network IO model under Linux". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Why does Redis,Nginx,Netty,Node.js smell so good? These technologies are accompanied by system calls that provide efficient processing of network requests in Linux kernel iterations.

I _ INPUT/OUTPUT, including file I _ I _ O, network _ I _ I _ O. Speed disdain in the computer world:

Memory read data: nanosecond level.

Gigabit network card read data: subtle level. 1 microsecond = 1000 nanoseconds, the network card is a thousand times slower than memory.

Disk read data: millisecond level. 1 millisecond = 100,000 nanoseconds, the hard disk is 100000 times slower than memory.

CPU a clock cycle of about 1 nanosecond, memory is relatively close to CPU, the other can not afford to wait.

The speed of CPU processing data is much faster than the speed of preparing data. Any programming language will encounter the problem that the CPU processing speed does not match the Imax O speed!

How to optimize the network Ihand O in network programming? How to efficiently use CPU for network data processing?

Related concepts

How to understand the network Ithumb O from the operating system level? The world of computers has a set of self-defined concepts.

If you don't understand these concepts, you can't really understand the design ideas and nature of technology. So in my opinion, these concepts are the basis for understanding technology and the computer world.

Synchronous and asynchronous, blocking and non-blocking

Understand the topics that can not be avoided by network Ipaw O: synchronous and asynchronous, blocking and non-blocking.

Using Shanzhi to boil water, for example, (Shanzhi's behavior is like a user program, and boiling water is like a system call provided by the kernel), these two sets of concepts can be translated into vernacular:

Synchronous / asynchronous is concerned with whether I need to deal with the water after it boils.

Blocking / non-blocking is concerned with whether something else is done while the water is boiling.

Synchronous blocking: after ignition, wait foolishly, do not do anything until the water opens resolutely (blocking), the water turns off the fire (synchronous).

Synchronous non-blocking: after ignition, go to watch TV (non-blocking), from time to time to see if the water is on, and turn off the fire after the water is turned on (synchronous).

Asynchronous blocking: after pressing the switch, wait for the water to turn on (blocking), and the power is automatically cut off after the water is turned on (asynchronous).

A model that does not exist in network programming.

Asynchronous non-blocking: after pressing the switch, do what you have to do (non-blocking), and automatically turn off the power after the water is turned on (asynchronous).

Kernel space, user space

Kernel space and user space are shown in the figure above:

The kernel is responsible for reading and writing network and file data.

The user program obtains the data of the network and files through system calls.

Kernel state and user state are shown in the figure above:

The program has to make system calls in order to read and write data.

Through the system call interface, the thread switches from the user mode to the kernel mode, and then switches back after the kernel reads and writes the data.

Different spatial states of a process or thread.

Thread switching is shown in the figure above. Switching between user mode and kernel mode is time-consuming and resource-consuming (memory, CPU).

Optimization recommendations:

Fewer switches.

Shared space.

Socket: Socket

Sockets serve the following purposes:

Network programming can only be carried out with sockets.

The application sets up a connection, receives and sends data (iCandle O) through a system call socket ().

Socket supports non-blocking, applications can make non-blocking calls, and applications can call asynchronously.

File descriptors: FD handle

Network programming needs to know FD??? What the heck is FD? Linux: everything is a file, and FD is a reference to a file.

Is it like everything is an object in Java? What is operated in the program is a reference to the object. The number of objects created in Java is limited by memory, and the number of FD is also limited.

Linux needs to turn FD on and off when dealing with files and network connections.

Every process has a default FD:

0 standard input stdin

1 standard output stdout

2 error output stderr

The process of processing a network request by a server

The process of the server processing the network request is shown in the figure above:

After the connection is established.

Wait for the data to be ready (CPU idle).

Copy data from the kernel into the process (CPU idle).

How to optimize it? For an iCandle O access (for example, read), the data is copied to the buffer of the operating system kernel before it is copied from the buffer of the operating system kernel to the address space of the application.

So, when a read operation occurs, it goes through two phases:

Wait for data preparation (Waiting for the data to be ready).

Copy data from the kernel into the Copying the data from the kernel to the process.

It is because of these two phases that the following three network mode solutions have emerged in the iteration of Linux system upgrade.

Ipaw O model

Blocking I/O:Blocking I Dot O

Introduction: the most original network Istroke O model. The process blocks until the data copy is complete.

Disadvantages: when there is high concurrency, the server connects with the client peering.

Problems caused by too many threads:

Waste of CPU resources, context switching.

The cost of memory has risen geometrically, and the cost of JVM per thread is about 1MB.

Public static void main (String [] args) throws IOException {ServerSocket ss = new ServerSocket (); ss.bind (new InetSocketAddress (Constant.HOST, Constant.PORT)); int idx = 0; while (true) {final Socket socket = ss.accept (); / / blocking method new Thread (()-> {handle (socket)) }, "thread [" + idx+ "]) .start ();}} static void handle (Socket socket) {byte [] bytes = new byte [1024]; try {String serverMsg =" server sss [thread: "+ Thread.currentThread (). GetName () +"] "; socket.getOutputStream () .write (serverMsg.getBytes ()) / / blocking method socket.getOutputStream () .flush ();} catch (Exception e) {e.printStackTrace ();}}

Non-blocking I/O:Non Blocking IO

Introduction: the process repeats the system call and returns the result immediately.

Disadvantages: when the process has 1000fds, polling on behalf of the user process occurs 1000 system calls to kernel, switching back and forth between user mode and kernel mode, resulting in a geometric increase in cost.

Public static void main (String [] args) throws IOException {ServerSocketChannel ss = ServerSocketChannel.open (); ss.bind (new InetSocketAddress (Constant.HOST, Constant.PORT)); System.out.println ("NIO server started..."); ss.configureBlocking (false); int idx = 0; while (true) {final SocketChannel socket = ss.accept () / / blocking method new Thread (()-> {handle (socket);}, "thread [" + idx+ "]") .start ();}} static void handle (SocketChannel socket) {try {socket.configureBlocking (false); ByteBuffer byteBuffer = ByteBuffer.allocate (1024); socket.read (byteBuffer) ByteBuffer.flip (); System.out.println ("request:" + new String (byteBuffer.array (); String resp = "Server response"; byteBuffer.get (resp.getBytes ()); socket.write (byteBuffer);} catch (IOException e) {e.printStackTrace ();}}

ICompo Multiplexing: IO multiplexing

Summary: a single thread can handle multiple network connections at the same time. The kernel is responsible for polling all Socket and notifies the user process when data from a Socket arrives.

In the iterative process of Linux kernel code, multiplexing supports three kinds of calls in turn, namely, Select, Poll and Epoll. The following drawing will be combined with Java code interpretation.

① I Band O Multiplexing: Select

Introduction: there is a connection request arrived and then check the processing.

The disadvantages are as follows:

Upper limit of handle: 1024 FD can be opened by default.

Repeat initialization: each time you call select (), you need to copy the FD collection from the user state to the kernel state, and the kernel traverses it.

It is not efficient to check all FD states one by one.

The Select of the server is like a socket full of sockets. The connection of the Client is connected to one of the sockets to establish a channel, and then the read and write events are registered in the channel in turn.

Be sure to delete when a ready, read, or write event is handled, or you can handle it next time.

Public static void main (String [] args) throws IOException {ServerSocketChannel ssc = ServerSocketChannel.open (); / / Pipeline ServerSocket ssc.socket (). Bind (new InetSocketAddress (Constant.HOST, Constant.PORT)); ssc.configureBlocking (false); / / set non-blocking System.out.println ("NIO single server started, listening on:" + ssc.getLocalAddress ()); Selector selector = Selector.open () Ssc.register (selector, SelectionKey.OP_ACCEPT); / / on the established pipeline, register event-ready while (true) {selector.select (); Set keys = selector.selectedKeys (); Iterator it = keys.iterator (); while (it.hasNext ()) {SelectionKey key = it.next () Events handled by it.remove (); / / must delete handle (key);} private static void handle (SelectionKey key) throws IOException {if (key.isAcceptable ()) {ServerSocketChannel ssc = (ServerSocketChannel) key.channel (); SocketChannel sc = ssc.accept () Sc.configureBlocking (false); / / set non-blocking sc.register (key.selector (), SelectionKey.OP_READ); / / on the established pipeline, register events of interest} else if (key.isReadable ()) {/ / flip SocketChannel sc = null; sc = (SocketChannel) key.channel () ByteBuffer buffer = ByteBuffer.allocate; buffer.clear (); int len = sc.read (buffer); if (len! =-1) {System.out.println ("[" + Thread.currentThread () .getName () + "] recv:" + new String (buffer.array (), 0, len) } ByteBuffer bufferToWrite = ByteBuffer.wrap ("HelloClient" .getBytes ()); sc.write (bufferToWrite);}}

② I Band O Multiplexing: Poll

Introduction: design a new data structure (linked list) to provide efficiency.

Poll does not change much compared to Select, except that Poll does not have the limit of the maximum number of file descriptors in Select mode.

Disadvantages: it is not efficient to troubleshoot all FD states one by one.

③ I Band O Multiplexing: Epoll

Summary: there is no limit on the number of FD. You only need to copy the user mode to the kernel state once, and use the event notification mechanism to trigger it.

Register the FD through epoll_ctl, and once the FD is ready, the corresponding FD will be activated through the Callback callback mechanism to perform the related FD O operation.

The disadvantages are as follows:

Cross-platform, Linux supports best.

The underlying implementation is complex.

Synchronize.

Public static void main (String [] args) throws Exception {final AsynchronousServerSocketChannel serverChannel = AsynchronousServerSocketChannel.open () .bind (new InetSocketAddress (Constant.HOST, Constant.PORT)); serverChannel.accept (null, new CompletionHandler () {@ Override public void completed (final AsynchronousSocketChannel client, Object attachment) {serverChannel.accept (null, this); ByteBuffer buffer = ByteBuffer.allocate (1024) Client.read (buffer, buffer, new CompletionHandler () {@ Override public void completed (Integer result, ByteBuffer attachment) {attachment.flip (); client.write (ByteBuffer.wrap ("HelloClient" .getBytes () / / Business Logic} @ Override public void failed (Throwable exc, ByteBuffer attachment) {System.out.println (exc.getMessage ()); / / failure handling}}) } @ Override public void failed (Throwable exc, Object attachment) {exc.printStackTrace (); / / failure handling}}); while (true) {/ / No while true main method ends in an instant}}

Of course, compared with the above disadvantages, its advantages can be ignored. JDK provides asynchronous implementation, but in the actual Linux environment, the underlying layer is still Epoll, but with an extra layer of loop, it is not really asynchronous non-blocking.

And just like the code call in the figure above, the code that handles the network connection is not decoupled from the business code well enough.

Netty provides a concise, decoupled, well-structured API.

Public static void main (String [] args) {new NettyServer () .serverStart (); System.out.println ("Netty server started!");} public void serverStart () {EventLoopGroup bossGroup = new NioEventLoopGroup (); EventLoopGroup workerGroup = new NioEventLoopGroup (); ServerBootstrap b = new ServerBootstrap () B.group (bossGroup, workerGroup) .channel (NioServerSocketChannel.class) .childHandler (new ChannelInitializer () {@ Override protected void initChannel (SocketChannel ch) throws Exception {ch.pipeline () .addLast (new Handler ());}}) Try {ChannelFuture f = b.localAddress (Constant.HOST, Constant.PORT). Bind (). Sync (); f.channel (). CloseFuture (). Sync ();} catch (InterruptedException e) {e.printStackTrace ();} finally {workerGroup.shutdownGracefully (); bossGroup.shutdownGracefully () } class Handler extends ChannelInboundHandlerAdapter {@ Override public void channelRead (ChannelHandlerContext ctx, Object msg) throws Exception {ByteBuf buf = (ByteBuf) msg; ctx.writeAndFlush (msg); ctx.close ();} @ Override public void exceptionCaught (ChannelHandlerContext ctx, Throwable cause) throws Exception {cause.printStackTrace (); ctx.close ();}}

The butler (s) of the bossGroup who handles the network request will hand it over to the workGroup workers when the network connection is ready.

This is the end of the content of "how to understand the Network IO Model under Linux". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.