Shulou(Shulou.com)06/03 Report--

I _ INPUT OUTPUT, including file I _ I _ O, network _ I _ I _ O.

Speed disdain in the computer world:

Memory read data: nanosecond level. Gigabit network card read data: subtle level. 1 microsecond = 1000 nanoseconds, the network card is a thousand times slower than memory. Disk read data: millisecond level. 1 millisecond = 100000 nanoseconds, the hard disk is 100000 times slower than memory. CPU a clock cycle of about 1 nanosecond, memory is relatively close to CPU, the other can not afford to wait.

The speed of CPU processing data is much faster than the speed of preparing data.

Any programming language will encounter the problem that the CPU processing speed does not match the Imax O speed!

How to optimize network IhampGo in network programming: how to efficiently use CPU for network data processing?

I. related concepts

How to understand the network Ithumb O from the operating system level? The world of computers has a set of self-defined concepts. If you don't understand these concepts, you can't really understand the design ideas and nature of technology. So in my opinion, these concepts are the basis for understanding technology and the computer world.

1.1 synchronous and asynchronous, blocking and non-blocking

Understand the topics that can not be avoided by network Ipaw O: synchronous and asynchronous, blocking and non-blocking.

Using Shanzhi to boil water, for example, (Shanzhi's behavior is like a user program, and boiling water is like a system call provided by the kernel), these two sets of concepts can be translated into vernacular.

Synchronous / asynchronous is concerned with whether I need to deal with the water after it boils. Blocking / non-blocking is concerned with whether something else is done while the water is boiling. 1.1.1 synchronous blocking

After lighting the fire, wait foolishly, do not do anything until the water opens firmly (blocking), the water turns off the fire (synchronous).

1.1.2 synchronous non-blocking

After lighting the fire, go to watch TV (non-blocking), from time to time to see if the water is on, and turn off the fire after the water is turned on (synchronous).

1.1.3 Asynchronous blocking

After pressing the switch, wait for the water to open (block), and then automatically turn off the power (asynchronous).

A model that does not exist in network programming.

1.1.4 Asynchronous non-blocking

After pressing the switch, do what you have to do (non-blocking), and automatically turn off the power after the water is turned on (asynchronous).

1.2 Kernel space, user space

The kernel is responsible for reading and writing network and file data. The user program obtains the data of the network and files through system calls. 1.2.1 user mode in kernel mode

The program has to make system calls in order to read and write data. Through the system call interface, the thread switches from the user mode to the kernel mode, and then switches back after the kernel reads and writes the data. Different spatial states of a process or thread. 1.2.2 Thread switching

Switching between user mode and kernel mode is time-consuming and resource-consuming (memory, CPU)

Optimization recommendations:

Fewer switches. Shared space. 1.3Sockets-socket

Network programming can only be carried out with sockets. The application establishes a connection, receives and sends data (I / O) through a system call to socket (). SOCKET supports non-blocking, the application can make non-blocking calls, and asynchronous support enables the application to asynchronously call the 1.4 file descriptor-FD handle

Network programming needs to know FD??? What the heck is FD?

Linux: everything is a file, and FD is a reference to a file. Is it like everything is an object in JAVA? What is operated in the program is a reference to the object. The number of objects created in JAVA is limited by memory, and the number of FD is also limited.

Linux needs to turn FD on and off when dealing with files and network connections.

Every process has a default FD:

0 standard input stdin1 standard output stdout2 error output process of stderr1.5 server processing network request

After the connection is established. Wait for the data to be ready (CPU idle). Copy data from the kernel into the process (CPU idle).

How to optimize it?

For an iCandle O access (for example, read), the data is copied to the buffer of the operating system kernel before it is copied from the buffer of the operating system kernel to the address space of the application.

So, when a read operation occurs, it goes through two phases:

Wait for data preparation (Waiting for the data to be ready). Copy data from the kernel into the Copying the data from the kernel to the process.

It is because of these two phases that the following three network mode solutions have emerged in the iteration of Linux system upgrade.

2. Introduction of IO model 2.1blocking I-Blocking O-blocking I-side O

Introduction: the most original network Istroke O model. The process blocks until the data copy is complete.

Disadvantages: when there is high concurrency, there are many problems caused by the peering connection between the server and the client:

Waste of CPU resources, context switching. The cost of memory has risen geometrically, and the cost of JVM per thread is about 1MB. Public static void main (String [] args) throws IOException {ServerSocket ss = new ServerSocket (); ss.bind (new InetSocketAddress (Constant.HOST, Constant.PORT)); int idx = 0; while (true) {final Socket socket = ss.accept (); / / blocking method new Thread (()-> {handle (socket)) }, "thread [" + idx+ "]) .start ();}} static void handle (Socket socket) {byte [] bytes = new byte [1024]; try {String serverMsg =" server sss [thread: "+ Thread.currentThread (). GetName () +"] "; socket.getOutputStream () .write (serverMsg.getBytes ()) / / blocking method socket.getOutputStream (). Flush ();} catch (Exception e) {e.printStackTrace ();}} 2.2 non-blocking Imax O-Non Blocking IO

Introduction: the process repeats the system call and returns the result immediately.

Disadvantages: when the process has 1000fds, polling on behalf of the user process occurs 1000 system calls to kernel, switching back and forth between user mode and kernel mode, resulting in a geometric increase in cost.

Public static void main (String [] args) throws IOException {ServerSocketChannel ss = ServerSocketChannel.open (); ss.bind (new InetSocketAddress (Constant.HOST, Constant.PORT)); System.out.println ("NIO server started..."); ss.configureBlocking (false); int idx = 0; while (true) {final SocketChannel socket = ss.accept () / / blocking method new Thread (()-> {handle (socket);}, "thread [" + idx+ "]") .start ();}} static void handle (SocketChannel socket) {try {socket.configureBlocking (false); ByteBuffer byteBuffer = ByteBuffer.allocate (1024); socket.read (byteBuffer) ByteBuffer.flip (); System.out.println ("request:" + new String (byteBuffer.array (); String resp = "Server response"; byteBuffer.get (resp.getBytes ()); socket.write (byteBuffer);} catch (IOException e) {e.printStackTrace () }} 2.3 Icano Multiplexing-IO multiplexing

Summary: a single thread can handle multiple network connections at the same time. The kernel is responsible for polling all socket and notifies the user process when data from a socket arrives. In the iterative process of Linux kernel code, multiplexing supports three kinds of calls in turn, namely, SELECT, POLL and EPOLL. The following drawing will be combined with Java code interpretation.

2.3.1 Icano Multiplexing-select

Introduction: there is a connection request arrived and then check the processing.

Disadvantages:

Handle limit-there is a limit of 1024 FD opened by default. Repeat initialization-each time select () is called, the fd collection needs to be copied from the user state to the kernel state, and the kernel traverses it. It is not efficient to check all FD states one by one.

The select of the server is like a socket full of sockets. The connection of the client is connected to one of the sockets to establish a channel, and then the read and write events are registered in the channel in turn. Be sure to delete when a ready, read, or write event is handled, or you can handle it next time.

Public static void main (String [] args) throws IOException {ServerSocketChannel ssc = ServerSocketChannel.open (); / / Pipeline ServerSocket ssc.socket (). Bind (new InetSocketAddress (Constant.HOST, Constant.PORT)); ssc.configureBlocking (false); / / set non-blocking System.out.println ("NIO single server started, listening on:" + ssc.getLocalAddress ()); Selector selector = Selector.open (); ssc.register (selector, SelectionKey.OP_ACCEPT) / on the established pipeline, register event ready while (true) {selector.select (); Set keys = selector.selectedKeys (); Iterator it = keys.iterator (); while (it.hasNext ()) {SelectionKey key = it.next (); it.remove () / / events handled must delete handle (key);} private static void handle (SelectionKey key) throws IOException {if (key.isAcceptable ()) {ServerSocketChannel ssc = (ServerSocketChannel) key.channel (); SocketChannel sc = ssc.accept (); sc.configureBlocking (false) / / set non-blocking sc.register (key.selector (), SelectionKey.OP_READ); / / on the established pipeline, register events of interest} else if (key.isReadable ()) {/ / flip SocketChannel sc = null; sc = (SocketChannel) key.channel (); ByteBuffer buffer = ByteBuffer.allocate Buffer.clear (); int len = sc.read (buffer); if (len! =-1) {System.out.println ("[" + Thread.currentThread () .getName () + "] recv:" + new String (buffer.array (), 0, len) } ByteBuffer bufferToWrite = ByteBuffer.wrap ("HelloClient" .getBytes ()); sc.write (bufferToWrite);} 2.3.2 Ideband O Multiplexing-poll

Introduction: design a new data structure (linked list) to provide efficiency.

Poll does not change much compared to select, except that poll does not have the limit of the maximum number of file descriptors in select mode.

Disadvantages: it is not efficient to troubleshoot all FD states one by one.

2.3.3 Icano Multiplexing-epoll

Summary: there is no limit on the number of fd. You only need to copy the user mode to the kernel state once, and use the event notification mechanism to trigger it. Register the fd through epoll_ctl, and once the fd is ready, the corresponding fd will be activated through the callback callback mechanism to perform the related fd O operation.

Disadvantages:

Cross-platform, Linux supports best. The underlying implementation is complex. Synchronize. Public static void main (String [] args) throws Exception {final AsynchronousServerSocketChannel serverChannel = AsynchronousServerSocketChannel.open () .bind (new InetSocketAddress (Constant.HOST, Constant.PORT)); serverChannel.accept (null, new CompletionHandler () {@ Override public void completed (final AsynchronousSocketChannel client, Object attachment) {serverChannel.accept (null, this); ByteBuffer buffer = ByteBuffer.allocate (1024) Client.read (buffer, buffer, new CompletionHandler () {@ Override public void completed (Integer result, ByteBuffer attachment) {attachment.flip (); client.write (ByteBuffer.wrap ("HelloClient" .getBytes () / / Business Logic} @ Override public void failed (Throwable exc, ByteBuffer attachment) {System.out.println (exc.getMessage ()); / / failure handling}}) } @ Override public void failed (Throwable exc, Object attachment) {exc.printStackTrace (); / / failure handling}}); while (true) {/ / No while true main method ends in an instant}}

Of course, the above shortcomings can be ignored compared with its advantages. JDK provides asynchronous implementation, but in the actual Linux environment, the underlying layer is still epoll, but with an extra layer of loop, it is not really asynchronous non-blocking. And just like the code call in the figure above, the code that handles the network connection is not decoupled from the business code well enough. Netty provides a concise, decoupled, well-structured API.

Public static void main (String [] args) {new NettyServer () .serverStart (); System.out.println ("Netty server started!");} public void serverStart () {EventLoopGroup bossGroup = new NioEventLoopGroup (); EventLoopGroup workerGroup = new NioEventLoopGroup (); ServerBootstrap b = new ServerBootstrap () B.group (bossGroup, workerGroup) .channel (NioServerSocketChannel.class) .childHandler (new ChannelInitializer () {@ Override protected void initChannel (SocketChannel ch) throws Exception {ch.pipeline () .addLast (new Handler ());}}) Try {ChannelFuture f = b.localAddress (Constant.HOST, Constant.PORT). Bind (). Sync (); f.channel (). CloseFuture (). Sync ();} catch (InterruptedException e) {e.printStackTrace ();} finally {workerGroup.shutdownGracefully (); bossGroup.shutdownGracefully () } class Handler extends ChannelInboundHandlerAdapter {@ Override public void channelRead (ChannelHandlerContext ctx, Object msg) throws Exception {ByteBuf buf = (ByteBuf) msg; ctx.writeAndFlush (msg); ctx.close ();} @ Override public void exceptionCaught (ChannelHandlerContext ctx, Throwable cause) throws Exception {cause.printStackTrace (); ctx.close ();}}

The butler (s) of the bossGroup who handles the network request will hand it over to the workGroup workers when the network connection is ready.

Third, summarize and review synchronous / asynchronous, after the connection is established, when the user program reads and writes, if the user program finally needs to call the system read () to read the data, it is synchronous, and vice versa. Windows implements true asynchronism, and the kernel code is complex, but transparent to user programs. Blocking / non-blocking, after the connection is established, can the user program do something else while waiting for it to be readable and writable? If possible, it is non-blocking, otherwise blocking. Supported by most operating systems. Why does Redis,Nginx,Netty,Node.js smell so good?

These technologies are accompanied by system calls that provide efficient processing of network requests in Linux kernel iterations. Only by understanding the underlying knowledge of the computer can we have a deeper understanding of Imax O, know what it is, and even know why it is. Share with you!

Article Source: Citic Institute of Technology & Technology sharing of CITIC payment and settlement team No. 8-Zhou Shengshuai, Senior engineer of payment Research and Development team of Credit payment settlement Department, "understanding Linux's Network IO Model from the operating system level"

Shared by: Zhou Shengshuai, senior engineer of payment R & D team in Credit payment settlement Department

The original text was first published on the technical official account "wild pointer" of the payment and settlement team.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.