Shulou(Shulou.com)06/01 Report--

This article shows you how to deeply understand the Linux high-performance network architecture, the content is concise and easy to understand, it will definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

1. Lonely Blackie.

It was very cold in Beijing last week. I went to the subway station after work on Friday night and received a Wechat from my good friend Xiao Hei:

So Dabai turned around and swept a bike to Wudaokou, and Blackie reliably chose a good position.

Blackie: you got off work early today!

Dabai: as far as we are aware, we have a job in mind, and there are desks everywhere. Don't stick to formality.

I can obviously feel that Brother Black seems to be tired recently. I couldn't see the flash of bulingbuling in my eyes before.

Dabai: which one did you go to this afternoon? What position? How's it going?

Xiao Hei: it is a start-up company that does autopilot. The website is that the introduction of the team is not bad, so I went to see it. I was not very prepared this time. I am actually familiar with many questions, but I did not answer them properly.

Dabai: Oh, I see, that is, I didn't understand well at that time, and I was confused in the past. Now I suddenly ask, and I can't remember the main point.

Blackie: pretty much. Ask me what high-performance network framework models I've done, that is, IO and event-driven.

Having said that, Xiao Hei took a big sip of beer, and Daobai saw that there was some loneliness in Xiao Hei's heart. After all, in Didu, competition and work pressure, as well as the trivia of life have always been around us, but money and good luck have skillfully avoided us.

Think of here, Dabai also took a deep drink, my life is not from the sky, open the whole!

Brother Hei, you say this question is really difficult to answer. It's all terminology and slightly ambiguous things. I think we should just grasp the essence and explain it.

Blackie: come on, please start your performance. I'll study.

Dabai decided to have a good chat with Blackie. There are some things in the high-performance network framework commonly used in Linux development. The hot pot makes the night and the weather less cold.

Through this article, you will learn the following:

IO events and IO reuse

Architecture of threading model and event-driven model

Detailed explanation of Reactor pattern based on event-driven

Introduction to synchronous IO and Asynchronous IO

2. IO events and IO reuse

2.1 what is an IO event

IO refers to input Input/ and output Output, but from a Chinese point of view, output and input are relative, so we need a reference. Here our reference is chosen as the main storage space when the program is running, which usually includes network cards, disks and so on. With the above settings, it is much easier to understand. Let's take a look at it together:

The essence of IO is the flow of data, which can be written from the network card to the program memory or from the program memory to the network card, and so is the disk operation.

So common IO can be divided into:

Network IO: data interaction between memory and network card

File IO: data interaction between memory and disk

So what is the IO incident? An event can be understood as a state or action, that is, the transition of a state triggers a corresponding action. Events in a network IO usually include:

Readable event

Writable event

Abnormal event

It is necessary to understand readable and writable events. Generally speaking, an socket is writable most of the time, but not always readable. Readable generally represents a new connection or an old connection with new data interaction, which is also a key event for server-side programs.

2.2 what is IO reuse

Imagine that if there are tens of thousands of IO events, how should the application manage it? This is about to mention IO reuse. IO reuse essentially means that the application registers many types of IO events with the kernel with the help of IO reuse functions. When these registered IO events change, the kernel notifies the application through IO reuse functions.

As you can see from the figure, what is reused in IO reuse is a thread responsible for listening and managing these IO events. It is possible for a thread to manage hundreds of IO events because only a small number of IO events are triggered at some point in most of the time.

It's something like this: a big dog on the prairie can take care of dozens of sheep, because most of the time only a few sheep run around unruly, and the others eat grass obediently.

3. Design elements of network framework

To understand what the network framework is, it is necessary to know what the network framework has accomplished.

Give a general description of the process of this request:

The remote machine A sends a HTTP request to server B, where the server B network card receives the data and generates an IO readable event

Let's take synchronous IO as an example, where the kernel notifies the application's Listen thread of the readable event

The Listen thread throws the task to the Handler thread, and Handler copies the data from the kernel read buffer to the user space read buffer

Request packets are calculated and processed within the application and encapsulated with response packets

The Handler thread waits for the arrival of writable events

When the connection is writable, the data is copied from the user-mode write buffer to the kernel buffer and sent out through the network card.

Note: the above example takes synchronous IO as an example, and the roles in the thread are divided into Listen thread, Handler thread and Worker thread to complete different tasks, which will be expanded in detail later.

So we can see that several large pieces of content are involved to complete a data interaction:

IO event snooping

Data copy

Data processing and calculation

Dabai believes that these three pieces of content, no matter what form of framework can not be bypassed, is also the key to understanding the network architecture.

4. Practice of High performance Network Framework

4.1 based on threading model

In the early scenarios where concurrency was low, there was an architectural pattern for One Request One Thread. In this mode, each time a new request is received, a processing thread is created. Although the thread does not consume much resources, but thousands of requests call, the performance is also unbearable.

This is a relatively primitive architecture, and the idea is very clear. Multiple threads are created to provide processing power, but there are few applications in a highly concurrent production environment, so this article will not expand.

4.2 based on event-driven model

At present, the popular IO reuse model is based on event-driven, which has obvious advantages over the multi-thread model.

Let's first understand what event-driven Event-Drive-Model is.

Event-driven programming is a programming paradigm, the execution flow of the program is determined by external events, it contains an event loop, when external events occur, the callback mechanism is used to trigger the corresponding processing.

In popular terms, there is a loop device that has been waiting for the arrival of various events, and puts the arriving events into the queue, and then a sorting device calls the corresponding processing device to respond.

4.3 Reactor reactor mode

I was confused when I heard this model for the first time. What on earth is a reactor? After studying it, it is found that the reactor is a concept of nuclear physics, which looks something like this:

A nuclear reactor is the heart of a nuclear power plant. It works like this: atoms are made up of nuclei and extra-nuclear electrons, and nuclei are made up of protons and neutrons.

When the nucleus of uranium 235 is bombarded by foreign neutrons, one nucleus absorbs one neutron and splits into two smaller nuclei, emitting 2-3 neutrons at the same time.

The neutrons produced by this fission bombard the other nuclei of uranium 235, causing new fission, and this continuation is the chain reaction of fission.

Combined with this fission diagram, it seems that a request is called, and there are a lot of branches within the server to complete the response, one to two, two to four, or even more, it really feels like a reactor. Next, let's take a look at how the reactor model builds a high-performance network framework.

5. Detailed explanation of reactor model

The reactor model is an idea, but there are many forms.

5.1 what is the nature of the reactor model

In essence, no matter what network framework, there are two parts of operation:

IO operation: reading and writing of packets

CPU operation: processing and encapsulation of data requests

Therefore, who does the above problems and how many threads to do, give rise to a lot of forms, so do not be confused by superficial phenomena, there must be a reason, only after tracing back can we really master it.

Reactor modes are divided into the following categories according to the difference in the number of processing IO links and processing data links:

Single Reactor thread

Single Reactor thread and thread pool

Multiple Reactor threads and thread pools

Let's take a look at the characteristics, principles, advantages and disadvantages, application scenarios of these three common modes.

5.2 single Reactor threading mode

This mode is the most concise, where a thread listens to a connection, receives a new connection, processes a connection, reads data, and writes data. Because only one thread is used, the utilization rate for multi-core is low, but the programming is simple. Do you think there is no market for this single-threaded model? That may not be true. Look at Redis.

In this mode, IO operation and CPU operation are not separated, and both are done by one thread. Obviously, if a request is timed out in Handler, it will block the client's normal connection. In Redis, because of the memory operation, the speed is very fast, this bottleneck exists but is not obvious enough.

5.3 single Reactor thread and thread pool mode

In order to solve the mismatch between IO operation and CPU operation, that is, IO operation and CPU operation are performed serially within a thread, which lowers the efficiency of CPU operation.

One solution is to use separate threads to perform IO and CPU operations without affecting each other. It is a solution that single Reactor thread completes IO operation and reuses worker thread pool to complete CPU operation.

In this mode, the reader thread manages the connection and reads & writes back the data, and is fully in charge of the IO operation. The worker thread pool handles tasks from upstream distribution, decodes, calculates, encodes the data, and returns it to the Reactor thread to interact with the client. This model makes effective use of multi-cores, but a single Reactor thread to complete IO operations can still be a bottleneck in high concurrency scenarios. In other words, there are so many connections that a Reactor thread is too busy to make a new connection and respond to an old one, so the Reactor thread also needs a few helpers.

5.4 Multi-Reactor thread and thread pool mode

Horizontal scaling is often an effective way to provide performance.

We extend the Reactor thread, one Reactor thread is responsible for handling the new connection, and multiple Reactor threads are responsible for reading and writing successful IO data. That is to say, snooping & creating connections and processing connections are completed by two or more threads respectively, which further improves the efficiency of the IO operation part.

This model is a relatively high-end version, and it is also used in the actual production environment.

5.5 extension: synchronous IO and Asynchronous IO

We can easily distinguish between blocking IO and non-blocking IO, so what are synchronous IO and asynchronous IO? A very important part of the Reactor mode mentioned earlier is to call the read/write function to complete the data copy, which is done by the application itself, and the kernel is only responsible for notifying that the monitoring event is coming, so essentially the Reactor mode belongs to non-blocking synchronous IO. There is also a Preactor mode, with the help of the asynchronous IO feature of the system itself, the operating system copies the data and notifies the application to retrieve it after completion, which is more efficient, but the underlying layer needs to be implemented with the help of the asynchronous IO mechanism of the kernel.

The underlying asynchronous IO mechanism may be implemented with the help of DMA and Zero-Copy technology, which has higher performance in theory. At present, the Windows system implements the real asynchronous I-Windows O through IOCP, but the asynchronous I-sign O in the Linux system is not perfect. For example, the boost.asio module in Linux is the support of asynchronous IO, but at present the Linux system is mainly based on non-blocking synchronous IO based on Reactor mode.

6. Summary

Starting from IO events and IO reuse, this paper describes the lowest composition of the network architecture. Continue to expand the characteristics and design elements of the network framework based on threading model and event-driven model. After that, it focuses on the core nature of the reactor model and the various forms in the production environment. Finally, it briefly introduces the difference between synchronous IO and asynchronous IO, as well as the advantages of Preactor mode. I hope that readers can abandon professional terms and expressions, grasp the nature and focus of the problem, and find a suitable way of thinking to understand and master the design of high-performance network architecture. Perhaps the high-performance network framework is just a paper tiger.

The above is how to gain an in-depth understanding of Linux high-performance network architecture. Have you learned any knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.