Shulou(Shulou.com)05/31 Report--

This article will explain in detail how the high-concurrency and high-performance server is realized, and the content of the article is of high quality, so the editor will share it with you for reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

It's easy to say, isn't it just a user request? According to the request, the server fished out the article from the database and sent it back over the network.

Complex as well as complex, how does the server process thousands of user requests in parallel? What technologies are involved?

Multiple processes

The earliest and simplest way in history to process multiple requests in parallel is to take advantage of multiple processes.

For example, in the Linux world, we can use system calls such as fork and exec to create multiple processes. We can receive the user's connection request in the parent process, and then create a child process to process the user's request, like this:

The advantages of this method are:

The programming is simple and easy to understand.

Because the address space of each process is isolated from each other, the crash of one process will not affect other processes.

Make full use of multi-core resources

The advantages of multi-process parallel processing are obvious, but the disadvantages are also obvious:

Each process address space is isolated from each other, and this advantage will also become a disadvantage, that is, communication between processes will become more difficult, you need to use inter-process communication (IPC,interprocess communications) mechanism, think about which inter-process communication mechanisms you now know, and then let you implement it in code? Obviously, the programming of interprocess communication is relatively complex, and performance is also a big problem.

We know that the overhead of creating processes is higher than that of threads, and frequent creation and destruction of processes will undoubtedly increase the burden on the system.

Fortunately, in addition to processes, we have threads.

Multithreading

Isn't it expensive to create a process? Isn't it difficult to communicate between processes? None of this is a problem for threads.

What? You do not understand threads, read this article "do not understand threads and thread pool you hit me", here explains in detail how the concept of threads came into being.

Because threads share process address space, communication between threads naturally does not need any communication mechanism, just read memory directly.

The overhead of thread creation and destruction is also reduced, you know, threads are like hermit crabs, the house (address space) is all process, they are only a tenant, so they are very lightweight, and the cost of creating and destroying is also very small.

We can create a thread for each request, and even if one thread is blocked and paused for performing an Icano operation, such as reading a database, and so on, it will not affect other threads, like this:

But are threads perfect and panacea? obviously, the computer world has never been that simple.

Because threads share process address space, it not only brings convenience to communication between threads, but also brings endless trouble.

It is precisely because the address space is shared between threads that a thread crash will cause the whole process to crash and exit. At the same time, the communication between threads is so simple that it only needs to read memory directly, and it is also so simple that it is extremely easy to have problems, such as deadlocks, synchronization mutual exclusion between threads, and so on, which are very easy to produce bug. Countless programmers spend a considerable part of their valuable time solving the endless problems caused by multithreading.

Although threads also have disadvantages, threads have more advantages than multi-processes, but it is impractical to solve the problem of high concurrency by simply using multi-threads.

Because although thread creation costs less than processes, it still has overhead. For highly concurrent servers with tens of thousands of links, there are performance problems in creating tens of thousands of threads, including memory footprint and switching between threads. That is, the cost of scheduling.

Therefore, we need to think further.

Event Loop: event driven

So far, when we mention the word "parallel", we will think of processes and threads. However, parallel programming can only rely on these two technologies? this is not the case.

There is another parallel technology widely used in GUI programming and server programming. This is event-driven programming, event-based concurrency, which is very popular in recent years.

Do not think that this is a difficult technology to understand, in fact, the principle of event-driven programming is very simple.

This technology requires two raw materials:

Event

A function that handles event, which is often referred to as event handler

The rest is simple:

You just need to wait quietly for event to arrive. When event arrives, check the type of event and find the corresponding event handler, event handler, according to that type, and then call the event handler directly.

That's it!

That's what event-driven programming is all about, isn't it very simple!

As you can see from the above discussion, we need to constantly receive event and then process event, so we need a loop (either a while or for loop), which is called Event loop.

This is what happens when you use pseudo-code to represent:

While (true) {event = getEvent (); handler (event);}

What needs to be done in Event loop is actually very simple, just wait for the event to be brought in, and then call the corresponding event handler.

Note that this code only needs to run in a single thread or process, and only this event loop is needed to process multiple user requests at the same time.

Some students still don't understand why such an event loop can handle multiple requests at the same time.

The reason is simple: for web servers, most of the time when processing a user's request is actually spent on I _ swap O operations, such as database read and write, file read and write, network read and write, and so on. When a request arrives, we may need to query the database and other Icano operations after simple processing. We know that IUnip O is very slow, and we can proceed to process the next user request without waiting for it to complete.

Now you can see that although we can actually process the next user request before the last user request has been processed, this is also parallelism, which can be handled by event-driven programming.

Just like a restaurant waiter, a waiter can't wait for one customer to place an order, serve, eat, and pay for the next customer. How does the waiter do it? When a customer places an order and deals with the next customer directly, when the customer has finished eating, he will come back to pay the bill himself.

See, the same waiter can handle multiple customers at the same time, which is equivalent to the Event loop here, even if the event loop runs in a single thread (process), it can handle multiple user requests at the same time.

I believe you already have a clear understanding of event-driven programming, so the next question is event-driven, event-driven, so how to get this event, that is, event?

Event source: IO multiplexer

In the article "finally understand, a thorough understanding of Linux/Unix O multiplexing", we know that in the world of Linux/Unix, everything is a file, and our programs operate through file descriptors, of course, for socket is no exception, so how do we deal with multiple file descriptors at the same time?

IO multiplexing technology is used to solve this problem. Through IO multiplexing technology, we can monitor multiple file descriptions at a time, and we can be notified when a file (socket) is readable or writable.

In this way, IO multiplexing technology has become the raw material supplier of event loop, continuously providing us with a variety of event, so the problem about the source of event is solved.

Of course, for a detailed explanation of IO multiplexing technology, please see "finally understand, a thorough understanding of Icano multiplexing".

So far, have you solved all the problems about using event drivers to implement concurrent programming? the source of event is solved, and when you get the event, call the corresponding handler, and it looks like you're done.

Think about it. Are there any other questions?

Problem: blocking IO

Now, we can use one thread (process) to do parallel programming based on event-driven, and there are no more annoying locks, synchronous mutexes, deadlocks, and so on.

However, there has never been a technology in computer science that can solve all problems, not now, and will not exist in the foreseeable future.

Is there anything wrong with the above methods?

Don't forget that our event loop runs on a thread (process), which solves the multithreading problem, but what if we need to do IO operations when dealing with an event?

In the article "what does the program go through when reading a file?" we explain how the most commonly used file reading is implemented at the bottom. This IO method most commonly used by programmers is called blocking IO, that is, when we perform IO operations, such as reading a file, if the file read is not completed, then our program (thread) will be blocked and suspended, which is not a problem in multithreading. Because the operating system can also schedule other threads.

But there is a problem in single-threaded event loop, because when we perform a blocking IO operation in event loop, the entire thread (event loop) will be suspended, and the operating system will have no other threads to schedule, because only one event loop in the system is processing user requests, so when the event loop thread is blocked and paused, all user requests cannot be processed. Can you imagine that your request is suspended when the server is processing other users' requests to read the database?

Therefore, it is important to note that blocking IO is not allowed when based on event-driven programming.

Some students may ask, if you can not initiate blocking IO, then how to carry out IO operation?

If there is blocking IO, there is non-blocking IO.

Non-blocking IO

In order to overcome the problems caused by blocking IO, modern operating systems began to provide a new way to initiate IO requests, this method is asynchronous IO, corresponding, blocking IO is synchronous IO, about synchronous and asynchronous these two concepts can refer to "from childhood to master, you need to understand synchronous and asynchronous".

When asynchronous IO, suppose you call the aio_read function (for specific asynchronous IO API, please refer to the specific operating system platform), that is, asynchronous reading. When we call this function, we can return immediately and continue other things, although the file may not have been read at this time, so that the calling thread will not be blocked. In addition, the operating system provides other methods for the calling thread to detect whether the IO operation is complete.

In this way, with the help of the operating system, the problem of blocking calls to IO is also solved.

The difficulties of event-based programming

Although there is an asynchronous IO to solve the problem that event loop may be blocked, event-based programming is still difficult.

First of all, we mentioned that event loop runs in a single thread. Obviously, one thread cannot make full use of multi-core resources. Some students may say that it is not OK to create multiple event loop instances, so there will be multiple event loop threads, but then the multi-thread problem will appear again.

Another point is programming. In the article "from childhood to master, you need to understand synchronization and asynchronism", asynchronous programming needs to be combined with callback functions (for callback functions, please refer to "how programmers should thoroughly understand callback functions"). This programming method needs to divide the processing logic into two parts, one is handled by the caller himself, and the other is handled in the callback function. This change in the way of programming increases the burden on programmers' understanding, and event-based programming projects can be difficult to expand and maintain later.

So is there a better way?

To find a better way, we need to solve the nature of the problem, so what is the essence of the problem?

A better way

Why should we use asynchronous programming in such an incomprehensible way?

This is because blocking programming is easy to understand but can cause threads to be blocked and suspended.

So smart you must ask, is there a way to combine the simple understanding of synchronous IO without blocking threads due to synchronous calls?

The answer is yes, this is the user-mode thread, user level thread, that is, the famous collaborative process, about the collaborative process is worth a separate article to explain, in the next article.

Although event-based programming has some disadvantages, event-based programming is still very popular on today's high-performance and high-concurrency servers, but it is no longer purely event-driven based on a single thread, but event loop + multi thread + user level thread.

On how to achieve high-concurrency high-performance server is shared here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.