Shulou(Shulou.com)06/02 Report--

This article introduces the relevant knowledge of "what is the general solution to deal with high concurrency systems". In the operation of actual cases, many people will encounter such a dilemma. Next, let the editor lead you to learn how to deal with these situations! I hope you can read it carefully and be able to achieve something!

Soul torture:

Are there any general solutions to deal with high concurrency systems?

What problems do these solutions solve?

What are the advantages and disadvantages of these schemes?

The tireless pursuit of performance is the fundamental driving force of the continuous development of Internet technology, from the initial mainframe to the current microcomputer, is also born for performance in essence. There is a similar phenomenon in the software system, a system from the initial small number of access requests to the later large concurrent requests, which requires us to provide a series of solutions to improve performance. Like the original Taobao, it is only an outsourced product. With the continuous development of the business, the concurrency of Taobao increases exponentially and poses severe challenges to the system. This has gradually created a high concurrency system like Taobao, which can support tens of millions of people online at the same time.

When it comes to dealing with high concurrency, everyone can more or less name several solutions. The design charm of high concurrency system is that we can rely on programmers' ingenuity to design ingenious solutions to cope with the impact of huge traffic. From the known schemes, they can be summarized into the following categories.

Improve the performance of a single machine

To improve the performance of a single machine as much as possible is an eternal topic, whether it is distributed or other solutions, the improvement of single machine performance is only beneficial to a system. In terms of programming languages, programs written in c or C++ are theoretically more efficient than those written by java or net,Python, of course, based on the normal operation of the program. The simplest and roughest way to improve stand-alone performance is to improve hardware performance. To take a simple example: if the server memory of database DB is 8 gigabytes, as the amount of data increases, you will find that some sql execution will slow down, because database indexes or data cannot be stored in memory and need to be written back to disk. Some queries cannot be hit in memory, resulting in some sql querying data on disk. If you increase the server's memory to 16 gigabytes at this time, you will find that these slow sql have disappeared out of thin air, which is a typical case of hardware performance improvement.

The same is true for the running program, optimize the program to the extreme as much as possible, maybe a single machine can achieve the performance effect of other people's distributed deployment, of course, this requires us to think carefully when writing the code.

"at any time, I think it is necessary to improve the performance of a single machine.

Horizontal expansion

When a stand-alone system can not resist the impact of huge traffic, one of the most simple and effective solutions is horizontal expansion, which refers to dividing the huge traffic into several relatively small traffic, so as to solve the performance problems of highly concurrent systems. in essence, horizontal expansion belongs to the theory of divide and conquer and belongs to the concept of distribution.

To take a very simple example, assuming that the current number of requests processed by a single machine is 200pm, when the number of requests per second reaches 1000, a single machine will definitely encounter a bottleneck. At this time, if the number of servers handling requests is increased to 5 or more, this will easily solve the performance problem. Of course, the convenient scale-out depends on the specific system design. If the system is stateless, there is no problem in theory, but some stateful services may involve work such as state migration. This is one reason why many architects advocate stateless services.

The scale-out of an application can be realized through load balancing, such as Aliyun's SLB service and nginx's reverse proxy function, all of which can easily achieve the scale-out of the application. However, for DB systems such as databases such as mysql, unlimited scale-out may just be a goal. Most DB uses master-slave or multi-master and multi-master to solve the horizontal expansion problem, the master node is responsible for the write operation, and the slave node is responsible for the read operation. Of course, the mechanism of master-slave synchronization and the delay of master-slave synchronization are involved here. Students who are interested can study them in depth.

So when should we choose to scale out? Generally speaking, scale-out is considered at the beginning of the design of the system, because this scheme is simple enough that the problem that can be solved by stacking hardware is not a problem. Now I dare say that more than 90% of the systems were deployed similar to load balancing when the first version was launched, and many of them took advantage of nginx's reverse proxy feature.

Of course, scale-out is not without negative effects. Like a stand-alone system, scale-out also takes into account the loss of a node's down, so monitoring and health check are now a necessary means of the system, and will be in the overall architecture at the beginning of the system design. As I said in previous articles, since scale-out belongs to the distributed category, it is necessary to consider the issues that distributed systems need to consider:

The problem of distributed system

Caching in addition to the scale-out scheme mentioned above, another effective and simple enough scheme is caching. There is no doubt that caching can be found in every corner of a system, from operating systems to browsers, from cpu to disks, from databases to message queues, and in any slightly more complex service and component.

Why can caching greatly improve performance? In terms of the bottleneck of the system, in the life cycle of a request from the client, the response time of the request is severely limited to the slowest link, which is similar to the bucket effect (the amount of water a barrel can hold depends on the shortest plank).

To take a very simple example: when the client requests a commodity information from the mall, the request arrives at a certain port of the server through the http protocol, and the server program unpacks the request and then requests the database, which is not only on another server, but also needs to load data from disk. The so-called DB cache does not hit. In this whole process, the process of requesting the disk is the slowest. The ordinary disk is composed of a mechanical arm, a magnetic head, a rotating shaft, and a disk. When the disk queries data, the head takes a long time to find the path. Of course, the speed of SSD is much faster than the ordinary disk, but it is still several orders of magnitude slower than the memory. The process we want most is like this: when a request arrives at the server, it can extract information from a device as soon as possible and return it to the client. This device can never be a disk. This device is relatively balanced in speed and capacity. It should be memory.

"caching is much more semantic, and any intermediate storage that can reduce response time can be called a cache. For example, CPU's first-level cache, second-level cache, third-level cache, browser cache and so on. Caching mainly solves the problem of speed mismatch between upstream and downstream devices.

There is an old saying in the programming world: putting data in the place closest to the user is the fastest. That's what CDN essentially does. As far as caching is concerned, we often hear the concepts of browser cache, in-process cache, out-of-process cache and so on. At present, the general caching strategy for the server is to use third-party kv storage devices, such as redis,Memcache and so on. Of course, in systems that are extremely demanding on performance, I recommend using in-process caching.

When it comes to asynchrony, we must talk about synchronization. Synchronous invocation means that the caller has to block and wait for the callee to finish execution before returning. The system now generally uses multithreading to provide system throughput (multi-process mode is rare now, but it does not mean there is no, for example: nodejs,nginx). In this way, if the response time of the callee is too long, it will cause the caller's thread to wait for a long time, and the thread utilization will be greatly reduced. Threads are very expensive resources for the system. It is unwise to create a large number of threads to deal with high concurrency, not only wasting memory, but also increasing the cost of thread context cpu switching.

In a high-throughput system, in theory, all threads should work all the time and squeeze the cpu resources to the maximum. For an IO-intensive operation, using asynchronous mode can greatly improve the system throughput. Async can execute other logic without waiting for the callee to complete, and feedback to the caller by notifying the caller after the callee has finished execution.

"Asynchronous is essentially a programming idea, a programming model. It improves the overall throughput of the system, but the response time of requests is slightly longer than that of synchronization.

Like the most commonly used message queue, it also belongs to the asynchronous programming model. The caller will throw the message into the queue, and then directly return to execute other services. The callee receives the message and processes it, and then according to the specific business to see whether it needs to give the result reply. There are many second-kill systems that use message queues to peak traffic, which is one of the advantages of asynchronism.

I need to say one more thing here: async is not without cost. In most cases, async will write more code than synchronous, and it will take more time to find bug. But for a highly concurrent system, the benefits of asynchrony are worth it, as long as you apply it correctly.

This is the end of the content of "what is the general solution for dealing with high concurrency systems". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.