Shulou(Shulou.com)06/01 Report--

How to analyze the source code of libgo, I believe that many inexperienced people are at a loss about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

Gossip

Collaborative programming is a very early concept, and it has been used on a large scale in the game industry in the early years. The cooperative programming primitives in languages such as lua and go have been relatively perfect. Generally speaking, it is good to use them directly, but in the background development of the system, it does not appear for a long time.

I am doing the backstage development of C++. At present, some domestic companies have also opened up some C++ protocol libraries, but at present, they are still in the stage of gradual improvement. The earliest contact with the C++ protocol library is the libco of Tencent Wechat, which can be said to be quite lightweight, and there are relatively many articles on the implementation of libco on the Internet, which will not be described too much here.

Look for information about libgo on the Internet, and there are not many articles about the implementation of the code, so I intend to read the code summary by myself. In the following article, I may compare some of the functions of libgo and libco. I hope the reader will point out the shortcomings.

Background

Due to the need of work, the asynchronous framework of the previous system is no longer so scalable. Asynchronism makes the originally simple logic (read-> process-> write) to be split into multiple phases and respond to each event through callbacks. So use a co-program instead.

Why did you choose libgo instead of the lighter libco? Someone on the Internet has given a performance comparison between the two. Considering the number of spin locks, collaborators, stack space, thread support and other aspects, it seems that libgo has won a complete victory.

The picture comes from the Internet.

The performance comparison data come from the network, which is not to say that libco is not good. Perhaps the scenarios of their respective applications are slightly different. With thousands of lines of code in libco, you can achieve a relatively complete collaboration and withstand the huge data traffic in Wechat backend, so it has its own advantages. Since the code for libgo is under study, we will not comment on both for the time being.

Cooperative process

No matter what kind of protocol it is, the core content is that the upper layer actively gives up the CPU when the system is blocked, and switches the context of the ready protocol. To sum up briefly, there are the following aspects:

Implementation of context switching

Hook of system function

Cooperative scheduling

Time management

Libgo source code

Https://github.com/yyzybb537/libgo

Libgo directory structure description muhui@ASIAYANG-MB0:~/code/libgo/libgo-master$ lltotal 64 muhui staff RW muhui RW muhui @ 1 muhui staff 5913 11 7 11:20 CMakeLists.txt-rw-r--r--@ 1 muhui staff 1084 11 7 11:20 LICENSE-rw-r--r--@ 1 muhui staff 8758 11 7 11:20 README.md-rw-r--r--@ 1 muhui staff 4230 11 7 11:20 TODOdrwxr-xr-x@ 4 muhui Staff 128 11 7 11:20 imgsdrwxr-xr-x@ 15 muhui staff 480 11 7 11:20 libgodrwxr-xr-x@ 8 muhui staff 256 11 7 11:20 testdrwxr-xr-x@ 6 muhui staff 192 11 7 11:20 third_partydrwxr-xr-x@ 20 muhui staff 640 11 7 11:20 tutorialdrwxr-xr-x@ 4 muhui staff 128 11 7 11:20 vs_projmuhui@ASIAYANG-MB0:~/code/libgo/libgo-master$ cd libgo/muhui@ASIAYANG-MB0:~ / code/libgo/libgo-master/libgo$ lltotal 16drwxr muhui staff 7 11:20 clsdrwxr-xr-x@ 19 muhui staff 608 11 7 11:20 commondrwxr-xr-x@ 6 muhui staff 192 11 7 11:20 context-rw-r--r--@ 1 muhui staff 1848 11 7 11:20 coroutine.hdrwxr-xr-x@ 5 muhui staff 160 11 7 11:20 debugdrwxr-xr-x@ 4 muhui staff 128 11 7 11:20 defer- Rw-r--r--@ 1 muhui staff 36 11 7 11:20 libgo.hdrwxr-xr-x@ 4 muhui staff 128 11 7 11:20 netiodrwxr-xr-x@ 5 muhui staff 160 11 7 11:20 pooldrwxr-xr-x@ 8 muhui staff 256 11 7 11:20 schedulerdrwxr-xr-x@ 7 muhui staff 224 11 7 11:20 syncdrwxr-xr-x@ 4 muhui staff 128 11 7 11:20 taskdrwxr-xr-x@ 4 muhui staff 128 11 7 11:20 timer

One of the better things that libgo has done is to increase the support for the windows environment, etc., we only do research on the Linux environment.

Functions that will be gradually improved or added by TODO:libgo in the future

Libgo: the home directory of the source code implementation, where the implementation of cooperative programs and scheduling policies are located.

Test: test code

Tutorial:libgo uses tutorial code

How to use libgo in vs_proj:VS environment.

Under the libgo directory

Task: the related implementation of the cooperative process

Scheduler: the implementation of Cooperative scheduling

Debug:libgo 's own debugging function (for locating the state of the co-program, etc.)

Coroutine.h: some common alignment methods have been redefined.

System call of netio:hook

Context: context switching

Connection pooling implemented by pool:libgo

Overview of libgo scheduling principle

We know that the co-program is a user-mode thread, so libco does not support threads. But in libgo, threads are also supported, depending on how they are scheduled.

The first point we want to say is that in libgo, the scheduling of each task,libgo is not directly affected by the co-program, but is implemented through indirect scheduling.

There are concepts of scheduler and processer in libgo:

The executor is directly responsible for the co-program scheduling, which will cut out the context when the co-program is blocked and cut into the context of a ready co-program to continue processing. when there is no executable co-program, the executor will block waiting and continue to process when there is a new task.

The scheduler is responsible for managing the executor. for the scheduler, each actuator is a separate thread. The most important job of the scheduler is to balance the number of collaborators in each actuator to prevent hunger effect. Some actuators are too busy, but some actuators do not have task to execute. In addition, if an actuator is stuck, the scheduler will also take out the runnable co-programs in the actuator. Put it on the actuator with the lowest load.

Of course, the number of schedulers supports dynamic expansion.

As shown below:

After reading the above, have you mastered how to analyze the source code of libgo? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.