Shulou(Shulou.com)06/03 Report--

This article focuses on "how to understand signal-based preemptive scheduling in Go". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "how to understand signal-based preemptive scheduling in Go".

Identifying the nature of the accident and showing it with a very simple example is a manifestation of skill. The cause of the accident can be reduced to the following demo:

Demo-1

Let me briefly explain the above program. In the main goroutine, first use the GoMAXPROCS function to get the logical core number threads of the CPU. This means that the Go process creates a P of the number of threads. Then, the goroutine of the number of threads is started, and each goroutine is executing an infinite loop, and the infinite loop is simply xdistinct +.

Then, the main goroutine sleep takes 1 second; finally, the value of x is printed.

You can think for yourself, what will the output be?

If you have come up with the answer, then look at the following demo:

Demo-2

Let me also explain that in the main goroutine, only one goroutine is started (although a for loop is used in the program, it is actually only looped once, just to look more compatible with the previous demo), and an infinite for loop is also executed.

The difference from the previous demo is that in the main goroutine, we manually execute the GC; once and finally print the value of x.

If you can answer the first question correctly, chances are you can also answer the second question correctly.

Now I will find out the answer.

Actually, I left a hole, and I didn't say which version of Go to use to run the code. So, the correct answer is:

The Go version of demo-1demo-21.13 is stuck 1.1400.

This is actually the trap of the Go scheduler.

Suppose there are four Ps in demo-1, so four goroutine are created. When the main goroutine executes sleep, the four goroutine just created immediately occupy the four Ps, execute an endless loop, and there is no function call, only a simple assignment statement. There is nothing Go 1.13 can do about this situation, there is no way to stop these goroutine, and the process appears to "crash" externally.

Demo-1 schematic diagram

Because Go 1.14 implements signal-based preemptive scheduling, these goroutine executing infinite loops will be "taken down" by the scheduler and P will be empty. So when the main goroutine sleep time is up, you can immediately get P and be able to print out the value of x. As for why x outputs 0, it's hard to explain, because it's an undefined (data competition, normally locked) behavior, and one possible reason is that CPU's cache didn't have time to update, but it's not easy to verify.

After understanding this demo, the second demo is actually a similar truth:

Demo-2 schematic diagram

When the main goroutine actively triggers GC, you need to stop all currently running goroutine, that is, stw (stop the world), but goroutine is executing an infinite loop and can't stop it. Of course, Go 1.14 can still preempt the goroutine and print out the value of x, which is also 0.

In Go versions prior to 1.14, it was important to preempt a goroutine that was executing an endless loop:

Whether it can be preempted or not depends not on whether the function is called, but on whether the preface of the function has inserted stack expansion detection instructions.

If the function is not called, it will certainly not be preempted.

Although some functions are called, they do not insert detection instructions and will not be preempted at this time.

Like the previous two demo, there is no chance to actively relinquish the right to use CPU during function stack expansion detection, thus completing preemption, because there are no function calls. After the specific process, there is an opportunity to write another article in detail, this paper mainly looks at how to achieve signal-based preemptive scheduling.

Preemptone

On the one hand, when the Go process starts, it will open a background thread, sysmon, to monitor the goroutine that takes too long to execute, and then issue preemption. On the other hand, when GC executes stw, it stops all goroutine, which is actually preemption. Both call the preemptone () function.

The preemptone () function follows this path:

Preemptone- > preemptM- > signalM- > tgkill

Send a SIGURG signal to the M (or thread) that the running goroutine is bound to.

Register for sighandler

Each M sets the signal processing function during initialization:

Initsig- > setsig- > sighandler signal execution process

Let's take a look at the signal execution process from a "macro" level:

Signal execution process

The main program (thread) is "diligently" executing the instruction: it has finished executing instruction m, and then it is about to execute instruction mSecret1. Unfortunately, at this point, the thread receives a signal that corresponds to the ① in the figure.

The kernel then takes over the execution flow and instead executes the preset signal processor program, which corresponds to the Go, which executes sighandler, which corresponds to the ② and ③ in the figure.

Finally, the execution flow is handed over to the thread and continues to execute instruction mSecret1, which corresponds to the ④ in the figure.

In fact, this involves some on-site protection and recovery, the kernel has helped us take care of it, we do not have to worry about.

DosigPreempt

When the thread receives the SIGURG signal, it executes the sighandler function, the core of which is the doSigPreempt function.

Func sighandler (sig uint32, info * siginfo, ctxt unsafe.Pointer, gp * g) {. If sig = = sigPreempt & & debug.asyncpreemptoff = = 0 {doSigPreempt (gp, c)}...}

The doSigPreempt function is actually very short, and it will be finished in a minute.

Func doSigPreempt (gp * g, ctxt * sigctxt) {. If ok, newpc: = isAsyncSafePoint (gp, ctxt.sigpc (), ctxt.sigsp (), ctxt.siglr ()); ok {/ / Adjust the PC and inject a call to asyncPreempt. Ctxt.pushCall (funcPC (asyncPreempt), newpc)}...}

The isAsyncSafePoint function returns whether the current goroutine can be preempted, and from which instruction the preemption begins, and the returned newpc indicates the secure preemptive address.

Next, pushCall adjusts the SP, sets the values of several registers and returns. Supposedly, after returning, we will continue to execute the instruction mSecret1, but how can we achieve preemption? In fact, the magic is all in the pushCall function.

PushCall

Before analyzing this function, we need to review the calling specification of the Go function, focusing on the CALL and RET instructions.

Call and ret instructions

The call instruction can be simply understood as push ip + JMP. This ip is actually the return address, that is, the address of the next instruction to execute when the subfunction is called. So push ip just pushes the return address into the stack before call a subfunction, and then JMP to the address of the subfunction to execute.

The ret instruction, in contrast to the call instruction, pop the return address from the stack to the IP register, allowing CPU to continue execution from that address.

After understanding call and ret, let's analyze the pushCall function:

Func (c * sigctxt) pushCall (targetPC, resumePC uintptr) {/ / Make it look like we called target at resumePC. Sp: = uintptr (c.rsp ()) sp-= sys.PtrSize * (* uintptr) (unsafe.Pointer (sp)) = resumePC c.set_rsp (uint64 (sp)) c.set_rip (uint64 (targetPC))}

Notice the note on this line:

/ / Make it look like we called target at resumePC.

It clearly illustrates the purpose of this function: to mislead CPU into thinking that resumePC called targetPC. This resumePC is the newpc returned by calling the isAsyncSafePoint function in the previous step, which represents the instruction address of our preemptive goroutine.

The first two lines of code move SP down eight bytes and put resumePC on the stack (note that it's actually a return address), then set targetPC to the ip register and sp to the SP register. This makes it possible to return to user-mode execution from the kernel, not from instruction mSecret1, but directly from targetPC, and then return to resumePC to continue execution when targetPC is finished. The whole process is like resumePC calling targetPC. And targetPC is actually funcPC (asyncPreempt), that is, the preemption function.

So we can see that the signal processor program sighandler just "inserts" an asynchronous preemption function, while the real preemption process is done in the asyncPreempt function.

Asynchronous preemption

When the sighandler is executed, the execution flow goes back to the thread again. Because sighandler inserts a function call to asyncPreempt, the original task of goroutine is not advanced and executes asyncPreempt instead:

AsyncPreempt call Link

The function of mcall (fn) is to cut to the G0 stack to execute the function fn, and fn never returns. In mcall (gopreempt_m), fn is gopreempt_m.

Gopreempt_m calls goschedImpl directly:

GoschedImpl

Dropg

The best part is the goschedImpl function. It first changes the state of goroutine from running to runnable;, then calls dropg to unbind g and m, and then calls globrunqput to throw goroutine into the global runnable queue, which needs to be locked because it is a global runnable queue. Finally, the schedule () function is called to enter the scheduling loop. You can read this article about scheduling loops.

The schedule function runs with the G0 stack, which looks for other runnable goroutine, including fetching from the current P local runnable queue, fetching from the global runnable queue, and stealing from other P to find the next runnable goroutine and execute it.

At this point, the thread moves on to execute another goroutine, and the current goroutine is preempted.

When will the preempted goroutine be executed again?

Because it has been thrown into the global runnable queue, its priority is lowered and its chances of getting scheduled are reduced, but there is always a chance to execute again, and it will be executed from the next instruction that calls mcall.

Remember what the mcall function does? It will cut to the G0 stack to execute the gopreempt_m, and naturally it will also save the execution progress of the goroutine, which is actually the value of the SP, BP, and PC registers. When the goroutine is scheduled to execute again, it will continue execution from the original breakpoint of the execution flow.

At this point, I believe you have a deeper understanding of "how to understand signal-based preemptive scheduling in Go". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.