Shulou(Shulou.com)06/02 Report--

This article shows you the principle of Go defer and source code analysis is how, the content is concise and easy to understand, absolutely can make your eyes bright, through the detailed introduction of this article hope you can get something.

There is a very useful reserved word defer in the Go language that can call a function whose execution is deferred until the function that wraps it returns.

The function called by the defer statement is either because the function that wraps it executes the return statement and reaches the end of the function body, or because panic occurs in the corresponding goroutine.

In actual go language programs, defer statements can replace try in other languages. Catch... It can also be used to handle closing operations such as releasing resources, such as closing file handles, closing database connections, and so on.

1. Compiler compiling defer process

Defer dosomething (x)

Simply put, executing the defer statement actually registers a function to be executed later, determines the function name and parameters, but does not call immediately, but defers the calling process until the current function return or panic occurs.

Let's first take a look at the data structures related to defer.

1) struct _ defer data structure

Each call to defer in a go language program generates a _ defer structure.

Type _ defer struct {memory size of siz int32 / / parameters and return values started boul heap boul / / distinguishes whether the structure is allocated on the stack or on the allocated sp uintptr / / sp counter value, stack pointer; pc uintptr / / pc counter value, program counter The address of the function passed in by fn * funcval / / defer, that is, the delayed function; _ panic * _ panic / / panic that is running defer link * _ defer / / linked list}

We use go version 1.13 source code by default, and other versions are similar.

There can be multiple defer calls within a function, so you naturally need a data structure to organize these _ defer structures. _ defer occupies 48 bytes of memory according to the alignment rule. In the link field in the _ defer structure, this field strands all the _ defer into a linked list, with the header hanging from the _ defer field of Goroutine.

The chain structure of _ defer is as follows:

_ defer.siz is used to specify the parameters of the delay function and the space for the return value, the size specified by _ defer.siz, and the value of this block of memory is filled when the defer keyword is executed.

The parameters of the defer delay function are precomputed, allocating space on the stack. The memory layout allocated on the stack for each defer call is shown in the following figure:

Where _ defer is a pointer to a struct _ defer object that may be allocated on the stack or on the heap.

2) struct _ defer memory allocation

The following is an example of using defer with the file name test_defer.go:

Package mainfunc doDeferFunc (x int) {println (x)} func doSomething () int {var x = 1 defer doDeferFunc (x) x + = 2 return x} func main () {x: = doSomething () println (x)}

Compile the above code, plus remove optimization and inner chain options:

Go tool compile-N-l test_defer.go

Export the assembly code:

Go tool objdump test_defer.o

Let's take a look at the compiled binary code:

We can see from the assembly instructions that when the compiler encounters the defer keyword, it adds some runtime functions: deferprocStack and deferreturn.

The release of the official version 1.13 of go has improved the performance of defer, claiming to have improved performance by 30% for defer scenarios.

Defer statements in versions prior to go 1.13 are translated by the compiler into two procedures: callback registration function procedures: deferproc and deferreturn

.

Go 1.13 brings the deferprocStack function, which is the core means of this 30% performance improvement. The purpose of both deferprocStack and deferproc is to register callback functions, but the difference is that deferprocStatck allocates struct _ defer structure on stack memory, while deferproc allocates structure memory on the heap. Most of our scenarios can be allocated on the stack, so the overall performance of nature is improved. Allocating memory on the stack is naturally much faster than alignment, and you only need to change the value of the rsp register to allocate.

So when is it allocated on the stack and when is it allocated on the heap?

In the compiler-related file (src/cmd/compile/internal/gc/ssa.go), there is a condition judgment:

Func (s * state) stmt (n * Node) {case ODEFER: d: = callDefer if n.Esc = = EscNever {d = callDeferStack}}

N.Esc is the result of ast.Node 's escape analysis, so when will n.Esc be set to EscNever?

This is in the escape analysis function esc (src/cmd/compile/internal/gc/esc.go):

Func (e * EscState) esc (n * Node, parent * Node) {case ODEFER: if e.loopdepth = = 1 {/ / top level n.Esc = EscNever / / force stack allocation of defer record (see ssa.go) break}}

Here, when e.loopdepth is equal to 1, it will be set to EscNever, and the e.loopdepth field is used to detect the nested loop scope. In other words, if the defer is in the context of the nested scope, it may cause struct _ defer to be allocated on the heap, as follows:

Package mainfunc main () {for I: = 0; I

< 10; i++ { defer func() { _ = i }() }} 编译器生成的则是 deferproc ： 当 defer 外层出现显式（for）或者隐式（goto）的时候，将会导致 struct _defer 结构体分配在堆上，性能就会变差，这个编程的时候要注意。 编译器就能决定 _defer 结构体分配在栈上还是堆上，对应函数分别是 deferprocStatck 和 deferproc 函数，这两个函数都很简单，目的一致：分配出 struct _defer 的内存结构，把回调函数初始化进去，挂到链表中。 3) deferprocStack 栈上分配 deferprocStack 函数做了哪些事情呢？ // 进入这个函数之前，就已经在栈上分配好了内存结构func deferprocStack(d *_defer) { gp := getg() // siz 和 fn 在进入这个函数之前已经赋值 d.started = false // 表明是栈的内存 d.heap = false // 获取到 caller 函数的 rsp 寄存器值，并赋值到 _defer 结构 sp 字段中 d.sp = getcallersp() // 获取到 caller 函数的 rip 寄存器值，并赋值到 _defer 结构 pc 字段中 // 根据函数调用的原理，我们就知道 caller 的压栈的 pc (rip) 值就是 deferprocStack 的下一条指令 d.pc = getcallerpc() // 把这个 _defer 结构作为一个节点，挂到 goroutine 的链表中 *(*uintptr)(unsafe.Pointer(&d._panic)) = 0 *(*uintptr)(unsafe.Pointer(&d.link)) = uintptr(unsafe.Pointer(gp._defer)) *(*uintptr)(unsafe.Pointer(&gp._defer)) = uintptr(unsafe.Pointer(d)) // 注意，特殊的返回，不会触发延迟调用的函数 return0()} 小结： 由于是栈上分配内存的，所以调用到 deferprocStack 之前，编译器就已经把 struct _defer 结构的函数准备好了； _defer.heap 字段用来标识这个结构体分配在栈上； 保存上下文，把 caller 函数的 rsp，pc（rip） 寄存器的值保存到 _defer 结构体； _defer 作为一个节点挂接到链表。注意：表头是 goroutine 结构的 _defer 字段，而在一个协程任务中大部分有多次函数调用的，所以这个链表会挂接一个调用栈上的 _defer 结构，执行的时候按照 rsp 来过滤区分；4) deferproc 堆上分配 堆上分配的函数为 deferproc ，简化逻辑如下： func deferproc(siz int32, fn *funcval) { // arguments of fn fullow fn // 获取 caller 函数的 rsp 寄存器值 sp := getcallersp() argp := uintptr(unsafe.Pointer(&fn)) + unsafe.Sizeof(fn) // 获取 caller 函数的 pc（rip） 寄存器值 callerpc := getcallerpc() // 分配 struct _defer 内存结构 d := newdefer(siz) if d._panic != nil { throw("deferproc: d.panic != nil after newdefer") } // _defer 结构体初始化 d.fn = fn d.pc = callerpc d.sp = sp switch siz { case 0: // Do nothing. case sys.PtrSize: *(*uintptr)(deferArgs(d)) = *(*uintptr)(unsafe.Pointer(argp)) default: memmove(deferArgs(d), unsafe.Pointer(argp), uintptr(siz)) } // 注意，特殊的返回，不会触发延迟调用的函数 return0()} 小结： 与栈上分配不同，struct _defer 结构是在该函数里分配的，调用 newdefer 分配结构体，newdefer 函数则是先去 poul 缓存池里看一眼，有就直接取用，没有就调用 mallocgc 从堆上分配内存； deferproc 接受入参 siz，fn ，这两个参数分别标识延迟函数的参数和返回值的内存大小，延迟函数地址； _defer.heap 字段用来标识这个结构体分配在堆上； 保存上下文，把 caller 函数的 rsp，pc（rip） 寄存器的值保存到 _defer 结构体； _defer 作为一个节点挂接到链表； 5) 执行 defer 函数链 编译器遇到 defer 语句，会插入两个函数： 分配函数：deferproc 或者 deferprocStack ； 执行函数：deferreturn 。 包裹 defer 语句的函数退出的时候，由 deferreturn 负责执行所有的延迟调用链。 func deferreturn(arg0 uintptr) { gp := getg() // 获取到最前的 _defer 节点 d := gp._defer // 函数递归终止条件（d 链表遍历完成） if d == nil { return } // 获取 caller 函数的 rsp 寄存器值 sp := getcallersp() if d.sp != sp { // 如果 _defer.sp 和 caller 的 sp 值不一致，那么直接返回； // 因为，就说明这个 _defer 结构不是在该 caller 函数注册的 return } switch d.siz { case 0: // Do nothing. case sys.PtrSize: *(*uintptr)(unsafe.Pointer(&arg0)) = *(*uintptr)(deferArgs(d)) default: memmove(unsafe.Pointer(&arg0), deferArgs(d), uintptr(d.siz)) } // 获取到延迟回调函数地址 fn := d.fn d.fn = nil // 把当前 _defer 节点从链表中摘除 gp._defer = d.link // 释放 _defer 内存（主要是堆上才会需要处理，栈上的随着函数执行完，栈收缩就回收了） freedefer(d) // 执行延迟回调函数 jmpdefer(fn, uintptr(unsafe.Pointer(&arg0)))} 代码说明： 遍历 defer 链表，一个个执行，顺序链表从前往后执行，执行一个摘除一个，直到链表为空； jmpdefer 负责跳转到延迟回调函数执行指令，执行结束之后，跳转回 deferreturn 里执行； _defer.sp 的值可以用来判断哪些是当前 caller 函数注册的，这样就能保证只执行自己函数注册的延迟回调函数； 例如，a() ->

B ()-> c (), a calls brecom b to call c, and the three functions of arecine bpenc all have defer registration delay functions, so naturally, when the c () function returns, the callback of c is executed.

2. Defer pass parameters

1) pre-calculated parameters

In the previous description of the _ defer data structure, the memory structure is as follows:

_ defer acts as a header on the stack, and the parameter and return value of the deferred callback function (defer) are placed immediately after _ defer, and this parameter value is set when the defer is executed, that is, the parameter is precomputed, rather than getting it when the defer function is executed.

For example, when you execute defer func (xrecoery y), these two arguments are calculated, and the function calls in Go are value passing. Then the value of x _ defer y will be copied to the _ copy structure. Look at another example:

Package mainfunc main () {var x = 1 defer println (x) x + = 2 return}

What is the output of this program? Is it one or three? The answer is 1. The function executed by defer is println, the println parameter is x, and the value passed in by the value of x is confirmed when the defer statement is executed.

2) Parameter preparation of defer

The parameters executed by the defer delay function have been saved in the contiguous block of memory along with _ defer. So where do the parameters come from when the defer function is executed? Of course, not directly to the address of _ defer. Because here is the standard function call.

In the Go language, the parameters of a function are prepared by the caller function, for example, a main ()-> A (7)-> B (a) forms a stack frame similar to the following:

So, in addition to jumping to the defer function instruction, deferreturn needs to do one more thing: prepare the parameters (space and value) needed for the defer delay callback function. Then the line of sight is done by the following code:

Func deferreturn (arg0 uintptr) {switch d.siz {case 0: / / Do nothing. Case sys.PtrSize: * (* uintptr) (unsafe.Pointer & arg0) = * (* uintptr) (deferArgs (d)) default: memmove (unsafe.Pointer (& arg0), deferArgs (d), uintptr (d.siz))}}

Arg0 is the stack address that caller uses to place defer parameters and return values. This code means to copy the pre-prepared parameters of _ defer to some address (arg0) of the caller stack frame.

3. Execute multiple defer

As explained in detail earlier, _ defer is a linked list with a goroutine._defer structure in the header. A function of a co-program registers the same linked list and distinguishes the functions according to rsp when executed. Also, the linked list inserts new elements in the header and executes from the front to the back, so this leads to a LIFO feature that registers the defer function first and then executes.

4. Defer and return running order

When the function containing the defer statement returns, set the return value first or execute the defer function first?

1) the calling process of the function

To understand this process, you first need to know the procedure of the function call:

The one-line function call statement of go is actually not an atomic operation, and corresponds to multiple lines of assembly instructions, including 1) parameter setting and 2) call instruction execution.

There are also two contents of the call assembly instruction: the return address stack (which causes the rsp value to grow down, rsp-0x8), and the callee function address is loaded into the pc register

In fact, the one-line function return return statement of go is not an atomic operation, which corresponds to multiple lines of assembly instructions, including 1) return value setting and 2) ret instruction execution.

There are two ret assembly instructions, the instruction pc register is restored to the address saved at the top of the rsp stack, the rsp is reduced upward, and rsp+0x8

The parameter is set in the caller function, and the return value is set in the callee function

Rsp and rbp are the two most important registers of the stack frame. These two values delimit the stack frame.

The most important point: the statement call of Go's return is a compound operation, which can correspond to two operation sequences:

Set the return value

Ret instruction jumps to caller function

2) should I return the value first or execute the defer function first after return?

The official Golang documentation clearly states:

That is, if the surrounding function returns through an explicit return statement, deferred functions are executedafter any result parameters are set by that return statementbutbefore the function returns to its caller.

That is, the function chain call of defer is after the return value is set, but before the run instruction context returns to the caller function.

So it contains the functions registered by defer, and after executing the return statement, it executes three operation sequences:

Set the return value

Execute defer linked list

Ret instruction jumps to caller function

So, according to this principle, let's analyze the following behavior:

Func F1 () (r int) {t: = 1 defer func () {t = t + 5} () return t} func f2 () (r int) {defer func (r int) {r = r + 5} (r) return 1} func f3 () (r int) {defer func () {r = r + 5} () return 1}

What are the return values of these three functions?

Answer: F1 ()-> 1 recorder f2 ()-> 1 recorder f3 ()-> 6.

A) after function F1 executes the return t statement:

Set the return value r = t, when the value of the local variable t is equal to 1, so r = 1

Execute the defer function, t = tweak 5, after which the value of the local variable t is 6

Execute the assembly ret instruction and jump to the caller function

So, the return value of F1 () is 1

B) after function f2 executes the return 1 statement:

Set the return value r = t, when the value of the local variable t is equal to 1, so r = 1

Execute the defer function, t = tweak 5, after which the value of the local variable t is 6

Execute the assembly ret instruction and jump to the caller function

So, the return value of f2 () is still 1.

C) after function f3 executes the return 1 statement:

Set the return value r = 1

Execute the defer function, r = rsqu5, and then return the value variable r with a value of 6 (this is a closure function, note that it is distinguished from f2)

Execute the assembly ret instruction and jump to the caller function

Therefore, the return value of F1 () is 6.

The defer keyword executes the corresponding _ defer data structure, which is always allocated on the heap during the go1.1-go1.12 period, and is optimized to allocate the _ defer structure on the stack after go1.13, resulting in a significant improvement in performance.

_ defer most scenarios are assigned on the stack, but scenarios that encounter loop nesting will be assigned to the heap, so you should pay attention to defer usage scenarios when programming, otherwise performance problems may occur.

_ defer corresponds to a registered deferred callback function (defer). The parameters and return values of the defer function are closely followed by _ defer, which can be understood as header,_defer and function parameters. The memory where the return value is located is a continuous space, where _ defer.siz indicates the space occupied by the parameters and return values.

The functions registered by defer in the same protocol are all hung in a linked list with the header goroutine._defer

The new element is inserted at the front, and the traversal is executed from the back to the back. Therefore, the defer registration function has the feature of LIFO, that is, the registration is executed first.

Different functions are on this linked list, distinguished by _ defer.sp

The parameter of defer is precomputed, that is, when the defer keyword is executed, the parameter is confirmed and assigned after the memory block of _ defer. When executing, copy to the corresponding position of the stack frame

Return corresponds to the compound operation of three actions: setting the return value, executing the linked list of defer functions, and jumping the ret instruction.

The above is what the principle of Go defer and source code analysis is like, have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.