Shulou(Shulou.com)06/02 Report--

Editor to share with you Golang running and Plan9 assembly example analysis, I believe that most people do not know much, so share this article for your reference, I hope you will learn a lot after reading this article, let's go to know it!

The operating environment of Golang

When we run the compiled Go code, it appears in the system as a process. Then start processing requests, data, and we'll see that the process takes up memory consumption, cpu percentage, and so on. The purpose of this article is to explain how memory, CPU, operating system (and, of course, other hardware, not to mention) cooperate in the process of running the program to accomplish what our code specifies.

Memory

First of all, let's talk about memory. Let's first take a look at a go process that we are running.

The code is as follows:

Package main import ("fmt"log"net/http") func main () {http.HandleFunc ("/", sayHello) err: = http.ListenAndServe (": 9999", nil) if err! = nil {log.Fatal ("ListenAndServe:", err)}} func sayHello (w http.ResponseWriter, r * http.Request) {fmt.Printf ("fibonacci:% d\ n" Fibonacci (1000)) _, _ = fmt.Fprint (w, "Hello World!")} func fibonacci (num int) int {if num

Ps aux USER PID% CPU% MEM VSZ RSS TT STAT STARTED TIME COMMAND xxxxx 3584 99.2 0.1 4380456 4376 s003 R + 8:33 05.81 p.m. / myhttp

Let's first look at VSZ and RSS without paying attention to other indicators.

VSZ: refers to the virtual address, which is the actual memory of the program. Contains the allocation of memory that is not yet used.

RSS: the actual physical memory, including stack memory and heap memory.

Each process runs in its own memory sandboxie, and the address assigned to the program is "virtual memory". The physical memory is actually invisible to the program developer, and the virtual address is much larger than the actual physical address of the process. We often program to take the address corresponding to the pointer is actually a virtual address. It is important to distinguish between virtual memory and physical memory. Take a picture and feel it.

The main purpose of this picture is to illustrate two problems:

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

Programs use virtual memory, but the operating system maps virtual memory to physical memory; you will find that the VSZ of all processes on your machine is much larger

Physical memory can be shared by multiple processes, and even different addresses within a process may map the same physical memory address.

The above knows exactly what the memory in the program refers to, and then explains how the program uses memory (virtual memory). To put it bluntly, memory is a piece of hardware that accesses faster than a hard disk. In order to facilitate memory management, the operating system divides the memory allocated to the process into different functional blocks. As we often say: code area, static data area, stack area, stack area and so on.

Let's borrow a picture from the Internet to take a look.

This is the distribution of our programs (processes) in virtual memory.

Code area: what is stored is our compiled machine code, which can only be read-only in general.

Static data area: global variables and constants are stored. The addresses of these variables are determined at compile time (this is also the benefit of using virtual addresses, which are impossible to determine at compile time if they are physical addresses). Both Data and BSS belong to this part. This part will be destroyed only if the program is aborted (kill, crasg, etc.).

Stack area: mainly where functions, methods, and their local variables are stored in Golang. This part is allocated with the execution of the function and method, and is released after running, with special attention that the release here does not empty the memory. Later articles will talk about memory allocation in more detail; there is another point to remember that the stack is generally allocated from the high address to the low address, in other words: the high address belongs to the low address of the stack, the low address belongs to the bottom of the stack, and its allocation direction is opposite to that of the heap.

Heap area: in languages like Chammer Cobb +, the heap is completely controlled by the programmer himself. But because of the GC mechanism in Golang, we don't need to worry about whether memory is allocated on the stack or on the heap when writing code. Golang will determine for itself that if the lifecycle of a variable cannot be destroyed after the function exits, or if resources on the stack are not allocated enough, it will be put on the heap. The performance of the heap is worse than that of the stack. The reasons are also left to the articles related to memory allocation to introduce to you.

The structure of memory has been figured out, and it takes the operating system to direct our programs to be loaded into memory in order to run correctly.

Add a more important concept:

Addressing space: generally speaking, it refers to the ability of CPU to address memory. Generally speaking, it is a question of how much memory can be used. For example: 32 address lines (32-bit machines), then the total address space is 2 ^ 32, if it is a 64-bit machine, it is 2 ^ 64 address space. You can use uname-a to view the digits supported by your system.

Operating system, CPU and memory cooperate with each other.

In order to explain the running and calling of the program, we must first clarify the relationship among the operating system, memory, CPU, and registers.

CPU: the brain of a computer that can understand and execute instructions

Register: strictly speaking, register is a part of CPU, which is mainly responsible for temporarily storing data when CPU calculates. Of course, CPU also has multi-level cache, which is not very relevant to us here, so we just skip it. As we all know, its purpose is to make up the gap between memory and CPU speed.

Memory: like the memory above, the memory is divided into different areas, and each part stores different data; of course, the division of these areas and the mapping between virtual memory and physical memory are done by the operating system.

Operating system: control all kinds of hardware resources, provide operation interface (system call) and management for other running programs.

Here the operating system is a software, CPU, registers, memory (physical memory) are real hardware. Although the operating system is also written by a pile of code. But she is the interface of the hardware to other applications. Generally speaking, the operating system controls all hardware resources through system calls, and it dispatches other programs to CPU for other programs to execute, but in order to give each program a chance to use CPU,CPU, it gives control to the operating system through time interruptions.

So that the operating system can control our programs, the programs we write need to follow the rules of the operating system. In this way, the operating system can control the execution of programs, switch processes and other operations.

Finally, after our code is compiled into machine code, it is essentially a piece of instruction. What we expect is for CPU to execute these instructions and then complete the task. On the other hand, the operating system can help us get CPU to execute the code and provide the calling interface (system call) of the required resources. Isn't it very simple?

Calling specification of Go program

We know that the whole virtual memory is divided into code area, static data area, stack area and heap area. The next call specification of the Go program (in fact, the rules for the operation of functions and methods) is mainly related to the stack part mentioned above (the heap part will be discussed in the memory allocation article). And how the hard and soft parts of the computer work together. Next, let's take a look at how the basic unit functions and methods of the program are executed and called each other.

The distribution of functions on the stack

In this part, we first understand some theories, and then use a practical example to analyze it. Let's take a look at how functions are distributed on the stack in Golang.

Several professional terms involved:

Stack: the stack here is consistent with the above explanation. Processes, threads, and goroutine all have their own call stacks

Stack frame: it can be understood to be the area assigned to the function on the stack when the function is called

Caller: caller, for example: a function calls b function, then an is the caller.

Transferee: callee, or the above example, b is the callee.

Stack frame

What this picture shows is the structure of a stack frame. It can also be said that the stack frame is the stack space allocated to a function by the stack, which includes the function caller address, local variables, return value address, caller parameters and other information.

Here are a few notes. The BP and SP in the figure all represent the corresponding registers.

BP: the base address pointer register (extended base pointer), also known as the frame pointer, holds a pointer indicating the beginning of the function stack.

SP: stack pointer register (extended stack pointer), which stores a pointer and stores the top of the function stack space, that is, where the function stack space allocation ends. Note that this is a hardware register, not a pseudo register in Plan9.

BP and SP are put together, one for the beginning (top of the stack) and one for the end (low stack).

With the basics above, let's use a practical example to verify it.

Call instance of Go

Just to start, let's start with a simple function to analyze the entire function call process (the following involves Plan9 assembly, please do not panic, most of them can understand, and I will also write notes).

Package main func main () {a: = 3 b: = 2 returnTwo (a, b)} func returnTwo (a, b int) (c, d int) {tmp: = 1 / / the main purpose of this line is to ensure that the stack frame is not 0, and it is convenient to analyze c = a + b d = b-tmp return}.

There are two functions above, main defines two local variables, and then calls the returnTwo function. The returnTwo function takes two arguments and two return values. The two return values are designed to take a look at how the multiple return values of golang are implemented. Next, let's show the assembly code corresponding to the above code.

There are a few lines of code that need to be specifically explained.

0x0000 00000 (test1.go:3) TEXT "" .main (SB), ABIInternal, $56-0

Key information in this line: $56-0. 56 represents the stack frame size of the function (two local variables, two parameters are of type int, two return values are of type int, one holds base pointer, a total of 7 * 8 = 56), and 0 represents the parameter and return value of the mian function. You can check its return value in returnTwo later.

Let's take a look at how the computer allocates size on the stack.

0x000f 00015 (test1.go:3) SUBQ $56, SP / / allocation, the size of 56 is defined in the first line above. 0x004b 00075 (test1.go:7) ADDQ $56, SP / / is released, but not empty

These two lines, one is distribution, the other is release. Why can it be allocated using the SUBQ instruction? And ADDQ is released? Remember what we said before? The SP is a pointer register and points to the top of the stack, and the stack is allocated from the high address to the low address. So if you subtract it once, does it mean moving the pointer from a high address to a low address? The same goes for release, where an addition operation returns the SP to its original state.

Let's take a look at the operation on the BP register.

0x0013 00019 (test1.go:3) MOVQ BP, 48 (SP) / / Save BP 0x0018 00024 (test1.go:3) LEAQ 48 (SP), BP / / BP stores the new address... 0x0046 00070 (test1.go:7) MOVQ 48 (SP), BP / / restore BP address

Do these three lines of code feel awkward? Writing and writing makes people feel foggy. I'll describe it in words first, and then explain it with a picture later.

Let's make the following assumption: at this point, the value BP points to is: 0x00ffrecover48 (SP), the address is: 0x0008.

The first instruction MOVQ BP, 48 (SP) is to write 0x00ff to the location of 48 (SP).

The second instruction, LEAQ 48 (SP), BP is the update register pointer that tells BP to hold the address of location 48 (SP), that is, the value 0x00ff.

The third instruction, MOVQ 48 (SP), BP, because at the beginning 48 (SP) holds the original value 0x00ff of BP, so here we restore BP again.

The role of these lines of code is very important, which is why when we execute, we can find the beginning of the function and return to the position where the function is called, so that it can continue to execute (if you feel forgiven, let it go first, there is a picture behind it, and then come back to understand). Let's take a look at the returnTwo function.

Here NOSPLIT | ABIInternal, $0-32 indicates that the stack frame size of this function is 0. Because there are two int parameters and two int return values, the total size is 4 bytes 8 = 32 bytes, is it matched with the above main function?

Is there any confusion that the size of the stack frame of the returnTwo function is 0? Doesn't this function need stack space? In fact, the main reason is that both the parameter passing and return values of golang require the use of stacks (which is why go can support multiple parameter returns). So the space required for both the parameter and the return value is provided by caller.

Next, we demonstrate the calling process with a complete diagram.

This picture has been drawn for nearly an hour. I hope it will be helpful for everyone to understand.

The whole process is: initialize-> call main function-> call returnTwo function-> returnTwo return-> main return.

Through this picture, combined with my above text explanation, I believe you can understand. But here are a few caveats:

BP and SP are registers, which hold the address on the stack, so you can do operations on SP to find the location of the next instruction.

The stack is recycled ADDQ $56, SP, but only changes the location pointed to by SP. The data in memory will not be emptied, but will be emptied only the next time it is allocated for use.

The parameters and return value memory of callee are all allocated by caller.

When returnTwo ret, the stack position of the next instruction of call returnTwo will be popped up, that is, the instruction saved by the 0x0d00 address in the figure, so when the returnTwo function returns, SP points to the 0x0d08 address.

Since the above involves some knowledge of Plan9, I will introduce some of its grammar by the way. If you talk about grammar directly, it will be very boring. The following will be combined with some practical situations to introduce. Not only gain but also learn grammar.

Go assembler plan9

The compilation of our entire program will eventually be translated into machine code, and assembly can be regarded as the text form of machine code, and they can correspond to each other one by one. So if we can understand the compilation a little bit, we can analyze a lot of practical problems.

The developers of go language are the most TOP programmers in the world. They choose to keep pretending. They don't use standard AT&T or Intel assembler. They want to develop their own set. No way, who makes others awesome? The compilation of Golang is based on the compilation of Plan9, and I think it is too complicated to fully understand it, because it involves a lot of low-level knowledge. But you can do it if you just want to understand it. Let's give some examples to try.

PS: it's not necessary to learn this thing completely. The input-output ratio is too low, as long as an application engineer can understand it.

Before we officially begin, we would like to add some necessary information, some of which have already been mentioned above, which will be introduced here as a whole for the sake of completeness.

Several important pseudo registers

SB: is a virtual register that holds the static base address (static-base) pointer, the start address of our program address space

NOSPLIT: a leading instruction to indicate to the compiler that stack-split should not be inserted to check for stack expansion

FP: refers to the input parameters of a function in a way such as symbol+offset (FP)

The SP register of SP:plan9 points to the start position of the local variable of the current stack frame, using a way such as symbol+offset (SP) to refer to the local variable of the function. Note: this register is different from the register above, here are pseudo registers, and what we show are hardware registers.

There are other operating instructions, most of which can be seen according to the name, so we will no longer introduce them and start to do it directly.

View the translation function corresponding to the go application code

Package main func main () {} func test () [] string {a: = make ([] string, 10) return a}-".test STEXT size=151 args=0x18 locals=0x40 0x0000 00000 (test1.go:6) TEXT" .test (SB), ABIInternal, $64-24 / / stack frame size And parameter, return value size 0x0000 00000 (test1.go:6) MOVQ (TLS), CX 0x0009 00009 (test1.go:6) CMPQ SP, 16 (CX) 0x000d 00013 (test1.go:6) JLS 1410x000f 00015 (test1.go:6) SUBQ $64, SP 0x0013 00019 (test1.go:6) MOVQ BP 56 (SP) 0x0018 00024 (test1.go:6) LEAQ 56 (SP), BP... 0x001d 00029 (test1.go:6) MOVQ $0, ". ~ r0x72 (SP) 0x0026 00038 (test1.go:6) XORPS X0, X0 0x0029 00041 (test1.go:6) MOVUPS X0 ".". ~ r0380 (SP) 0x002e 00046 (test1.go:7) PCDATA $2, $1 0x002e 00046 (test1.go:7) LEAQ type.string (SB), AX 0x0035 00053 (test1.go:7) PCDATA $2, $0 0x0035 00053 (test1.go:7) MOVQ AX, (SP) 0x0039 00057 (test1.go:7) MOVQ $10 8 (SP) 0x0042 00066 (test1.go:7) MOVQ $10,16 (SP) 0x004b 00075 (test1.go:7) CALL runtime.makeslice (SB) / / corresponding underlying runtime function. 0x008c 00140 (test1.go:8) RET 0x008d 00141 (test1.go:8) NOP 0x008d 00141 (test1.go:6) PCDATA $0 $- 1 0x008d 00141 (test1.go:6) PCDATA $2, $- 1 0x008d 00141 (test1.go:6) CALL runtime.morestack_noctxt (SB) 0x0092 00146 (test1.go:6) JMP 0

According to the corresponding number of lines of code and the name, it is obvious that the make written in the application layer corresponds to the underlying makeslice.

Escape analysis

Let's start with the concept of escape analysis. Stack and heap allocation are involved here. If the variable is assigned to the stack, it will be automatically recycled along with the function call, and the allocation efficiency is very high; secondly, if the variable is assigned to the heap, GC is required to mark the collection. The so-called escape means that the variable escapes from the stack to the heap (many people are talking about escape analysis when they are not clear about this concept, and they have met several times in the interview?)

Package main func main () {} func test () * int {t: = 3 return & t}-".test STEXT size=98 args=0x8 locals=0x20 0x0000 00000 (test1.go:6) TEXT" .test (SB), ABIInternal, $32-8 0x0000 00000 (test1.go:6) MOVQ (TLS), CX 0x0009 00009 (test1.go:6) CMPQ SP 16 (CX) 0x000d 00013 (test1.go:6) JLS 91 0x000f 00015 (test1.go:6) SUBQ $32, SP 0x0013 00019 (test1.go:6) MOVQ BP, 24 (SP) 0x0018 00024 (test1.go:6) LEAQ 24 (SP), BP. 0x001d 00029 (test1.go:6) MOVQ $0 ". ~ r0,40 (SP) 0x0026 00038 (test1.go:7) PCDATA $2, $1 0x0026 00038 (test1.go:7) LEAQ type.int (SB), AX 0x002d 00045 (test1.go:7) PCDATA $2, $0 0x002d 00045 (test1.go:7) MOVQ AX (SP) 0x0031 00049 (test1.go:7) CALL runtime.newobject (SB) / / allocate space on the heap It means to run away.

If you are using assembly for escape analysis of slice here, it will not be very intuitive. Because you will only see that the runtime.makeslice function is called, and the runtime.mallocgc function is actually called inside the function, the memory allocated by this function is actually the memory on the heap (if there is enough storage on the stack, you will not see the call to the runtime.makslice function).

The actual go also provides a more convenient command for escape analysis: go build-gcflags= "- m". If you are really doing escape analysis, it is recommended to use this command instead of assembler.

Pass value or pointer

There is no need to say much about the basic types in golang: strings, integers, and Boolean types. It must be value passing, so is it value passing or pointer passing for structures and pointers?

Package main type Student struct {name string age int} func main () {jack: = & Student {"jack", 30} test (jack)} func test (s * Student) * Student {return s}-".test STEXT nosplit size=20 args=0x10 locals=0x0 0x0000 00000 (test1.go:14) TEXT" .test (SB), NOSPLIT | ABIInternal $0-16... 0x0000 00000 (test1.go:14) MOVQ $0, ". ~ r1q16 (SP) / / the initial return value is 0 0x0009 00009 (test1.go:15) PCDATA $2, $1 0x0009 00009 (test1.go:15) PCDATA $0, $1 0x0009 00009 (test1.go:15) MOVQ" .s + 8 (SP) AX / / copy the reference address to the AX register 0x000e 00014 (test1.go:15) PCDATA $2, $0 0x000e 00014 (test1.go:15) PCDATA $0, $2 0x000e 00014 (test1.go:15) MOVQ AX, ". ~ r1q16 (SP) / / copy the reference address of AX to the return address 0x0013 00019 (test1.go:15) RET

You can see here that in go, only values are passed, because it is still underneath by copying the corresponding values.

The above is all the contents of the article "sample Analysis running in Golang and Plan9 Assembly". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.