Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "how to deal with the stack in Go language". Many people will encounter this dilemma in the operation of actual cases, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

1. Introduction of thread stack (thread stacks)

Before we look at the stack handling of Go, let's take a look at how traditional languages such as C do stack management.

When you start a C-implemented thread, the C standard library is responsible for allocating a piece of memory as the thread's stack. The standard library allocates this piece of memory, tells the kernel its location, and lets the kernel handle the execution of the thread. However, when this piece of memory is insufficient, the problem arises. Let's take a look at the following function:

Int a (int m, int n) {if (m = = 0) {return n + 1;} else if (m > 0 & & n = = 0) {return a (m-1,1);} else {return a (m-1, a (m, n-1));}}

This function uses a lot of recursion, and executing a (4,5) reduces all stack memory depletion. To solve this problem, you can adjust the size of the memory blocks allocated by the standard library to the thread stack. But increasing the stack size across the board means that each thread increases the memory usage of the stack, that is, they are not heavily recursive. In this way, you will use up all the memory, even if your program has not yet used the memory on the stack.

Another alternative is to determine the stack size separately for each thread. In this way, you have to complete the task of estimating the size of each thread's stack memory according to its needs. It will be more difficult to create threads than we expected. It is not feasible to figure out how much memory a thread stack requires in general, even if it is usually very difficult.

2. How does Go deal with this problem

Instead of allocating a fixed amount of stack space for each goroutine, the Go runtime tries to provide goroutine with the stack space they need on demand. This frees programmers from the annoyance of determining the size of stack space. But the Go core team is trying to switch to another solution, and here I'm going to try to explain the old solution and its shortcomings, the new one, and why the change was made.

3. Segment stack (Segmented Stacks)

Segmented stack (segmented stacks) is a scheme originally used by the Go language to deal with stacks. When creating a goroutine, the Go runtime allocates 8K bytes of memory for the goroutine stack to run, and we let goroutine complete its task processing on this stack.

When we used up the 8K bytes of stack space, the problem followed. To solve this problem, each go function has a small piece of code (called prologue) at the entrance to the function, which checks to see if the allocated stack space is used up, and if so, it calls the morestack function.

The morestack function allocates a new piece of memory to use as stack space, and then writes all kinds of data information about the stack to a struct at the bottom of the stack, including the address of the previous stack. Somewhat we have a new stack segment (stack segment), and we will restart goroutine, starting with the function that causes the stack space to run out (Foobar in the following figure). This is called "stack stack split".

The following stack intention is exactly what happens after we split the stack:

At the bottom of the new stack, we insert a stack entry function lessstack. We will not call this function, but set this function to be used when we return from the function that caused us to use the Foobar space. When that function returns, we go back to lessstack (the stack frame), and lessstack looks for the struct at the bottom of stack and adjusts the stack pointer (stack pointer) so that we return to the previous stack space. After doing so, we can release this new stack segment (stack segment) and continue to execute our program.

The problem of segmented stack (Segmented stacks)

The segmented stack gives us the ability to scale on demand. Programmers don't have to worry about the size of the stack, it's cheap to start a new goroutine and programmers don't know how big the stack will grow.

This is the way the Go language has handled stack growth until now, but there is a flaw in this approach. That is, stack reduction can be a relatively expensive operation. If you encounter stack stack split in a loop, you will feel it most. A function increases stack space, splits the stack, returns and frees the stack segment (stack segment). If you do this in a loop, you will pay a high price (performance).

This is the so-called "hot split" problem. It is also the main reason why the Go core development team has been changed to a new stack management scheme-stack copy (stack copying).

Stack copy (stack copying)

The initial stage of stack copy is similar to a segmented stack. Goroutine is running on the stack, and when using the stack space, it encounters the same stack overflow check as in the old scheme. But unlike the old scheme, which retains a link that returns the previous stack, the new scheme creates a new stack that is twice the size of the original stack and copies the old stack to it. This means that when the actual space used by the stack shrinks to its original size, the go runtime does not have to do anything. Stack reduction is a cost-free operation. In addition, when the stack grows again, the runtime does not need to do anything, we just need to reuse the previously allocated free space.

6. How is the stack copied

Copying a stack sounds simple, but in fact it is a difficult thing. Because the variables on the stack in Go have their own addresses, once you have a pointer to the variables on the stack, you won't be able to do what you want. When you move the stack, the pointer to the original stack becomes invalid.

Fortunately, only pointers assigned on the stack can point to the address on the stack. This is extremely necessary for memory security, otherwise the program may access the address on the stack that is no longer in use.

Because we need to know the location of pointers that need to be reclaimed by the garbage collector, we know which parts of the stack are pointers. When we move the stack, we can update the pointer in the stack to point to the new destination address, and all related pointers should be taken care of.

Because we use garbage collection information to help complete the stack copy, all functions that appear on the stack must have this information. But this is not always the case. Because most of the Go runtime code is written in C, a large number of runtime calls do not have pointer information available, so they cannot be copied. Once this happens, we have to go back to the segmented stack scheme and accept the high price paid for it.

This is why the current Go runtime developers rewrite Go runtime on a large scale. Code that cannot be rewritten with Go, such as the kernel of the scheduler and garbage collector, will be executed on a special stack whose size is calculated and determined by the runtime developer alone.

In addition to making stack copying possible, this approach will also enable us to implement features such as concurrent garbage collection in the future.

VII. About virtual memory

Another different way of stack processing is to allocate large memory segments in virtual memory. Since physical memory is allocated only when it is actually used, it looks as if you can allocate a large memory segment and let the operating system process it. Here are some problems with this approach

First of all, 32-bit systems can only support 4G bytes of virtual memory, and applications can only use 3G space. Since it's not uncommon to run millions of goroutines at the same time, you can probably run out of virtual memory, even if we assume that the stack of each goroutine is only 8K.

Second, however, we can allocate a large amount of memory in a 64-bit system, which depends on excessive memory usage. Overuse means that when you allocate more memory than physical memory, you rely on the operating system to ensure that physical memory can be allocated when needed. However, allowing overuse may lead to some risks. Because some processes allocate more memory than the physical memory of the machine, the operating system will have to replenish the memory for these processes if they use more memory. This causes the operating system to put some segments of memory into the disk cache, which often increases unpredictable processing delays. It is for this reason that some new systems have turned off support for overuse.

This is the end of the content of "how to deal with the stack in Go language". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.