Shulou(Shulou.com)06/01 Report--

Today, the editor will share with you the relevant knowledge points about how to solve golang's block and race. The content is detailed and the logic is clear. I believe most people still know too much about this, so share this article for your reference. I hope you can get something after reading this article. Let's take a look at it.

There are two kinds of concurrent bug, deadlock (block) and competition (race).

When a deadlock occurs, go run will report an error directly.

When race occurs, you need to add race to run the Times warning.

Add the-race parameter after go run xxx.go

$go run-race race.go

=

WARNING: DATA RACE

Write at 0x00c0000a2000 by goroutine 6:

Main.main.func2 ()

/ Users/harryhare/git/go_playground/src/race.go:15 + 0x38

Previous write at 0x00c0000a2000 by goroutine 5:

Main.main.func1 ()

/ Users/harryhare/git/go_playground/src/race.go:9 + 0x38

Goroutine 6 (running) created at:

Main.main ()

/ Users/harryhare/git/go_playground/src/race.go:13 + 0x9c

Goroutine 5 (running) created at:

Main.main ()

/ Users/harryhare/git/go_playground/src/race.go:7 + 0x7a

Package main

Import "time"

Func main () {

Var x int

Go func () {

For {

Xero1

}

} ()

Go func () {

For {

Xerox 2

}

} ()

Time.Sleep (100*time.Second)

}

This command outputs Warning, telling us that contention is triggered when goroutine5 runs to line 11 and main goroutine runs to line 13.

And goroutine5 is generated at line 12.

Formation condition

In general, it is due to the competition condition for the same variable operation when multiple coprograms operate without locking.

Solution method

Method 1: use mutex sync.Mutex

Method 2: use pipes

The efficiency of using pipeline is higher than that of mutex, and it also accords with the design idea of Go language.

Before writing if to detect race, first understand the first question, what is race?

When multiple goroutine are performing conflicting read and write operations on the same variable at the same time, the result is uncertain, which is called race. For example, if goroutine1 is writing a when reading goroutine1 gorgeous 2, if he is not sure whether the result read by race is the value before or after writing by goroutine2, it will be called race.

Var x int

Go func () {

V: = x

} ()

X = 5

Is the value of v above 0 or 5? I don't know, there is a race in this code. This is a more oral description, a rigorous formal description, it is necessary to talk about the memory model of Go.

Go's memory model describes the condition that "reading a variable in one groutine can detect write operations on that variable in other goroutine".

Suppose An and B represent two operations performed by a multithreaded program. If A happens-before B, the memory impact of the An operation will be visible to the thread executing B (and before executing B).

With the formal description of happens before, whether there is race is equivalent to whether there are conflicting operations that cannot determine happens before for the same block of memory access. That is to say:

For the previous code, the two operations v: = x and x = 5 access the same block of memory x, and there is no guarantee that v: = x is happens before x = 5, so this code has race.

Then the problem of "implementing race dectect" translates into "the problem of happens before event detection".

How do I detect happens before events?

We can write "which thread id, at what time, which memory block, read or write", as long as we record all memory access events, and then traverse to verify the sequence of these operations. Once it is found that, for example, reading and writing two operation records are not enough to read happens before and write, race is detected.

But it seems a bit intimidating to record all memory access operations. In fact, it is only to record the variables that may be accessed concurrently, not all variables. The g below is a local variable, so there is no need to record it.

Func f () {

G: = 3

}

But the price still seems to be high? Indeed. Well, I'm not sure whether it will be 10 times slower or 100 times slower, but the online code won't run on race anyway. Now that Go has done it, it must be possible to do it.

There need to be two parts, in Go-race compilation option will do the corresponding processing. The compilation part needs to insert instructions to record events where memory access is involved; the runtime detects the happens before between events.

A memory access event can be recorded in 8 bytes: 16-bit thread id,42-bit timestamp, 5-bit memory location, and 1-bit mark whether to read or write.

Thread id does not need to be interpreted, nor do read and write tokens. The timestamp is a logical clock, not the real time each time.

How do you record the memory location with only 5 bits? Here are some tricks, and the same technique is used in Go's memory management. For the actual use of a memory area, map another "shadow" memory area, the mapping is the real "shadow".

For example, there is an array A [1000] whose "shadow" is B [1000]. What happened in A [I] can only be recorded in B [I]. Note that the two do not need to be the same size, such as

Int A [1000]; / / arrays that are actually used

Char B [1000]; / / is used to record the operations that occur in the An array. If only 1 bit is read / written, 8 bits may not be used.

Similarly, for the actual memory area used is [0x7fffffffffff 0x7f0000000000], its "shadow" area can be [0x1fffffffffff 0x180000000000], 5 bits can represent 64 units, if the actual memory usage is aligned by 8 bytes, it is sufficient to represent a group.

It seems a little confusing, so let's put it this way: three bits can represent the status of eight units, right? The third power of 2 equals 8.

A [8 8-byte units] = > B [3 bits]

Whether there is a read or write operation in An is recorded with the 0 or 1 of the bit in B. It shows that a large number of events can be recorded with only a small amount of memory!

Going back to the record format of the event, a record occupies 8 bytes, of which 5 bits record the memory location. 5 bits can record 64 8 bytes, that is, the space overhead of race dectect is 1x8 of the memory used (it is not, because a set of events in the same memory is recorded).

Looking at an example, we recorded the first event, thread T1, at the E1 timestamp, accessing the memory area [02], and performing a write operation:

(T1, E1, 0RO, 2Pol W)

The second event, thread T2, in E2 timestamp, read memory area [4 8]:

(T2, E2, 4, 14, 8, R)

Because there is no intersection of positions, there is no conflict.

The third event, thread T3, in E3 timestamp, read memory area [0 4]:

(T3memeE3pyrrine 0GRO 4jrr)

This area intersects the area of the first event, so assuming that E1 does not meet happens before E3, then a conflict is detected.

Type hchan struct {

Qcount uint / / total data in the queue the number of data in the current queue

The size of the dataqsiz uint / / size of the circular queue channel ring queue

Buf unsafe.Pointer / / pointer to the circular queue where the data is stored in points to an array of dataqsiz elements

Size of the data type stored in elemsize uint16 / / channel | that is, the size of each element

Indication of whether closed uint32 / / channel is closed or not

The type of element stored in elemtype * _ type / / element type channel

Sendx uint / / send index the subscript pointer of the current sending element to the channel ring queue

Recvx uint / / receive index currently receives a subscript pointer to the channel ring queue

Recvq waitq / / list of recv waiters waits for the goroutine queue to receive elements

Sendq waitq / / list of send waiters waiting for the goroutine queue of elements to be sent

/ / lock protects all fields in hchan, as well as several

/ / fields in sudogs blocked on this channel.

/ /

/ / Do not change another G's status while holding this lock

/ / (in particular, do not ready a G), as this can deadlock

/ / with stack shrinking.

/ / do not change the state of another G while maintaining this lock (in particular, not ready G), as this may lead to a deadlock due to stack shrinkage.

Lock mutex

}

A brief description:

Buf is a structure unique to buffered channel that is used to store cached data. It's a circular linked list.

Sendx and recvx are used to record the ~ index sent or received in the circular linked list of buf

Lock is a mutex.

Recvq and sendq are receivers (

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.