Shulou(Shulou.com)05/31 Report--

In this issue, the editor will bring you about how to achieve virtual memory in Linux. The article is rich in content and analyzes and narrates it from a professional point of view. I hope you can get something after reading this article.

How to realize Linux Virtual memory

The first example: the following program will print out the program's pid (process number) and hang.

# include

# include

# include

Intmain () {

Printf ("run`pmap% d`\ n", getpid ())

Pause ()

}

Save the above code to a file mem_munch.c and run the following program to compile and execute:

$gccmem_munch.c-omem_munch

$. / mem_munch

Run`pmap25681`

The above process number is 25681, maybe the result of your experiment will be different.

Let's take a look at the memory usage of this Mini Program through the pmap command.

$pmap25681

25681:./mem_munch

00000000004000004Kr-x--/home/user/mem_munch

00000000006000004Kr----/home/user/mem_munch

00000000006010004Krw---/home/user/mem_munch

00007fcf5af880001576Kr-x--/lib/x86_64-linux-gnu/libc-2.13.so

00007fcf5b1120002044K-/lib/x86_64-linux-gnu/libc-2.13.so

00007fcf5b31100016Kr----/lib/x86_64-linux-gnu/libc-2.13.so

00007fcf5b3150004Krw---/lib/x86_64-linux-gnu/libc-2.13.so

00007fcf5b31600024Krwmuri-[anon]

00007fcf5b31c000132Kr-x--/lib/x86_64-linux-gnu/ld-2.13.so

00007fcf5b51200012Krwmuri-[anon]

00007fcf5b53900012Krwmuri-[anon]

00007fcf5b53c0004Kr----/lib/x86_64-linux-gnu/ld-2.13.so

00007fcf5b53d0008Krw---/lib/x86_64-linux-gnu/ld-2.13.so

00007fff7efd8000132Krwmurmuri-[stack]

00007fff7efff0004Krmuri-[anon]

Ffffffffff6000004Kr-x-- [anon]

Total3984K

The above result is the memory usage of the program, or rather the memory usage that the program thinks it uses. From the above results, we can see that when you access the libc library, it is actually access to the memory address 00007fcf5af88000, and when you access the ld library, it is actually access to the memory address 00007fcf5b31c000.

The above output may be quite abstract, so let's modify the above program so that we put a piece of data on the stack and one piece of data on the stack.

# include

# include

# include

# include

Intmain () {

Inton_stack,*on_heap

/ / Local variables are placed on the stack, so the address of on_stack is the initial address of the stack.

On_stack=42

Printf ("stackaddress:%p\ n", & on_stack)

/ / malloc's memory is allocated on the heap

On_heap= (int*) malloc (sizeof (int))

Printf ("heapaddress:%p\ n", on_heap)

Printf ("run`pmap% d`\ n", getpid ())

Pause ()

}

Compile and run:

$. / mem_munch

Stackaddress:0x7fff497670bc

Heapaddress:0x1b84010

Run`pmap11972`

Then check the memory usage with the pmap command:

$pmap11972

11972:./mem_munch

00000000004000004Kr-x--/home/user/mem_munch

00000000006000004Kr----/home/user/mem_munch

00000000006010004Krw---/home/user/mem_munch

0000000001b84000132KrwMurray-[anon]

00007f3ec4d980001576Kr-x--/lib/x86_64-linux-gnu/libc-2.13.so

00007f3ec4f220002044K-/lib/x86_64-linux-gnu/libc-2.13.so

00007f3ec512100016Kr----/lib/x86_64-linux-gnu/libc-2.13.so

00007f3ec51250004Krw---/lib/x86_64-linux-gnu/libc-2.13.so

00007f3ec512600024Krwmuri-[anon]

00007f3ec512c000132Kr-x--/lib/x86_64-linux-gnu/ld-2.13.so

00007f3ec532200012KrwMurray-[anon]

00007f3ec534900012Krwmuri-[anon]

00007f3ec534c0004Kr----/lib/x86_64-linux-gnu/ld-2.13.so

00007f3ec534d0008Krw---/lib/x86_64-linux-gnu/ld-2.13.so

00007fff49747000132KrwMurray-[stack]

00007fff497bb0004Krmurxmuri-[anon]

Ffffffffff6000004Kr-x-- [anon]

Total4116K

This time there is an extra red line above, and the red content is the starting position of the heap:

0000000001b84000132KrwMurray-[anon]

There is also a line of red output in the output of our program, which is the memory address of this address in the program:

Heapaddress:0x1b84010

The two addresses are basically the same, where anon is an abbreviation for Anonymous, indicating that there is no file mapping for this piece of memory.

Let's look at the two green lines above, corresponding to the above, which are the stack start address seen by the pmap and the application, respectively:

00007fff49747000132KrwMurray-[stack]

Stackaddress:0x7fff497670bc

The memory usage mentioned above is only what the program thinks it is using. In fact, the program does not know the state of the system memory when allocating memory. So the above output is just the memory usage seen from the program's own point of view. For example, in the above example, we see that the memory address space of the program ranges from 0 × 0000000000400000 to all addresses of 0xffffffffff600000 (while addresses between 0xffffffffff600000 and 0 × 00007fffffffffffffff are of special use, which are not discussed here). In this way, we can use a total of 10 million terabytes of memory.

But in fact, no hardware can have 10 million terabytes of physical memory at present. Why is the operating system designed in this way? There are many reasons, as you can see here, but also because of this, we can use memory space far beyond the size of physical memory.

Memory mapping

The principle of memory mapping is to let the operating system map a file to a section of memory, and then manipulate the file memory as if it were memory. For example, we create a file with completely random content, and then map it to a section of memory space by memory mapping. So if we take any bit in this piece of memory, we are equivalent to getting a random number. Let's do this experiment, first generate a file with random content using the following command.

$ddif=/dev/urandombs=1024count=1000000of=/home/user/random

1000000+0recordsin

1000000+0recordsout

1024000000bytes (1.0GB) copied,123.293s,8.3MB/s

$ls-lhrandom

-rw-r--r--1useruser977M2011-08-2916:46random

Then we use the following program to map the contents of this file to memory and extract random numbers from it

# include

# include

# include

# include

# include

Intmain () {

Char*random_bytes

FILE*f

Intoffset=0

/ / open "random" forreading

F=fopen ("/ home/user/random", "r")

If (! F) {

Perror ("couldn'topenfile")

Return-1

}

/ / wewanttoinspectmemorybeforemappingthefile

Printf ("run`pmap% d`, thenpress", getpid ())

Getchar ()

Random_bytes=mmap (0pc10000000mPROTIMACEREAD READ MAPTREAD SHAREDJING fileno (f), 0)

If (random_bytes==MAP_FAILED) {

Perror ("errormappingthefile")

Return-1

}

While (1) {

Printf ("randomnumber:%d (pressfornextnumber)", * (int*) (random_bytes+offset))

Getchar ()

Offset+=4

}

}

Then run the program:

$. / mem_munch

Run`pmap12727`, thenpress

How to realize Linux Virtual memory

Let's read the random number from this file by pressing the enter key again and again. After pressing the enter key several times, we can check its memory space through pmap:

$pmap12727

12727:./mem_munch

00000000004000004Kr-x--/home/user/mem_munch

00000000006000004Kr----/home/user/mem_munch

00000000006010004Krw---/home/user/mem_munch

000000000147d000132KrwMurray-[anon]

00007fe261c6f000976564Kr--s-/home/user/random

00007fe29d61c0001576Kr-x--/lib/x86_64-linux-gnu/libc-2.13.so

00007fe29d7a60002044K-/lib/x86_64-linux-gnu/libc-2.13.so

00007fe29d9a500016Kr----/lib/x86_64-linux-gnu/libc-2.13.so

00007fe29d9a90004Krw---/lib/x86_64-linux-gnu/libc-2.13.so

00007fe29d9aa00024Krwmuri-[anon]

00007fe29d9b0000132Kr-x--/lib/x86_64-linux-gnu/ld-2.13.so

00007fe29dba600012KrwMurray-[anon]

00007fe29dbcc00016KrwMurray-[anon]

00007fe29dbd00004Kr----/lib/x86_64-linux-gnu/ld-2.13.so

00007fe29dbd10008Krw---/lib/x86_64-linux-gnu/ld-2.13.so

00007ffff29b2000132KrwMurray-[stack]

00007ffff29de0004Krmuri xmuri-[anon]

Ffffffffff6000004Kr-x-- [anon]

Total980684K

The above output is more or less the same as the previous one, but with an extra red line above. This is our random file above mapped to memory in memory. When we use the pmap-x option to look at the program's memory usage, we get the following, where the RSS (residentsetsize) column represents the actual memory usage.

Pmap-x12727

12727:./mem_munch

AddressKbytesRSSDirtyModeMapping

0000000000400000040r-x--mem_munch

0000000000600000044r----mem_munch

0000000000601000044rw---mem_munch

000000000147d000044rwmuri-[anon]

00007fe261c6f000040r--s-random

00007fe29d61c00002880r-x--libc-2.13.so

00007fe29d7a6000000-libc-2.13.so

00007fe29d9a500001616r----libc-2.13.so

00007fe29d9a9000044rw---libc-2.13.so

00007fe29d9aa000016rwmuri-[anon]

00007fe29d9b000001080r-x--ld-2.13.so

00007fe29dba6000012rwMurray-[anon]

00007fe29dbcc000016rwMurray-[anon]

00007fe29dbd0000044r----ld-2.13.so

00007fe29dbd1000088rw---ld-2.13.so

00007ffff29b200001212rwMurray-[stack]

00007ffff29de000040rMurxmuri-[anon]

Ffffffffff600000000r-x-- [anon]

--

TotalkB980684508100

If your virtual memory footprint (the Kbytes column above) is 0, don't worry, this is a bug for pmap-x commands on Debian/Ubuntu systems. The total occupancy of the output on the last line is correct.

Now you can look at the RSS column, which is the actual memory footprint. On random files, your program can actually access gigabytes of memory before 00007fe261c6f000, but as long as you access an address that exceeds 4KB, the operating system will look for content on disk. This means that only 4KB's physical memory is actually used. Only when accessing this 4KB's stuff is the real memory operation. Other parts although you also use memory operation functions to access it, but because it is not loaded into memory, so when these contents are accessed, the operating system will first go to the disk to read the contents of random into memory.

If we modify the program again, change it to the following, and let the program access the entire random file.

# include

# include

# include

# include

# include

Intmain () {

Char*random_bytes

FILE*f

Intoffset=0

/ / open "random" forreading

F=fopen ("/ home/user/random", "r")

If (! F) {

Perror ("couldn'topenfile")

Return-1

}

Random_bytes=mmap (0pc10000000mPROTIMACEREAD READ MAPTREAD SHAREDJING fileno (f), 0)

If (random_bytes==MAP_FAILED) {

Printf ("errormappingthefile\ n")

Return-1

}

For (offset=0;offset

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.