Shulou(Shulou.com)06/01 Report--

Linux system sequential writing, random writing example analysis, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain for you in detail, people with this need can come to learn, I hope you can gain something.

I. Preface

● random writing will lead to continuous channel change of magnetic heads, resulting in a great reduction in efficiency; sequential write heads rarely need to change channels, or the time to change channels is very short ● this article discusses the specific differences between the two and the corresponding kernel calls

II. Environmental preparation

Component version OSUbuntu 16.04.4 LTSfio2.2.10 III. Introduction to fio

Through the fio test, the status can be reflected in reading and writing, we need to focus on several key indicators in the fio output report: slat: refers to the time from the submission to the actual execution of the fio (Submission latency) clat: refers to the time from the submission of the fio to the completion of the lat (Completion latency): refers to the total time from the creation of the fio to the completion of the bw: throughput iops: the number of times per second

Fourth, synchronous write test

(1) synchronous random writing

Fio is mainly used as the test tool, in order to see the system calls, use the strace tool, the command looks like this:

Let's test a random write first.

Strace-f-tt-o / tmp/randwrite.log-D fio-name=randwrite-rw=randwrite\-direct=1-bs=4k-size=1G-numjobs=1-group_reporting-filename=/tmp/test.db extract key information

Root@wilson-ubuntu:~# strace-f-tt-o / tmp/randwrite.log-D fio- name=randwrite-rw=randwrite\ >-direct=1-bs=4k-size=1G-numjobs=1-group_reporting-filename=/tmp/test.db randwrite: (Gambi0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1 fio-2.2.10 Starting 1 process. Randwrite: (groupid=0, jobs=1): err= 0: pid=26882: Wed Aug 14 10:39:02 2019 write: io=1024.0MB, bw=52526KB/s, iops=13131, runt= 19963msec clat (usec): min=42, max=18620, avg=56.15, stdev=164.79 lat (usec): min=42, max=18620, avg=56.39, stdev=164.79. Bw (KB / s): min=50648, max=55208, per=99.96%, avg=52506.03, stdev=1055.83... Run status group 0 (all jobs): WRITE: io=1024.0MB, aggrb=52525KB/s, minb=52525KB/s, maxb=52525KB/s, mint=19963msec, maxt=19963msec Disk stats (read/write):... Sda: ios=0/262177, merge=0/25, ticks=0/7500, in_queue=7476, util=36.05% list the information we need to focus on:

(1) clat, with an average duration of about 56ms

(2) lat, the average duration is about 56ms

(3) bw, throughput, about 52m

Let's take a look at the kernel call information:

Root@wilson-ubuntu:~# more / tmp/randwrite.log... 26882 10 lseek 38 lseek (3, 665198592, SEEK_SET) = 665198592 26882 10 38 write 41.919920 write (3, "\ 220\ 240 @\ 6\ 371\ 341\ 277 > 0\ 200\ 36\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 343\ 340\ 370\ 370\ 330\ 30e\ 30"... 4096) = 4096 26882 1038 38 38 38 1491 19969 lseek (3,4313088) SEEK_SET) = 4313088 26882 10 lseek 38 lseek 41.919985 lseek (3, "\ 220\ 240@\ 6\ 371\ 341\ 277) = 455880704 26p\ 340\ 370\ 330\ 30e\ 30"... (3,455880704, SEEK_SET) = 455880704 26882 1038 write "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0200\ 36\ 31\ 0\ 0\ 0\ 0202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 4096 26882 103838 lseek 41.920096 write (3, 338862080, SEEK_SET) = 338862080 26882 10 3841.920112 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 4096 26882 10VRV 3841.920161 lseek (3, 739086336, SEEK_SET) = 739086336 26882 10RV 3838 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 4096 26882 10write 3841.920229 lseek (3, 848175104, SEEK_SET) = 848175104 26882 10 write 3841.920245 "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 4096 26882 10147200 26882 103841.920312 write (3, 1060147200, SEEK_SET) = 1060147200 26882 1038312 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 4096 26882 10VRV 3841.920362 lseek (3, 863690752, SEEK_SET) = 863690752 26882 10RV 3841.920377 write "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 4096 26882 10shuffle 3841.920428 lseek (3, 279457792, SEEK_SET) = 279457792 26882 10 3841.920444 write (3 "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 4096 26882 10shuffle 3841.920492 lseek (3, 271794176, SEEK_SET) = 271794176 26882 10 3841.920508 write (3 "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 4096 26882 10night 3841.920558 lseek (3, 1067864064, SEEK_SET) = 1067864064 26882 1038 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 4096. Lseek is used to locate the current file offset before each random read.

Synchronous sequential write

Use the previous method to test the sequential write

Root@wilson-ubuntu:~# strace-f-tt-o / tmp/write.log-D fio-name=write-rw=write\-direct=1-bs=4k-size=1G-numjobs=1-group_reporting-filename=/tmp/test.db write: (grub0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=sync Iodepth=1 fio-2.2.10 Starting 1 process Jobs: 1 (fission 1): [W (1)] [100.0% done] [0KB/70432KB/0KB / s] [0/17.7K/0 iops] [eta 00m:00s] write: (groupid=0, jobs=1): err= 0: Wed Aug 14 10:53:02 2019 write: io=1024.0MB, bw=70238KB/s, iops=17559, runt= 14929msec clat (usec): min=43, max=7464, avg=55.95, stdev=56.24 lat (usec): min=43, max=7465, avg=56.15 Stdev=56.25... Bw (KB / s): min=67304, max=72008, per=99.98%, avg=70225.38, stdev=1266.88... Run status group 0 (all jobs): WRITE: io=1024.0MB, aggrb=70237KB/s, minb=70237KB/s, maxb=70237KB/s, mint=14929msec, maxt=14929msec Disk stats (read/write):... Sda: ios=0/262162, merge=0/10, ticks=0/6948, in_queue=6932, util=46.49% can see:

The throughput has increased to about 70m.

Let's take a look at the kernel calls:

Root@wilson-ubuntu:~# more / tmp/write.log... 27046 10 write 54 write 28.194508 (3, "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 write 54194568 (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5422 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 54ver 28.194687 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5447 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 54ver 28.194807 write (3 "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5448 68 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5428 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5488 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5449 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 220\ 24\ 0\ 0\ 0\ 0\ 0\ 0\ 320\ 17\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 Dev 5415 28.195110 write (3 "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 220\ 24\ 0\ 0\ 0\ 0\ 0\ 0\ 320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 54ver 28.195197 write (3 "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5422 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5440 28.195330 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5424 28.195426 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 Dev 54ver 28.195497 write (3 "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 54ver 28.195567 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5437 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5440 28.195704 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5457 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 220\ 24\ 0\ 0\ 0\ 0\ 0\ 0\ 320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 54ver 28.195807 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 220\ 24\ 0\ 0\ 0\ 0\ 0\ 0\ 320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5459 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5415 28.195910 write (3 "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5451 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5412 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 Dev 5415 28.196062 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 220\ 24\ 0\ 0\ 0\ 0\ 0\ 0\ 320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 10 5412 write (3 "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 1054 28.196162 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 1054 28.196213 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 3 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 1054 28.196265 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 3 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 1054 28.196314 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 1054 28.196363 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 3 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 1054 28.196414 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 1054 fret 28.196472 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 3 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 1054 28.196524 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 4096 27046 105415 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0", 4096) = 4096. Because of sequential reading, there is no need to repeatedly locate the file offset, so you can focus on the write operation.

5. Slat indicators

From the above test, we did not find slat in the test report of fio. That is because the above operations are all synchronous operations. For synchronous Imax O, since the submission and completion of Imax O are the same action, slat is actually the time it takes for it to complete.

Write asynchronously, adding commands written in synchronous order to-ioengine=libaio:

Root@wilson-ubuntu:~# fio- name=write-rw=write-ioengine=libaio-direct=1-bs=4k-size=1G-numjobs=1-group_reporting-filename=/tmp/test.db write: (gumb0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1 fio-2.2.10 Starting 1 process Jobs: 1 (fission 1): [W (1)] [100.0% done] [0KB/119.3MB/0KB / s] [0/30.6K/0 iops] [eta 00m:00s] write: (groupid=0) Jobs=1): err= 0: pid=27258: Wed Aug 14 11:14:36 2019 write: io=1024.0MB, bw=120443KB/s, iops=30110, runt= 8706msec slat (usec): min=3, max=70, avg= 4.31, stdev= 1.56 clat (usec): min=0, max=8967, avg=28.13, stdev=55.68 lat (usec): min=22, max=8976, avg=32.53, stdev=55.72. Bw (KB / s): min=118480, max=122880, per=100.00%, avg=120467.29, stdev=1525.68... Run status group 0 (all jobs): WRITE: io=1024.0MB, aggrb=120442KB/s, minb=120442KB/s, maxb=120442KB/s, mint=8706msec, maxt=8706msec Disk stats (read/write):... Sda: ios=0/262147, merge=0/1, ticks=0/6576, in_queue=6568, util=74.32% can see that the slat index appears, and lat is approximately equal to the sum of slat + clat (avg average), and after switching to asynchronous io, the throughput has been greatly improved, about 120m.

VI. Summary

● fio should be used as a baseline tool for disks. When you get a machine (physical machine or cloud machine), you should do a baseline test on the disk of the machine as soon as possible. ● all the tests in this article bypass caching. In practical applications, the impact of caching needs to be taken into account.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.