Shulou(Shulou.com)06/02 Report--

I. Preface

● random writing will lead to continuous channel change of magnetic heads, resulting in a great reduction in efficiency; sequential write heads rarely need to change channels, or the time of changing channels is very short.

● this article discusses the specific differences between the two and the corresponding kernel calls

II. Environmental preparation

Component version OSUbuntu 16.04.4 LTSfio2.2.10

III. Introduction to fio

Through the fio test, which can reflect the status in read and write, we need to focus on several key indicators in the output report of fio:

Slat: refers to the length of time (Submission latency) from the submission of Iamp O to the actual execution of it.

Clat: refers to the length of time (Completion latency) that was submitted from iCompo to IWeiO.

Lat: refers to the total time from the creation of the fio to the completion of the fio

Bw: throughput

Iops: the number of iUnites per second

Fourth, synchronous write test

(1) synchronous random writing

Fio is mainly used as the test tool, in order to see the system calls, use the strace tool, the command looks like this:

Let's test a random write first.

Strace-f-tt-o / tmp/randwrite.log-D fio-name=randwrite-rw=randwrite\-direct=1-bs=4k-size=1G-numjobs=1-group_reporting-filename=/tmp/test.db

Extract key information

Root@wilson-ubuntu:~# strace-f-tt-o / tmp/randwrite.log-D fio- name=randwrite-rw=randwrite\ >-direct=1-bs=4k-size=1G-numjobs=1-group_reporting-filename=/tmp/test.dbrandwrite: (grub0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1fio-2.2.10Starting 1 process...randwrite: (groupid=0, jobs=1): err= 0: pid=26882: Wed Aug 14 10:39:02 2019 write: io=1024.0MB, bw=52526KB/s, iops=13131, runt= 19963msec clat (usec): min=42, max=18620 Avg=56.15, stdev=164.79 lat (usec): min=42, max=18620, avg=56.39, stdev=164.79... Bw (KB/s): min=50648, max=55208, per=99.96%, avg=52506.03, stdev=1055.83...Run status group 0 (all jobs): WRITE: io=1024.0MB, aggrb=52525KB/s, minb=52525KB/s, maxb=52525KB/s, mint=19963msec, maxt=19963msecDisk stats (read/write):. Sda: ios=0/262177, merge=0/25, ticks=0/7500, in_queue=7476, util=36.05%

Lists the information we need to focus on:

(1) clat, with an average duration of about 56ms

(2) lat, the average duration is about 56ms

(3) bw, throughput, about 52m

Let's take a look at the kernel call information:

Root@wilson-ubuntu:~# more / tmp/randwrite.log...26882 10 lseek 38 lseek (3, 665198592, SEEK_SET) = 66519859226882 10 lseek 41.919920 write (3, "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 200\ 31\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26P\ 340\ 370\ 330\ 30e\ 30"... 4096) = 409626882 10 38 Freight 41.919969 lseek (3,4313088) SEEK_SET) = 431308826882 10lseek 38 lseek 41.919985 write (3, "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 200\ 36\ 0\ 0\ 0\ 0202\ 7\ 320\ 343\ 6H\ 340\ 277\ 370\ 330\ 30e\ 30"..., 4096) = 409626882 10 Switzerland 38 41.920032 write (3,455880704, SEEK_SET) = 45588070426882 10140 38 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0200\ 36\ 31\ 0\ 0\ 0\ 0202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 409626882 10VRV 3841.920096 lseek (3, 338862080, SEEK_SET) = 33886208026882 1038 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 409626882 10VRV 38RV 41.920161 lseek (3, 739086336, SEEK_SET) = 73908633626882 10RV 3841.920177 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 409626882 1038 lseek 41.920229 write (3, 848175104, SEEK_SET) = 84817510426882 1038 write 41.920245 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 409626882 10Rich 38RH 41.920296 lseek (3, 1060147200, SEEK_SET) = 106014720026882 10 3841.920312 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 409626882 10VRV 38RV 41.920362 lseek (3, 863690752, SEEK_SET) = 86369075226882 10140 3841.920377 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 409626882 10RV 38RV 41.920428 lseek (3, 279457792, SEEK_SET) = 27945779226882 10RN 3841.920444 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 409626882 1038 lseek 41.920492 write (3, 271794176, SEEK_SET) = 27179417626882 10 3841.920508 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 409626882 10VRV 38RV 41.920558 lseek (3, 1067864064, SEEK_SET) = 106786406426882 10 38R41.920573 write (3) "\ 220\ 240@\ 6\ 371\ 341\ 277 >\ 0\ 2402\ 24\ 0\ 0\ 0\ 0\ 202\ 2\ 7\ 320\ 343\ 6H\ 26p\ 340\ 277\ 370\ 330\ 30e\ 30", 4096) = 4096.

Lseek is used to locate the current file offset before each random read.

Synchronous sequential write

Use the previous method to test the sequential write

Root@wilson-ubuntu:~# strace-f-tt-o / tmp/write.log-D fio-name=write-rw=write\-direct=1-bs=4k-size=1G-numjobs=1-group_reporting-filename=/tmp/test.dbwrite: (grub0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=sync Iodepth=1fio-2.2.10Starting 1 processJobs: 1 (fission 1): [W (1)] [100.0% done] [0KB/70432KB/0KB / s] [0/17.7K/0 iops] [eta 00m:00s] write: (groupid=0, jobs=1): err= 0: Wed Aug 14 10:53:02 2019 write: io=1024.0MB, bw=70238KB/s, iops=17559, runt= 14929msec clat (usec): min=43, max=7464, avg=55.95, stdev=56.24 lat (usec): min=43, max=7465, avg=56.15, stdev=56.25... Bw (KB/s): min=67304, max=72008, per=99.98%, avg=70225.38, stdev=1266.88...Run status group 0 (all jobs): WRITE: io=1024.0MB, aggrb=70237KB/s, minb=70237KB/s, maxb=70237KB/s, mint=14929msec, maxt=14929msecDisk stats (read/write):. Sda: ios=0/262162, merge=0/10, ticks=0/6948, in_queue=6932, util=46.49%

You can see:

The throughput has increased to about 70m.

Let's take a look at the kernel calls:

Root@wilson-ubuntu:~# more / tmp/write.log...27046 10 write 54 write 28.194508 write (3, "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0360\ t\ 0\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0..., 4096) = 409627046 10write 5428 194568 (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.194627 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.194687 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.194747 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.194807 write (3 "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.194868 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.194928 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.194988 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195049 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195110 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195197 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195262 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195330 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195426 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195497 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195567 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0\ 0\ 320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054v 28.195637 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195704 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195757 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195807 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195859 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195910 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.195961 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.196012 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.196062 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 0220\ 24\ 0\ 0\ 0\ 0\ 0320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.196112 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.196162 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054 28.196213 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054 28.196265 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.196314 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.196363 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 20\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.196414 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054 virtual 28.196472 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0 320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054virtual 28.196524 write (3) "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0\ 0\ 0\ 320\ 17\ 0\ 0\ 0\ 0\ 0", 4096) = 409627046 1054 write 28.196573 "\ 0\ 0\ 23\ 0\ 0\ 0\ 0\ 0\ 0\ 300\ 16\ 0\ 0\ 0\ 0\ 0\ 26\ 0\ 0\ 0\ 0\ 0", 4096) = 4096.

Because of sequential reading, there is no need to repeatedly locate the file offset, so you can focus on the write operation.

5. Slat indicators

From the above test, we did not find slat in the test report of fio. That is because the above operations are all synchronous operations. For synchronous Imax O, since the submission and completion of Imax O are the same action, slat is actually the time it takes for it to complete.

Write asynchronously, adding commands written in synchronous order to-ioengine=libaio:

Root@wilson-ubuntu:~# fio- name=write-rw=write-ioengine=libaio-direct=1-bs=4k-size=1G-numjobs=1-group_reporting-filename=/tmp/test.dbwrite: (gumb0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1fio-2.2.10Starting 1 processJobs: 1 (fission 1): [W (1)] [100.0% done] [0KB/119.3MB/0KB / s] [0/30.6K/0 iops] [eta 00m:00s] write: (groupid=0) Jobs=1): err= 0: pid=27258: Wed Aug 14 11:14:36 2019 write: io=1024.0MB, bw=120443KB/s, iops=30110, runt= 8706msec slat (usec): min=3, max=70, avg= 4.31, stdev= 1.56 clat (usec): min=0, max=8967, avg=28.13, stdev=55.68 lat (usec): min=22, max=8976, avg=32.53, stdev=55.72... Bw (KB/s): min=118480, max=122880, per=100.00%, avg=120467.29, stdev=1525.68...Run status group 0 (all jobs): WRITE: io=1024.0MB, aggrb=120442KB/s, minb=120442KB/s, maxb=120442KB/s, mint=8706msec, maxt=8706msecDisk stats (read/write):. Sda: ios=0/262147, merge=0/1, ticks=0/6576, in_queue=6568, util=74.32%

It can be seen that the slat index appears, and lat is approximately equal to the sum of slat + clat (avg average), and after being replaced by asynchronous io, the throughput has been greatly improved, about 120m.

VI. Summary

● fio should be used as a baseline tool for disks. When you get a machine (physical machine or cloud machine), you should do a baseline test on the disk of the machine as soon as possible, so that you have a good idea.

● all the tests in this article bypass the cache, and the impact of the cache needs to be taken into account in practical application.

The above is the linux disk written sequentially and randomly introduced by the editor. I hope it will be helpful to you. If you have any questions, please leave me a message and the editor will reply to you in time. Thank you very much for your support to the website!

If you think this article is helpful to you, you are welcome to reprint it, please indicate the source, thank you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.