Shulou(Shulou.com)06/01 Report--

Editor to share with you what FIO commands are commonly used in the Linux system. I hope you will get something after reading this article. Let's discuss it together.

FIO is a very good tool for testing IOPS. It is used for stress testing and verification of hardware. It supports 13 different Imax O engines. It can simulate various io operations through multiple threads or processes.

FIO tool common parameters: parameter description: filename=/dev/sdb1 test file name, usually select the data directory of the disk to be tested. Whether direct=1 uses directIO or not, the test process bypasses the buffer that comes with OS to make the results of the test disk more realistic. When Linux reads and writes, the kernel maintains the cache. The data is written to the cache first, and then to SSD in the background. When reading, priority is also given to reading the data in the cache. This speed can be accelerated, but once the power is off, the data in the cache will be gone. So there is a mode called DirectIO, which skips the cache and reads and writes SSD directly. Rw=randwrite Test Random write I rw=randrw Test Random write and read I bsrange=512 O bs=16k single io block file size is 16k bsrange=512-2048 ditto, set block size range size=5G the amount of data read and written per thread is 5GB. Numjobs=1 each job (task) opens 1 thread, which is used here, followed by a few thread tests with each task specified by-name. So the final number of threads = number of tasks (several name=jobx) * numjobs. Name=job1: the name of a task, it's okay to repeat it. If fio-name=job1-name=job2, two tasks are established, sharing the parameters before-name=job1. -name is followed by parameters unique to the job2 task. Thread uses pthread_create to create threads, and the other is fork to create processes. Processes are more expensive than threads, and thread testing is generally used. The runtime=1000 test time is 1000 seconds. If you don't write, you will continue to write 5g files in 4k each time. Ioengine=libaio specifies that the io engine uses libaio mode. Libaio:Linux local asynchronous IBO. Note that Linux may only support queuing behavior with unbuffered I direct=1 O (set to "direct=1" or "buffered=0"); rbd: the depth of direct access to the CEPH Rados iodepth=16 queue through librbd is 16. 5. In asynchronous mode, CPU cannot always issue commands to SSD indefinitely. For example, if stutters occur in SSD read and write, it is possible that the system will keep issuing thousands or even tens of thousands of commands, so on the one hand, SSD can not handle it, on the other hand, so many commands will take up a lot of memory, and the system will hang up. In this way, it brings a parameter called queue depth. Block Devices (RBD), no kernel RBD driver (rbd.ko) is required. This parameter contains a lot of ioengine, such as: libhdfs/rdma and other rwmixwrite=30 in mixed read-write mode, write accounts for 30% of the group_reporting about displaying the results, summarizing the information of each process. In addition, lockmem=1g uses only 1g of memory for testing. Zero_buffers initializes the system buffer with 0. Nrfiles=8 the number of files generated by each process. Test points commonly used for disk read and write: 1. Read=100% Ramdon=100% rw=randread (100% random read) 2. Read=100% Sequence=100% rw=read (100% sequential read) 3. Write=100% Sequence=100% rw=write (100% sequential write) 4. Write=100% Ramdon=100% rw=randwrite (100% random write) 5. Read=70% Sequence=100% rw=rw, rwmixread=70, rwmixwrite=30 (70% sequential read, 30% sequential write) 6. Read=70% Ramdon=100% rw=randrw, rwmixread=70, rwmixwrite=30 (70% random read, 30% random write) fio example:

[root@docker sda] # fio- ioengine=libaio-bs=4k-direct=1-thread-rw=read-filename=/dev/sda-name= "BS 4KB read test"-iodepth=16-runtime=60 BS 4KB read test: (iodepth=16 0): rw=read, bs= (R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 fio-3.7 Starting 1 thread Jobs: 1 (fission 1): [R (1)] [100.0%] Eta 00m:00s] [eta 00m:00s] BS 4KB read test: (groupid=0, jobs=1): err= 0: pid=18557: Thu Apr 11 13:08:11 2019 read: IOPS=22.7k, BW=88.5MiB/s (92.8MB/s) (5313MiB/60001msec) slat (nsec): min=901, max=168330, avg=6932.34, stdev=1348.82 clat (usec): min=90, max=63760, avg=698.08, stdev=240.83 lat (usec): min=97, max=63762, avg=705.17 Stdev=240.81 clat percentiles (usec): | 1.00th = [619,5.00th = [627,10.00th = [627,20.00th], | 30.00th= [635,40.00th= [685], 50.00th= [717,60.00th= [725], 70.00th= [725,80.00th= [725], 90.00th= [734], 95.00th= [816,99.00th= [1004]] | 99.50th = [1020], 99.90th= [1057], 99.95th= [1057], | 99.99th= [1860] bw (KiB/s): min=62144, max=91552, per=100.00%, avg=90669.02, stdev=3533.77, samples=120 iops: min=15536, max=22888, avg=22667.27, stdev=883.44, samples=120 lat (usec): 100mm 0.01%, 250mm 0.01%, 500mm 0.01%, 750mm 93.85%, 1000mm 5.14% lat (msec): 2mm 0.99%, 40.01% 10 percent 0.01%, 50 percent 0.01 percent, 100 percent 0.01 percent cpu: usr=5.35%, sys=23.17%, ctx=1359692, majf=0, minf=17 IO depths: 1 percent 0.1%, 2 percent 0.1 percent, 4 percent 0.1 percent, 8 percent 0.1 percent, 16 percent 100 percent, 32 percent 0. 0%, > = 64 percent 0. 0% submit: 0 percent, 4 percent 100 percent, 8 percent 0 percent, 16 percent 0 percent, 32 percent 0 percent, 64 percent 0. 0%, > = 64 percent 0. 0 percent complete: 0 percent. 440% 100%, 8% 0%, 16% 0.1%, 32% 0%, 64% 0%, > = 64% 0.0% issued rwts: total=1360097,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency: target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): READ: bw=88.5MiB/s (92.8MB/s), 88.5MiB/s-88.5MiB/s (92.8MB/s-92.8MB/s), io=5313MiB (5571MB) Run=60001-60001msec Disk stats (read/write): sda: how many M's of IO are executed by ios=1357472/0, merge=70/0, ticks=949141/0, in_queue=948776, util=99.88%io=

Bw= average IO bandwidth iops=IOPS runt= thread run time slat= submission delay, time spent submitting the IO request to kernel (excluding kernel processing time) clat= completion delay, processing time spent after submitting the IO request to kernel, lat= response time bw= bandwidth cpu= utilization IO depths=io queue IO submit= number of IO to be submitted by a single IO submission IO complete=Like the above submit number, but for completions instead. IO issued=The number of read/write requests issued, and how many of them were short. Distribution of IO latencies=IO completion delay

Total number of size executed by io=. The total IO aggrb=group bandwidth of minb= is minimum. Average bandwidth. The maximum average bandwidth of maxb=. The minimum running time of a thread in mint=group. The maximum running time of a thread in maxt=group.

The total number of IO executed by all group of ios=. The total number of IO mergers that occurred in merge=. Ticks=Number of ticks we kept the disk busy. The total time io_queue= spent in the queue. Util= disk utilization

Fio has many test task configuration files. In the git project examples folder, we can use command line parameters to configure directly, or we can configure the contents of a test directly through the configuration file.

For more detailed instructions on fio output, please refer to the blog post: Fio Output Explained

IO status Monitoring:

When doing disk testing, we can use monitoring tools such as iostat to see the current read and write status of all disks (this tool is included in sysstat-11.7.3-2.fc29.x86_64 on fedora systems).

Monitor disk IO command: iostat-mx 1

Iostat introduces that iostat is mainly used to monitor the IO load of system devices. The first time iostat runs, it displays statistics from the start of the system, and then running iostat displays statistics since the last time the command was run. Users can obtain the required statistics by specifying the number and time of statistics.

Grammar

Iostat [- c] [- d] [- h] [- k |-m] [- t] [- V] [- x] [- z] [device [...] | ALL] [- p [device [,...] | ALL]] [interval [count]] iostat usage example:

Iostat-d-x-k 1 10 Device: rrqm/s wrqm/s r util sda s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm% util sda 1.56 28.31 7.80 31.49 42.51 2.92 21.26 1.46 1.16 0.03 0.79 2.62 10.28 Device: rrqm/s wrqm/s r / s wswap s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm% util sda 2.00 20.00 381.00 7.00 12320.00 216.00 6160.00 108.00 32.31 1.75 4.50 2.17 84.20 murd: parameters showing the status of the device

-x: is a more common option that will be used to display extended data related to io.

-k: statically displays statistics per second (in kilobytes)

1: the first number indicates that the data display is refreshed every 1 second.

10: the second number represents the total number of refreshes

The meaning of rrqm/s: how many read requests related to this device have been Merge per second (when the system call needs to read data, VFS sends the request to each FS. If FS finds that different read requests read data from the same Block, FS will merge this request with Merge); wrqm/s: how many write requests related to this device have been Merge per second. Merge s: the number of read requests per second completed by the device (after merge merging) wUnip s: the number of write requests completed by the device per second (after merge merging) rsec/s: the number of sectors read per second; the number of sectors written by wsec/: per second. RKB/s: total number of read requests sent to the device per second wKB/s: total number of write requests sent to the device per second avgrq-sz average request sector size avgqu-sz is the average request queue length. There is no doubt that the shorter the queue, the better. Await: the average processing time (in microseconds and milliseconds) of each IO request. It can be understood as the response time of IO. Generally speaking, the response time of system IO should be lower than that of 5ms, and it is larger if it is greater than 10ms. This time includes queue time and service time, that is to say, in general, await is greater than svctm, and the smaller the difference between them, the shorter the queue time, and the greater the reverse difference, the longer the queue time, indicating that there is something wrong with the system. Svctm: represents the average service time (in milliseconds) for each device Istroke O operation. If the value of svctm is very close to that of await, it means that there is almost no Imax O waiting, and the disk performance is good. If the value of await is much higher than the value of svctm, it means that the queue wait of Imax O is too long, and the applications running on the system will be slower. % util: all processing IO times during the statistical time divided by the total statistical time. For example, if the statistical interval is 1 second, the device has 0.8 seconds of processing IO and 0.2 seconds of idle, then the% util of the device = 0.8 shock 1 = 80%, so this parameter indicates how busy the device is. Generally speaking, if this parameter is 100%, it means that the device is running at nearly full capacity (of course, if it is multiple disks, even if the% util is 100%, because of the concurrency of disks, disk usage may not be a bottleneck). After reading this article, I believe you have a certain understanding of "what are the common FIO commands in the Linux system". If you want to know more about it, you are welcome to follow the industry information channel. Thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.