linux vps 使用fio全面测试磁盘性能-starstao

注：文章末尾提供测试多个磁盘使用场景的脚本

安装fio

centos

如果要使用异步测试则要安装libaio

yum install fio libaio

安装libaio建议先yum search libaio, 查看libaio的名字是什么
我的系统显示名字是libaio

yum search libaio
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.dal.nexril.net
 * epel: d2lzkl7pfhq30w.cloudfront.net
 * extras: mirrors.oit.uci.edu
 * updates: mirrors.oit.uci.edu
============================================================================================================================== N/S matched: libaio ==============================================================================================================================
libaio.i686 : Linux-native asynchronous I/O access library
libaio.x86_64 : Linux-native asynchronous I/O access library
libaio-devel.i686 : Development files for Linux-native asynchronous I/O access
libaio-devel.x86_64 : Development files for Linux-native asynchronous I/O access

  Name and summary matches only, use "search all" for everything.

ubuntu

如果要使用异步测试则要安装libaio

apt install fio libaio1

安装libaio建议先apt search libaio, 查看libaio的名字是什么
我的系统显示名字是libaio1

apt search libaio
Sorting... Done
Full Text Search... Done
libaio-dev/jammy 0.3.112-13build1 amd64
  Linux kernel AIO access library - development files

libaio1/jammy,now 0.3.112-13build1 amd64 [installed]
  Linux kernel AIO access library - shared library

参数讲解

FIO 核心参数

参数	说明	示例/取值
`-filename`	指定测试的目标文件或设备路径	`-filename=/dev/sdb1`
`-direct`	是否绕过系统缓存（`1`启用，`0`禁用）	`-direct=1`
`-iodepth`	I/O 队列深度（异步引擎有效，高并发建议增大）	`-iodepth=32`
`-rw`	I/O 模式：`read`/`write`（顺序）、`randread`/`randwrite`（随机）等	`-rw=randrw`
`-ioengine`	I/O 引擎：`sync`（同步）、`libaio`（异步）、`psync`（类似sync）	`-ioengine=libaio`
`-bs`	块大小（影响吞吐和延迟）	`-bs=4k`
`-size`	总 I/O 数据量（支持单位如 `2G`、`500M`）	`-size=10G`
`-numjobs`	并发线程/进程数（模拟多客户端）	`-numjobs=16`
`-runtime`	测试运行时间（秒）	`-runtime=300`
`-group_reporting`	合并所有任务的统计结果	`-group_reporting`
`-name`	测试任务名称（用于日志标识）	`-name=mytest`

高级参数

参数	说明	示例/取值
`-rwmixread`	混合读写中读操作占比（仅 `rw=randrw` 有效）	`-rwmixread=70`
`-time_based`	强制按 `-runtime` 运行，即使提前完成 `-size`	`-time_based`
`-ramp_time`	预热时间（秒），跳过初始阶段的统计	`-ramp_time=10`
`-sync`	启用同步 I/O（`O_SYNC`，强制数据落盘）	`-sync=1`
`-fsync`	每次 I/O 后执行 `fsync()`（严格一致性）	`-fsync=1`
`-write_iops_log`	输出 IOPS 日志到文件	`-write_iops_log=iops.log`
`-stonewall`	所有任务完成后统一报告	`-stonewall`
`-norandommap`	禁用随机文件偏移（全盘覆盖）	`-norandommap`

小文件读写测试

同步16k大小10线程顺序读（异步的libaio更能模拟真实高并发高负载情况）

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=read -ioengine=psync -bs=16k -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest

输出

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=read -ioengine=psync -bs=16k -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest
mytest: (g=0): rw=read, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=1
...
fio-3.28
Starting 10 processes
mytest: Laying out IO file (1 file / 2048MiB)
Jobs: 10 (f=10): [R(10)][99.2%][r=184MiB/s][r=11.8k IOPS][eta 00m:01s]
mytest: (groupid=0, jobs=10): err= 0: pid=1192510: Mon Mar 24 03:26:16 2025
  read: IOPS=10.5k, BW=164MiB/s (172MB/s)(20.0GiB/125023msec)
    clat (usec): min=122, max=54421, avg=947.18, stdev=1542.74
     lat (usec): min=123, max=54421, avg=947.68, stdev=1542.82
    clat percentiles (usec):
     |  1.00th=[  221],  5.00th=[  273], 10.00th=[  314], 20.00th=[  371],
     | 30.00th=[  433], 40.00th=[  506], 50.00th=[  603], 60.00th=[  717],
     | 70.00th=[  865], 80.00th=[ 1090], 90.00th=[ 1549], 95.00th=[ 2311],
     | 99.00th=[ 7635], 99.50th=[10945], 99.90th=[20055], 99.95th=[23987],
     | 99.99th=[34866]
   bw (  KiB/s): min=41596, max=316672, per=100.00%, avg=168041.21, stdev=5517.09, samples=2490
   iops        : min= 2598, max=19792, avg=10500.73, stdev=344.86, samples=2490
  lat (usec)   : 250=2.79%, 500=36.34%, 750=23.36%, 1000=14.15%
  lat (msec)   : 2=17.00%, 4=3.96%, 10=1.79%, 20=0.50%, 50=0.10%
  lat (msec)   : 100=0.01%
  cpu          : usr=1.19%, sys=5.49%, ctx=1325984, majf=0, minf=207
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=1310720,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=164MiB/s (172MB/s), 164MiB/s-164MiB/s (172MB/s-172MB/s), io=20.0GiB (21.5GB), run=125023-125023msec

Disk stats (read/write):
  sda: ios=1310350/88, merge=449/39, ticks=936239/98, in_queue=936344, util=99.18%

同步16k大小10线程顺序写

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=write -ioengine=psync -bs=16k -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest

输出

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=write -ioengine=psync -bs=16k -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest
mytest: (g=0): rw=write, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=1
...
fio-3.7
Starting 10 processes
Jobs: 1 (f=1): [_(6),W(1),_(3)][97.2%][r=0KiB/s,w=124MiB/s][r=0,w=7932 IOPS][eta 00m:05s]          
mytest: (groupid=0, jobs=10): err= 0: pid=9911: Mon Mar 24 12:46:42 2025
  write: IOPS=7517, BW=117MiB/s (123MB/s)(20.0GiB/174353msec)
    clat (usec): min=81, max=71717k, avg=887.97, stdev=196690.96
     lat (usec): min=82, max=71717k, avg=889.24, stdev=196690.99
    clat percentiles (usec):
     |  1.00th=[   88],  5.00th=[   92], 10.00th=[   94], 20.00th=[   95],
     | 30.00th=[   97], 40.00th=[   99], 50.00th=[  101], 60.00th=[  105],
     | 70.00th=[  112], 80.00th=[  121], 90.00th=[  130], 95.00th=[  143],
     | 99.00th=[  429], 99.50th=[  693], 99.90th=[ 5932], 99.95th=[10552],
     | 99.99th=[11076]
   bw (  KiB/s): min= 3552, max=146688, per=83.44%, avg=100367.26, stdev=24669.85, samples=362
   iops        : min=  222, max= 9168, avg=6272.66, stdev=1541.96, samples=362
  lat (usec)   : 100=47.55%, 250=51.08%, 500=0.43%, 750=0.54%, 1000=0.09%
  lat (msec)   : 2=0.06%, 4=0.10%, 10=0.08%, 20=0.06%, 50=0.01%
  cpu          : usr=0.81%, sys=3.56%, ctx=2529819, majf=0, minf=346
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,1310720,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=117MiB/s (123MB/s), 117MiB/s-117MiB/s (123MB/s-123MB/s), io=20.0GiB (21.5GB), run=174353-174353msec

Disk stats (read/write):
  vda: ios=4/1308938, merge=0/62, ticks=4/113706, in_queue=113710, util=91.07%

同步16k大小10线程随机读

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=randread -ioengine=psync -bs=16k -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest

输出

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=randread -ioengine=psync -bs=16k -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest
mytest: (g=0): rw=randread, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=1
...
fio-3.28
Starting 10 processes
Jobs: 10 (f=10): [r(10)][99.5%][r=181MiB/s][r=11.6k IOPS][eta 00m:01s]
mytest: (groupid=0, jobs=10): err= 0: pid=1255986: Mon Mar 24 04:56:57 2025
  read: IOPS=6654, BW=104MiB/s (109MB/s)(20.0GiB/196966msec)
    clat (usec): min=201, max=88175, avg=1493.36, stdev=1994.21
     lat (usec): min=202, max=88175, avg=1493.88, stdev=1994.32
    clat percentiles (usec):
     |  1.00th=[  445],  5.00th=[  523], 10.00th=[  578], 20.00th=[  685],
     | 30.00th=[  783], 40.00th=[  889], 50.00th=[ 1012], 60.00th=[ 1156],
     | 70.00th=[ 1352], 80.00th=[ 1663], 90.00th=[ 2376], 95.00th=[ 3851],
     | 99.00th=[10552], 99.50th=[13698], 99.90th=[23200], 99.95th=[28443],
     | 99.99th=[41681]
   bw (  KiB/s): min=18091, max=189984, per=100.00%, avg=106608.79, stdev=2958.74, samples=3918
   iops        : min= 1130, max=11874, avg=6660.70, stdev=184.93, samples=3918
  lat (usec)   : 250=0.01%, 500=3.48%, 750=23.57%, 1000=21.88%
  lat (msec)   : 2=37.35%, 4=8.98%, 10=3.63%, 20=0.94%, 50=0.17%
  lat (msec)   : 100=0.01%
  cpu          : usr=0.99%, sys=3.76%, ctx=1322074, majf=0, minf=160
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=1310720,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=104MiB/s (109MB/s), 104MiB/s-104MiB/s (109MB/s-109MB/s), io=20.0GiB (21.5GB), run=196966-196966msec

Disk stats (read/write):
  sda: ios=1310636/163, merge=1/46, ticks=1624274/289, in_queue=1624584, util=99.17%

同步16k大小10线程随机写

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=randwrite -ioengine=psync -bs=16k -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest

输出

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=randwrite -ioengine=psync -bs=16k -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest
mytest: (g=0): rw=randwrite, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=1
...
fio-3.28
Starting 10 processes
Jobs: 8 (f=8): [_(1),w(4),_(1),w(4)][100.0%][w=88.2MiB/s][w=5640 IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=10): err= 0: pid=1263789: Mon Mar 24 05:08:54 2025
  write: IOPS=5313, BW=83.0MiB/s (87.1MB/s)(20.0GiB/246673msec); 0 zone resets
    clat (usec): min=145, max=91262, avg=1870.67, stdev=2339.70
     lat (usec): min=146, max=91263, avg=1872.01, stdev=2339.86
    clat percentiles (usec):
     |  1.00th=[  251],  5.00th=[  334], 10.00th=[  420], 20.00th=[  701],
     | 30.00th=[  914], 40.00th=[ 1123], 50.00th=[ 1319], 60.00th=[ 1565],
     | 70.00th=[ 1909], 80.00th=[ 2376], 90.00th=[ 3326], 95.00th=[ 4883],
     | 99.00th=[11863], 99.50th=[16057], 99.90th=[26870], 99.95th=[31851],
     | 99.99th=[44303]
   bw (  KiB/s): min=16690, max=139296, per=100.00%, avg=85239.96, stdev=2012.44, samples=4907
   iops        : min= 1036, max= 8706, avg=5325.93, stdev=125.81, samples=4907
  lat (usec)   : 250=0.95%, 500=12.35%, 750=8.92%, 1000=12.08%
  lat (msec)   : 2=38.01%, 4=20.61%, 10=5.60%, 20=1.20%, 50=0.28%
  lat (msec)   : 100=0.01%
  cpu          : usr=0.86%, sys=3.10%, ctx=1320935, majf=0, minf=122
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,1310720,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=83.0MiB/s (87.1MB/s), 83.0MiB/s-83.0MiB/s (87.1MB/s-87.1MB/s), io=20.0GiB (21.5GB), run=246673-246673msec

Disk stats (read/write):
  sda: ios=8/1310689, merge=0/136, ticks=47/2228600, in_queue=2228725, util=99.48%

大文件读写测试

同步10M大小10线程顺序读

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=read -ioengine=psync -bs=10M -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest

输出

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=read -ioengine=psync -bs=10M -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest
mytest: (g=0): rw=read, bs=(R) 10.0MiB-10.0MiB, (W) 10.0MiB-10.0MiB, (T) 10.0MiB-10.0MiB, ioengine=psync, iodepth=1
...
fio-3.28
Starting 10 processes
Jobs: 10 (f=10): [R(10)][92.3%][r=1715MiB/s][r=171 IOPS][eta 00m:01s]
mytest: (groupid=0, jobs=10): err= 0: pid=1267546: Mon Mar 24 05:10:24 2025
  read: IOPS=174, BW=1740MiB/s (1825MB/s)(19.9GiB/11722msec)
    clat (msec): min=7, max=318, avg=57.10, stdev=18.73
     lat (msec): min=7, max=318, avg=57.10, stdev=18.73
    clat percentiles (msec):
     |  1.00th=[   23],  5.00th=[   40], 10.00th=[   44], 20.00th=[   49],
     | 30.00th=[   52], 40.00th=[   54], 50.00th=[   56], 60.00th=[   58],
     | 70.00th=[   61], 80.00th=[   64], 90.00th=[   70], 95.00th=[   77],
     | 99.00th=[   99], 99.50th=[  106], 99.90th=[  313], 99.95th=[  317],
     | 99.99th=[  317]
   bw (  MiB/s): min= 1015, max= 1998, per=99.80%, avg=1736.86, stdev=21.15, samples=230
   iops        : min=  100, max=  199, avg=170.13, stdev= 2.10, samples=230
  lat (msec)   : 10=0.15%, 20=0.54%, 50=24.90%, 100=73.53%, 250=0.54%
  lat (msec)   : 500=0.34%
  cpu          : usr=0.02%, sys=2.15%, ctx=2764, majf=0, minf=25725
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=2040,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=1740MiB/s (1825MB/s), 1740MiB/s-1740MiB/s (1825MB/s-1825MB/s), io=19.9GiB (21.4GB), run=11722-11722msec

Disk stats (read/write):
  sda: ios=42233/7, merge=847/5, ticks=1742404/281, in_queue=1742712, util=98.73%

同步10M大小10线程顺序写

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=write -ioengine=psync -bs=10M -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest

输出

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=write -ioengine=psync -bs=10M -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest
mytest: (g=0): rw=write, bs=(R) 10.0MiB-10.0MiB, (W) 10.0MiB-10.0MiB, (T) 10.0MiB-10.0MiB, ioengine=psync, iodepth=1
...
fio-3.28
Starting 10 processes
Jobs: 5 (f=5): [W(1),_(2),W(3),_(2),W(1),_(1)][100.0%][w=781MiB/s][w=78 IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=10): err= 0: pid=1268000: Mon Mar 24 05:11:14 2025
  write: IOPS=84, BW=846MiB/s (887MB/s)(19.9GiB/24125msec); 0 zone resets
    clat (msec): min=7, max=490, avg=115.36, stdev=60.60
     lat (msec): min=8, max=491, avg=116.22, stdev=60.73
    clat percentiles (msec):
     |  1.00th=[   13],  5.00th=[   41], 10.00th=[   58], 20.00th=[   71],
     | 30.00th=[   80], 40.00th=[   88], 50.00th=[   99], 60.00th=[  116],
     | 70.00th=[  134], 80.00th=[  159], 90.00th=[  199], 95.00th=[  234],
     | 99.00th=[  309], 99.50th=[  334], 99.90th=[  439], 99.95th=[  443],
     | 99.99th=[  489]
   bw (  KiB/s): min=304774, max=1537312, per=100.00%, avg=875198.12, stdev=29389.30, samples=468
   iops        : min=   22, max=  150, avg=81.78, stdev= 2.98, samples=468
  lat (msec)   : 10=0.59%, 20=1.37%, 50=4.80%, 100=44.36%, 250=45.25%
  lat (msec)   : 500=3.63%
  cpu          : usr=0.70%, sys=1.08%, ctx=3910, majf=0, minf=123
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,2040,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=846MiB/s (887MB/s), 846MiB/s-846MiB/s (887MB/s-887MB/s), io=19.9GiB (21.4GB), run=24125-24125msec

Disk stats (read/write):
  sda: ios=0/44983, merge=0/1301, ticks=0/2303581, in_queue=2303646, util=98.53%

同步10M大小10线程随机读

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=randread -ioengine=psync -bs=10M -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest

输出

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=randread -ioengine=psync -bs=10M -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest
mytest: (g=0): rw=randread, bs=(R) 10.0MiB-10.0MiB, (W) 10.0MiB-10.0MiB, (T) 10.0MiB-10.0MiB, ioengine=psync, iodepth=1
...
fio-3.28
Starting 10 processes
Jobs: 10 (f=10): [r(10)][100.0%][r=2262MiB/s][r=226 IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=10): err= 0: pid=1268662: Mon Mar 24 05:11:55 2025
  read: IOPS=218, BW=2181MiB/s (2287MB/s)(19.9GiB/9353msec)
    clat (msec): min=7, max=118, avg=45.51, stdev= 9.85
     lat (msec): min=7, max=118, avg=45.51, stdev= 9.85
    clat percentiles (msec):
     |  1.00th=[   23],  5.00th=[   32], 10.00th=[   36], 20.00th=[   40],
     | 30.00th=[   42], 40.00th=[   43], 50.00th=[   45], 60.00th=[   47],
     | 70.00th=[   50], 80.00th=[   52], 90.00th=[   57], 95.00th=[   62],
     | 99.00th=[   73], 99.50th=[   83], 99.90th=[  114], 99.95th=[  116],
     | 99.99th=[  118]
   bw (  MiB/s): min= 1847, max= 2459, per=99.95%, avg=2180.04, stdev=16.11, samples=180
   iops        : min=  179, max=  244, avg=213.56, stdev= 1.64, samples=180
  lat (msec)   : 10=0.20%, 20=0.64%, 50=73.09%, 100=25.88%, 250=0.20%
  cpu          : usr=0.04%, sys=2.27%, ctx=2644, majf=0, minf=25709
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=2040,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=2181MiB/s (2287MB/s), 2181MiB/s-2181MiB/s (2287MB/s-2287MB/s), io=19.9GiB (21.4GB), run=9353-9353msec

Disk stats (read/write):
  sda: ios=44018/10, merge=698/0, ticks=1475692/113, in_queue=1475806, util=98.89%

同步10M大小10线程随机写

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=randwrite -ioengine=psync -bs=10M -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest

输出

fio -filename=iotestfile -direct=1 -iodepth=1 -rw=randwrite -ioengine=psync -bs=10M -size=2G -numjobs=10 -runtime=300 -group_reporting -name=mytest
mytest: (g=0): rw=randwrite, bs=(R) 10.0MiB-10.0MiB, (W) 10.0MiB-10.0MiB, (T) 10.0MiB-10.0MiB, ioengine=psync, iodepth=1
...
fio-3.28
Starting 10 processes
Jobs: 5 (f=5): [w(1),_(2),w(3),_(1),w(1),_(2)][100.0%][w=741MiB/s][w=74 IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=10): err= 0: pid=1269022: Mon Mar 24 05:12:42 2025
  write: IOPS=75, BW=756MiB/s (793MB/s)(19.9GiB/26968msec); 0 zone resets
    clat (msec): min=7, max=395, avg=129.15, stdev=54.76
     lat (msec): min=8, max=396, avg=129.90, stdev=54.76
    clat percentiles (msec):
     |  1.00th=[   15],  5.00th=[   50], 10.00th=[   72], 20.00th=[   90],
     | 30.00th=[  102], 40.00th=[  111], 50.00th=[  122], 60.00th=[  133],
     | 70.00th=[  148], 80.00th=[  167], 90.00th=[  199], 95.00th=[  228],
     | 99.00th=[  309], 99.50th=[  342], 99.90th=[  388], 99.95th=[  393],
     | 99.99th=[  397]
   bw (  KiB/s): min=423450, max=1205167, per=100.00%, avg=783377.17, stdev=18729.34, samples=522
   iops        : min=   32, max=  116, avg=73.14, stdev= 1.88, samples=522
  lat (msec)   : 10=0.25%, 20=1.32%, 50=3.48%, 100=24.22%, 250=67.60%
  lat (msec)   : 500=3.14%
  cpu          : usr=0.61%, sys=0.91%, ctx=4263, majf=0, minf=123
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,2040,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=756MiB/s (793MB/s), 756MiB/s-756MiB/s (793MB/s-793MB/s), io=19.9GiB (21.4GB), run=26968-26968msec

Disk stats (read/write):
  sda: ios=0/44870, merge=0/946, ticks=0/2417776, in_queue=2417838, util=98.99%

IOPS测试

测试最大随机读IOPS

fio -filename=iotestfile -direct=1 -iodepth=32 -rw=randread -ioengine=libaio -bs=4k -size=100M -numjobs=10 -runtime=60 --time_based -group_reporting -name=mytest

输出

fio -filename=iotestfile -direct=1 -iodepth=32 -rw=randread -ioengine=libaio -bs=4k -size=100M -numjobs=10 -runtime=60 --time_based -group_reporting -name=mytest
mytest: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
...
fio-3.28
Starting 10 processes
Jobs: 10 (f=10): [r(10)][100.0%][r=113MiB/s][r=29.0k IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=10): err= 0: pid=1288318: Mon Mar 24 05:40:26 2025
  read: IOPS=22.4k, BW=87.4MiB/s (91.7MB/s)(5248MiB/60032msec)
    slat (usec): min=5, max=372056, avg=176.31, stdev=2229.40
    clat (usec): min=7, max=393445, avg=14097.86, stdev=17651.81
     lat (usec): min=195, max=397013, avg=14277.30, stdev=17838.24
    clat percentiles (usec):
     |  1.00th=[   668],  5.00th=[  1012], 10.00th=[  1369], 20.00th=[  2933],
     | 30.00th=[  5014], 40.00th=[  7046], 50.00th=[  9241], 60.00th=[ 11863],
     | 70.00th=[ 15270], 80.00th=[ 20579], 90.00th=[ 30802], 95.00th=[ 43254],
     | 99.00th=[ 85459], 99.50th=[107480], 99.90th=[181404], 99.95th=[214959],
     | 99.99th=[312476]
   bw (  KiB/s): min=11267, max=191163, per=100.00%, avg=89730.90, stdev=3674.42, samples=1189
   iops        : min= 2813, max=47787, avg=22430.38, stdev=918.61, samples=1189
  lat (usec)   : 10=0.01%, 50=0.01%, 100=0.01%, 250=0.01%, 500=0.19%
  lat (usec)   : 750=1.52%, 1000=3.16%
  lat (msec)   : 2=10.58%, 4=9.44%, 10=28.23%, 20=26.18%, 50=16.98%
  lat (msec)   : 100=3.09%, 250=0.60%, 500=0.03%
  cpu          : usr=2.73%, sys=9.82%, ctx=66608, majf=0, minf=449
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued rwts: total=1343593,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=87.4MiB/s (91.7MB/s), 87.4MiB/s-87.4MiB/s (91.7MB/s-91.7MB/s), io=5248MiB (5503MB), run=60032-60032msec

Disk stats (read/write):
  sda: ios=1333985/150, merge=7067/14, ticks=7631314/1042, in_queue=7632363, util=95.11%

测试最大随机写IOPS

fio -filename=iotestfile -direct=1 -iodepth=32 -rw=randwrite -ioengine=libaio -bs=4k -size=100M -numjobs=10 -runtime=60 --time_based -group_reporting -name=mytest

输出

fio -filename=iotestfile -direct=1 -iodepth=32 -rw=randwrite -ioengine=libaio -bs=4k -size=100M -numjobs=10 -runtime=60 --time_based -group_reporting -name=mytest
mytest: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
...
fio-3.28
Starting 10 processes
Jobs: 10 (f=10): [w(10)][100.0%][w=29.6MiB/s][w=7575 IOPS][eta 00m:00s]
mytest: (groupid=0, jobs=10): err= 0: pid=1289318: Mon Mar 24 05:42:01 2025
  write: IOPS=8099, BW=31.6MiB/s (33.2MB/s)(1900MiB/60045msec); 0 zone resets
    slat (usec): min=6, max=121191, avg=838.62, stdev=3584.21
    clat (usec): min=202, max=507908, avg=38648.45, stdev=30290.40
     lat (usec): min=276, max=507937, avg=39488.46, stdev=30667.66
    clat percentiles (msec):
     |  1.00th=[    7],  5.00th=[   11], 10.00th=[   14], 20.00th=[   18],
     | 30.00th=[   22], 40.00th=[   27], 50.00th=[   31], 60.00th=[   36],
     | 70.00th=[   43], 80.00th=[   53], 90.00th=[   72], 95.00th=[   95],
     | 99.00th=[  159], 99.50th=[  188], 99.90th=[  268], 99.95th=[  313],
     | 99.99th=[  401]
   bw (  KiB/s): min=12230, max=57632, per=100.00%, avg=32474.45, stdev=975.92, samples=1190
   iops        : min= 3055, max=14407, avg=8116.55, stdev=244.01, samples=1190
  lat (usec)   : 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.02%, 4=0.07%, 10=4.52%, 20=20.42%, 50=52.67%
  lat (msec)   : 100=18.03%, 250=4.14%, 500=0.13%, 750=0.01%
  cpu          : usr=1.02%, sys=2.84%, ctx=186417, majf=0, minf=125
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=99.9%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,486340,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: bw=31.6MiB/s (33.2MB/s), 31.6MiB/s-31.6MiB/s (33.2MB/s-33.2MB/s), io=1900MiB (1992MB), run=60045-60045msec

Disk stats (read/write):
  sda: ios=0/477759, merge=0/7579, ticks=0/14805520, in_queue=14805773, util=99.14%

执行多个fio测试命令并且汇总结果

需要安装好fio和libaio，并且安装了python3，python3的版本要在3.6.0及以上
执行如下命令获取测试脚本iotest.py，脚本所在github仓库地址: iotest仓库地址

curl -O https://raw.githubusercontent.com/starstao/iotest/refs/heads/main/iotest.py

然后再执行python3 iotest.py进行测试

测试结果

python3 iotest.py 

正在执行测试：小文件随机读

正在执行测试：小文件随机写

正在执行测试：大文件随机读

正在执行测试：大文件随机写

正在执行测试：异步测试最大读IOPS

正在执行测试：异步测试最大写IOPS

========================== 测试结果汇总 ==========================
测试类型           | 读带宽(MiB/s) |   读IOPS | 写带宽(MiB/s) |  写IOPS
-----------------------------------------------------------
小文件随机读       |        140.77 |  9009.60 |          0.00 |    0.00
小文件随机写       |          0.00 |     0.00 |         86.46 | 5533.47
大文件随机读       |       2261.64 |   226.16 |          0.00 |    0.00
大文件随机写       |          0.00 |     0.00 |        718.59 |   71.86
异步测试最大读IOPS |         98.98 | 25337.84 |          0.00 |    0.00
异步测试最大写IOPS |          0.00 |     0.00 |         37.23 | 9529.94

=========================== 峰值性能 ===========================
最大读IOPS:             25337.84
最大写IOPS:             9529.94

目录CONTENT

linux vps 使用fio全面测试磁盘性能