io系统的监控工具-blktrace

author：skate
time：2012/04/11

io系统的监控工具-blktrace

blktrace是一个可以显示block的io详细信息的工具，但他的输出信息太专业了，很难看懂，可以同通过blkiomon、blkparse等工具来查看。

下载
[root@dhdb tmp]# wget ftp://mirror.switch.ch/pool/3/mirror/centos/5.8/os/x86_64/CentOS/blktrace-1.0.0-6.el5.x86_64.rpm
--14:20:16-- ftp://mirror.switch.ch/pool/3/mirror/centos/5.8/os/x86_64/CentOS/blktrace-1.0.0-6.el5.x86_64.rpm
=> `blktrace-1.0.0-6.el5.x86_64.rpm'
Resolving mirror.switch.ch... 130.59.10.36, 2001:620:0:8::20
Connecting to mirror.switch.ch|130.59.10.36|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD /pool/3/mirror/centos/5.8/os/x86_64/CentOS ... done.
==> SIZE blktrace-1.0.0-6.el5.x86_64.rpm ... 809191
==> PASV ... done. ==> RETR blktrace-1.0.0-6.el5.x86_64.rpm ... done.
Length: 809191 (790K)

100%[===========================================================================================>] 809,191 34.5K/s in 21s k

14:20:47 (37.0 KB/s) - `blktrace-1.0.0-6.el5.x86_64.rpm' saved [809191]

或者直接通过yum下载
[root@dhdb tmp] yum -y install blktrace

查看下载的文件
[root@dhdb tmp]# ll
total 7044
-rw-r--r-- 1 root root 809191 Apr 11 14:20 blktrace-1.0.0-6.el5.x86_64.rpm
-rw-r--r-- 1 root root 649605 Apr 2 23:58 dhdb_120402_0000.nmon
-rw-r--r-- 1 root root 649687 Apr 3 23:58 dhdb_120403_0000.nmon

安装文件
[root@dhdb tmp]# rpm -ivh blktrace-1.0.0-6.el5.x86_64.rpm
warning: blktrace-1.0.0-6.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID e8562897
Preparing... ########################################### [100%]
1:blktrace ########################################### [100%]
[root@dhdb tmp]#

开始测试
[root@dhdb tmp]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 38G 24G 13G 65% /
/dev/sda1 4.8G 143M 4.4G 4% /boot
tmpfs 7.9G 0 7.9G 0% /dev/shm
/dev/sda8 1.3T 1.2T 13G 99% /oracle
/dev/sda7 9.5G 158M 8.9G 2% /tmp
/dev/sda6 9.5G 4.3G 4.8G 48% /usr
/dev/sda5 19G 540M 18G 3% /var
[root@dhdb tmp]# blktrace /dev/sda -a issue -a complete -w 120 -o - | blkiomon -I 2 -h -
/sys/kernel/debug does not appear to be a debug filesystem

报这个错误的原因是blktrace在接受kernel数据时，并要存储在debugfs文件系统上，所以按如下挂载即可

[root@dhdb tmp]# mount -t debugfs none /sys/kernel/debug

对设备/dev/sda的io监控180秒，每10秒显示一次
[root@dhdb tmp]# blktrace /dev/sda -a issue -a complete -w 180 -o - | blkiomon -I 10 -h -

time: Wed Apr 11 14:31:29 2012
device: 8,0
sizes read (bytes): num 2, min 4096, max 4096, sum 8192, squ 33554432, avg 4096.0, var 0.0
sizes write (bytes): num 53, min 4096, max 28672, sum 655360, squ 9730785280, avg 12365.3, var 30699498.1
d2c read (usec): num 2, min 321, max 45066, sum 45387, squ 2031047397, avg 22693.5, var 500528756.2
d2c write (usec): num 53, min 42, max 164, sum 4353, squ 396051, avg 82.1, var 727.0
throughput read (bytes/msec): num 2, min 90, max 12760, sum 12850, squ 162825700, avg 6425.0, var 40132225.0
throughput write (bytes/msec): num 53, min 40554, max 263045, sum 8334568, squ 1560140448150, avg 157256.0, var 4707162693.2
sizes histogram (bytes):
0: 0 1024: 0 2048: 0 4096: 12
8192: 12 16384: 30 32768: 1 65536: 0
131072: 0 262144: 0 524288: 0 1048576: 0
2097152: 0 4194304: 0 8388608: 0 > 8388608: 0
d2c histogram (usec):
0: 0 8: 0 16: 0 32: 0
64: 11 128: 37 256: 5 512: 1
1024: 0 2048: 0 4096: 0 8192: 0
16384: 0 32768: 0 65536: 1 131072: 0
262144: 0 524288: 0 1048576: 0 2097152: 0
4194304: 0 8388608: 0 16777216: 0 33554432: 0
>33554432: 0
bidirectional requests: 0

说明
sizes read (bytes):
num 2 ：表示read 2次io
min 4096:表示read最小io 4096bytes
max 4096：表示read最大io 4096bytes
sum 8192：表示read总共io 8192bytes
avg 4096.0：表示read平均io大小 4096bytes

d2c write (usec)
num 53：表示write 53次io
min 42：表示最小write io的响应时间是42微秒
max 164：表示最大write io的响应时间是164微秒
sum 4353：表示write总共io响应时间是 4353微秒
avg 82.1：表示write io平均响应时间是82.1微秒

d2c histogram是io响应时间的分布图，256毫秒以内的io次数是53（11+37+5=53），从上面的响应时间分布图可以看到
写的io最快响应时间是42微秒，最慢的响应延迟164微秒，所以这53个io里大部分都是写io。

如果没有io活动，可以通过dd命令来模拟io活动
在一个窗口
[root@dhdb tmp]# dd if=/dev/zero of=skate.dat bs=4096 count=1000
另一个窗口监控
[root@dhdb tmp]# blktrace /dev/sda -a issue -a complete -w 180 -o - | blkiomon -I 10 -h -

参考：
http://blog.yufeng.info/archives/1786
blktrace帮助文档

----end------