????

??? vdbench ???????????????????vdbench ? Henk Vandenbergh ????????????????????????Java ? JNI ??????? Solaris(SPARC and x86) ?? Windows ? Linux, VMware, Mac OS X ?????????????????????????? Solaris ? Linux ?????????????????? Sun ????????????????????? 5.00 ?????????????????????????????????????????????????? ??? bonnie-64, bonnie++, Iometer, IOzone, FileBench, IOR ????????vdbench ???????????????????????????????????????????

  • ????????????????????????
  • RAW ????????????????? I/O ????????????
  • ??????????????????????? I/O ????????????
  • Read / Write ???????? I/O ????????????
  • I/O ???????? I/O ????????????
  • IOPS ???????????????????
  • ??????????????I/O ?????????????????????
  • ???????????? I/O ????????????
  • ???????? I/O ????????????
  • Solaris ? Linux ?? O_SYNC, O_DSYNC, O_RSYNC ??????????????????????
  • ????????????I/O ??????????????????????????????????
vdbench ? http://blogs.sun.com/henk/entry/vdbench_a_disk_and_tape ? http://www.sun.com/storage/white-pap...ottlenecks.pdf ???????????????????????

vdbench ???????

vdbench ????????????? http://vdbench.sourceforge.net/ ?????????????2009 ? 8 ???????????? 5.01 ???

vdbench ???????

Solaris ????????

?????????????????????

# mkdir /var/tmp/vdbench # cp vdbench501.zip /var/tmp/vdbench # cd /var/tmp/vdbench # unzip vdbench501.zip Linux ?????????

CentOS(64bit) ??????? OpenJDK ? vdbench ??????????????? Linux ? JavaVM ?????????? http://java.sun.com ????? JDK ? 64bit Linux ???????????????????????

# mkdir /var/tmp/vdbench # cp vdbench501.zip /var/tmp/vdbench # cd /var/tmp/vdbench # unzip vdbench501.zip vdbench ????

???? vdbench ??????????????????????????????? vdbench ????????????? vdbench.pdf ???????

vdbench ???????

?????????????? vdbench ???????????./vdbench -t ???????????????????????????

# ./vdbench -t Vdbench distribution: vdbench500 For documentation, see 'vdbench.pdf'. 01:07:59.273 input argument scanned: '-f/tmp/parmfile' 01:07:59.327 Starting slave: /var/tmp/vdbench/vdbench SlaveJvm -m 10.16.67.4 -n localhost-10- 090723-01.07.58.208 -l localhost-0 -p 5570 01:07:59.660 All slaves are now connected 01:08:00.763 Inserted 'rd=File_format_for_sd=sd1' to initialize new file for sd=sd1,lun=/tmp/quick_vdbench_test,size=10485760 01:08:02.003 Starting RD=File_format_for_sd=sd1; I/O rate: 5000; Elapsed=(none); For loops: threads=8 01:08:02.089 All sequential workloads on all slaves are done. 01:08:02.089 This triggers end of run inspite of possibly some non-sequential workloads that are still running. Jul 23, 2009 interval i/o MB/sec bytes read resp resp resp cpu% cpu% rate 1024**2 i/o pct time max stddev sys+usr sys 01:08:03.051 1 9.00 9.00 1048576 0.00 11.178 15.601 5.275 2.7 1.4 01:08:03.057 avg_2-1 0.00 0.00 0 0.00 0.000 0.000 0.000 01:08:04.003 Starting RD=rd1; I/O rate: 100; Elapsed=5; For loops: None Jul 23, 2009 interval i/o MB/sec bytes read resp resp resp cpu% cpu% rate 1024**2 i/o pct time max stddev sys+usr sys 01:08:05.009 1 84.00 0.33 4096 53.57 0.009 0.040 0.005 1.1 0.3 01:08:06.047 2 89.00 0.35 4096 50.56 0.008 0.018 0.003 0.1 0.0 01:08:07.048 3 99.00 0.39 4096 50.51 0.008 0.017 0.003 0.1 0.0 01:08:08.047 4 94.00 0.37 4096 51.06 0.008 0.017 0.003 0.1 0.0 01:08:09.047 5 111.00 0.43 4096 53.15 0.008 0.024 0.003 0.1 0.1 01:08:09.050 avg_2-5 98.25 0.38 4096 51.40 0.008 0.024 0.003 0.1 0.0 01:08:09.357 Slave localhost-0 terminated 01:08:09.409 Vdbench execution completed successfully. Output directory: /var/tmp/vdbench/output
  • vdbench -t ? /tmp ????????????????? output ??????????????????????? output/summary.html ???????????
vdbench ??????????

??????????????????????? "./vdbench -f -o +" ????????????????????? myconf.txt ?????????????????????? /var/tmp/resultXXX ????????????????????????????????????????resultXXX ? XXX ????????2 ????? 000 ?? 999 ?????????????????????????????????????????????????? ????????????

# ./vdbench -f myconf.txt -o /var/tmp/result+ vdbench ???????????????????????????????????????

usage: ./vdbench [compare][gui] [-f xxx] [-o xxx] [-e nn] [-i nn] [-j] [-jr] [-v] [-vq] [-s] [-k] [- "parmfile parameters"] '-f xxx': Workload parameter file name(s). Default 'parmfile' in current directory 'gui': Start Vdbench Graphical User Interface '-o xxx': Output directory for reporting. Default 'output' in current directory '-e nn': Override elapsed=seconds. '-i nn': Override interval=seconds. '-v': Activate Data validation. '-vq': Activate Data validation, validate lba and data key (saves cpu) '-j': Activate Data validation with Journaling. '-jr': Recover existing Journal, Validate data and run workload '-s': Simulate execution. Do everything but I/O. '-k': Solaris only: Report kstat statistics on console. ??????????

I/O ?????????????????????????????????????????????????

??????

vdbench ??????????????????????????????????* ????????????????????????vdbench -f ???????????????? vdbench ?????????????????????????????????????????????????? ?????????????????????????????

  • Example 1 *Example 1: Single run, one raw disk *SD: Storage Definition *WD: Workload Definition *RD: Run Definition * sd=sd1,lun=/dev/rdsk/c0t0d0s0 wd=wd1,sd=sd1,xfersize=4096,rdpct=100 rd=run1,wd=wd1,iorate=100,elapsed=10,interval=1 *Single raw disk, 100% random read of 4k records at i/o rate of 100 for 10 seconds
  • Example 2 *Example 2: Single run, two raw disk, two workloads. sd=sd1,lun=/dev/rdsk/c0t0d0s0 sd=sd2,lun=/dev/rdsk/c0t0d1s0 wd=wd1,sd=sd1,xfersize=4k,rdpct=80,skew=40 wd=wd2,sd=sd2,xfersize=8k,rdpct=0 rd=run1,wd=wd*,iorate=200,elapsed=10,interval=1 *Two raw disks: sd1 does 80 i/o's per second, read-to-write ratio 4:1, 4k records. sd2 does 120 i/o's per second, 100% write at 8k records.
  • Example 3(Filesystem) fsd=fsd1,anchor=/dir1,depth=2,width=2,files=2,size=128k fwd=fwd1,fsd=fsd1,operation=read,xfersize=4k,filei o=sequential,fileselect=random,threads=2 rd=rd1,fwd=fwd1,fwdrate=max,format=yes,elapsed=10, interval=1
Raw ??????????????????

vdbench ? Raw ???????????????? I/O ????????????????? Raw ???????????????????????????????

  • sd ???
???????????????????????????? sd= ????????????sd ? Storage Definition ?????????????????????????????????????????????????? ?

sd=sd1 ???????????????????????????????????????????? lun= ?????????lun ? Logical Unit Number ??????????? Solaris ?? /dev/rdsk/c0t0d0s0 ? /dev/md/rdsk/d10 ????????????????????????Linux ?? raw ????? /dev/raw/... ??????????????

sd=sd1,lun=/dev/rdsk/c0t0d0s0 ????????????????????? fdisk -l ????????

sd=sd1,lun=/dev/raw/raw1,size=733468426240 ??????????????????????????????????????

sd=sd1,lun=/dev/rdsk/c0t0d0s0,threads=32 ??????wd ? Workload Definition ?????

wd=wd1 ??????????????????????????????????????????? sd ???????????????? sd= ???????????? sd ???????????????? sd=(sd1,sd2) ????????????????? sd=(sd1-sd10) ?????????????????

wd=wd1,sd=sd1 ??????????????? seekpct=??????????? rhpct=?whpct= ?????????seekpct ??????????????????????????????????????? 0 ??? sequential ????????????????100 ??? random ???????????????????seekpct=0 ?????????????????????????????????????????

wd=wd1,sd=sd1,xfersize=32k,rdpct=100,seekpct=100 ????????????rd ? Run Definition ?????

rd=rd1 ???????????? wd ??????????? wd=(wd1, wd2) ?????????????????????wd=(wd1-wd10) ????????????????????? wd ?????????wd ?????? skew ???????????? wd ?????? I/O ?????????????

rd=rd1,wd=wd1 ???????? interval= ???????

rd=rd1,wd=wd1,iorate=max,elapsed=600,interval=30 ?forrdpct=?forseekpct= ??????????????forxferseize=(4k,8k,12k,16k) ???????I/O ???? 4KB, 8KB, 12KB, 16KB ??????????????????????????????????????????

??????????????????????I/O ?????????????? 180 ?????????? 10 ????????????* ??????????????

* Test Pattern 1 - Sequential Write, 4KB * 2009/07/31 sd=sd1,lun=/dev/rdsk/c0t0d0s0 sd=sd2,lun=/dev/rdsk/c0t1d0s0 wd=wd1,sd=(sd1,sd2),xfersize=4k,rdpct=0,seekpct=0 rd=rd1,wd=wd1,iorate=max,elapsed=180,interval=10
  • ???
???????????vdbench ??????????????????????????????? sd, wd, rd ?????????????????????????????????????????????????? ??????????????????????????????????? ??????=? ????????????* ?????????????

* ???? ???????=???,??????=?,??????=?,... ??????????????????????

?????????????????? sd ????? fsd, wd ????? fwd ??????????rd ????????????????

  • fsd ???
fsd ? Filesystem Definition ?????????????????????????????????????????????????? ?????????????????????????????

fsd=fsd1 ??????????????????? files= ??????????????? sizes= ?????????????????????sizes=(32k,30,2m,70) ???????? 32KB ?????? 30%, 2MB ?????? 70% ???????????

fsd=fsd1,anchor=/mnt/test,files=100,sizes=2m ???????????????????????????depth ?????????????????????????????????????? width ???????????????????????????????????? files ??????????????????????????????????? "width ^ depth * files" ???????width=2,depth=2,files=2 ???? 4 ????????? 8 ????????????width=2, depth=3, files=4 ???? 8 ????????????? 32 ??????????????

fsd=fsd1,anchor=/mnt/test,width=2,depth=3,files=4,sizes=10m ???????I/O ??? random ??? sequential ?????????? random ?????????? I/O ??sequential ????????? I/O ??????close() ??????? I/O ??????? stopafter= ????????????????? fileselect= ???????????? random ??? sequential ????????????I/O ????? xfersizes= ???????????????????????????xfersizes=(8k,30,128k,7 0) ?????????? 8KB ??? I/O ? 30% ? 128KB ??? I/O ? 70% ??????I/O ?????? operation= ??????????????? read / write / open / close / create / delete / mkdir / rmdir / setattr / getattr ?????????????????? read ????????????????? write ??????????????I/O ????? threads= ???????

???????????????????????????8KB ???????????????????????? 1000 ??? 8 ??????????????????????????????????????

fsd=fsd1,anchor=/mnt/test01/,files=10,sizes=1g fwd=fwd1,fsd=fsd1,fileio=sequential,stopafter=1000 ,fileselect=sequential,xfersizes=8k,operation=writ e,threads=8
  • rd ???
????? rd ???????rd ? Run Definition ?????Raw I/O ???? rd ???????????????????????rd ???????? fwd ?????????????????

rd=rd1 ???????? fwd ? rd ??????????????????????? I/O ????????????????????? fwdrate= ??????????? max ?????????????????? I/O ?????????format=yes ?????????????????????????????????????????????????? ?????????????????????????????????????????????????? ?????????????????????????????????????????????????? elapsed= ????????????????????????????????? 30 ?????????????????????????????????????????????????i nterval= ????????????????????????????Solaris ? Linux ?? openflags= ?????? open() ??????????????????????????? o_sync, o_dsync, o_rsync ??????????? Vdb/OpenFlags.java ????????????????????????????? O_DSYNC ??????????????????????

forsizes= ??????????????????????????????????????forsizes=(12 8k,2m,1g) ??????????? 128KB ???????????????? 2MB ????????????? 1GB ??????????????????forfiles= ?????????????????????????????????forfiles=(300,500 ,1000) ?????? 300 ???????????500 ???????????1000 ????????????????????

??????????????????????????????????????? vdbench ????? -f ?????????????????????????????????????????

fsd=fsd1,anchor=/mnt/test01/,files=10,sizes=1g fwd=fwd1,fsd=fsd1,fileio=sequential,stopafter=1000 ,fileselect=sequential,xfersizes=8k,operation=writ e,threads=8 rd=rd1,fwd=fwd1,fwdrate=max,elapsed=600,interval=1 0,format=yes
  • ??
?????????????????????????????????????????????????? ??????????????????????

  • ???
Raw ????????? sd, wd, rd ???????????????????????????????? fsd, fwd, rd ?????????????????????????????????????????????????? ??????????????????????????????????

vdbench ??????

?????????????Raw ??????????????? SD(Storage Definition), WD(Workload Definition), RD(Run Definition) ????????????????????????? FSD(Filesystem Storage Definition), FWD(Filesystem Workload Definition), RD(Run Definition) ??????????????????????? General Parameters ? HD(Host Definition) ???????????????????????????????????????????????

????????

Raw ???

??????????????? summary.html ?????????????????????????

14:55:08.000 Starting RD=rd1; I/O rate: Uncontrolled MAX; Elapsed=60; For loops: xfersize=32768 Aug 04, 2009 interval i/o MB/sec bytes read resp resp resp cpu% cpu% rate 1024**2 i/o pct time max stddev sys+usr sys 14:55:18.023 1 9978.40 311.82 32768 0.00 0.798 52.837 1.960 4.5 3.8 14:55:28.011 2 8517.60 266.18 32768 0.00 0.936 26.145 2.138 4.0 3.3 14:55:38.011 3 8661.60 270.68 32768 0.00 0.921 33.382 2.112 4.1 3.3 14:55:48.012 4 7870.60 245.96 32768 0.00 1.013 30.333 2.250 3.6 3.1 14:55:58.010 5 8523.80 266.37 32768 0.00 0.935 87.823 2.317 3.9 3.3 14:56:08.011 6 8365.10 261.41 32768 0.00 0.954 34.503 2.170 3.9 3.2 14:56:08.014 avg_2-6 8387.74 262.12 32768 0.00 0.951 87.823 2.198 3.9 3.2 14:56:09.296 Vdbench execution completed successfully
  • interval ?????????????????????????????????????????????
  • i/o rate ? I/O ??? (IOPS) ???
  • MB/sec ? I/O ?????? (megabytes per second) ???
  • bytes i/o ? I/O ?????????????????? xfersize=32768 ??????????? I/O ???? 32768 bytes ??????
  • read pct ? Read ??????Write Only ??? 0 ??????
  • resp time ???????????????????
  • resp max ?????????????????????????????
  • resp stddev ?????????????????????????
  • cpu% sys+usr ???????? CPU ?????
  • cpu% sys ? OS ????????? CPU ?????
  • avg_2-6 ?????? 2 ??????? 6 ????????????????????????? i/o rate ? MB/sec ?????????????????????
  • ????????????????????????????????????
??????????????

??????????????? summary.html ???????????????????????????? summary.html ??????????????

... 15:15:15.001 Starting RD=rd1; Elapsed=60; fwdrate=max. For loops: None 15:15:25.025 Interval ....Ops..... ...cpu%... ....read.... ...write.... ..mb/sec.. .xfer. ... 15:15:25.025 rate resp total sys rate resp rate resp read write size ... 15:15:25.025 1 10228 0.5 4.6 4.1 0.0 0.0 10228 0.5 0.0 79.9 8192 ... 15:15:35.011 2 3443.1 2.8 1.2 1.0 0.0 0.0 3443.1 2.8 0.0 26.9 8192 ... 15:15:45.013 3 14297 0.6 3.8 3.2 0.0 0.0 14297 0.6 0.0 111.7 8192 ... 15:15:55.009 4 8678.4 0.9 3.5 3.1 0.0 0.0 8678.4 0.9 0.0 67.8 8192 ... 15:16:05.011 5 5030.4 1.0 2.2 2.0 0.0 0.0 5030.4 1.0 0.0 39.3 8192 ... 15:16:15.009 6 3532.8 3.0 1.8 1.6 0.0 0.0 3532.8 3.0 0.0 27.6 8192 ... 15:16:15.012 avg_2-6 6996.5 1.2 2.5 2.2 0.0 0.0 6996.5 1.2 0.0 54.7 8192 ... 15:16:17.044 Vdbench execution completed successfully
  • Interval ????????????????????????????????
  • Ops ? I/O ?????????????Ops ? rate ?????????resp ?????????????
  • cpu% ? CPU ???????????total ? CPU ???? sys ??????????????
  • read ????????????????rate ???? read ???resp ?????????????
  • write ????????????????rate ???? write ???resp ?????????????
  • mb/sec ?????????? (megabytes per second) ???read ??????????write ????????????
  • xfer size ? I/O ?????????? byte ???
  • avg_2-6 ?????? 2 ??????? 6 ????????????????????????? read rate, write rate ? mb/sec ?????????????????????
  • ????????????????????????????????????
????????

Solaris ? Raw ????????

vdbench ???? Solaris ? Raw ????? Sequential Write ???????????????????????????

# cat conf/test01.conf * START "test01.conf" : sequential write - 8KB, 16KB, 24KB, 32KB sd=sd1,lun=/dev/rdsk/c0t2d0s2,size=733468426240 wd=wd1,sd=sd1,rdpct=0,seekpct=0 rd=rd1,wd=wd1,iorate=max,elapsed=180,interval=10,f orxfersize=(8k,16k,24k,32k) * END ???????????????????????????????????????

# ./vdbench -f conf/test01 -o /var/tmp/output+ ... Aug 17, 2009 interval i/o MB/sec bytes read resp resp resp cpu% cpu% rate 1024**2 i/o pct time max stddev sys+usr sys ... 16:43:16.030 avg_2-18 37145.06 290.20 8192 0.00 0.212 5.279 0.049 16.7 14.1 ... 16:46:17.017 avg_2-18 16963.02 265.05 16384 0.00 0.468 85.172 1.356 7.7 6.5 ... 16:49:18.013 avg_2-18 11330.82 265.57 24576 0.00 0.703 72.644 1.928 5.2 4.3 ... 16:52:19.015 avg_2-18 8746.64 273.33 32768 0.00 0.911 102.431 2.199 4.1 3.5 ?????? MB/sec ? avg ????????????????????????????

8KB write 16KB write 24KB write 32KB write ----------------------------------------------------------- 290MB/sec 265.05MB/sec 265.57MB/sec 273.33MB/sec Linux ? Raw ????????

Solaris ???????? Linux ?? Sequential Write ????????????Linux ?? Solaris ? /dev/rdsk/... ?????????????????????????????

# ls /dev/raw ls: /dev/raw: No such file or directory