Software RAID question - Hardware

This is a discussion on Software RAID question - Hardware ; There is an article on Linux.com that has a counter-intuitive result, it claims that RAID1 is faster than RAID0 and significantly faster than a single drive in a multitasking environment. http://www.linux.com/feature/124256 There are a number of possible explanations for this, ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: Software RAID question

  1. Software RAID question

    There is an article on Linux.com that has a counter-intuitive result, it
    claims that RAID1 is faster than RAID0 and significantly faster than a
    single drive in a multitasking environment.

    http://www.linux.com/feature/124256

    There are a number of possible explanations for this, 1) the article is
    wrong. 2) Software RAID1 distributes read requests across both drives so
    that in a multitasking situation the disk subsystem can do twice as many
    read transactions. 3) Software RAID1 makes a simultaneous read request to
    both drives and then uses the first to return. 4) The software is able to
    to determine the head positions on both drives and then makes the read
    request to the drive that has it's head closest to the desired block.

    Can anyone tell me what the RAID software is really doing?


  2. Re: Software RAID question

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    General Schvantzkopf wrote:
    > There is an article on Linux.com that has a counter-intuitive result,
    > it claims that RAID1 is faster than RAID0 and significantly faster
    > than a single drive in a multitasking environment.
    >
    > http://www.linux.com/feature/124256
    >
    > There are a number of possible explanations for this, 1) the article
    > is wrong. 2) Software RAID1 distributes read requests across both
    > drives so that in a multitasking situation the disk subsystem can do
    > twice as many read transactions. 3) Software RAID1 makes a
    > simultaneous read request to both drives and then uses the first to
    > return. 4) The software is able to to determine the head positions on
    > both drives and then makes the read request to the drive that has
    > it's head closest to the desired block.
    >
    > Can anyone tell me what the RAID software is really doing?


    There are a number of faults with that study.

    Firstly, unless the author used different hard disk controllers for the
    then RAID1 should be at best equal to single drive performance. In my
    experience there is a very slight overhead that while is measurable is
    not noticeable in most deployments. I would suggest the source of this
    authors unexplained results are more than likely down to 'sampling
    error'. If the author went through the trouble to be scientific and get
    lots of samples then this type of error would be very unlikely to have
    occurred.

    Secondly, the author uses the the term 'significantly faster' very
    flippantly. Scientifically, something is 'significantly different' if a
    manipulation by an experimenter reliably gives differences in samples
    from the same population. The size of the difference makes NO
    contribution to whether something is 'significantly different'.
    Furthermore, looking at the means between the samples that he presented
    in the graphs (and he probably only did one sample per cohort!) although
    there are probably 'significant' performance differences between the
    means, that ACTUAL difference (the size of the difference) didn't look
    like anything writing home about to me. Ask yourself, if a single drive
    takes 1 minute 43 seconds to do one task, is the fact that a RAID0 setup
    would take 1 minute 35 seconds really a perceptible gain? If that is a
    perceptible gain, is it actually worth it given the cost and other
    disadvantages? (See my final point for disadvantages of RAID0.)

    Thirdly while I could look up the fact that the hard disk he used were
    7200rpm, he didn't test 10000rpm for an objective comparison. There may
    be further differences that are larger or possible smaller.

    Fourthly real world difference, given the type of tasks he set and how
    good modern hard disks are capable of deal with multiple-requests for
    data, even if a desktop machine is laden with background tasks indexing
    and prelinking for example, it is unlikely that most of the normal tasks
    a desktop user does (document composing, checking email, browsing) would
    be impacted.

    Fifthly, proper RAID cards that are worth their salt probably start
    around the price range of 75 and above; the one that came 'free' on
    your motherboard is in a different class! This is NOT the type of RAID
    card we use out in the real world for proper setups!

    Overall RAID0 for desktops is a really, really bad idea in the vast
    majority of circumstances. Minor performance gains are available to
    people usually with double costs when you include a decent RAID card in
    comparison to just buying a big and fast disk in the first place. You
    also draw lots of extra power (another hard disk and the RAID card needs
    power too) and you DOUBLE the risk of losing your data to hard disk failure.

    Don't be tempted by RAID0; buy a large fast hard disk and backup often
    instead. Read articles about fast hard disks or solid state drives
    instead

    - --
    Regards,
    Sheridan Hutchinson
    Sheridan@Shezza.org
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFHlL7GnBrliHqz8aARAi5AAJ48qv5i5CBERWHX148JV/Ikg1SBNgCePBQg
    2ki6QwYDOEDXhOvhqG94jOM=
    =j1x8
    -----END PGP SIGNATURE-----

  3. Re: Software RAID question

    General Schvantzkopf writes:
    >There are a number of possible explanations for this, 1) the article is
    >wrong. 2) Software RAID1 distributes read requests across both drives so
    >that in a multitasking situation the disk subsystem can do twice as many
    >read transactions. 3) Software RAID1 makes a simultaneous read request to
    >both drives and then uses the first to return. 4) The software is able to
    >to determine the head positions on both drives and then makes the read
    >request to the drive that has it's head closest to the desired block.


    2 sounds most plausible to me. 4 and (to a lesser degree) 3 would be
    cool, but I doubt that the md driver does this. For 4 one would
    probably not try to query the current head position; this is probably
    not possible, and even if it is, is not a good idea, because the block
    interface and (with command queuing) the disk interface is
    asynchronous (it may serve other requests before serving the request
    under consideration); instead, one might look at what requests have
    been sent to the n block devices recently, and send the new request to
    the device that has had the closest block request recently; however,
    one has to balance this with load balancing.

    I also note that one can configure the Barcelona-based AMD CPUs
    (Phenoms and quad-core Opterons) to access the two memory channels in
    an unganged fashion as an alternative to the traditional ganged setup.
    In the unganged setup a cache line is read from only one channel,
    which can have the following positive effects: the other channel is
    free to serve a different request, and there are more different open
    pages in the DIMMs (with the ganges setup each pair of DIMMs has the
    same pages open). From what I read, which setup is faster depends on
    the workload, but for many multi-tasking loads the unganged setup is a
    little faster.

    In a similar vein, I note that the new ATI RV670 graphics chips
    (Radeon 3850, 3870) reduces the bits in the memory interface compared
    to the R600 (Radeon 2900) by keeping the same number (8) of memory
    controllers/channels, but making each channel narrower (from 64 to 32
    bits), and hardly loses performance from this.

    - anton
    --
    M. Anton Ertl Some things have to be seen to be believed
    anton@mips.complang.tuwien.ac.at Most things have to be believed to be seen
    http://www.complang.tuwien.ac.at/anton/home.html

  4. Re: Software RAID question

    General Schvantzkopf writes:
    >There is an article on Linux.com that has a counter-intuitive result, it
    >claims that RAID1 is faster than RAID0 and significantly faster than a
    >single drive in a multitasking environment.
    >
    >http://www.linux.com/feature/124256


    Inspired by this, I did a little micro-benchmarking on a machine that
    has both a software RAID0 and a software RAID1 on the same disks (but
    of course, in different regions, so there may be differences coming
    from that). The disks are a WDC WD4000KD-55NSB2 and a ST3400620AS;
    the kernel is the Debian 2.6.18-4-amd64 kernel.

    First benchmark: Create 2 20GB files in parallel, with:

    for i in 1 2; do time dd if=/dev/zero of=$i bs=1024k count=20480 & done

    Results: the numbers produced by dd:

    process 1 process2
    RAID1 869.537s (24.7 MB/s) 935.002s (23.0MB/s)
    RAID0 583.512s (36.8 MB/s) 601.59s (35.7MB/s)

    No surprise here: RAID1 has to write to two disks, so it is slower.

    Next benchmark: Read these files in parallel, with:

    for i in 1 2; do time dd if=$i bs=1024k count=20480|wc -c & done

    Results: the numbers produced by dd:

    process 1 process2
    RAID1 696.336s (30.8MB/s) 703.792s (30.5MB/s)
    RAID0 460.54s (46.6MB/s) 466.837s (46.0MB/s)

    So, even though it's theoretically possible to be as fast with RAID1
    in this case as with RAID0, this does not happen.

    However, things may be different for accesses that are seek-limited
    instead of accesses like these that are bandwidth-limited. It's a
    little more involved to benchmark that, so I won't do that for now.

    Just for comparison, when doing a similar test with a single large
    file, I saw 47MB/s on the RAID1 and 89MB/s on the RAID0.

    - anton
    --
    M. Anton Ertl Some things have to be seen to be believed
    anton@mips.complang.tuwien.ac.at Most things have to be believed to be seen
    http://www.complang.tuwien.ac.at/anton/home.html

  5. Re: Software RAID question

    anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
    >However, things may be different for accesses that are seek-limited
    >instead of accesses like these that are bandwidth-limited. It's a
    >little more involved to benchmark that, so I won't do that for now.


    But I got around to it: I created a 50GB file as follows:

    dd if=/dev/zero of=x bs=1024k count=51200

    Then I produced random accesses from multiple processes to this file,
    e.g. as follows:

    for i in 0 1 2 3 4 5 6 7 8; do /usr/bin/time -f %es gforth random.fs -e "9$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait

    I.e., each process does 100 random accesses within the file. I always
    varied the seed to avoid accessing blocks that had already been
    accessed. Here's the raw data:

    #proc RAID1
    1 1.98s
    2 1.46s 1.53s
    3 2.25s 2.41s 2.58s
    4 2.79s 2.94s 2.96s 2.97s
    5 3.23s 3.60s 3.76s 4.09s 4.15s
    6 3.70s 4.21s 4.43s 4.67s 4.76s 4.77s
    7 4.47s 5.13s 5.51s 5.62s 5.66s 5.74s 5.83s
    8 5.65s 5.82s 5.84s 5.86s 5.98s 6.17s 6.19s 6.21s
    9 6.67s 6.76s 6.84s 6.86s 6.97s 7.00s 7.20s 7.26s 7.43s

    #proc RAID0
    1 1.60s
    2 2.17s 2.21s
    3 2.78s 3.28s 3.32s
    4 3.58s 3.70s 4.03s 4.14s
    5 3.92s 4.30s 4.36s 4.87s 4.92s
    6 4.37s 4.62s 4.99s 5.14s 5.27s 5.34s
    7 5.80s 5.97s 6.35s 6.37s 6.59s 6.79s 6.80s
    8 5.64s 5.86s 6.06s 6.41s 6.63s 6.67s 6.85s 6.86s
    9 6.60s 6.84s 6.98s 7.09s 7.21s 7.29s 7.31s 7.43s 7.48s

    I used the same seeds for both file systems.

    So, it looks like md RAID1 is faster than md RAID0 for seek-limited
    workloads with a few (but more than 1) independent threads/processes.
    It's not clear why this should be the case for such small accesses
    (1K),

    Another relevant hardware detail on this setup: I think that native
    command queuing is only available (and apparently enabled) on only one
    of the two disk drives on the tested system.

    - anton
    --
    M. Anton Ertl Some things have to be seen to be believed
    anton@mips.complang.tuwien.ac.at Most things have to be believed to be seen
    http://www.complang.tuwien.ac.at/anton/home.html

  6. Re: Software RAID question

    On Tue, 22 Jan 2008 20:27:54 +0000, Anton Ertl wrote:

    > anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
    >>However, things may be different for accesses that are seek-limited
    >>instead of accesses like these that are bandwidth-limited. It's a
    >>little more involved to benchmark that, so I won't do that for now.

    >
    > But I got around to it: I created a 50GB file as follows:
    >
    > dd if=/dev/zero of=x bs=1024k count=51200
    >
    > Then I produced random accesses from multiple processes to this file,
    > e.g. as follows:
    >
    > for i in 0 1 2 3 4 5 6 7 8; do /usr/bin/time -f %es gforth random.fs -e
    > "9$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do
    > 52428800 random block drop loop ; foo bye" & done; wait
    >
    > I.e., each process does 100 random accesses within the file. I always
    > varied the seed to avoid accessing blocks that had already been
    > accessed. Here's the raw data:
    >
    > #proc RAID1
    > 1 1.98s
    > 2 1.46s 1.53s
    > 3 2.25s 2.41s 2.58s
    > 4 2.79s 2.94s 2.96s 2.97s
    > 5 3.23s 3.60s 3.76s 4.09s 4.15s 6 3.70s 4.21s 4.43s 4.67s
    > 4.76s 4.77s 7 4.47s 5.13s 5.51s 5.62s 5.66s 5.74s 5.83s 8
    > 5.65s 5.82s 5.84s 5.86s 5.98s 6.17s 6.19s 6.21s 9 6.67s 6.76s
    > 6.84s 6.86s 6.97s 7.00s 7.20s 7.26s 7.43s
    >
    > #proc RAID0
    > 1 1.60s
    > 2 2.17s 2.21s
    > 3 2.78s 3.28s 3.32s
    > 4 3.58s 3.70s 4.03s 4.14s
    > 5 3.92s 4.30s 4.36s 4.87s 4.92s 6 4.37s 4.62s 4.99s 5.14s
    > 5.27s 5.34s 7 5.80s 5.97s 6.35s 6.37s 6.59s 6.79s 6.80s 8
    > 5.64s 5.86s 6.06s 6.41s 6.63s 6.67s 6.85s 6.86s 9 6.60s 6.84s
    > 6.98s 7.09s 7.21s 7.29s 7.31s 7.43s 7.48s
    >
    > I used the same seeds for both file systems.
    >
    > So, it looks like md RAID1 is faster than md RAID0 for seek-limited
    > workloads with a few (but more than 1) independent threads/processes.
    > It's not clear why this should be the case for such small accesses (1K),
    >
    > Another relevant hardware detail on this setup: I think that native
    > command queuing is only available (and apparently enabled) on only one
    > of the two disk drives on the tested system.
    >
    > - anton


    Interesting. Do you want to post your code somewhere so that other people
    can try it.

  7. Re: Software RAID question

    General Schvantzkopf writes:
    >Interesting. Do you want to post your code somewhere so that other people
    >can try it.


    The code is in my posting (you may have to "apt-get install gforth" or
    similar). In order to spare you coming up with the variations
    yourself, here are the commands for all numbers of processes:

    dd if=/dev/zero of=x bs=1024k count=51200

    #now do something to flush the cache; umounting and remounting the
    #file system may work.

    for i in 3; do /usr/bin/time -f %es gforth random.fs -e "200$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
    for i in 6 7; do /usr/bin/time -f %es gforth random.fs -e "200$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
    for i in 0 1 2; do /usr/bin/time -f %es gforth random.fs -e "3$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
    for i in 0 1 2 3; do /usr/bin/time -f %es gforth random.fs -e "4$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
    for i in 0 1 2 3 4; do /usr/bin/time -f %es gforth random.fs -e "5$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
    for i in 0 1 2 3 4 5; do /usr/bin/time -f %es gforth random.fs -e "6$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
    for i in 0 1 2 3 4 5 6; do /usr/bin/time -f %es gforth random.fs -e "7$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
    for i in 0 1 2 3 4 5 6 7; do /usr/bin/time -f %es gforth random.fs -e "8$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait
    for i in 0 1 2 3 4 5 6 7 8; do /usr/bin/time -f %es gforth random.fs -e "9$i seed ! rnd drop use x 52428800 to block-limit : foo 100 0 do 52428800 random block drop loop ; foo bye" & done; wait

    - anton
    --
    M. Anton Ertl Some things have to be seen to be believed
    anton@mips.complang.tuwien.ac.at Most things have to be believed to be seen
    http://www.complang.tuwien.ac.at/anton/home.html

+ Reply to Thread