Rackmount RAID With Consistent 20 MB/sec+ Average Read and Write Speeds? - Storage

This is a discussion on Rackmount RAID With Consistent 20 MB/sec+ Average Read and Write Speeds? - Storage ; What low end fibre channel rackmount RAID arrays will give me a consistent 20 MB / second average read and write speed with a reasonable number of 15K RPM SCSI drives in a RAID 5 or RAID 6 configuration? I've ...

+ Reply to Thread
Results 1 to 5 of 5

Thread: Rackmount RAID With Consistent 20 MB/sec+ Average Read and Write Speeds?

  1. Rackmount RAID With Consistent 20 MB/sec+ Average Read and Write Speeds?

    What low end fibre channel rackmount RAID arrays will give me a consistent
    20 MB / second average read and write speed with a reasonable number of 15K
    RPM SCSI drives in a RAID 5 or RAID 6 configuration?

    I've been using Compaq RA4x000 arrays as a cheap disk source for
    applications that do not require very high disk performance. I had
    assumed the RA4x000 RAID systems would bottleneck on the 1 Gbit fibre
    channel interface, which is far more throughput than I could get out even
    six drives in a RAID 0 array. Well, I was wrong. I measured performance
    today on a Proliant 6400R with 4 GB of memory, copying from a RAID 5 array
    of five 15K rpm SCSI drives to a RAID 5 array with six 15K rpm SCSI drives.
    Each array is located on a separate RA4x00 locally attached by a separate
    Compaq FC 64-bit PCI card. I am getting an absolutely miserable 2.5 MB per
    second average write performance at the destination RA4x00 array.

    How is the above result possible? Each drive should pull data at minimum 3
    MB per second, and five 15K drives in a RAID 5 should give me at minimum 10
    MB/second read and write performance. I don't have bottlenecks in system
    memory, on the PCI bus, on the fibre channel bus. I just don't see where
    the 2.5MB/sec could be coming from unless it's hard coded into the array
    itself. I was measuring performance on the "physicaldrive" in Windows
    performance monitor, read and write, on each drive defined by the separate
    RA4x000 arrays.

    With my commodity home-user SATA arrays I am getting 30 MB/second read
    performance, over gigabit ethernet at that. It just shouldn't be that
    hard to find a business focused RAID array that can give consistent 20
    MB/second or faster average read and write speeds? What are my options?

    --
    Will



  2. Re: Rackmount RAID With Consistent 20 MB/sec+ Average Read and WriteSpeeds?

    Will wrote:
    > What low end fibre channel rackmount RAID arrays will give me a consistent
    > 20 MB / second average read and write speed with a reasonable number of 15K
    > RPM SCSI drives in a RAID 5 or RAID 6 configuration?
    >
    > I've been using Compaq RA4x000 arrays as a cheap disk source for
    > applications that do not require very high disk performance. I had
    > assumed the RA4x000 RAID systems would bottleneck on the 1 Gbit fibre
    > channel interface, which is far more throughput than I could get out even
    > six drives in a RAID 0 array. Well, I was wrong. I measured performance
    > today on a Proliant 6400R with 4 GB of memory, copying from a RAID 5 array
    > of five 15K rpm SCSI drives to a RAID 5 array with six 15K rpm SCSI drives.
    > Each array is located on a separate RA4x00 locally attached by a separate
    > Compaq FC 64-bit PCI card. I am getting an absolutely miserable 2.5 MB per
    > second average write performance at the destination RA4x00 array.
    >
    > How is the above result possible? Each drive should pull data at minimum 3
    > MB per second, and five 15K drives in a RAID 5 should give me at minimum 10
    > MB/second read and write performance. I don't have bottlenecks in system
    > memory, on the PCI bus, on the fibre channel bus. I just don't see where
    > the 2.5MB/sec could be coming from unless it's hard coded into the array
    > itself. I was measuring performance on the "physicaldrive" in Windows
    > performance monitor, read and write, on each drive defined by the separate
    > RA4x000 arrays.
    >
    > With my commodity home-user SATA arrays I am getting 30 MB/second read
    > performance, over gigabit ethernet at that. It just shouldn't be that
    > hard to find a business focused RAID array that can give consistent 20
    > MB/second or faster average read and write speeds? What are my options?


    Well, it could be that you know what you're talking about and just
    didn't think it necessary to provide all the details. But just in case:

    The theoretical worst-case write bandwidth of a RAID-5 array of 15k rpm
    drives is something like 50 kilobytes per second if you're performing
    single-sector (512-byte) updates: it takes an average of about 6 ms. to
    randomly position to and read in the target sector (and its associated
    parity on a different disk, which we'll assume can be accessed in
    parallel) and another 4 ms. to rotate around again to update the sector
    and its associated parity, which means about 10 ms. for each 512 bytes
    transferred - Q.E.D. If your array is accumulating such updates in a
    non-volatile write-back cache it can leverage the disks' queue
    optimization to at least double that performance (to over 100 KB/sec).
    If you were using 4KB transfers rather than 512-byte transfers this
    would bring worst-case throughput (with the aid of the non-volatile
    write-back cache) up to a princely 1 MB/sec or so.

    So it's possible that the problem is not with your array but with the
    way you're using it - e.g., copying lots of very small files, perhaps.
    If so, and that's the way you need to use the array, the only way you'll
    get higher speeds is to go to solid-state disks (or perhaps you could
    move to Solaris and use its new ZFS file system, which might do well on
    this kind of workload because it can aggregate small writes into large
    ones, much as a log-structured file system does).

    On the other hand, if you're transferring large contiguous chunks of
    data and still seeing this level of performance, then something is wrong
    with your array: fix it, and it'll fix your problem.

    - bill

  3. Re: Rackmount RAID With Consistent 20 MB/sec+ Average Read and Write Speeds?

    "Bill Todd" wrote in message
    news:36OdneA6NJx_hp_bnZ2dnUVZ_vCknZ2d@metrocastcab levision.com...
    > On the other hand, if you're transferring large contiguous chunks of data
    > and still seeing this level of performance, then something is wrong with
    > your array: fix it, and it'll fix your problem.


    The performance of 2.5 MB/sec average write speed was seen while reading and
    writing a single 250MB file, and the file was all contiguous on a 72 GB
    logical drive that had about 40 GB unused. I examined the file's
    allocation units on the drive with a file defrag application to verify that
    everything was contiguous.

    So it kind of does look like something basic is broken - possibly the array
    just is not very good. I did try copying in both directions from and to
    the same array, so it's not a single bad piece of hardware.

    --
    Will



  4. Re: Rackmount RAID With Consistent 20 MB/sec+ Average Read and WriteSpeeds?

    Will wrote:
    > "Bill Todd" wrote in message
    > news:36OdneA6NJx_hp_bnZ2dnUVZ_vCknZ2d@metrocastcab levision.com...
    >> On the other hand, if you're transferring large contiguous chunks of data
    >> and still seeing this level of performance, then something is wrong with
    >> your array: fix it, and it'll fix your problem.

    >
    > The performance of 2.5 MB/sec average write speed was seen while reading and
    > writing a single 250MB file, and the file was all contiguous on a 72 GB
    > logical drive that had about 40 GB unused. I examined the file's
    > allocation units on the drive with a file defrag application to verify that
    > everything was contiguous.
    >
    > So it kind of does look like something basic is broken - possibly the array
    > just is not very good. I did try copying in both directions from and to
    > the same array, so it's not a single bad piece of hardware.


    Hmmm. You don't say what OS you're using. Most recent versions of
    Windows tend to write large contiguous files in 64 KB chunks - but even
    if your array had its write-back cache disabled, and even if Windows
    weren't queuing up requests in advance (such that they would be combined
    at the disk - though that could get thwarted if the RAID stripe were
    also 64 KB per stripe segment), it should still get something more like
    8 MB/sec throughput under those conditions (64 KB every 8 ms. or so,
    even if the write requests didn't align with the RAID stripe segments
    such that each write spanned two disks).

    If the array and OS were cooperating as they should be doing, you ought
    to be able to write large contiguous files (which should be being
    streamed out to the array without waiting for it to finish each request
    before queuing the next) at something close to N-1 times 40 MB/sec
    (where N is the number of disks in the RAID stripe) - or in your case
    around 100 MB/sec as limited by the FC link. So something isn't right.

    - bill

  5. Re: Rackmount RAID With Consistent 20 MB/sec+ Average Read and Write Speeds?

    In article ,
    "Will" wrote:

    > What low end fibre channel rackmount RAID arrays will give me a consistent
    > 20 MB / second average read and write speed with a reasonable number of 15K
    > RPM SCSI drives in a RAID 5 or RAID 6 configuration?
    >
    > I've been using Compaq RA4x000 arrays as a cheap disk source for
    > applications that do not require very high disk performance. I had
    > assumed the RA4x000 RAID systems would bottleneck on the 1 Gbit fibre
    > channel interface, which is far more throughput than I could get out even
    > six drives in a RAID 0 array. Well, I was wrong. I measured performance
    > today on a Proliant 6400R with 4 GB of memory, copying from a RAID 5 array
    > of five 15K rpm SCSI drives to a RAID 5 array with six 15K rpm SCSI drives.
    > Each array is located on a separate RA4x00 locally attached by a separate
    > Compaq FC 64-bit PCI card. I am getting an absolutely miserable 2.5 MB per
    > second average write performance at the destination RA4x00 array.
    >
    > How is the above result possible? Each drive should pull data at minimum 3
    > MB per second, and five 15K drives in a RAID 5 should give me at minimum 10
    > MB/second read and write performance. I don't have bottlenecks in system
    > memory, on the PCI bus, on the fibre channel bus. I just don't see where
    > the 2.5MB/sec could be coming from unless it's hard coded into the array
    > itself. I was measuring performance on the "physicaldrive" in Windows
    > performance monitor, read and write, on each drive defined by the separate
    > RA4x000 arrays.
    >
    > With my commodity home-user SATA arrays I am getting 30 MB/second read
    > performance, over gigabit ethernet at that. It just shouldn't be that
    > hard to find a business focused RAID array that can give consistent 20
    > MB/second or faster average read and write speeds? What are my options?


    I've done a lot of write performance testing on SATA drives lately, and
    750 GB Seagates consistently get 70-80 megabytes per second and that's
    on their own. A 4-disk RAID 6 of these drives is similar, maybe a tad
    quicker.

    I'm bypassing its page cache (Linux) and using an 800 MB file which
    overpowers the RAID card's own cache, but if I weren't bypassing the
    page cache, the numbers would be definitely be better. It'd have no
    trouble saturating a gigabit link.

    Also, in my other setup, I have a failover cluster, which consists of
    two systems each with their own SCSI to SATA based RAID 6 and I'm doing
    network RAID 1 between them. That averages about 60 megabytes per second
    to take data on one system and write it to both RAIDs.

+ Reply to Thread