Re: Building a dedicated NAS box - Hardware

This is a discussion on Re: Building a dedicated NAS box - Hardware ; In article , Kevin Snodgrass wrote: > Jack Snodgrass wrote: > > Hi cousin! :-) > > > not sure what comp.os.linux.* group this question should go to... > > > > I want to build a dedicated, gigabit NAS ...

+ Reply to Thread
Results 1 to 6 of 6

Thread: Re: Building a dedicated NAS box

  1. Re: Building a dedicated NAS box

    In article ,
    Kevin Snodgrass wrote:
    > Jack Snodgrass wrote:
    >
    > Hi cousin! :-)
    >
    > > not sure what comp.os.linux.* group this question should go to...
    > >
    > > I want to build a dedicated, gigabit NAS box for my home network.
    > > I know I can go out and buy one, but I want to build one myself.
    > > I don't have to have the fastest.... I just want fast and cheap.
    > >
    > > I'm thinking that I need a
    > > motherboard, cpu, memory
    > > gigabit lan card ( or two ) that support jumbo frames
    > > hard drives
    > > plus a few more ods and ends.
    > >
    > > I've done some inital tests and it seems like a gigabit
    > > attached device is almost as fast as a local drive. The
    > > disk i/o delays are longer than the network transfer
    > > speed ( when using gigabit ). I would have expected that
    > > to be the other way around, but it's not.

    >
    > gigabit ethernet ~= 100MByte/sec. A typical hard drive can't do that.


    Well, how much traffic does it take to saturate gigabit ethernet, 35%
    (WAG)? That's 350 Mb/s, which is about 44 MB/s. A bog-standard ATAPI
    drive will do that easily.

    --
    -eben QebWenE01R@vTerYizUonI.nOetP royalty.no-ip.org:81

    And we never failed to fail / It was the easiest thing to do -- CSN


  2. Re: Building a dedicated NAS box

    Hactar wrote:

    > In article ,
    > Kevin Snodgrass wrote:


    >> gigabit ethernet ~= 100MByte/sec. A typical hard drive can't do that.

    >
    > Well, how much traffic does it take to saturate gigabit ethernet, 35%
    > (WAG)? That's 350 Mb/s, which is about 44 MB/s. A bog-standard ATAPI
    > drive will do that easily.
    >


    Hi,

    Not necessarily... Point to point switched ethernet is far more efficient
    than the bus topology used with hubs or coax of yonder days. There is no
    opportunity for collisions between the host and switch and a decent switch
    will queue packets from multiple sources destined for one host - so
    efficiency is higher that you suggest.

    Notable problems in practice are:

    1) Host bus bandwidth

    2) Network filesystem used

    3) Disks

    4) NIC

    It is quite possible to get 44MB/sec out of a PATA/SATA disk *for sequential
    reads/writes at the block device level*. Once you start doing random seeks
    and adding the overhead of a filesystem, you will no longer get that.

    My old server managed to copy a gig file within the same XFS filesystem at a
    grand total of 23MB/sec - and that was a fairly optimised operation, and
    the test was a bit duff as I didn't use fsync or o_direct so it gains some
    speed advantage by using the cache. This is a Seagate Baracuda 7200 PATA
    disk.

    Last "proper" NFS server I built which was designed for raw speed managed 2
    gigabits/sec over bonded links before the twin Xeon CPUs topped out
    servicing NFS and NIC interrupts. Jumbo frames weren't an option, but the
    rest of the box was optimised as much as possible. That server had 10GB of
    RAM so that nearly everything commonly wanted could be served from cache.
    Anything required from the disk came from a fast hardware raid array over 2
    gigabit fibre channel. Opteron would almost certainly have done better.

    It's not actually that easy to serve network filesystems at gig speeds and
    the OP is right to query their hardware requirements.

    Raid 5 will help, especially if you have a reasonable number of disks, say 4
    minimum. Striping would help even more, without the benefit of redundancy.

    In my own tests, with a decent CPU and bus, linux software raid (2.6
    dm-mapper) performs very well - in fact better than most of the cheap
    hardware RAID cards and better than lower end, but otherwise respectable,
    external SATA-SCSI RAID units, though you do lose features (eg decent
    online array expansion and handling of a failing disk).

    Make sure the disks support native command queuing and the host uses it.
    Mobo with a decent architecture and a decent NIC and as much RAM as
    possible. Tune the system.

    Cheers

    Tim

  3. Re: Building a dedicated NAS box

    On 2007-01-07, Hactar wrote:
    >
    > Well, how much traffic does it take to saturate gigabit ethernet, 35%
    > (WAG)? That's 350 Mb/s, which is about 44 MB/s. A bog-standard ATAPI
    > drive will do that easily.


    Erm, no. I've seen >90MB/s using NFS over gigabit.

    --
    Joshua Baker-LePain
    Department of Biomedical Engineering
    Duke University

  4. Re: Building a dedicated NAS box

    Joshua Baker-LePain wrote:

    > On 2007-01-07, Hactar wrote:
    >>
    >> Well, how much traffic does it take to saturate gigabit ethernet, 35%
    >> (WAG)? That's 350 Mb/s, which is about 44 MB/s. A bog-standard ATAPI
    >> drive will do that easily.

    >
    > Erm, no. I've seen >90MB/s using NFS over gigabit.
    >


    You can get almost 100 MB/s on a single 1000TX link (not a complete
    network). For a fully switch based network and other decent traffic
    getting over 80 MB/s is rare.

    And what, pray tell, was the complete configuration of the end devices?
    Mobo, mobo interfaces, disks, disk interfaces, LVM and RAID implementations
    (if any)? I find it uncommon (with the well known case of some
    bioinformatics and genome data sets) to be able to feed Gigabit networks
    that fast for very long.

    --
    JosephKK
    Gegen dummheit kampfen die Gotter Selbst, vergebens.**
    --Schiller

  5. Re: Building a dedicated NAS box

    On 2007-01-11, joseph2k wrote:
    > Joshua Baker-LePain wrote:
    >
    >> On 2007-01-07, Hactar wrote:
    >>>
    >>> Well, how much traffic does it take to saturate gigabit ethernet, 35%
    >>> (WAG)? That's 350 Mb/s, which is about 44 MB/s. A bog-standard ATAPI
    >>> drive will do that easily.

    >>
    >> Erm, no. I've seen >90MB/s using NFS over gigabit.
    >>

    >
    > You can get almost 100 MB/s on a single 1000TX link (not a complete
    > network). For a fully switch based network and other decent traffic
    > getting over 80 MB/s is rare.
    >
    > And what, pray tell, was the complete configuration of the end devices?
    > Mobo, mobo interfaces, disks, disk interfaces, LVM and RAID implementations
    > (if any)? I find it uncommon (with the well known case of some
    > bioinformatics and genome data sets) to be able to feed Gigabit networks
    > that fast for very long.


    Server:
    Dual Opteron 246s on Supermicro H8DAE, 24 300GB 7200RPM SATA drives attached
    to 2 3ware 9550SX boards. 3wares set to RAID5 w/ a hot spare, and a software
    RAID0 stripe across the 2 hardware arrays. OS=CentOS-4, FS=ext3.

    Clients:
    Dual Opteron 270s on Supermicro H8DAR.

    Network:
    Server and clients attached to SMC8748L2 switch dedicated to the cluster.
    Everybody using MTU=9000.

    --
    Joshua Baker-LePain
    Department of Biomedical Engineering
    Duke University

  6. Re: Building a dedicated NAS box

    Joshua Baker-LePain wrote:

    > Server:
    > Dual Opteron 246s on Supermicro H8DAE, 24 300GB 7200RPM SATA drives
    > attached
    > to 2 3ware 9550SX boards. 3wares set to RAID5 w/ a hot spare, and a
    > software
    > RAID0 stripe across the 2 hardware arrays. OS=CentOS-4, FS=ext3.
    >
    > Clients:
    > Dual Opteron 270s on Supermicro H8DAR.
    >
    > Network:
    > Server and clients attached to SMC8748L2 switch dedicated to the cluster.
    > Everybody using MTU=9000.
    >


    Pretty nice

    From my experiences with dual-Xeon, depending on your RAM (for caching) and
    possibly provided that the 3ware cards were on the best choice of the 2
    PCI buses, I reckon you could saturate 2-3 gigabit/sec bonded connection on
    your system.

    Lucky you!

    Cheers

    Tim

+ Reply to Thread