Seeking recommendation for fast GigEthernet card - Hardware

This is a discussion on Seeking recommendation for fast GigEthernet card - Hardware ; I've a couple of projects where network performance is an issue: a NAS and a Router. Both are actually clusters, but I'm pretty sure that that's beside the point (though I mention it Just In Case {8^). The NAS is ...

+ Reply to Thread
Results 1 to 8 of 8

Thread: Seeking recommendation for fast GigEthernet card

  1. Seeking recommendation for fast GigEthernet card


    I've a couple of projects where network performance is an issue: a NAS
    and a Router. Both are actually clusters, but I'm pretty sure that
    that's beside the point (though I mention it Just In Case {8^).

    The NAS is is actually a "head", speaking NFS on the front and iSCSI on
    the back. The router is the usual IP device.

    Everything works using the built-in GigE interfaces. But I'm thinking
    that I can squeeze more performance out of my boxes if I push more of the
    processing out of the system and onto the network interface cards.
    So...any suggestions?

    The one absolute requirement in both cases is that I not lose the ability
    to "speak" 802.1q to a switch. Having multiple ports would be a nice
    option, though I can always use multiple cards if necessary.

    The distribution is CentOS5/RHEL5.

    Thanks...Andrew

  2. Re: Seeking recommendation for fast GigEthernet card

    On Sun, 17 Feb 2008 22:48:49 +0000, Andrew Gideon wrote:

    > I've a couple of projects where network performance is an issue: a NAS
    > and a Router. Both are actually clusters, but I'm pretty sure that
    > that's beside the point (though I mention it Just In Case {8^).
    >
    > The NAS is is actually a "head", speaking NFS on the front and iSCSI on
    > the back. The router is the usual IP device.
    >
    > Everything works using the built-in GigE interfaces. But I'm thinking
    > that I can squeeze more performance out of my boxes if I push more of
    > the processing out of the system and onto the network interface cards.
    > So...any suggestions?
    >
    > The one absolute requirement in both cases is that I not lose the
    > ability to "speak" 802.1q to a switch. Having multiple ports would be a
    > nice option, though I can always use multiple cards if necessary.
    >
    > The distribution is CentOS5/RHEL5.
    >
    > Thanks...Andrew


    What's the CPU utilization on those boxes? If you are getting near
    gigabit speeds and the CPUs aren't saturated a smarter gigabit card isn't
    going to help you. If you a really want more bandwidth out of the box
    then get a 10G card. However that will only help you if you aren't disk
    limited.

  3. Re: Seeking recommendation for fast GigEthernet card

    On Sun, 17 Feb 2008 17:06:29 -0600, General Schvantzkopf wrote:

    >
    >
    > What's the CPU utilization on those boxes?


    That's a legitimate question, but I'm not pushing any production through
    these yet so I don't have a real answer. I'm just queuing up my
    "response" in case this does become an issue. Also, since I'm planning
    to buy new boxes for these, I want to be sure to be buying boxes with the
    "right" bus type for the cards I may be purchasing.

    > If you are getting near
    > gigabit speeds and the CPUs aren't saturated a smarter gigabit card
    > isn't going to help you. If you a really want more bandwidth out of the
    > box then get a 10G card.


    I have to admit: I'd not thought of that. I cannot envision hitting a
    real bandwidth limit on the Router cluster in the near future, but that
    may be a very good idea for the NAS cluster.

    Any recommended 10GigE cards?

    Thanks...

    Andrew

  4. Re: Seeking recommendation for fast GigEthernet card

    Andrew Gideon writes:

    > On Sun, 17 Feb 2008 17:06:29 -0600, General Schvantzkopf wrote:
    >
    >>
    >>
    >> What's the CPU utilization on those boxes?

    >
    > That's a legitimate question, but I'm not pushing any production through
    > these yet so I don't have a real answer. I'm just queuing up my
    > "response" in case this does become an issue. Also, since I'm planning
    > to buy new boxes for these, I want to be sure to be buying boxes with the
    > "right" bus type for the cards I may be purchasing.


    I'd go for a PCI express based system. As for the NICs, Intel is
    generally a safe bet.

    --
    Måns Rullgård
    mans@mansr.com

  5. Re: Seeking recommendation for fast GigEthernet card

    On Sun, 17 Feb 2008 23:39:23 +0000, Andrew Gideon wrote:

    > On Sun, 17 Feb 2008 17:06:29 -0600, General Schvantzkopf wrote:
    >
    >
    >>
    >> What's the CPU utilization on those boxes?

    >
    > That's a legitimate question, but I'm not pushing any production through
    > these yet so I don't have a real answer. I'm just queuing up my
    > "response" in case this does become an issue. Also, since I'm planning
    > to buy new boxes for these, I want to be sure to be buying boxes with
    > the "right" bus type for the cards I may be purchasing.
    >
    >> If you are getting near
    >> gigabit speeds and the CPUs aren't saturated a smarter gigabit card
    >> isn't going to help you. If you a really want more bandwidth out of the
    >> box then get a 10G card.

    >
    > I have to admit: I'd not thought of that. I cannot envision hitting a
    > real bandwidth limit on the Router cluster in the near future, but that
    > may be a very good idea for the NAS cluster.
    >
    > Any recommended 10GigE cards?
    >
    > Thanks...
    >
    > Andrew


    I can't help you with the 10G cards, I haven't tried any. As for 1G, the
    MACs in Nvidia chipsets have IP offload engines so you can get IP
    acceleration even in a desktop system, Intel MACs also have acceleration.
    However a NAS box doesn't use much horsepower, any Core2 will have way
    more cycles than it needs so even a crappy 1G card should be able to run
    near wire speeds.


  6. Re: Seeking recommendation for fast GigEthernet card

    Andrew Gideon writes:
    >
    >I've a couple of projects where network performance is an issue: a NAS
    >and a Router. Both are actually clusters, but I'm pretty sure that
    >that's beside the point (though I mention it Just In Case {8^).
    >
    >The NAS is is actually a "head", speaking NFS on the front and iSCSI on
    >the back. The router is the usual IP device.
    >
    >Everything works using the built-in GigE interfaces. But I'm thinking
    >that I can squeeze more performance out of my boxes if I push more of the
    >processing out of the system and onto the network interface cards.
    >So...any suggestions?


    Some experimental data:

    The setup: The NFS server is a 2.66GHz Pentium 4 with an onboard Intel
    e1000 interface; the data is a directory containing 354112 KB of
    files. I first warmed up the NFS server's cache by reading this data
    on a machine that I did not use later in the test. Then I read the
    data on various machines with:

    time tar cf - |cat >/dev/null

    Running this on a 3GHz Xeon 5160 machine with an Intel e1000
    interface:

    real 0m5.163s
    user 0m0.044s
    sys 0m0.364s

    A 2GHz Opteron 246 machine with a Tigon3 interface:

    real 0m5.226s
    user 0m0.102s
    sys 0m1.693s

    An Opteron 270 machine running at 1GHz (clocked down) with a Tigon3 interface:

    real 0m4.959s
    user 0m0.080s
    sys 0m1.760s

    A 2.2GHz Athlon 64 X2 4400+ machine with an Nvidia (forcedeth) interface:

    real 0m4.956s
    user 0m0.040s
    sys 0m1.196s

    An Athlon 64 X2 4600+ machine running at 1GHz (clocked down) with an
    Nvidia (forcedeth) interface:

    real 0m5.012s
    user 0m0.076s
    sys 0m2.180s

    All these machines run a 2.6.18 kernel, except the Opteron 246
    (2.6.10) and the NFS server (2.6.13).

    Looks like the e1000 produces the least CPU load, but none of these
    cards produce so much CPU load that you need to worry about it if the
    machine is a dedicated NFS server.

    I also watched the NFS server with top during one of these jobs, and
    the load stayed low.

    Note also that having all the data cached on the NFS server is a
    worst-case setting for CPU load; more typically the server will wait
    much of the time for disk seeks to complete.

    >The one absolute requirement in both cases is that I not lose the ability
    >to "speak" 802.1q to a switch.


    I cannot help you there. I don't know what 802.1q is.

    Followups set to comp.os.linux.hardware.

    - anton
    --
    M. Anton Ertl Some things have to be seen to be believed
    anton@mips.complang.tuwien.ac.at Most things have to be believed to be seen
    http://www.complang.tuwien.ac.at/anton/home.html

  7. Re: Seeking recommendation for fast GigEthernet card

    Andrew Gideon wrote:

    >
    > I've a couple of projects where network performance is an issue: a NAS
    > and a Router. Both are actually clusters, but I'm pretty sure that
    > that's beside the point (though I mention it Just In Case {8^).
    >
    > The NAS is is actually a "head", speaking NFS on the front and iSCSI on
    > the back. The router is the usual IP device.
    >
    > Everything works using the built-in GigE interfaces. But I'm thinking
    > that I can squeeze more performance out of my boxes if I push more of the
    > processing out of the system and onto the network interface cards.
    > So...any suggestions?
    >
    > The one absolute requirement in both cases is that I not lose the ability
    > to "speak" 802.1q to a switch. Having multiple ports would be a nice
    > option, though I can always use multiple cards if necessary.
    >
    > The distribution is CentOS5/RHEL5.
    >
    > Thanks...Andrew


    This is Linux, right? :') I'd find myself some mid-priced dual or quad port
    1g nics and switches that support jumboframes and bond them... Whay more
    affordable that going 10g unless ofcourse your budget is really fat to
    begin with.



    --

    Jerry McBride (jmcbride@mail-on.us)

  8. Re: Seeking recommendation for fast GigEthernet card

    On Mon, 18 Feb 2008 10:09:34 +0000, Anton Ertl wrote:

    > I cannot help you there. I don't know what 802.1q is.


    Just an FYI: This is support for VLAN trunking. It means that the device
    in question can participate in multiple LANs (ie. "broadcast domains")
    without either multiple NICs or routing.

    Very convenient. For example, we used to route ridiculous amounts of
    traffic merely in doing backups. The backup server now has [virtual]
    ports on every LAN, and no backup traffic is routed.

    Linux has had support for 802.1q since at least RH9 (though that may have
    required a patch; I don't recall the details).

    - Andrew

+ Reply to Thread