2,000+ clients - NFS

This is a discussion on 2,000+ clients - NFS ; I have been told that there are serious problems with NFS supporting 2,000+ clients, specifically in the area of volume locking. Now, I know that USED to be the case, but is it really still the case? Note that I ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: 2,000+ clients

  1. 2,000+ clients


    I have been told that there are serious problems with NFS supporting
    2,000+ clients, specifically in the area of volume locking. Now, I
    know that USED to be the case, but is it really still the case?

    Note that I am not talking about performance degradation as seen by
    the client. It is somewhat unreasonable to expect a single NFS
    file server to be able to handle 2,000+ clients, all thrashing it
    to hell and back again at full gigabit Ethernet speed :-)

    Specifically, is the following specification REALLY unreasonable for
    an NFS server in this day and age?

    Ability to handle 2,000+ clients (simultaneous connexions) to
    a single 10 TB file system.

    Ability to deliver 750 MB/sec aggregate to such clients, each of
    which uses gigabit Ethernet.

    An unspecified aggregating switch which may use multiple links,
    of 10 gigabit Ethernet or anything else it likes.


    Regards,
    Nick Maclaren.

  2. Re: 2,000+ clients

    Nick Maclaren wrote:
    > I have been told that there are serious problems with NFS supporting
    > 2,000+ clients, specifically in the area of volume locking. Now, I
    > know that USED to be the case, but is it really still the case?


    On one of our systems, the number of active NFS/TCP connections:

    % netstat -na | grep 2049 | wc -l
    647

    Because an idle connection will be dropped after just a few
    minutes, there is probably 5x as many "mounts".

    This is an 8-way USIII box that is not very busy:

    12:15pm up 7 day(s), 15:49, 47 users, load average: 1.65, 1.94, 2.26

    And it is doing non-NFS stuff as well. So it will handle a lot
    of connections..

    > Specifically, is the following specification REALLY unreasonable for
    > an NFS server in this day and age?
    >
    > Ability to handle 2,000+ clients (simultaneous connexions) to
    > a single 10 TB file system.


    Yes, given a reasonable filesystem (e.g. UFS may be stretched by this).

    > Ability to deliver 750 MB/sec aggregate to such clients, each of
    > which uses gigabit Ethernet.


    The limit will probably be the I/O bus speeds. 750MBps out of
    a PCI-X will not cut it.

    > An unspecified aggregating switch which may use multiple links,
    > of 10 gigabit Ethernet or anything else it likes.


    The classic cheap Opteron box will not, there are few small
    systems that can handle 10G, you will need multiple cores and
    multiple buses. But it is doable.

    -David

  3. Re: 2,000+ clients

    David Robinson wrote:
    > Nick Maclaren wrote:
    >> Specifically, is the following specification REALLY unreasonable for
    >> an NFS server in this day and age?
    >>
    >> Ability to handle 2,000+ clients (simultaneous connexions) to
    >> a single 10 TB file system.


    > Yes, given a reasonable filesystem (e.g. UFS may be stretched by
    > this).


    David - you flipped the sense of Nick's question I think that in
    response to "would it be REALLY undreasonable" you meant to say "no"
    yes?-)

    >> Ability to deliver 750 MB/sec aggregate to such clients, each of
    >> which uses gigabit Ethernet.


    > The limit will probably be the I/O bus speeds. 750MBps out of
    > a PCI-X will not cut it.


    If we ass-u-me MB == 1048576 bytes here, and if I've done my math
    correctly, 750 MBps is ~6.3 billion bits per second. You might
    squeeze that through a 133 MHz PCI-X, but you would not have much
    headroom. In basic "netperf" and the like tests folks have claimed
    7.X gigabits per second with 10G NICs on PCI-X 133 (multiple streams).

    rick jones
    --
    Process shall set you free from the need for rational thought.
    these opinions are mine, all mine; HP might not want them anyway...
    feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

  4. Re: 2,000+ clients

    In article ,
    Rick Jones wrote:
    >
    >>> Ability to deliver 750 MB/sec aggregate to such clients, each of
    >>> which uses gigabit Ethernet.

    >
    >> The limit will probably be the I/O bus speeds. 750MBps out of
    >> a PCI-X will not cut it.

    >
    >If we ass-u-me MB == 1048576 bytes here, and if I've done my math
    >correctly, 750 MBps is ~6.3 billion bits per second. You might
    >squeeze that through a 133 MHz PCI-X, but you would not have much
    >headroom. In basic "netperf" and the like tests folks have claimed
    >7.X gigabits per second with 10G NICs on PCI-X 133 (multiple streams).


    Yes. But nowhere did I say that it was limited to a single such
    connexion! If we leave out the matter of the filesystem and number
    of spindles, the bottleneck is in the server-switch connexion. And
    there really isn't any reason not to use a SMP server with multiple
    PCI-X slots and multiple connexions to 'the switch' (which might be
    multiple switches connected in parallel).


    Regards,
    Nick Maclaren.

+ Reply to Thread