Question regarding NFS performance - NFS

This is a discussion on Question regarding NFS performance - NFS ; Hi, I set up an eight machine cluster with their /home directory mounted from the /home of an nfs. However, the runtime for my programs, which do a lot of file I/O, has doubled with writing on the nfs server, ...

+ Reply to Thread
Results 1 to 8 of 8

Thread: Question regarding NFS performance

  1. Question regarding NFS performance

    Hi,

    I set up an eight machine cluster with their /home directory mounted
    from the /home of an nfs. However, the runtime for my programs, which
    do a lot of file I/O, has doubled with writing on the nfs server, as
    compared to writing on the local machine. I was expecting an overhead,
    but not really over 100%. Any way I can enchance performance?

    The exports file on the nfs server reads as:

    /home *.Lui(rw,async,no_root_squash)

    The relevant entry in the fstab file reads as:
    172.16.70.30:/home /home nfs
    rw,async,noatime,rsize=8192,wsize=8192,remount 0 0

    I set the clients to noatime and increased rsize/wsize to double their
    default values, hoping to get a performance improvement. But no such
    luck. its still over a 100% increase in run time.

    Anyone got any comments about what I might be doing wrong?

    Thanks

    Lui Ultz

  2. Re: Question regarding NFS performance

    NFS normally writes synchronously, instead of deferring writes as disk
    channels typically do, meaning applications with frequent writes take a
    heavy penalty.

    This is a design choice in the original NFS v1/v2 way back around 1988, made
    for safety considerations, and NFS v3 allows asynchronous writes, which your
    use of 'async' normally triggers. I would check whether your mounts actually
    use it by tracing the protocol: negotiation might fail and revert to sync
    writes.

    "Lui Ultz" a écrit dans le message de
    news:36c7d993.0407262246.241ed3f@posting.google.co m...
    > Hi,
    >
    > I set up an eight machine cluster with their /home directory mounted
    > from the /home of an nfs. However, the runtime for my programs, which
    > do a lot of file I/O, has doubled with writing on the nfs server, as
    > compared to writing on the local machine. I was expecting an overhead,
    > but not really over 100%. Any way I can enchance performance?
    >
    > The exports file on the nfs server reads as:
    >
    > /home *.Lui(rw,async,no_root_squash)
    >
    > The relevant entry in the fstab file reads as:
    > 172.16.70.30:/home /home nfs
    > rw,async,noatime,rsize=8192,wsize=8192,remount 0 0
    >
    > I set the clients to noatime and increased rsize/wsize to double their
    > default values, hoping to get a performance improvement. But no such
    > luck. its still over a 100% increase in run time.
    >
    > Anyone got any comments about what I might be doing wrong?
    >
    > Thanks
    >
    > Lui Ultz




  3. Re: Question regarding NFS performance

    What is the type of local storage, and what is the network link
    between your systems when you are using NFS?

    rick jones
    --
    a wide gulf separates "what if" from "if only"
    these opinions are mine, all mine; HP might not want them anyway...
    feel free to post, OR email to raj in cup.hp.com but NOT BOTH...

  4. Re: Question regarding NFS performance

    Hi Frederick, Rick,

    Thanx for the replies.

    So, the async mode doesn't really enforce asynchronous writes?
    bugger.. But how can I check this? I didn't get the part about
    'tracing the protocol'.

    Regarding the network, its a Fast Ethernet (100Mbps) switched
    environment. The disk storage is a single Hard-drive (a 80GB Western
    Digital drive).

    Actually, my experiments, which are for a term project focus on
    parallelization of Genetic Algorithms to VLSI cell placement. But if
    the NFS writes are going to increase my runtime by over a 100%, I'm
    pretty sure that'd mess up any notion of speedup I'd get through
    parallelization.

    I'd really like to know how others have their disk storage
    architecture for clusters. Thanx

    _Lui_

  5. Re: Question regarding NFS performance

    Lui Ultz wrote:
    > So, the async mode doesn't really enforce asynchronous writes?
    > bugger.. But how can I check this? I didn't get the part about
    > 'tracing the protocol'.


    That would mean using tcpdump or ethereal to watch the requests going
    across the network, and looking at the decoded messages to see what
    flavor of write they happen to be.

    > Regarding the network, its a Fast Ethernet (100Mbps) switched
    > environment. The disk storage is a single Hard-drive (a 80GB Western
    > Digital drive).


    OK, well that means you have at most a 10 MB/s network connection. We
    still don't know what flavor of disc is in use - unless Western
    Digital makes only one flavor (ATA, SCSI, etc) of 80 GB drive, in
    which case I show my ignorance

    What is your best local disc I/O rate to that drive?

    > Actually, my experiments, which are for a term project focus on
    > parallelization of Genetic Algorithms to VLSI cell placement. But if
    > the NFS writes are going to increase my runtime by over a 100%, I'm
    > pretty sure that'd mess up any notion of speedup I'd get through
    > parallelization.


    Although it would make an interesting observation and could help serve
    to demonstrate again to folks that simply increasing one resource in a
    "system" by N% doesn't necessarily mean the whole thing will run N%
    faster... Various metaphors come to mind - peeling the onion,
    squeezing the ballon etc etc.

    rick jones
    --
    denial, anger, bargaining, depression, acceptance, rebirth...
    where do you want to be today?
    these opinions are mine, all mine; HP might not want them anyway...
    feel free to post, OR email to raj in cup.hp.com but NOT BOTH...

  6. Re: Question regarding NFS performance

    AFAIK, WD makes three 80 Gb ATA drives: 800BB : PATA100/2MB cache, 800JB :
    PATA100/8MB cache, 800JD: SATA160/8MB cache. They do not seem to manufacture
    SCSI drives these days.

    To see if NFSv3 with async writes is used, you can check in the protocol
    dump for COMMIT RPC calls: these are only used is async mode is actually
    used. v1, v2 and v3 sync only used WRITE RPC calls, since all write are
    supposed to be synchronous to disk.


    "Rick Jones" a écrit dans le message de
    news:W2TNc.6754$aN3.101@news.cpqcorp.net...
    > Lui Ultz wrote:
    > > So, the async mode doesn't really enforce asynchronous writes?
    > > bugger.. But how can I check this? I didn't get the part about
    > > 'tracing the protocol'.

    >
    > That would mean using tcpdump or ethereal to watch the requests going
    > across the network, and looking at the decoded messages to see what
    > flavor of write they happen to be.
    >
    > > Regarding the network, its a Fast Ethernet (100Mbps) switched
    > > environment. The disk storage is a single Hard-drive (a 80GB Western
    > > Digital drive).

    >
    > OK, well that means you have at most a 10 MB/s network connection. We
    > still don't know what flavor of disc is in use - unless Western
    > Digital makes only one flavor (ATA, SCSI, etc) of 80 GB drive, in
    > which case I show my ignorance
    >
    > What is your best local disc I/O rate to that drive?
    >
    > > Actually, my experiments, which are for a term project focus on
    > > parallelization of Genetic Algorithms to VLSI cell placement. But if
    > > the NFS writes are going to increase my runtime by over a 100%, I'm
    > > pretty sure that'd mess up any notion of speedup I'd get through
    > > parallelization.

    >
    > Although it would make an interesting observation and could help serve
    > to demonstrate again to folks that simply increasing one resource in a
    > "system" by N% doesn't necessarily mean the whole thing will run N%
    > faster... Various metaphors come to mind - peeling the onion,
    > squeezing the ballon etc etc.
    >
    > rick jones
    > --
    > denial, anger, bargaining, depression, acceptance, rebirth...
    > where do you want to be today?
    > these opinions are mine, all mine; HP might not want them anyway...
    > feel free to post, OR email to raj in cup.hp.com but NOT BOTH...




  7. Re: Question regarding NFS performance

    Frederic G. MARAND wrote:
    > AFAIK, WD makes three 80 Gb ATA drives: 800BB : PATA100/2MB cache,
    > 800JB : PATA100/8MB cache, 800JD: SATA160/8MB cache. They do not
    > seem to manufacture SCSI drives these days.


    For those of us unversed in the ways of ATA drives, what does that
    translate to in rough handwaving terms for throughput to a mech? Is
    it much more than 10 MB/s?

    rick jones
    --
    The glass is neither half-empty nor half-full. The glass has a leak.
    The real question is "Can it be patched?"
    these opinions are mine, all mine; HP might not want them anyway...
    feel free to post, OR email to raj in cup.hp.com but NOT BOTH...

  8. Re: Question regarding NFS performance

    Rick Jones wrote:

    > Frederic G. MARAND wrote:
    > > AFAIK, WD makes three 80 Gb ATA drives: 800BB : PATA100/2MB cache,
    > > 800JB : PATA100/8MB cache, 800JD: SATA160/8MB cache. They do not
    > > seem to manufacture SCSI drives these days.

    >
    > For those of us unversed in the ways of ATA drives, what does that
    > translate to in rough handwaving terms for throughput to a mech? Is
    > it much more than 10 MB/s?
    >
    > rick jones


    The ATA100 is rated for 100MB/s, but that of course is maximum peak
    transfer. PATA is parallel ATA and SATA is the serial ATA which is
    rated at 150MB/s. Actual transfer of data is still limited by many
    other hardware bottleneck, such as track seeking, other programs and
    drivers on the computer, etc.



+ Reply to Thread