NFS v4 and pNFS - NFS

This is a discussion on NFS v4 and pNFS - NFS ; Nick Maclaren wrote: > In article , > Rick Jones writes: > |> > |> Ah, so "frankengrams" > > Well, back in the days when I was dealing with FTP transfers between > different systems (MVS, MacOS, Unix etc.), ...

+ Reply to Thread
Page 3 of 3 FirstFirst 1 2 3
Results 41 to 50 of 50

Thread: NFS v4 and pNFS

  1. Re: NFS v4 and pNFS


    Nick Maclaren wrote:
    > In article ,
    > Rick Jones writes:
    > |>
    > |> Ah, so "frankengrams"
    >
    > Well, back in the days when I was dealing with FTP transfers between
    > different systems (MVS, MacOS, Unix etc.), I got quite a lot of
    > "frankenfiles". I never did track down exactly why, but the TCP
    > layer was clearly involved, given that the transfer code in FTP itself
    > would have a major job achieving that effect, and the blocks were not
    > network-sized. Most of the time, it was zeroed blocks, swapped blocks
    > and duplicated blocks.
    >
    > That was one of the main reasons that I wrote a portable, decent
    > quality, checksum program :-)
    >
    > I haven't seen it as commonly since, so I suspect it was because the
    > implementations had some seriously substandard error recovery, but
    > I have seen such corruption on a fair number of Unix-Unix transfers
    > since, albeit rarely. I am not convinced that TCP is as reliable as
    > many people claim, but can't say whether the problem is in the design,
    > the Berkeley reference implementation, the operating systems, POSIX,
    > the C standard (sic) or what.
    >
    > My experience is that such corruption is actually LESS common with
    > NFS over UDP, but the issue is such that I don't read anything into
    > that. It might be merely that I have not tracked down such events
    > with NFS.


    If you are comparing NFS/UDP to ftp, then it is easy to explain NFS/UDP
    might have fewer corruptions. NFS transfers with multiple RPCs,
    each of which has to have XDR decode arguments and results, with
    control info like file handles and offsets. Ftp is basically one big
    RPC,
    with no offset or file id information coming with each TCP segment.
    So NFS/RPC can detect corruptions that ftp won't.

    >
    >
    > Regards,
    > Nick Maclaren.



  2. Re: NFS v4 and pNFS


    In article <1151553436.019984.201320@d56g2000cwd.googlegroups. com>,
    "Mike Eisler" writes:
    |>
    |> If you are comparing NFS/UDP to ftp, then it is easy to explain NFS/UDP
    |> might have fewer corruptions. NFS transfers with multiple RPCs,
    |> each of which has to have XDR decode arguments and results, with
    |> control info like file handles and offsets. Ftp is basically one big
    |> RPC, with no offset or file id information coming with each TCP segment.
    |> So NFS/RPC can detect corruptions that ftp won't.

    As are rsh and ssh, where I have also seen this - they take just the raw
    TCP stream. However, you missed two points:

    Firstly, many of the ones I saw were NOT corruptions that TCP failed
    to notice, but ones INTRODUCED by TCP. I never got a simultaneous snoop
    and corruption, so I can say what it did, but some could not have been
    produced by any realistic failure in the transport or IP layers.

    Secondly, some of the corruptions that I saw were ones that might well
    get through the NFS layer as well, if NFS were modified to take advantages
    of TCP's 'reliable delivery'. In particular, delayed block replication
    isn't good news.

    As I say, I don't have a clue of the cause, but am 99% sure that the
    cause is within TCP as currently implemented. The big problem here is
    that there are almost NO real TCP/IP experts, at the level of really
    understanding how the error recovery works, and what its failure modes
    might be. Checksums be damned - that is the easy bit - the killer is
    the retry and rebuild logic, timeouts, system problems and all.


    Regards,
    Nick Maclaren.

  3. Re: NFS v4 and pNFS


    Nick Maclaren wrote:
    > In article <1151553436.019984.201320@d56g2000cwd.googlegroups. com>,
    > "Mike Eisler" writes:
    > |>
    > |> If you are comparing NFS/UDP to ftp, then it is easy to explain NFS/UDP
    > |> might have fewer corruptions. NFS transfers with multiple RPCs,
    > |> each of which has to have XDR decode arguments and results, with
    > |> control info like file handles and offsets. Ftp is basically one big
    > |> RPC, with no offset or file id information coming with each TCP segment.
    > |> So NFS/RPC can detect corruptions that ftp won't.
    >
    > As are rsh and ssh, where I have also seen this - they take just the raw
    > TCP stream. However, you missed two points:
    >
    > Firstly, many of the ones I saw were NOT corruptions that TCP failed
    > to notice, but ones INTRODUCED by TCP. I never got a simultaneous snoop
    > and corruption, so I can say what it did, but some could not have been
    > produced by any realistic failure in the transport or IP layers.


    So you are claiming a bad TCP implementation introduced corruptions?
    >
    > Secondly, some of the corruptions that I saw were ones that might well
    > get through the NFS layer as well, if NFS were modified to take advantages
    > of TCP's 'reliable delivery'. In particular, delayed block replication
    > isn't good news.
    >
    > As I say, I don't have a clue of the cause, but am 99% sure that the


    Given you don't a have a clue of the cause, you can't be 99% sure.

    UDP's reliance on IP fragment re-assembly is demonstrably fragile.

    If you don't have any confidence in TCP, then you always fall back to
    DecNet, SNA, or Novell. :-)

    > cause is within TCP as currently implemented. The big problem here is
    > that there are almost NO real TCP/IP experts, at the level of really


    That's an amazing statement, but that's why I keep coming back to
    comp.protocols.nfs ... to be amazed by your writings. :-) You should
    ask to speak at an IETF plenary to make this claim.

    > understanding how the error recovery works, and what its failure modes
    > might be. Checksums be damned - that is the easy bit - the killer is
    > the retry and rebuild logic, timeouts, system problems and all.
    >
    >
    > Regards,
    > Nick Maclaren.



  4. Re: NFS v4 and pNFS


    In article <1151585031.205530.55860@m73g2000cwd.googlegroups.c om>,
    "Mike Eisler" writes:
    |>
    |> So you are claiming a bad TCP implementation introduced corruptions?

    No. Don't attempt to put words into my mouth. That is only one of
    several possible causes.

    |> Given you don't a have a clue of the cause, you can't be 99% sure.

    Try understanding what I post. I have reasons for my certainty about
    the area in which the problem lies, and I even mentioned two of them.

    |> > cause is within TCP as currently implemented. The big problem here is
    |> > that there are almost NO real TCP/IP experts, at the level of really
    |>
    |> That's an amazing statement, but that's why I keep coming back to
    |> comp.protocols.nfs ... to be amazed by your writings. :-) You should
    |> ask to speak at an IETF plenary to make this claim.

    Perhaps my standards are higher than yours.


    Regards,
    Nick Maclaren.

  5. Re: NFS v4 and pNFS


    Nick Maclaren wrote:
    > In article <1151585031.205530.55860@m73g2000cwd.googlegroups.c om>,
    > "Mike Eisler" writes:
    > |>
    > |> So you are claiming a bad TCP implementation introduced corruptions?
    >
    > No. Don't attempt to put words into my mouth. That is only one of
    > several possible causes.


    Well given your unsupported assertions, I'm left only to guess.

    >
    > |> Given you don't a have a clue of the cause, you can't be 99% sure.
    >
    > Try understanding what I post. I have reasons for my certainty about
    > the area in which the problem lies, and I even mentioned two of them.


    You explained nothing.

    >
    > |> > cause is within TCP as currently implemented. The big problem here is
    > |> > that there are almost NO real TCP/IP experts, at the level of really
    > |>
    > |> That's an amazing statement, but that's why I keep coming back to
    > |> comp.protocols.nfs ... to be amazed by your writings. :-) You should
    > |> ask to speak at an IETF plenary to make this claim.
    >
    > Perhaps my standards are higher than yours.


    Then why are you slumming around here?

    >
    >
    > Regards,
    > Nick Maclaren.



  6. Re: NFS v4 and pNFS


    In article <1151589950.636420.157920@p79g2000cwp.googlegroups. com>,
    "Mike Eisler" writes:
    |>
    |> Well given your unsupported assertions, I'm left only to guess.

    You could, of course, have read what I posted, where I listed 5 possible
    causes.

    |> You explained nothing.

    People who have learnt a bit more from experience usually find my remarks
    about the size of the corrupted blocks and types of corruption self-
    explanatory.

    |> > Perhaps my standards are higher than yours.
    |>
    |> Then why are you slumming around here?

    Because it amuses me? Because you aren't actually the owner of this
    group?


    Regards,
    Nick Maclaren.

  7. Re: NFS v4 and pNFS


    Nick Maclaren wrote:
    > In article <1151527622.210001.307920@j72g2000cwa.googlegroups. com>,
    > "Mike Eisler" writes:
    > |>
    > |> > |> > Bugs in the Cassini driver, showing apparent failure of NFS v4 and
    > |> > |> > not v3 - probably due to the TCP/UDP differences and not the NFS version.
    > |> > |>
    > |> > Anyway, if the situation is that using TCP causes your systems to have
    > |> > an unacceptably low MTBF, and the failure symptoms are such that they
    > |> > need an operator to log into the system controller to power cycle the
    > |> > domain, and that using UDP cures that, would you continue to use TCP?
    > |>
    > |> No, I'd switch to a client that works.
    >
    > Fascinating. Do you know what I was referring to by the Cassini driver,
    > and what it implies?


    I imagine it refers to a driver for an ethernet controller. As a short
    term
    solution, switching back to UDP is one thing. But to embrace it as a
    long term
    solution because the driver and/or the controller is buggy seems very
    wrong to me, given the demonstrable issues with using large UDP
    datagrams
    that have to be fragmented. I googled and found some noise that
    Cassini had a TOE mode. If this is true, is it on, and are results
    better when the TOE is disabled?

    I'm still waiting for your explanation that Kerberized NFSv4 is vastly
    different than
    Kerberized NFSv3.


  8. Re: NFS v4 and pNFS


    Nick Maclaren wrote:
    > In article <1151589950.636420.157920@p79g2000cwp.googlegroups. com>,
    > "Mike Eisler" writes:
    > |>
    > |> Well given your unsupported assertions, I'm left only to guess.
    >
    > You could, of course, have read what I posted, where I listed 5 possible
    > causes.


    I saw nothing that could be regarded as a cause.

    >
    > |> You explained nothing.
    >
    > People who have learnt a bit more from experience usually find my remarks
    > about the size of the corrupted blocks and types of corruption self-
    > explanatory.


    They should speak up then; we could use someone to interpret your
    writings.

    >
    > |> > Perhaps my standards are higher than yours.
    > |>
    > |> Then why are you slumming around here?
    >
    > Because it amuses me? Because you aren't actually the owner of this
    > group?


    True you are amusing.

    NFSv4 vs. NFSv3 Kerberos?


  9. Re: NFS v4 and pNFS

    Mike Eisler wrote:
    > Rick Jones wrote:
    >> Ah, so "frankengrams"
    >>
    >> > http://www.ietf.org/internet-drafts/...harmful-00.txt

    >>
    >> I get "The page you are looking for cannot be found." trying to follow
    >> the link above.
    >>
    >> > http://nfsworld.blogspot.com/2006/03...thon-2006.html

    >>
    >> Since I nitpit for fun and food:


    > You are both nit picky and apparently helpless.


    I tried 01 but didn't go farther than that.

    > i-ds get updated all the time. Try:



    > http://www.ietf.org/internet-drafts/...harmful-02.txt


    > and no promises that it won't be -03 or higher by the time
    > you click your browser (remember the days when we didn't have
    > browsers and we'd actually look for things?) ... this is
    > after all i-d season a week or two before an IETF meeting.


    >> Meanwhile, the client is busy doing other NFS/UDP things, and the
    >> datagram identifier gets re-used. The identifier is just 16 bits;
    >> assuming 32 kilo byte writes, giga bit/sec transmission speeds,
    >> then 2^16 * 32 * 1024 * 8 / 1000^3 is just 17.2 seconds. If the


    >> Um, _only_ if the UDP checksum happens to match... Now, it may still
    >> not be as "safe" as TCP, but it is quite a bit different from what is
    >> written above.


    > So you think it is 17.2 seconds * 2^16 between corruptions? The
    > Internet Draft says that the experiment with 10 TB dataset there were
    > 121 corruptions.
    > So at 1 giga bit/sec
    > 10 * 1024 ^ 4 * 8 / 1000^3 / 121 = 726 seconds between corruptions.


    I was not questioning any of the math, I was pointing-out that you
    need to include that the UDP checksum must not catch the frankengram.

    > Anyway whether your result is right, my result is right, or the i-d
    > is right, UDP is gonna really suck when 10 giga/sec ethernet is
    > used.


    It certainly has that potential.

    >> IIRC the Linux folks have added some IP fragment reassembly
    >> heuristics intended to reduce albeit not elimitate Frankengrams -
    >> if more than N other fragments have arrived without a particular
    >> datagram successfully reassembling, it is ass-u-me-d that one or
    >> more fragments of that datagram were lost and the whole thing is
    >> tossed.


    > Nothing like beating a dead horse, and UDP is a dead as they come.
    > Energy that would be better invested in making NFS, sftp, iscsi work
    > better over TCP.


    Liberal in what they accept I suppose.

    rick jones
    --
    oxymoron n, commuter in a gas-guzzling luxury SUV with an American flag
    these opinions are mine, all mine; HP might not want them anyway...
    feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

  10. Re: NFS v4 and pNFS


    In article ,
    Rick Jones writes:
    |>
    |> > Nothing like beating a dead horse, and UDP is a dead as they come.
    |> > Energy that would be better invested in making NFS, sftp, iscsi work
    |> > better over TCP.
    |>
    |> Liberal in what they accept I suppose.

    True. And there are requirements for which UDP is a perfect match,
    but almost all are concerned solely about having a low latency for
    short packets, so fragmentation isn't really an issue - except when
    the UDP is transferred over ATM, I suppose.

    One could imagine implementing and tuning NFS implemented directly on
    top of ATM - in one's worst nightmares :-)


    Regards,
    Nick Maclaren.

+ Reply to Thread
Page 3 of 3 FirstFirst 1 2 3