Passing floating point over TCPIP. - TCP-IP

This is a discussion on Passing floating point over TCPIP. - TCP-IP ; This has been talked about before, and I've read some posts going back to 1997. However, this is 2007, and the FP will pass between two wintel machine, two linux ia32/64 machines or wintel and linux ia32/64. Is there a ...

+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 31

Thread: Passing floating point over TCPIP.

  1. Passing floating point over TCPIP.

    This has been talked about before, and I've read some posts going back
    to 1997. However, this is 2007, and the FP will pass between two
    wintel machine, two linux ia32/64 machines or wintel and linux
    ia32/64.

    Is there a need to worry about formats between these two any more, if
    they adhere to IEEE floating point specifications?

    What about endianess? I suspect I have to worry here, maybe, but if
    again, the machines are Intel architecture, is it necessary to watch
    this?


  2. Re: Passing floating point over TCPIP.

    On Jul 5, 1:19 am, rsprawls wrote:

    > This has been talked about before, and I've read some posts going back
    > to 1997. However, this is 2007, and the FP will pass between two
    > wintel machine, two linux ia32/64 machines or wintel and linux
    > ia32/64.
    >
    > Is there a need to worry about formats between these two any more, if
    > they adhere to IEEE floating point specifications?
    >
    > What about endianess? I suspect I have to worry here, maybe, but if
    > again, the machines are Intel architecture, is it necessary to watch
    > this?


    You are approaching this problem backwards. Simply *specify* the
    format you are going to use and then *implement* the format on each
    machine. That will work no matter what, and as a bonus, if it ever
    doesn't work on some machine, you can go to the specification to see
    if that machine is to blame.

    DS


  3. Re: Passing floating point over TCPIP.

    On Thu, 05 Jul 2007 01:32:23 -0700, David Schwartz wrote:
    > On Jul 5, 1:19 am, rsprawls wrote:

    ....
    >> Is there a need to worry about formats between these two any more, if
    >> they adhere to IEEE floating point specifications?
    >>
    >> What about endianess? I suspect I have to worry here, maybe, but if
    >> again, the machines are Intel architecture, is it necessary to watch
    >> this?

    >
    > You are approaching this problem backwards. Simply *specify* the
    > format you are going to use and then *implement* the format on each
    > machine. That will work no matter what, and as a bonus, if it ever
    > doesn't work on some machine, you can go to the specification to see
    > if that machine is to blame.


    Good advise, and good rationale. As for the format, XDR is one
    popular and widely implemented choice:

    http://en.wikipedia.org/wiki/Externa...Representation

    /Jorgen

    --
    // Jorgen Grahn \X/ snipabacken.dyndns.org> R'lyeh wgah'nagl fhtagn!

  4. Re: Passing floating point over TCPIP.


    "Jorgen Grahn" wrote in message
    news:slrnfape14.52s.grahn+nntp@frailea.sa.invalid. ..
    > On Thu, 05 Jul 2007 01:32:23 -0700, David Schwartz
    > wrote:
    >> On Jul 5, 1:19 am, rsprawls wrote:

    > ...
    >>> Is there a need to worry about formats between these two any more, if
    >>> they adhere to IEEE floating point specifications?
    >>>
    >>> What about endianess? I suspect I have to worry here, maybe, but if
    >>> again, the machines are Intel architecture, is it necessary to watch
    >>> this?

    >>
    >> You are approaching this problem backwards. Simply *specify* the
    >> format you are going to use and then *implement* the format on each
    >> machine.


    By "implement" you mean serialization (yes?). IOW, what the wire protocol
    is. If you are careful, you can have your in-memory representation be the
    protocol data also. (But you'll get a lot of flak for it in the language
    groups).

    >> That will work no matter what, and as a bonus, if it ever
    >> doesn't work on some machine, you can go to the specification to see
    >> if that machine is to blame.

    >
    > Good advise, and good rationale. As for the format, XDR is one
    > popular and widely implemented choice:
    >
    > http://en.wikipedia.org/wiki/Externa...Representation


    Which is big-endian like TCP/IP over the wire (but the endpoints are
    probably on ethernet which is little-endian). Since most machines are
    little-endian, the choice of big-endian serialization that was made is
    unfortunate. On an upbeat note though, however you encode your protocol data
    inside of IP packets is up to you, but it will probably be little endian
    because the source and destination machines are mostly likely little-endian.
    Then you make only the big-endian machines pay the price for being different
    by doing marshalling on those.

    John


  5. Re: Passing floating point over TCPIP.

    On Tue, 31 Jul 2007 03:07:50 -0500, JohnQ wrote:
    >
    > "Jorgen Grahn" wrote in message
    > news:slrnfape14.52s.grahn+nntp@frailea.sa.invalid. ..
    >> On Thu, 05 Jul 2007 01:32:23 -0700, David Schwartz
    >> wrote:
    >>> On Jul 5, 1:19 am, rsprawls wrote:

    >> ...
    >>>> Is there a need to worry about formats between these two any more, if
    >>>> they adhere to IEEE floating point specifications?
    >>>>
    >>>> What about endianess? I suspect I have to worry here, maybe, but if
    >>>> again, the machines are Intel architecture, is it necessary to watch
    >>>> this?
    >>>
    >>> You are approaching this problem backwards. Simply *specify* the
    >>> format you are going to use and then *implement* the format on each
    >>> machine.

    >
    > By "implement" you mean serialization (yes?). IOW, what the wire protocol
    > is.


    That's surely what he means -- although "serialization" isn't the term
    everybody uses (IMHO, it focuses the mind to much on the details of
    shuffling internal data structures, rather than on designing a stable,
    published language -- something which is often very useful to do).

    If you are careful, you can have your in-memory representation be the
    > protocol data also. (But you'll get a lot of flak for it in the language
    > groups).
    >
    >>> That will work no matter what, and as a bonus, if it ever
    >>> doesn't work on some machine, you can go to the specification to see
    >>> if that machine is to blame.

    >>
    >> Good advise, and good rationale. As for the format, XDR is one
    >> popular and widely implemented choice:
    >>
    >> http://en.wikipedia.org/wiki/Externa...Representation

    >
    > Which is big-endian like TCP/IP over the wire (but the endpoints are
    > probably on ethernet which is little-endian).


    Uh, ethernet shuffles blocks of octets. It doesn't care about the
    endianness inside that block.

    > Since most machines are
    > little-endian, the choice of big-endian serialization that was made is
    > unfortunate.


    Well, at the time, most networked machines /were/ big-endian. And they
    might still have been if Microsoft hadn't insisted on supporting
    Windows only on little-endian architectures.

    I doubt this matters much -- I am told htons() and friends compile to
    a single inline instruction on x86. (Big-endian machines have less
    incentive to provide such a byte-swapping instruction, so sticking to
    little-endian is likely to hurt them more than it helps x86.)

    > On an upbeat note though, however you encode your protocol data
    > inside of IP packets is up to you, but it will probably be little endian
    > because the source and destination machines are mostly likely little-endian.
    > Then you make only the big-endian machines pay the price for being different
    > by doing marshalling on those.


    But he cannot use standard, optimized and portable tools like htons()
    and the XDR to implement this. If I were him, I'd only do that if I
    needed extreme speed, and measurements showed that byte-swapping on
    x86 was a bottleneck.

    /Jorgen

    --
    // Jorgen Grahn \X/ snipabacken.dyndns.org> R'lyeh wgah'nagl fhtagn!

  6. Re: Passing floating point over TCPIP.


    "Jorgen Grahn" wrote in message
    news:slrnfaudm3.52s.grahn+nntp@frailea.sa.invalid. ..
    > On Tue, 31 Jul 2007 03:07:50 -0500, JohnQ
    > wrote:
    >>
    >> "Jorgen Grahn" wrote in message
    >> news:slrnfape14.52s.grahn+nntp@frailea.sa.invalid. ..
    >>> On Thu, 05 Jul 2007 01:32:23 -0700, David Schwartz
    >>>
    >>> wrote:
    >>>> On Jul 5, 1:19 am, rsprawls wrote:
    >>> ...
    >>>>> Is there a need to worry about formats between these two any more, if
    >>>>> they adhere to IEEE floating point specifications?
    >>>>>
    >>>>> What about endianess? I suspect I have to worry here, maybe, but if
    >>>>> again, the machines are Intel architecture, is it necessary to watch
    >>>>> this?
    >>>>
    >>>> You are approaching this problem backwards. Simply *specify* the
    >>>> format you are going to use and then *implement* the format on each
    >>>> machine.

    >>
    >> By "implement" you mean serialization (yes?). IOW, what the wire protocol
    >> is.

    >
    > That's surely what he means -- although "serialization" isn't the term
    > everybody uses (IMHO, it focuses the mind to much on the details of
    > shuffling internal data structures, rather than on designing a stable,
    > published language -- something which is often very useful to do).


    You meant 'protocol' rather than 'language', yes?

    > If you are careful, you can have your in-memory representation be the
    >> protocol data also. (But you'll get a lot of flak for it in the language
    >> groups).
    >>
    >>>> That will work no matter what, and as a bonus, if it ever
    >>>> doesn't work on some machine, you can go to the specification to see
    >>>> if that machine is to blame.
    >>>
    >>> Good advise, and good rationale. As for the format, XDR is one
    >>> popular and widely implemented choice:
    >>>
    >>> http://en.wikipedia.org/wiki/Externa...Representation

    >>
    >> Which is big-endian like TCP/IP over the wire (but the endpoints are
    >> probably on ethernet which is little-endian).

    >
    > Uh, ethernet shuffles blocks of octets. It doesn't care about the
    > endianness inside that block.


    But the frames must have data that is specified in some endianness right?
    That's the only part of TCP/IP (well and the protocols based upon it) I was
    referring to above.

    >
    >> Since most machines are
    >> little-endian, the choice of big-endian serialization that was made is
    >> unfortunate.

    >
    > Well, at the time, most networked machines /were/ big-endian. And they
    > might still have been if Microsoft hadn't insisted on supporting
    > Windows only on little-endian architectures.
    >
    > I doubt this matters much -- I am told htons() and friends compile to
    > a single inline instruction on x86. (Big-endian machines have less
    > incentive to provide such a byte-swapping instruction, so sticking to
    > little-endian is likely to hurt them more than it helps x86.)


    Well they should change their architectures to little-endian. Do most new
    Sparcs get set up as little-endian machines or big-endian machines?

    >
    >> On an upbeat note though, however you encode your protocol data
    >> inside of IP packets is up to you, but it will probably be little endian
    >> because the source and destination machines are mostly likely
    >> little-endian.
    >> Then you make only the big-endian machines pay the price for being
    >> different
    >> by doing marshalling on those.

    >
    > But he cannot use standard, optimized and portable tools like htons()
    > and the XDR to implement this. If I were him, I'd only do that if I
    > needed extreme speed, and measurements showed that byte-swapping on
    > x86 was a bottleneck.


    And I'd only use something like XDR in very special circumstances. I make
    the choice of a "native" platform and then only everything else gets
    penalized with the extra coding time and performance degradation. Most
    applications don't need to run on every platform and coding them that way is
    less than fun.

    John


  7. Re: Passing floating point over TCPIP.

    "JohnQ" wrote:

    > But the frames must have data that is specified in some endianness
    > right?
    > That's the only part of TCP/IP (well and the protocols based upon it)
    > I
    > was referring to above.


    Yes.

    > Well they should change their architectures to little-endian. Do most
    > new Sparcs get set up as little-endian machines or big-endian
    > machines?


    Big endian byte order over IP. That's the general rule, *not* the
    exception. See RFC 791 Appendix B.

    --------------Quoting--------------
    APPENDIX B: Data Transmission Order

    The order of transmission of the header and data described in this
    document is resolved to the octet level. Whenever a diagram shows a
    group of octets, the order of transmission of those octets is the normal
    order in which they are read in English. For example, in the following
    diagram the octets are transmitted in the order they are numbered.


    0 1 2 3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    | 1 | 2 | 3 | 4 |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    | 5 | 6 | 7 | 8 |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    | 9 | 10 | 11 | 12 |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    Transmission Order of Bytes

    Figure 10.

    Whenever an octet represents a numeric quantity the left most bit in the
    diagram is the high order or most significant bit. That is, the bit
    labeled 0 is the most significant bit. For example, the following
    diagram represents the value 170 (decimal).


    0 1 2 3 4 5 6 7
    +-+-+-+-+-+-+-+-+
    |1 0 1 0 1 0 1 0|
    +-+-+-+-+-+-+-+-+

    Significance of Bits

    Figure 11.

    Similarly, whenever a multi-octet field represents a numeric quantity
    the left most bit of the whole field is the most significant bit. When
    a multi-octet quantity is transmitted the most significant octet is
    transmitted first.
    ----------End quote------------

    Bert


  8. Re: Passing floating point over TCPIP.

    In article <3RZri.508$IE5.415@newssvr17.news.prodigy.net>,
    JohnQ wrote:
    >"Jorgen Grahn" wrote in message
    >news:slrnfaudm3.52s.grahn+nntp@frailea.sa.invalid. ..


    >> That's surely what he means -- although "serialization" isn't the term
    >> everybody uses (IMHO, it focuses the mind to much on the details of
    >> shuffling internal data structures, rather than on designing a stable,
    >> published language -- something which is often very useful to do).


    I think the standard terms, "marshalling" and "unmarshalling," do
    that job better than "serialization."
    You'll also find more instructive web pages if you ask Google about
    the standard terms.


    >>> Which is big-endian like TCP/IP over the wire (but the endpoints are
    >>> probably on ethernet which is little-endian).

    >>
    >> Uh, ethernet shuffles blocks of octets. It doesn't care about the
    >> endianness inside that block.

    >
    >But the frames must have data that is specified in some endianness right?
    >That's the only part of TCP/IP (well and the protocols based upon it) I was
    >referring to above.


    As stated, that is wrong. TCP/IP shuffles streams of octets without
    regard for endianness. Endianness is specified only for parts of TCP/IP
    that are inside the protocol and invisible to applications and the
    application programmer. Yes, there are the reasons why one sometimes
    uses [hn]to[sl]() on the args for some socket calls, but those are
    aspects of the socket interface instead of the TCP/IP protocol.


    >And I'd only use something like XDR in very special circumstances. I make
    >the choice of a "native" platform and then only everything else gets
    >penalized with the extra coding time and performance degradation. Most
    >applications don't need to run on every platform and coding them that way is
    >less than fun.


    Applications don't need to run on "every platform" to need specified
    byte order. Outside the wintel swamp, there is a surprisingly large
    number of big endian systems, which is why people with much experience
    do not have anything good to say about code written with implicit
    or unstated assumptions of byte order (and other issues). The
    assumptions are usually not realized by the person writing the code.

    I wonder if other person has ever used XDR, because the "extra
    coding time and performance degradation" are small. One does not
    write XDR (un)marshalling code manually but feeds a specification
    of the data structures to an XDR compiler.

    Big versus little endian is only one of the issues that competent
    programmers consider. Other issues include structure packing,
    alignment, and floating point format. One can just stuff Intel
    8087 style floating point numbers into the wire, but that might not
    work if the system at the other end of the wire expects some other
    format. For example, even 32-bit Intel floating point hardware
    supports three different formats, single precision, double precision,
    and the 80-bit, not exactly internal format.


    Vernon Schryver vjs@rhyolite.com

  9. Re: Passing floating point over TCPIP.

    On Wed, 1 Aug 2007 06:11:56 -0500, JohnQ wrote:
    >
    > "Jorgen Grahn" wrote in message
    > news:slrnfaudm3.52s.grahn+nntp@frailea.sa.invalid. ..
    >> On Tue, 31 Jul 2007 03:07:50 -0500, JohnQ
    >> wrote:
    >>>
    >>> "Jorgen Grahn" wrote in message
    >>> news:slrnfape14.52s.grahn+nntp@frailea.sa.invalid. ..
    >>>> On Thu, 05 Jul 2007 01:32:23 -0700, David Schwartz
    >>>>
    >>>> wrote:

    ....
    >>>>> You are approaching this problem backwards. Simply *specify* the
    >>>>> format you are going to use and then *implement* the format on each
    >>>>> machine.
    >>>
    >>> By "implement" you mean serialization (yes?). IOW, what the wire protocol
    >>> is.

    >>
    >> That's surely what he means -- although "serialization" isn't the term
    >> everybody uses (IMHO, it focuses the mind to much on the details of
    >> shuffling internal data structures, rather than on designing a stable,
    >> published language -- something which is often very useful to do).

    >
    > You meant 'protocol' rather than 'language', yes?


    Basically, yes. I got sidetracked by vague memories of my CS
    education: language = syntax + semantics, or whatever. It would have
    been simpler to just say 'protocol'.

    /Jorgen

    --
    // Jorgen Grahn \X/ snipabacken.dyndns.org> R'lyeh wgah'nagl fhtagn!

  10. Re: Passing floating point over TCPIP.


    "Vernon Schryver" wrote in message
    news:f8q3k7$jbt$1@calcite.rhyolite.com...
    > In article <3RZri.508$IE5.415@newssvr17.news.prodigy.net>,
    > JohnQ wrote:
    >>"Jorgen Grahn" wrote in message
    >>news:slrnfaudm3.52s.grahn+nntp@frailea.sa.invalid. ..

    >
    >>> That's surely what he means -- although "serialization" isn't the term
    >>> everybody uses (IMHO, it focuses the mind to much on the details of
    >>> shuffling internal data structures, rather than on designing a stable,
    >>> published language -- something which is often very useful to do).

    >
    > I think the standard terms, "marshalling" and "unmarshalling," do
    > that job better than "serialization."
    > You'll also find more instructive web pages if you ask Google about
    > the standard terms.
    >
    >
    >>>> Which is big-endian like TCP/IP over the wire (but the endpoints are
    >>>> probably on ethernet which is little-endian).
    >>>
    >>> Uh, ethernet shuffles blocks of octets. It doesn't care about the
    >>> endianness inside that block.

    >>
    >>But the frames must have data that is specified in some endianness right?
    >>That's the only part of TCP/IP (well and the protocols based upon it) I
    >>was
    >>referring to above.

    >
    > As stated, that is wrong. TCP/IP shuffles streams of octets without
    > regard for endianness. Endianness is specified only for parts of TCP/IP
    > that are inside the protocol and invisible to applications and the
    > application programmer.


    That's exactly what I meant. Having to use htons on addresses etc.

    > Yes, there are the reasons why one sometimes
    > uses [hn]to[sl]() on the args for some socket calls, but those are
    > aspects of the socket interface instead of the TCP/IP protocol.


    Yeah, i forget that sockets is a layer on top of TCP/IP. But no, not
    really. It just makes the raw frames and such easier to work with. The
    frames have specified endian-ness. (Not for the payload though of course).

    >>And I'd only use something like XDR in very special circumstances. I make
    >>the choice of a "native" platform and then only everything else gets
    >>penalized with the extra coding time and performance degradation. Most
    >>applications don't need to run on every platform and coding them that way
    >>is
    >>less than fun.

    >
    > Applications don't need to run on "every platform" to need specified
    > byte order.


    The specified byte order is the one of the native platform at the time of
    development.

    > Outside the wintel swamp, there is a surprisingly large
    > number of big endian systems, which is why people with much experience
    > do not have anything good to say about code written with implicit
    > or unstated assumptions of byte order (and other issues). The
    > assumptions are usually not realized by the person writing the code.


    Which, in a way, is more desireable than never making choices and thereby
    kludging up the code on all platforms rather than just on the non-compliant
    ones. YMMV.

    > I wonder if other person has ever used XDR, because the "extra
    > coding time and performance degradation" are small. One does not
    > write XDR (un)marshalling code manually but feeds a specification
    > of the data structures to an XDR compiler.


    Which are unnecessary extra steps either if your deployment target is a
    single platform or if you specify a "native" platform. I don't see the point
    of penalizing the primary platform with development costs and complexity and
    performance (the last of which is least important by far) if it is not
    necessary. A little forethought goes a long way.

    > Big versus little endian is only one of the issues that competent
    > programmers consider. Other issues include structure packing,
    > alignment,


    That's easily solved and of course must be when choosing to program without
    marshalling everywhere.

    > and floating point format.


    Again, the native platform formats will be specified. Non-primary platforms
    will have to marshall.

    > One can just stuff Intel
    > 8087 style floating point numbers into the wire, but that might not
    > work if the system at the other end of the wire expects some other
    > format. For example, even 32-bit Intel floating point hardware
    > supports three different formats, single precision, double precision,
    > and the 80-bit, not exactly internal format.


    John


  11. Re: Passing floating point over TCPIP.


    "Albert Manfredi" wrote in message
    news:JM3y8v.EA0@news.boeing.com...
    > "JohnQ" wrote:
    >
    >> But the frames must have data that is specified in some endianness right?
    >> That's the only part of TCP/IP (well and the protocols based upon it) I
    >> was referring to above.

    >
    > Yes.
    >
    >> Well they should change their architectures to little-endian. Do most new
    >> Sparcs get set up as little-endian machines or big-endian machines?

    >
    > Big endian byte order over IP. That's the general rule, *not* the
    > exception. See RFC 791 Appendix B.


    That wasn't the question. Obviously I'm well aware that over-the-wire is
    specified to be big-endian. The question I had is whether most of those with
    newer Sparc boxes run them in big-endian mode or little-endian mode.

    John


  12. Re: Passing floating point over TCPIP.

    "Vernon Schryver" wrote:

    > TCP/IP shuffles streams of octets without
    > regard for endianness. Endianness is specified only for parts of
    > TCP/IP
    > that are inside the protocol and invisible to applications and the
    > application programmer. Yes, there are the reasons why one sometimes
    > uses [hn]to[sl]() on the args for some socket calls, but those are
    > aspects of the socket interface instead of the TCP/IP protocol.


    We went over this at great length some moons ago, and I still disagree
    to a large degree.

    RFC 791 Appendix B refers to the IP header bytes and to the IP data
    bytes. And it specifies big endian byte order.

    Yes, the header bytes of higher layer protocols, like TCP or UDP, are
    part of the IP data payload. So Appendix B says that for sure, the
    headers of UDP and TCP must also be in big endian byte order. But the
    payload of those protocols is not exempt. So when carrying HTTP, SMTP,
    RTP, etc., you still expect big endian byte order in those headers. And
    so on.

    Ultimately, nothing exempts the final payload from also being assumed to
    be in big endian byte order as well. RFC 791 does not tyell us when the
    big endian byte order requirement ends. It's just some of us regular
    folk who claim to know.

    You'll note that RFC 2460, which is the equivalent of RFC 791 for IPv6,
    makes no mention of byte order at all. So I brought this up to the IPv6
    working group. The response was, in so many words, that no sane person
    would use anything other than big endian byte order. That is the obvious
    convention always used in IP. Whether the words of Appendix B will be
    incorporated in RFC 2460 in the future is uncertain, but the sentiment
    was anything but uncertain.

    > Applications don't need to run on "every platform" to need specified
    > byte order. Outside the wintel swamp, there is a surprisingly large
    > number of big endian systems, which is why people with much experience
    > do not have anything good to say about code written with implicit
    > or unstated assumptions of byte order (and other issues). The
    > assumptions are usually not realized by the person writing the code.


    I wholeheartedly agree with this part. Which is why this question keeps
    coming up.

    Amazingly, it seems a lot of folk are blind-sided by Appendix B or RFC
    791.

    Bert


  13. Re: Passing floating point over TCPIP.

    In article ,
    Albert Manfredi wrote:

    >You'll note that RFC 2460, which is the equivalent of RFC 791 for IPv6,
    >makes no mention of byte order at all. So I brought this up to the IPv6
    >working group. The response was, in so many words, that no sane person
    >would use anything other than big endian byte order. That is the obvious
    >convention always used in IP. Whether the words of Appendix B will be
    >incorporated in RFC 2460 in the future is uncertain, but the sentiment
    >was anything but uncertain.


    I suspect some poetic license in that report, because the statement
    about byte order sounds extreme to anyone who knows enough about Apollo/HP
    RPC to know that "receiver makes it right" wants little endian data
    between little endian hosts. Isn't the flavor of RPC that Microsoft
    uses closely related to the Apollo/HP RPC?

    See
    http://www.google.com/search?q=dce+r...es+it+right%22


    Vernon Schryver vjs@rhyolite.com

  14. Re: Passing floating point over TCPIP.

    "Albert Manfredi" writes:

    > RFC 791 Appendix B refers to the IP header bytes and to the IP data
    > bytes. And it specifies big endian byte order.


    You're misreading this appendix. It is simply describing the diagrams.
    It is not making any requirement of the data format of the payload.
    How could it? The format of the payload is defined by specification of
    the protocol being carried in the payload.

    > Yes, the header bytes of higher layer protocols, like TCP or UDP, are
    > part of the IP data payload. So Appendix B says that for sure, the
    > headers of UDP and TCP must also be in big endian byte order.


    No, it says if you draw one of these diagrams as used in the IP RFC
    (and, of course, exactly the same diagramming is used in most other
    RFCs), then your diagram should represent the bytes as being
    transmitted in that order. This does not, in fact, relieve your
    specification from explaining whether a two or four byte quantity
    shown in the diagram is loaded big-endian or little-endian before it
    is transmitted. (Fortunately, this isn't really an issue because all
    Internet protocols use big-endian.)

    We should perhaps stop at this point and mention *why* RFC-791 had to
    specify this. One would think it was obvious to any English language
    reader that the bytes would be transmitted left to right. The reason
    this had to be made clear is that *little-endian* hardware
    documentation almost always shows byte layouts with the lowest address
    on the *right* side of the diagram, so bytes transmitted from such a
    diagram would be send "backwards", from right to left.

    > But the
    > payload of those protocols is not exempt. So when carrying HTTP, SMTP,
    > RTP, etc., you still expect big endian byte order in those headers.
    > And so on.


    HTTP and SMTP send text, so endianness is not an issue.

    > You'll note that RFC 2460, which is the equivalent of RFC 791 for
    > IPv6, makes no mention of byte order at all.


    All those many years later, the standard protocol diagramming method
    had been well established, so RFC-2460 didn't have to explain it
    again.

    -don

  15. Re: Passing floating point over TCPIP.

    In article , "Albert Manfredi" writes:
    > "Vernon Schryver" wrote:
    >
    >> TCP/IP shuffles streams of octets without
    >> regard for endianness. Endianness is specified only for parts of
    >> TCP/IP
    >> that are inside the protocol and invisible to applications and the
    >> application programmer. Yes, there are the reasons why one sometimes
    >> uses [hn]to[sl]() on the args for some socket calls, but those are
    >> aspects of the socket interface instead of the TCP/IP protocol.

    >
    > We went over this at great length some moons ago, and I still disagree
    > to a large degree.


    The notion that RFC 791 applies to the byte order of IEEE floating
    point stored in a file on disk because that file might someday be
    passed over the Internet in a binary transfer using FTP is, shall we
    say, bizarre.

    >
    > RFC 791 Appendix B refers to the IP header bytes and to the IP data
    > bytes. And it specifies big endian byte order.
    >
    > Yes, the header bytes of higher layer protocols, like TCP or UDP, are
    > part of the IP data payload. So Appendix B says that for sure, the
    > headers of UDP and TCP must also be in big endian byte order.


    > But the
    > payload of those protocols is not exempt. So when carrying HTTP, SMTP,
    > RTP, etc., you still expect big endian byte order in those headers. And
    > so on.


    An interesting picture in the abstract. But we live in the real world.
    We have an installed base that does things differently.

    > Ultimately, nothing exempts the final payload from also being assumed to
    > be in big endian byte order as well. RFC 791 does not tyell us when the
    > big endian byte order requirement ends. It's just some of us regular
    > folk who claim to know.


    > You'll note that RFC 2460, which is the equivalent of RFC 791 for IPv6,
    > makes no mention of byte order at all. So I brought this up to the IPv6
    > working group. The response was, in so many words, that no sane person
    > would use anything other than big endian byte order. That is the obvious
    > convention always used in IP. Whether the words of Appendix B will be
    > incorporated in RFC 2460 in the future is uncertain, but the sentiment
    > was anything but uncertain.


    As a documentation convention for Internet drafts, standards and RFCs,
    big-endian is universally assumed.

    >> Applications don't need to run on "every platform" to need specified
    >> byte order. Outside the wintel swamp, there is a surprisingly large
    >> number of big endian systems, which is why people with much experience
    >> do not have anything good to say about code written with implicit
    >> or unstated assumptions of byte order (and other issues). The
    >> assumptions are usually not realized by the person writing the code.

    >
    > I wholeheartedly agree with this part. Which is why this question keeps
    > coming up.
    >
    > Amazingly, it seems a lot of folk are blind-sided by Appendix B or RFC
    > 791.


    Amazingly, you think it has force outside its domain of applicability.

  16. Re: Passing floating point over TCPIP.

    On Aug 2, 1:05 pm, don provan wrote:

    > > Yes, the header bytes of higher layer protocols, like TCP or UDP, are
    > > part of the IP data payload. So Appendix B says that for sure, the
    > > headers of UDP and TCP must also be in big endian byte order.

    >
    > No, it says if you draw one of these diagrams as used in the IP RFC
    > (and, of course, exactly the same diagramming is used in most other
    > RFCs), then your diagram should represent the bytes as being
    > transmitted in that order. This does not, in fact, relieve your
    > specification from explaining whether a two or four byte quantity
    > shown in the diagram is loaded big-endian or little-endian before it
    > is transmitted. (Fortunately, this isn't really an issue because all
    > Internet protocols use big-endian.)


    I feel like we're walking on parallel paths here.

    Yes, your last sentence is of course true, *because* RFC 791 Appendix
    B makes it so.

    Let me focus on the exact parts that state this (quoting from RFC 791
    Appendix B):

    "Whenever a diagram shows a group of octets, the order of transmission
    of those octets is the normal order in which they are read in
    English."

    Okay, so far the endianness of numerical quantities is not stated. I
    think this is what you mean in your first sentence.

    But then it says:

    "When a multi-octet quantity is transmitted the most significant octet
    is transmitted first."

    What am I missing here? Doesn't it state concisely that big endian
    byte order is required? And that's why it is NOT necssary in IP to
    specify word lengths. If Appendix B did not exist, then you would be
    correct in stating "This does not, in fact, relieve your specification
    from explaining whether a two or four byte quantity shown in the
    diagram is loaded big-endian or little-endian before it is
    transmitted."

    So in fact, as far as I can tell, you are relieved of having to
    specify word length.

    You will note that RFCs describing upper layer protocols never specify
    this endianness (well, at least none of the ones I checked). Why is
    that? It's because RFC 791 already did.

    Bert


  17. Re: Passing floating point over TCPIP.

    On Aug 2, 1:26 pm, bri...@encompasserve.org wrote:

    > The notion that RFC 791 applies to the byte order of IEEE floating
    > point stored in a file on disk because that file might someday be
    > passed over the Internet in a binary transfer using FTP is, shall we
    > say, bizarre.


    Indeed that would be bizarre. I never said that.

    I simply said that when the floating point or other numerical value is
    transmitted across an IP interface, it is supposed to be transmitted
    in big endian byte order. Put a logic analyzer on the interface and
    that's what you should see.

    How that gets stored in memory and moved around in local buses is of
    no concern of IP.

    However, check again the subject line of tis thread.

    Bert


  18. Re: Passing floating point over TCPIP.

    In article <1186090543.662276.177590@x40g2000prg.googlegroups. com>,
    Albert Manfredi wrote:

    >I simply said that when the floating point or other numerical value is
    >transmitted across an IP interface, it is supposed to be transmitted
    >in big endian byte order. Put a logic analyzer on the interface and
    >that's what you should see.


    Anyone who has actually used a logic analyzer on more than one or
    two "IP interfaces" knows that statement is not quite wrong.
    In most cases, to use a logic analyzer on anything close to an
    "IP interface" you will be looking at link layer packets, such as
    IEEE 802.3 (Ethernet) frames. The IP headers in frames will be
    big-endian, but the TCP or UDP payloads will be in arbitrary, often
    little endian byte order.

    It is more common to use a packet snooper than a logic analyzer to look
    at network packets. Anyone who has used tcpdump, Etherreal (or whatever
    it's now called), or some other packet snooper (and isn't merely a
    marketing professional) knows that plenty of numeric data in IP packets
    is little endian.

    Notice the little endian types in
    http://www.opengroup.org/onlinepubs/9629399/apdxf.htm
    Then recall that a certain large personal computer software vendor has
    shipped a lot of DCE code.


    Vernon Schryver vjs@rhyolite.com

  19. Re: Passing floating point over TCPIP.

    In article <1186090543.662276.177590@x40g2000prg.googlegroups. com>, Albert Manfredi writes:
    > On Aug 2, 1:26 pm, bri...@encompasserve.org wrote:
    >
    >> The notion that RFC 791 applies to the byte order of IEEE floating
    >> point stored in a file on disk because that file might someday be
    >> passed over the Internet in a binary transfer using FTP is, shall we
    >> say, bizarre.

    >
    > Indeed that would be bizarre. I never said that.
    >
    > I simply said that when the floating point or other numerical value is
    > transmitted across an IP interface, it is supposed to be transmitted
    > in big endian byte order. Put a logic analyzer on the interface and
    > that's what you should see.


    No. It's not. If the aforementioned file is stored on disk in
    little-endian format and transferred with binary FTP it goes on
    the wire in a little endian byte order.

    Try it.

  20. Re: Passing floating point over TCPIP.

    Albert Manfredi writes:

    > Yes, your last sentence is of course true, *because* RFC 791 Appendix
    > B makes it so.


    No, it really isn't. "The order of transmission of the header and data
    described *in this document* is resolved to the octet level." The
    entire appendix is limited to the IP protocol itself. Yes, it is true
    that all other internet protocol adopted the same standard, but that's
    because it was universally understood that consistency was far more
    important than any other consideration, so the somewhat arbitrary
    decision to always use big-endian was adopted. It was *not* because
    anyone thought Appendix B of RFC-791 mandated it.

    > "When a multi-octet quantity is transmitted the most significant octet
    > is transmitted first."


    The multi-octet quantities *defined in this document* are transmitted
    most significant octet first. IP is defined such that it has no clue
    whatsoever what multi-octet quantities it is carrying in its payload.
    There was absolutely no intention that this text would apply outside
    the IP specification.

    > What am I missing here? Doesn't it state concisely that big endian
    > byte order is required?


    You're missing that that a protocol specification is limited to
    dictating only its own values.

    > And that's why it is NOT necssary in IP to specify word lengths.


    Now you've lost me. What does IP have to do with "word lengths"? Word
    lengths are characteristics of specific CPUs, so, of course, IP
    couldn't have anything to say about them.

    > You will note that RFCs describing upper layer protocols never specify
    > this endianness (well, at least none of the ones I checked). Why is
    > that? It's because RFC 791 already did.


    Well, you got me there, but I still claim that was because the endian
    issue was settled, not because anyone thought RFC-791 Appendix B
    dictated it. At any rate, I can assure you that no IP implementation
    enforces your hypothetical restriction, so I don't think we really
    have to worry about it as a practical matter.

    -don

+ Reply to Thread
Page 1 of 2 1 2 LastLast