Passing floating point over TCPIP. - TCP-IP

This is a discussion on Passing floating point over TCPIP. - TCP-IP ; "don provan" wrote:.. > Albert Manfredi writes: > >> Yes, your last sentence is of course true, *because* RFC 791 Appendix >> B makes it so. > > No, it really isn't. "The order of transmission of the header and ...

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2
Results 21 to 31 of 31

Thread: Passing floating point over TCPIP.

  1. Re: Passing floating point over TCPIP.

    "don provan" wrote:..
    > Albert Manfredi writes:
    >
    >> Yes, your last sentence is of course true, *because* RFC 791 Appendix
    >> B makes it so.

    >
    > No, it really isn't. "The order of transmission of the header and data
    > described *in this document* is resolved to the octet level." The
    > entire appendix is limited to the IP protocol itself.


    What can I say? IMO, you're simply wrong. When "and data" is mentioned
    in RFC 791, it means the data payload of the IP datagram. And the data
    payload of ther IP datagram includes both the headers and the data
    payload of upper layer protocols. And these upper layer protocols go
    beyond just TCP, UDP, ICMP, or IGMP. They include FTP, HTTP, RTP, SMTP,
    and on and on, as well as THEIR payloads..

    I'm not saying that legacy data formats that might have used little
    endian don't exist, or that they are never present over IP interfaces.
    I'm simply reading, rather than deliberately misreading, RFC 791.

    > Yes, it is true
    > that all other internet protocol adopted the same standard, but that's
    > because it was universally understood that consistency was far more
    > important than any other consideration, so the somewhat arbitrary
    > decision to always use big-endian was adopted. It was *not* because
    > anyone thought Appendix B of RFC-791 mandated it.


    Again, if this were true, then upper layer protocols would have made
    mention of endianness. Please read RFCs 793 (TCP) or RFC 768 (UDP), just
    to name two of the most obvious candidates, and tell me where endianness
    is mentioned.

    In those early 1980s, when these documents were written, there were
    plenty of little endian interface specs around. So while you seem to
    think that the big endian order just sort of came about "for
    consistency" on its own, without ever being spelled out, I respectfully
    disagree.

    Bert


  2. Re: Passing floating point over TCPIP.

    In article ,
    "Albert Manfredi" wrote:

    > "don provan" wrote:..
    > > Albert Manfredi writes:
    > >
    > >> Yes, your last sentence is of course true, *because* RFC 791 Appendix
    > >> B makes it so.

    > >
    > > No, it really isn't. "The order of transmission of the header and data
    > > described *in this document* is resolved to the octet level." The
    > > entire appendix is limited to the IP protocol itself.

    >
    > What can I say? IMO, you're simply wrong. When "and data" is mentioned
    > in RFC 791, it means the data payload of the IP datagram. And the data
    > payload of ther IP datagram includes both the headers and the data
    > payload of upper layer protocols. And these upper layer protocols go
    > beyond just TCP, UDP, ICMP, or IGMP. They include FTP, HTTP, RTP, SMTP,
    > and on and on, as well as THEIR payloads..


    But how can this possibly be the intent? The IP layer doesn't know
    where the multi-byte data is in the payload, so how can it swap the
    bytes if necessary? The layer N+1 payload is opaque data to layer N.

    The only potentially reasonable way to adopt your interpretation is that
    it's telling the designers of higher-layer protocols how to design them,
    i.e. they must ALSO use big-endian format. Is that what you mean? I
    respectively submit that it's unlikely that the IP designers felt the
    need to dictate how higher-layer protocols format their data.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  3. Re: Passing floating point over TCPIP.

    "Barry Margolin" wrote:

    >> When "and data" is mentioned
    >> in RFC 791, it means the data payload of the IP datagram. And the
    >> data
    >> payload of ther IP datagram includes both the headers and the data
    >> payload of upper layer protocols. And these upper layer protocols go
    >> beyond just TCP, UDP, ICMP, or IGMP. They include FTP, HTTP, RTP,
    >> SMTP,
    >> and on and on, as well as THEIR payloads..

    >
    > But how can this possibly be the intent? The IP layer doesn't know
    > where the multi-byte data is in the payload, so how can it swap the
    > bytes if necessary? The layer N+1 payload is opaque data to layer N.


    Yes, and this is true above the IP, the layer above that, and so on.
    Unless and until you reach a layer at which there is an *explicit*
    contradiction of the big endian byte order convention of Internet
    Protocols.

    (Not that such an explicit contradiction is contemplated by RFC 791 --
    it clearly is not. But if such an explicitly stated contradiction should
    occur, at least one can still guarantee a solid interface.)

    Fundamentally, I don't think there's anything desirable about switching
    numerical values to little endian byte order as a convention, for a
    general purpose protocol such as IP. At least, not for folks who are
    used to working with numbers *and* text that are read from left to
    right.

    By the way, just for illustration purposes:

    In RS-232/422/423/485, each "byte" is typically sent in little endian
    *bit* order, agreed? Just like Ethernet. And often, but not by any means
    always, the bytes are sent in big endian byte order over these
    interfaces.

    However, *unlike* Ethernet, the bytes in these serial protocols may be
    specified as consisting of 5, 6, 7, or 8 data bits. The most common
    specs used are 7 or 8 data bits per byte.

    For these interfaces to work, one *must* specify the length of each
    byte. It isn't enough to say "little endian bit order."

    (Now please, I know that this is also due to the fact that parity is
    optional, and 1 or 2 stop bits may be used. But even if those parameters
    are not changed, the number of data bits in each byte is still very much
    to be specified, before the interface will work.)

    If you ponder this for a bit, I think this also applies when little
    endian byte order is specified. You now need to specify the word length,
    or the fact that any width variable is indeed sent as little endian. And
    then you need to clarify how ASCII text, including ASCII numbers,
    hex-ASCII, BCD, or ways of depicting numerical quantities, are meant to
    be sent. There is plenty of wisdom to sticking with big endian byte
    order, IMO.

    Bert


  4. Re: Passing floating point over TCPIP.


    "Albert Manfredi" wrote in message
    news:JMBno1.2tq@news.boeing.com...
    > "Barry Margolin" wrote:
    >
    >>> When "and data" is mentioned
    >>> in RFC 791, it means the data payload of the IP datagram. And the data
    >>> payload of ther IP datagram includes both the headers and the data
    >>> payload of upper layer protocols. And these upper layer protocols go
    >>> beyond just TCP, UDP, ICMP, or IGMP. They include FTP, HTTP, RTP, SMTP,
    >>> and on and on, as well as THEIR payloads..

    >>
    >> But how can this possibly be the intent? The IP layer doesn't know
    >> where the multi-byte data is in the payload, so how can it swap the
    >> bytes if necessary? The layer N+1 payload is opaque data to layer N.

    >
    > Yes, and this is true above the IP, the layer above that, and so on.
    > Unless and until you reach a layer at which there is an *explicit*
    > contradiction of the big endian byte order convention of Internet
    > Protocols.
    >
    > (Not that such an explicit contradiction is contemplated by RFC 791 -- it
    > clearly is not. But if such an explicitly stated contradiction should
    > occur, at least one can still guarantee a solid interface.)
    >
    > Fundamentally, I don't think there's anything desirable about switching
    > numerical values to little endian byte order as a convention, for a
    > general purpose protocol such as IP. At least, not for folks who are used
    > to working with numbers *and* text that are read from left to right.
    >
    > By the way, just for illustration purposes:
    >
    > In RS-232/422/423/485, each "byte" is typically sent in little endian
    > *bit* order, agreed? Just like Ethernet. And often, but not by any means
    > always, the bytes are sent in big endian byte order over these interfaces.
    >
    > However, *unlike* Ethernet, the bytes in these serial protocols may be
    > specified as consisting of 5, 6, 7, or 8 data bits. The most common specs
    > used are 7 or 8 data bits per byte.
    >
    > For these interfaces to work, one *must* specify the length of each byte.
    > It isn't enough to say "little endian bit order."
    >
    > (Now please, I know that this is also due to the fact that parity is
    > optional, and 1 or 2 stop bits may be used. But even if those parameters
    > are not changed, the number of data bits in each byte is still very much
    > to be specified, before the interface will work.)
    >
    > If you ponder this for a bit, I think this also applies when little endian
    > byte order is specified. You now need to specify the word length, or the
    > fact that any width variable is indeed sent as little endian. And then you
    > need to clarify how ASCII text, including ASCII numbers, hex-ASCII, BCD,
    > or ways of depicting numerical quantities, are meant to be sent. There is
    > plenty of wisdom to sticking with big endian byte order, IMO.


    In transmitting struct data (with no floating point or pointers), one needs
    to know on the receiving end:

    byte size: number of bits
    bit ordering: least-to-most significant or most-to-least significant
    word size: number of bytes
    byte ordering; little, big, other
    alignment rules: could get complex
    padding rules: could get complex
    integer sizes: short, int, long, long long

    A simplification is to align struct members on 1-byte boundaries (or use
    only data members that align "nicely"). Then the concerns are only:

    byte size
    bit ordering
    word size
    byte ordering
    integer sizes

    How one can accept data from a machine with 8-bit bytes on a machine with
    7-bit bytes, I have no idea. Having to reorder bits seems a bit too much to
    do also. So for those 2, it looks like a place for exclusion (can't
    run/compile the protocol on this machine because of the byte size and bit
    ordering). Gotta make some concessions sometimes.

    So, we're left with:

    word size
    byte ordering
    integer sizes

    Integer size is easy if you use C++: use the guaranteed width integers
    (__int32, for example). So the structs definitions will work across
    machines.

    Now we have:

    word size
    byte ordering (endianness)

    So what did you guys decide? Is word size only important for little endian
    or both or not at all?

    John


  5. Re: Passing floating point over TCPIP.

    JohnQ wrote:
    >
    > In transmitting struct data (with no floating point or pointers), one
    > needs to know on the receiving end:
    >
    > byte size: number of bits
    > bit ordering: least-to-most significant or most-to-least significant
    > word size: number of bytes
    > byte ordering; little, big, other
    > alignment rules: could get complex
    > padding rules: could get complex
    > integer sizes: short, int, long, long long


    I don't know that we were talking about sending structs actually, but,
    if we are, it should be noted that of the above, the alignment and
    padding rules are dependent on the hardware, the compiler, the compiler
    version, the surrounding #pragmas, *and* the compiler options in effect
    when the struct was compiled, and may not be reproducible at the target
    at all.

    Sending structs directly out of memory is basically infeasible unless
    you can guarantee that the same binary executable is deployed at both
    ends, or at least one that's built from exactly the same compiled object
    files.

  6. Re: Passing floating point over TCPIP.

    In article ,
    Esmond Pitt wrote:

    > Sending structs directly out of memory is basically infeasible unless
    > you can guarantee that the same binary executable is deployed at both
    > ends, or at least one that's built from exactly the same compiled object
    > files.


    Yep. Take a look at the history of the BSD "talk" program (one of the
    earliest network chat applications). The protocol was "defined" by the
    source code, which simply wrote structs to the network. When all the
    world was a Vax, things worked OK, but problems arose when Suns and
    x86-based systems started appearing.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  7. Re: Passing floating point over TCPIP.


    "Esmond Pitt" wrote in message
    news:s2Pui.17759$4A1.2648@news-server.bigpond.net.au...
    > JohnQ wrote:
    >>
    >> In transmitting struct data (with no floating point or pointers), one
    >> needs to know on the receiving end:
    >>
    >> byte size: number of bits
    >> bit ordering: least-to-most significant or most-to-least significant
    >> word size: number of bytes
    >> byte ordering; little, big, other
    >> alignment rules: could get complex
    >> padding rules: could get complex
    >> integer sizes: short, int, long, long long

    >
    > I don't know that we were talking about sending structs actually, but, if
    > we are, it should be noted that of the above, the alignment and padding
    > rules are dependent on the hardware, the compiler, the compiler version,
    > the surrounding #pragmas, *and* the compiler options in effect when the
    > struct was compiled, and may not be reproducible at the target at all.
    >
    > Sending structs directly out of memory is basically infeasible unless you
    > can guarantee that the same binary executable is deployed at both ends, or
    > at least one that's built from exactly the same compiled object files.


    Note that the set of issues in the following section that you clipped dealt
    with that.

    John


  8. Re: Passing floating point over TCPIP.


    "Barry Margolin" wrote in message
    news:barmar-B2D094.21314009082007@comcast.dca.giganews.com...
    > In article ,
    > Esmond Pitt wrote:
    >
    >> Sending structs directly out of memory is basically infeasible unless
    >> you can guarantee that the same binary executable is deployed at both
    >> ends, or at least one that's built from exactly the same compiled object
    >> files.

    >
    > Yep. Take a look at the history of the BSD "talk" program (one of the
    > earliest network chat applications). The protocol was "defined" by the
    > source code, which simply wrote structs to the network. When all the
    > world was a Vax, things worked OK, but problems arose when Suns and
    > x86-based systems started appearing.


    The protocol can still be defined by the structs along with the constraints
    made by a designated platform. Of course one should try to avoid those
    platform peculiarities and address the issues of alignment, padding, etc.
    But you're right, doing it blindly is, well, blind. I'm not sure yet if a
    programming language could make all the guarantees, but if so, then I find
    languages deficient in not enabling the pattern.

    John


  9. Re: Passing floating point over TCPIP.

    In article ,
    "JohnQ" wrote:

    > "Barry Margolin" wrote in message
    > news:barmar-B2D094.21314009082007@comcast.dca.giganews.com...
    > > In article ,
    > > Esmond Pitt wrote:
    > >
    > >> Sending structs directly out of memory is basically infeasible unless
    > >> you can guarantee that the same binary executable is deployed at both
    > >> ends, or at least one that's built from exactly the same compiled object
    > >> files.

    > >
    > > Yep. Take a look at the history of the BSD "talk" program (one of the
    > > earliest network chat applications). The protocol was "defined" by the
    > > source code, which simply wrote structs to the network. When all the
    > > world was a Vax, things worked OK, but problems arose when Suns and
    > > x86-based systems started appearing.

    >
    > The protocol can still be defined by the structs along with the constraints
    > made by a designated platform. Of course one should try to avoid those
    > platform peculiarities and address the issues of alignment, padding, etc.
    > But you're right, doing it blindly is, well, blind. I'm not sure yet if a
    > programming language could make all the guarantees, but if so, then I find
    > languages deficient in not enabling the pattern.
    >
    > John


    But no one ever bothered to formally "define" the protocol, they just
    wrote a program that implemented it. The program was included in the
    BSD distribution, so ended up on all the systems that derived from it.

    That program worked fine in Vax shops, and also worked fine in Sun
    shops. But if someone on a Sun tried to talk to someone on a Vax it
    didn't work, because neither was marshalling the data into a common
    format. Basically, there was no protocol, there was just an
    implementation.

    When this was noticed and was becoming a big enough problem (because
    heterogeneous environments were becoming more common), someone sat down
    and defined a protocol, and this was "ntalk".

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  10. Re: Passing floating point over TCPIP.

    JohnQ wrote:
    > Note that the set of issues in the following section that you clipped
    > dealt with that.


    It did, and I dealt that *that* with the clause 'may not be reproducible
    at the target at all'.

  11. Re: Passing floating point over TCPIP.


    "Esmond Pitt" wrote in message
    news:Zb9vi.19113$4A1.14813@news-server.bigpond.net.au...
    > JohnQ wrote:
    >> Note that the set of issues in the following section that you clipped
    >> dealt with that.

    >
    > It did, and I dealt that *that* with the clause 'may not be reproducible
    > at the target at all'.


    I think you're assuming that one would actually be silly enough to not
    consider whether the struct ends up as a widely usable protocol message,
    which, of course, is the goal. There IS a protocol, but it's what the struct
    would imply. The target doesn't have to work with the protocol message, as
    it can marshall if it needs to. It just needs to be able to SPEAK the
    protocol. Whatever it wants to transform/marshall the data that comes off
    the wire into, is fine. Obviously, the struct may not work on all platforms,
    but it should be able to be used with common settings (such as 1-byte
    alignment where available). The only time it's a concern is when the
    platform/tools are different. For example, if deploying only on Windows,
    that one communicating executable will work on all Windows machines.

    It's pretty obvious what needs to be done once you think about it. The
    benefit is that you don't have to marshal on the primary platform. If it's
    the only platform or a compatible platform all the better.

    John


+ Reply to Thread
Page 2 of 2 FirstFirst 1 2