Socket help - OS2

This is a discussion on Socket help - OS2 ; I am extending the "copy clipboard across machines" code of ClipView to other OSs. The OS/2 version simplistically used sockets and just threw the lot at a socket and the other end read it. This appeared to work whatever the ...

+ Reply to Thread
Results 1 to 16 of 16

Thread: Socket help

  1. Socket help

    I am extending the "copy clipboard across machines" code of ClipView to other
    OSs.

    The OS/2 version simplistically used sockets and just threw the lot at a socket
    and the other end read it. This appeared to work whatever the size of the clip.
    (The biggest I tried was about thirteen K.)

    I then tried Windows using the mingw libraries. I immediately found that
    sometimes it did not transfer all the data - So I changed the receive to keep
    reading until the socket was empty - I still did not get all the data. I then
    tried sending in 1K chunks - I still did not get them all. I finally tried
    sending back an OK after each 1K chunk and not sending more until I had the OK.
    This now seems rock solid either going Win to OS/2 or OS/2 to Win.

    I decided that this could not really be correct, so this morning I downloaded
    the code for FTP - Bugger me if it doesn't just send in chunks with the other
    end reading until it gets no more - Which is essentially what I was doing
    before I put the acknowledgements in.

    What am I missing?

    TIA

    --

    Regards

    Dave Saville

    NB Remove no-spam- for good email address



  2. Re: Socket help

    Dave Saville wrote:
    > I am extending the "copy clipboard across machines" code of ClipView to other
    > OSs.
    >
    > The OS/2 version simplistically used sockets and just threw the lot at a socket
    > and the other end read it. This appeared to work whatever the size of the clip.
    > (The biggest I tried was about thirteen K.)
    >
    > I then tried Windows using the mingw libraries. I immediately found that
    > sometimes it did not transfer all the data - So I changed the receive to keep
    > reading until the socket was empty - I still did not get all the data. I then
    > tried sending in 1K chunks - I still did not get them all. I finally tried
    > sending back an OK after each 1K chunk and not sending more until I had the OK.
    > This now seems rock solid either going Win to OS/2 or OS/2 to Win.
    >
    > I decided that this could not really be correct, so this morning I downloaded
    > the code for FTP - Bugger me if it doesn't just send in chunks with the other
    > end reading until it gets no more - Which is essentially what I was doing
    > before I put the acknowledgements in.
    >
    > What am I missing?


    Random guess... sounds like buffering on one end or the other. It's
    waiting for a full buffer before flushing that last little bit of data.
    Since Windows is really the only variable you've added to your already
    working code, try reading up on or playing around with socket IOCTLs in
    whatever library you're using. Maybe their default setup is braindead.

    --
    [Reverse the parts of the e-mail address to reply.]


  3. Re: Socket help

    On Wed, 02 Feb 2005 01:56:16 -0800, Marty wrote:

    >
    >
    >Dave Saville wrote:
    >> I am extending the "copy clipboard across machines" code of ClipView to other
    >> OSs.
    >>
    >> The OS/2 version simplistically used sockets and just threw the lot at a socket
    >> and the other end read it. This appeared to work whatever the size of the clip.
    >> (The biggest I tried was about thirteen K.)
    >>
    >> I then tried Windows using the mingw libraries. I immediately found that
    >> sometimes it did not transfer all the data - So I changed the receive to keep
    >> reading until the socket was empty - I still did not get all the data. I then
    >> tried sending in 1K chunks - I still did not get them all. I finally tried
    >> sending back an OK after each 1K chunk and not sending more until I had the OK.
    >> This now seems rock solid either going Win to OS/2 or OS/2 to Win.
    >>
    >> I decided that this could not really be correct, so this morning I downloaded
    >> the code for FTP - Bugger me if it doesn't just send in chunks with the other
    >> end reading until it gets no more - Which is essentially what I was doing
    >> before I put the acknowledgements in.
    >>
    >> What am I missing?

    >
    >Random guess... sounds like buffering on one end or the other. It's
    >waiting for a full buffer before flushing that last little bit of data.


    Don't think so - It stops well short, sometimes I only get three or four chunks
    instead of the 13 * 1024 + 50 I should be getting - for the size of the file I
    am sending.

    --

    Regards

    Dave Saville

    NB Remove no-spam- for good email address



  4. Re: Socket help

    Dave Saville infrared:
    >I am extending the "copy clipboard across machines" code of ClipView to other
    >OSs.
    >
    >The OS/2 version simplistically used sockets and just threw the lot at a socket
    >and the other end read it. This appeared to work whatever the size of the clip.
    >(The biggest I tried was about thirteen K.)
    >
    >I then tried Windows using the mingw libraries. I immediately found that
    >sometimes it did not transfer all the data - So I changed the receive to keep
    >reading until the socket was empty - I still did not get all the data. I then
    >tried sending in 1K chunks - I still did not get them all. I finally tried
    >sending back an OK after each 1K chunk and not sending more until I had the OK.
    >This now seems rock solid either going Win to OS/2 or OS/2 to Win.
    >
    >I decided that this could not really be correct, so this morning I downloaded
    >the code for FTP - Bugger me if it doesn't just send in chunks with the other
    >end reading until it gets no more - Which is essentially what I was doing
    >before I put the acknowledgements in.
    >
    >What am I missing?


    The "keep reading until socket is empty" should really be "keep
    reading until socket is closed" because the "empty" condition could
    simply mean that the data have not yet arrived because of network
    delays. Thus, it's important to ensure that the sender closes
    the socket when the transfer is done.

    But in any case, the Windows FTP code is not a good model because
    it appears to be broken in some way. In my tests, I have found
    that some Windows FTP clients (e.g. Internet Explorer) give
    strange results if the client is too close to the server. The
    problem goes away if the network delays are increased, so it looks
    as if the client is getting confused if the server is too fast.
    But it's an application-dependent problem, because I've seen at
    least one Windows FTP client (WS_FTP) that does not have this
    problem.

    --
    Peter Moylan peter at ee dot newcastle dot edu dot au
    http://eepjm.newcastle.edu.au (OS/2 and eCS information and software)

  5. Re: Socket help

    Dave Saville wrote:
    > I am extending the "copy clipboard across machines" code of ClipView to other
    > OSs.
    >
    > The OS/2 version simplistically used sockets and just threw the lot at a socket
    > and the other end read it. This appeared to work whatever the size of the clip.
    > (The biggest I tried was about thirteen K.)
    >
    > I then tried Windows using the mingw libraries. I immediately found that
    > sometimes it did not transfer all the data - So I changed the receive to keep
    > reading until the socket was empty - I still did not get all the data. I then
    > tried sending in 1K chunks - I still did not get them all. I finally tried
    > sending back an OK after each 1K chunk and not sending more until I had the OK.
    > This now seems rock solid either going Win to OS/2 or OS/2 to Win.
    >
    > I decided that this could not really be correct, so this morning I downloaded
    > the code for FTP - Bugger me if it doesn't just send in chunks with the other
    > end reading until it gets no more - Which is essentially what I was doing
    > before I put the acknowledgements in.
    >
    > What am I missing?
    >
    > TIA
    >
    > --
    >
    > Regards
    >
    > Dave Saville
    >
    > NB Remove no-spam- for good email address
    >
    >


    I am probably no help, but sockets on both ends are STREAM vice DGRAM right?

    --

    M Greene Voice Memeber #1356

    IRC (MikeG): irc.va.webbnet.info #os/2 and #learnc++
    ashburn.va.us.undernet.org #os/2warp irc.ecomstation.com #eCS
    ICQ:319474014 AIM:mikekg2003

    http://members.cox.net/greenemk/os2/index.html

  6. Re: Socket help

    On Wed, 02 Feb 2005 09:34:14 GMT, Dave Saville
    wrote:

    > I am extending the "copy clipboard across machines" code of ClipView to other
    > OSs.
    >
    > The OS/2 version simplistically used sockets and just threw the lot at a socket
    > and the other end read it. This appeared to work whatever the size of the clip.
    > (The biggest I tried was about thirteen K.)
    >
    > I then tried Windows using the mingw libraries. I immediately found that
    > sometimes it did not transfer all the data - So I changed the receive to keep
    > reading until the socket was empty - I still did not get all the data.


    Is this a blocking read or a non-blocking read? If the latter, then you
    can certainly get 0 bytes returned even though there is more to come.
    I think a blocking read only returns 0 when the socket gets closed. Check
    docs. to be sure.

    > I then
    > tried sending in 1K chunks - I still did not get them all. I finally tried
    > sending back an OK after each 1K chunk and not sending more until I had the OK.
    > This now seems rock solid either going Win to OS/2 or OS/2 to Win.


    This is ridiculous. You are paying TCP to do all this stuff, so you don't
    need to do it yourself. At least I assume you are... you didn't say.
    "Sockets" covers several possibilities.

    > the code for FTP - Bugger me if it doesn't just send in chunks with the other
    > end reading until it gets no more


    Exactly.

    > What am I missing?


    I don't know, apart from not using the calls correctly, or not checking
    the return codes properly, or perhaps there's a bug in that mingw thing,
    whatever it is.

  7. Re: Socket help

    Hi,

    From your discription I cannot tell if you are missing the last chunks of
    data to or from the windows box.
    could there be a firewall or virus blocker that is disrupting the path? It
    could be the kernel is dropping some of the data?

    I would suggest getting ethereal and monitor the traffic. I know there are
    builds of ethereal for windows. I have not looked for an OS/2 version. I
    have run it under Linux and it is a great debug tool.

    Regards,
    Bart

    On Wed, 02 Feb 2005 11:05:33 GMT, Dave Saville wrote:

    >
    >
    >On Wed, 02 Feb 2005 01:56:16 -0800, Marty wrote:
    >
    >>
    >>
    >>Dave Saville wrote:
    >>> I am extending the "copy clipboard across machines" code of ClipView to other
    >>> OSs.
    >>>
    >>> The OS/2 version simplistically used sockets and just threw the lot at a socket
    >>> and the other end read it. This appeared to work whatever the size of the clip.
    >>> (The biggest I tried was about thirteen K.)
    >>>
    >>> I then tried Windows using the mingw libraries. I immediately found that
    >>> sometimes it did not transfer all the data - So I changed the receive to keep
    >>> reading until the socket was empty - I still did not get all the data. I then
    >>> tried sending in 1K chunks - I still did not get them all. I finally tried
    >>> sending back an OK after each 1K chunk and not sending more until I had the OK.
    >>> This now seems rock solid either going Win to OS/2 or OS/2 to Win.
    >>>
    >>> I decided that this could not really be correct, so this morning I downloaded
    >>> the code for FTP - Bugger me if it doesn't just send in chunks with the other
    >>> end reading until it gets no more - Which is essentially what I was doing
    >>> before I put the acknowledgements in.
    >>>
    >>> What am I missing?

    >>
    >>Random guess... sounds like buffering on one end or the other. It's
    >>waiting for a full buffer before flushing that last little bit of data.

    >
    >Don't think so - It stops well short, sometimes I only get three or four chunks
    >instead of the 13 * 1024 + 50 I should be getting - for the size of the file I
    >am sending.
    >
    >--
    >
    >Regards
    >
    >Dave Saville
    >
    >NB Remove no-spam- for good email address
    >
    >





  8. Re: Socket help

    On 2 Feb 2005 23:58:05 GMT, Peter Moylan wrote:

    >The "keep reading until socket is empty" should really be "keep
    >reading until socket is closed" because the "empty" condition could
    >simply mean that the data have not yet arrived because of network
    >delays. Thus, it's important to ensure that the sender closes
    >the socket when the transfer is done.
    >


    Paul & Peter

    How do you tell the difference between reading nowt and socket closed? And how
    do you set blocking/non blocking?

    Very stripped code:

    Client Side

    if ( ( sock = socket(AF_INET, SOCK_STREAM, 0)) < 0 ) error

    if( connect(sock, (struct sockaddr *)&sockaddr_server, sizeof
    sockaddr_server)< 0) error

    if ( (i = write(sock, pClipText + j, k)) <= 0 ) error

    Server side

    if( (sock = socket(AF_INET, SOCK_STREAM, 0)) < 0) error

    if( bind(sock, &sockadr_server, addrsize) < 0 ) error

    listen(sock, 5);

    while( (newsock = accept(sock, &sockadr_server, &addrsize)) != -1 ) error
    {
    if ( (i = read(newsock, &tmp, BUFSIZE) <= 0 ) error


    >But in any case, the Windows FTP code is not a good model because
    >it appears to be broken in some way. In my tests, I have found
    >that some Windows FTP clients (e.g. Internet Explorer) give
    >strange results if the client is too close to the server. The
    >problem goes away if the network delays are increased, so it looks
    >as if the client is getting confused if the server is too fast.
    >But it's an application-dependent problem, because I've seen at
    >least one Windows FTP client (WS_FTP) that does not have this
    >problem.


    I never said it was Windows FTP code. In fact it is a copy of *the* original
    *nix code which was the first thing Google threw up.

    Without the ack's, which I agree Paul should not be required, the windows side
    sending just does not send all the data. Sometimes it does sometimes it
    doesn't. OS/2 is quite happy sending all 13K in one lump, but win will send say
    7K followed by 3K followed by some -tve number which must be an error code.
    Clearly it is some kind of timing issue. Oh, and it is all on the local LAN BTW
    so no firewall etc. in the way. Mingw is a free emx/gcc type thing for the
    win32 api.

    --

    Regards

    Dave Saville

    NB Remove no-spam- for good email address



  9. Re: Socket help

    On Thu, 03 Feb 2005 08:50:49 GMT, Dave Saville
    wrote:

    > How do you tell the difference between reading nowt and socket closed?


    Actually, I don't think you can, having just looked at the docs.
    If you're in non-blocking mode, you should check to see how many bytes are
    available first.
    If you're in blocking mode, then it just won't return until there is
    something to read, or the socket is closed in which case it will return 0
    or <0 if an error.

    > And how do you set blocking/non blocking?


    Check out ioctl() and select() in the MPTS Sockets Programming Reference.

    > Client Side
    >
    > if ( (i = write(sock, pClipText + j, k)) <= 0 ) error


    Are you sure you don't mean send() ? And have you checked what value i is
    getting i.e. how many characters did the network layer think it sent? Is
    this different to k?

    > Server side
    >
    > if ( (i = read(newsock, &tmp, BUFSIZE) <= 0 ) error


    Again, are you sure you don't mean recv() ?

    > Without the ack's, which I agree Paul should not be required, the windows side
    > sending just does not send all the data.


    Have you confirmed this with IPTRACE/IPFORMAT or a packet monitor?

    > OS/2 is quite happy sending all 13K in one lump, but win will send say
    > 7K followed by 3K


    How are you deducing this?

    > followed by some -tve number which must be an error code.


    What number?

    > Clearly it is some kind of timing issue.


    Not necessarily.

  10. Re: Socket help

    Suddenly, Dave Saville sprang forth and uttered these pithy words:
    > The OS/2 version simplistically used sockets and just threw the lot at a socket
    > and the other end read it. This appeared to work whatever the size of the clip.
    > (The biggest I tried was about thirteen K.)


    Welcome to TCP/IP. TCP gives you a stream which arbitrarily splits data
    up for transmission. It guarantees only that all the bytes will
    eventually get there in the right order.

    You must impose some structure on this stream. Since it is reliable, a
    simple one will do, like sending a 4 byte integer first to say the
    length of data to follow. YOu can then read until you get that much data
    (handling errors in case the socket is broken before all the data
    arrives).

    You might also need to adjust buffering (e.g. the settings for the
    "nagle" algorithm) to ensure that your data is sent promptly when you
    want.


    It's funny how everyone's first socket program makes the same mistake



    --
    aaronl at consultant dot com
    For every expert, there is an equal and
    opposite expert. - Arthur C. Clarke

  11. Re: Socket help

    On Fri, 4 Feb 2005 02:19:03 +1300, Aaron Lawrence wrote:

    >
    >
    >Suddenly, Dave Saville sprang forth and uttered these pithy words:
    >> The OS/2 version simplistically used sockets and just threw the lot at a socket
    >> and the other end read it. This appeared to work whatever the size of the clip.
    >> (The biggest I tried was about thirteen K.)

    >
    >Welcome to TCP/IP. TCP gives you a stream which arbitrarily splits data
    >up for transmission. It guarantees only that all the bytes will
    >eventually get there in the right order.
    >


    I know.

    >You must impose some structure on this stream. Since it is reliable, a
    >simple one will do, like sending a 4 byte integer first to say the
    >length of data to follow. YOu can then read until you get that much data
    >(handling errors in case the socket is broken before all the data
    >arrives).
    >


    I do - it doesn't all arrive.

    >You might also need to adjust buffering (e.g. the settings for the
    >"nagle" algorithm) to ensure that your data is sent promptly when you
    >want.
    >


    You lost me here.

    >
    >It's funny how everyone's first socket program makes the same mistake


    And that is?

    --

    Regards

    Dave Saville

    NB Remove no-spam- for good email address



  12. Re: Socket help

    On Thu, 03 Feb 2005 11:29:39 GMT, Paul Ratcliffe wrote:

    >
    >
    >On Thu, 03 Feb 2005 08:50:49 GMT, Dave Saville
    >wrote:
    >
    >> How do you tell the difference between reading nowt and socket closed?

    >
    >Actually, I don't think you can, having just looked at the docs.
    >If you're in non-blocking mode, you should check to see how many bytes are
    >available first.
    >If you're in blocking mode, then it just won't return until there is
    >something to read, or the socket is closed in which case it will return 0
    >or <0 if an error.
    >
    >> And how do you set blocking/non blocking?

    >
    >Check out ioctl() and select() in the MPTS Sockets Programming Reference.


    That's as maybe, but the EMX docs say nonblocking is not supported.

    >
    >> Client Side
    >>
    >> if ( (i = write(sock, pClipText + j, k)) <= 0 ) error

    >
    >Are you sure you don't mean send() ? And have you checked what value i is
    >getting i.e. how many characters did the network layer think it sent? Is
    >this different to k?
    >


    EMX allows read/write/close on sockets. Also the *nix source code I have for
    FTP also uses read/write. Not sure what, if any, is the difference between read
    and recv. Yes I print i out and it will be what I expect. 13k or chunks. On the
    receive end i is often less than k when trying to read the 13k in one chunk.
    Smaller chunks, 1k, i will equal k except for the odd length last block
    assuming it all transfers *and* assuming I stick in the ACK which we all agree
    is not needed.

    >> Server side
    >>
    >> if ( (i = read(newsock, &tmp, BUFSIZE) <= 0 ) error

    >
    >Again, are you sure you don't mean recv() ?
    >


    See above.

    >> Without the ack's, which I agree Paul should not be required, the windows side
    >> sending just does not send all the data.

    >
    >Have you confirmed this with IPTRACE/IPFORMAT or a packet monitor?
    >


    Not yet. Thanks for the idea.

    >> OS/2 is quite happy sending all 13K in one lump, but win will send say
    >> 7K followed by 3K

    >
    >How are you deducing this?


    By printing out the value(s) of i on both ends. That read is in a loop until I
    get an error or the number of bytes I am expecting - which is the first packet
    sent.

    >
    >> followed by some -tve number which must be an error code.

    >
    >What number?


    I don't have it to hand at present, but I would get two or maybe three blocks
    whose total sizes were nowhere near what I was sending followed by errors. That
    was when I was sending the 13K in one chunk. I thought TCP/IP could handle up
    to 32767? If I broke the sending down into 1k chunks then sometimes I got all
    thirteen sometimes less.
    >
    >> Clearly it is some kind of timing issue.

    >
    >Not necessarily.



    --

    Regards

    Dave Saville

    NB Remove no-spam- for good email address



  13. Re: Socket help - Solved - I think

    There must be some difference between send/write and read/recv despite what it
    says in the EMX docs. In OS/2 and *nix using read & write between the two boxes
    works. But Win does not let you use read/write on sockets so I used recv/send.
    The problem would appear to be Win using recv/send and OS/2 using write/read -
    or the other way around. Changing everything to use send/recv seems to have
    fixed the problem. I have taken out the ACK and I cannot now get it to fail.

    So for those of you that asked for it, there should soon be a clipboard
    interface OS/2 to Win as well as OS/2 to OS/2 in ClipView. And yes I know I
    mentioned *nix earlier, but I still have to figure out the damn X11 clipboard
    :-( AT least the transfer part works!

    Thanks for the help guys.

    --

    Regards

    Dave Saville

    NB Remove no-spam- for good email address



  14. Re: Socket help

    Suddenly, Dave Saville sprang forth and uttered these pithy words:
    > >You might also need to adjust buffering (e.g. the settings for the
    > >"nagle" algorithm) to ensure that your data is sent promptly when you
    > >want.
    > >

    >
    > You lost me here.


    There are socket options you can set.

    However, if you're saying it *never* arrives that sounds more serious.

    >
    > >
    > >It's funny how everyone's first socket program makes the same mistake

    >
    > And that is?


    Assuming that all the data they sent in one block will arrive in one
    block.

    Maybe you didn't make it, but it sounded like it. I've seen two other
    developers do it, independently

    --
    aaronl at consultant dot com
    For every expert, there is an equal and
    opposite expert. - Arthur C. Clarke

  15. Re: Socket help

    Suddenly, Dave Saville sprang forth and uttered these pithy words:
    > I am extending the "copy clipboard across machines" code of ClipView to other
    > OSs.
    >
    > The OS/2 version simplistically used sockets and just threw the lot at a socket
    > and the other end read it. This appeared to work whatever the size of the clip.
    > (The biggest I tried was about thirteen K.)
    >
    > I then tried Windows using the mingw libraries. I immediately found that
    > sometimes it did not transfer all the data - So I changed the receive to keep
    > reading until the socket was empty - I still did not get all the data. I then
    > tried sending in 1K chunks - I still did not get them all. I finally tried
    > sending back an OK after each 1K chunk and not sending more until I had the OK.
    > This now seems rock solid either going Win to OS/2 or OS/2 to Win.



    Hm, this reminds me of a problem I heard about in Windows - perhaps it
    was IIS? It would close the socket improperly and the last set of data
    would be discarded.
    --
    aaronl at consultant dot com
    For every expert, there is an equal and
    opposite expert. - Arthur C. Clarke

  16. Re: Socket help

    On Fri, 4 Feb 2005 23:45:08 +1300, Aaron Lawrence wrote:

    >
    >However, if you're saying it *never* arrives that sounds more serious.
    >


    It did not but see other post subject Solved I think :-)

    >>
    >> >
    >> >It's funny how everyone's first socket program makes the same mistake

    >>
    >> And that is?

    >
    >Assuming that all the data they sent in one block will arrive in one
    >block.
    >
    >Maybe you didn't make it, but it sounded like it. I've seen two other
    >developers do it, independently


    Yes I did initially - but at least I figured it out for myself :-) Problem is
    with OS/2, on a quiet LAN, you can throw huge chunks around and it nearly
    always gets it in one read - so it took some time for the problem to show up.

    --

    Regards

    Dave Saville

    NB Remove no-spam- for good email address



+ Reply to Thread