Is there a way to do asynchronous socket communication in UNIX? - Unix

This is a discussion on Is there a way to do asynchronous socket communication in UNIX? - Unix ; Is there a way to do asynchronous socket communication in UNIX? If so, what is it? Case in question: I am writing a small non-GUI application which needs to use a single socket as a client. It is mainly recv()-ing ...

+ Reply to Thread
Results 1 to 13 of 13

Thread: Is there a way to do asynchronous socket communication in UNIX?

  1. Is there a way to do asynchronous socket communication in UNIX?

    Is there a way to do asynchronous socket communication in UNIX?

    If so, what is it?

    Case in question: I am writing a small non-GUI application which needs
    to use a single socket as a client. It is mainly recv()-ing data - in
    a single thread. Initially, this application should run on Windows,
    but later should be ported to UNIX/Linux.

    When I first used recv() in the socket's default creation mode
    (blocking), I noticed that CPU use went up to 100%. Yes, it was
    yielding control to other processes in the system (Windows XP), but I
    must admit that seeing 100% CPU use for such a simple and low
    bandwidth communication program is an ugly experience...

    Changing the socket to non-blocking mode didn't help eliminate the
    100% use either. Yes, I guess I could have used sleep() to free some
    CPU but AFAIAC, sleep() is not the correct solution here: It doesn't
    really get me out of the synchronous realm.

    So, I tried to use select() - for that single socket... but this
    didn't help reducing the programs CPU use down from 100%. I re-checked
    the documentation and, sure enough, select() is also synchronous (at
    least in Microsoft Windows).

    So, in my quest for Asynchronous method for using sockets, I found
    that Windows offers no less than 4 different such methods:

    http://www.codeproject.com/internet/...ver_client.asp

    Great, but all these methods are Windows-specific and not portable to
    UNIX/Linux.

    So, I am wondering: is there an ASYNCHRONOUS method for socket
    communication in UNIX/Linux?

    If so, what is it?

    If there isn't one, then I wonder how this fundamental problem is
    addressed in UNIX. I am sure that I am not the first one to encounter
    this problem and I know that there must be a *good* (perhaps
    excellent) solution to this in UNIX.

    But what is it?

    Thanks,
    Victor


  2. Re: Is there a way to do asynchronous socket communication in UNIX?

    vc6 writes:

    [...]

    > When I first used recv() in the socket's default creation mode
    > (blocking), I noticed that CPU use went up to 100%.


    Application bug.

  3. Re: Is there a way to do asynchronous socket communication in UNIX?

    On Thu, 07 Jun 2007 13:13:33 +0200 Rainer Weikusat wrote:
    | vc6 writes:
    |
    | [...]
    |
    |> When I first used recv() in the socket's default creation mode
    |> (blocking), I noticed that CPU use went up to 100%.
    |
    | Application bug.

    Programmer error.

    --
    |---------------------------------------/----------------------------------|
    | Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
    | first name lower case at ipal.net / spamtrap-2007-06-07-0710@ipal.net |
    |------------------------------------/-------------------------------------|

  4. Re: Is there a way to do asynchronous socket communication in UNIX?

    On Jun 7, 8:10 am, phil-news-nos...@ipal.net wrote:
    > On Thu, 07 Jun 2007 13:13:33 +0200 Rainer Weikusat wrote:| vc6 writes:
    > |
    > |> When I first used recv() in the socket's default creation mode
    > |> (blocking), I noticed that CPU use went up to 100%.
    > |
    > | Application bug.
    >
    > Programmer error.
    >


    OK. Back to the basics: what could be wrong in the following code
    snippet:

    while( bytesRecv == SOCKET_ERROR ) {
    bytesRecv = recv( sd, recvbuf, 512, 0 );
    if ( bytesRecv == 0 || bytesRecv == WSAECONNRESET ) {
    printf( "Connection Closed.\n");
    break;
    }
    printf( "Bytes Recv: %ld\n", bytesRecv );
    }

    Assuming that the server on the other side sends over that socket 512
    bytes every 20 milliseconds, should I be seeing 100% CPU use if my
    main() program contains only the above snippet?

    The answer depends, of course, on how recv() is implemented. If it is
    implemented correctly (blocking in a manner that *suspends the thread/
    process* until recvbuf is filled), then there shouldn't be 100% CPU
    use.

    If, on the other hand, it constantly polls for arriving bytes,
    returning only when recvbuf is filled with 512 bytes (or an error),
    then that explains seeing the 100% CPU use.

    Could you help me please understand this fundamental issue? How does
    recv() implement "blocking" in UNIX?

    Thanks,
    Victor


  5. Re: Is there a way to do asynchronous socket communication in UNIX?

    On Jun 7, 9:00 am, vc6 wrote:
    > On Jun 7, 8:10 am, phil-news-nos...@ipal.net wrote:
    >
    > > On Thu, 07 Jun 2007 13:13:33 +0200 Rainer Weikusat wrote:| vc6 writes:
    > > |
    > > |> When I first used recv() in the socket's default creation mode
    > > |> (blocking), I noticed that CPU use went up to 100%.
    > > |
    > > | Application bug.

    >
    > > Programmer error.

    >
    > OK. Back to the basics: what could be wrong in the following code
    > snippet:
    >
    > while( bytesRecv == SOCKET_ERROR ) {
    > bytesRecv = recv( sd, recvbuf, 512, 0 );
    > if ( bytesRecv == 0 || bytesRecv == WSAECONNRESET ) {
    > printf( "Connection Closed.\n");
    > break;
    > }
    > printf( "Bytes Recv: %ld\n", bytesRecv );
    > }
    >
    > Assuming that the server on the other side sends over that socket 512
    > bytes every 20 milliseconds, should I be seeing 100% CPU use if my
    > main() program contains only the above snippet?
    >
    > The answer depends, of course, on how recv() is implemented. If it is
    > implemented correctly (blocking in a manner that *suspends the thread/
    > process* until recvbuf is filled), then there shouldn't be 100% CPU
    > use.
    >
    > If, on the other hand, it constantly polls for arriving bytes,
    > returning only when recvbuf is filled with 512 bytes (or an error),
    > then that explains seeing the 100% CPU use.
    >
    > Could you help me please understand this fundamental issue? How does
    > recv() implement "blocking" in UNIX?
    >
    > Thanks,
    > Victor


    Victor,

    Windoze sockets affect on CPU utilization most likely has more to do
    with how M$ implemented the IP stack, and for that matter the
    difference between Windoze process switching. UNIX is truly a
    preemptive kernel, whereas Windoze is an "event driven" kernel. "When
    in Rome, do as the Romans do." Use one of Winsock socket IO wait
    events -- I'll let you find out which is best to use since this is a
    UNIX group.

    I suspect you socket code is already UNIX ready. I too support
    Windoze and UNIX, and while I can keep a large portion of the source
    "portable," some areas like socket IO, are wrapped in non portable
    helper methods, such as msWaitSockets() which is a poll() on UNIX, and
    a Winsock async socket event on Windoze trying to minimize the non
    portable source.

    cj


  6. Re: Is there a way to do asynchronous socket communication in UNIX?

    phil-news-nospam@ipal.net writes:
    > On Thu, 07 Jun 2007 13:13:33 +0200 Rainer Weikusat wrote:
    > | vc6 writes:
    > |
    > | [...]
    > |
    > |> When I first used recv() in the socket's default creation mode
    > |> (blocking), I noticed that CPU use went up to 100%.
    > |
    > | Application bug.
    >
    > Programmer error.


    'Programmer error' would be the cause, and 'application bug' the
    effect.

  7. Re: Is there a way to do asynchronous socket communication in UNIX?

    In article <1181222323.397480.230290@w5g2000hsg.googlegroups.c om>,
    CJ wrote:

    > Windoze sockets affect on CPU utilization most likely has more to do
    > with how M$ implemented the IP stack, and for that matter the
    > difference between Windoze process switching. UNIX is truly a
    > preemptive kernel, whereas Windoze is an "event driven" kernel. "When
    > in Rome, do as the Romans do." Use one of Winsock socket IO wait
    > events -- I'll let you find out which is best to use since this is a
    > UNIX group.


    I'm as much a Unix bigot as the next guy here, and I admit I'm not an
    expert on Windows programming, but I very much doubt that they'd
    implement recv() in such a way that it pegs the CPU rather than blocking
    the process until something is available.

    I go with the others and say it's programmer error. Back to his sample
    code:

    > > while( bytesRecv == SOCKET_ERROR ) {
    > > bytesRecv = recv( sd, recvbuf, 512, 0 );
    > > if ( bytesRecv == 0 || bytesRecv == WSAECONNRESET ) {
    > > printf( "Connection Closed.\n");
    > > break;
    > > }
    > > printf( "Bytes Recv: %ld\n", bytesRecv );
    > > }


    Shouldn't that initial test be while (bytesRecv != SOCKET_ERROR)? I'm
    assuming SOCKET_ERROR is -1.

    I'm also curious what the printf() is printing. And is %ld the correct
    format for bytesRecv? recv() returns ssize_t, which is int on my
    system, not long; that printf() should use (long)bytesRecv for
    portability.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  8. Re: Is there a way to do asynchronous socket communication in UNIX?

    On Jun 7, 9:35 am, Barry Margolin wrote:
    > In article <1181222323.397480.230...@w5g2000hsg.googlegroups.c om>,
    >
    > CJ wrote:
    > > Windoze sockets affect on CPU utilization most likely has more to do
    > > with how M$ implemented the IP stack, and for that matter the
    > > difference between Windoze process switching. UNIX is truly a
    > > preemptive kernel, whereas Windoze is an "event driven" kernel. "When
    > > in Rome, do as the Romans do." Use one of Winsock socket IO wait
    > > events -- I'll let you find out which is best to use since this is a
    > > UNIX group.

    >
    > I'm as much a Unix bigot as the next guy here, and I admit I'm not an
    > expert on Windows programming, but I very much doubt that they'd
    > implement recv() in such a way that it pegs the CPU rather than blocking
    > the process until something is available.
    >
    > I go with the others and say it's programmer error. Back to his sample
    > code:
    >
    > > > while( bytesRecv == SOCKET_ERROR ) {
    > > > bytesRecv = recv( sd, recvbuf, 512, 0 );
    > > > if ( bytesRecv == 0 || bytesRecv == WSAECONNRESET ) {
    > > > printf( "Connection Closed.\n");
    > > > break;
    > > > }
    > > > printf( "Bytes Recv: %ld\n", bytesRecv );
    > > > }

    >
    > Shouldn't that initial test be while (bytesRecv != SOCKET_ERROR)? I'm
    > assuming SOCKET_ERROR is -1.
    >
    > I'm also curious what the printf() is printing. And is %ld the correct
    > format for bytesRecv? recv() returns ssize_t, which is int on my
    > system, not long; that printf() should use (long)bytesRecv for
    > portability.
    >
    > --
    > Barry Margolin, bar...@alum.mit.edu
    > Arlington, MA
    > *** PLEASE post questions in newsgroups, not directly to me ***
    > *** PLEASE don't copy me on replies, I'll read them in the group ***


    Barry,

    I did not say it was a problem with how Windoze implemented recv(),
    but the entire IP stack. M$ has put the stack in ring 0, for speed
    performance (they say). I suspect if Victor looks at taskmgr while
    his app is running, the bulk of CPU utilization will be in the
    "system" (Kernel) and NOT his application code.

    I only offer my real world, current day experience coding on both UNIX
    and Windoze, baed on empirical evidence collected over years of
    working both UNIX and Windows. Take the advice or not, your choice.

    cj


  9. Re: Is there a way to do asynchronous socket communication in UNIX?

    vc6 writes:
    > On Jun 7, 8:10 am, phil-news-nos...@ipal.net wrote:
    >> On Thu, 07 Jun 2007 13:13:33 +0200 Rainer Weikusat wrote:| vc6 writes:
    >> |> When I first used recv() in the socket's default creation mode
    >> |> (blocking), I noticed that CPU use went up to 100%.
    >> |
    >> | Application bug.
    >>
    >> Programmer error.
    >>

    >
    > OK. Back to the basics: what could be wrong in the following code
    > snippet:
    >
    > while( bytesRecv == SOCKET_ERROR ) {
    > bytesRecv = recv( sd, recvbuf, 512, 0 );
    > if ( bytesRecv == 0 || bytesRecv == WSAECONNRESET ) {
    > printf( "Connection Closed.\n");
    > break;
    > }
    > printf( "Bytes Recv: %ld\n", bytesRecv );
    > }
    >
    > Assuming that the server on the other side sends over that socket 512
    > bytes every 20 milliseconds, should I be seeing 100% CPU use if my
    > main() program contains only the above snippet?


    If every recv fails, yes.

  10. Re: Is there a way to do asynchronous socket communication in UNIX?

    On Thu, 07 Jun 2007 06:00:40 -0700, vc6 wrote:

    > On Jun 7, 8:10 am, phil-news-nos...@ipal.net wrote:
    >> On Thu, 07 Jun 2007 13:13:33 +0200 Rainer Weikusat
    >> wrote:| vc6 writes: |
    >> |> When I first used recv() in the socket's default creation mode |>
    >> (blocking), I noticed that CPU use went up to 100%. |
    >> | Application bug.
    >>
    >> Programmer error.
    >>
    >>

    > OK. Back to the basics: what could be wrong in the following code
    > snippet:


    Why are you posting obviously win32 code to a unix group? This code
    won't compile on unix. You've also "helpfully" not shown the relevant
    surrounding code.
    I assume you want/meant something like:

    int fd = ... ;
    char buf[512];
    ssize_t bytes = -1;

    while (bytes == -1)
    { /* assume blocking socket, or we'll busy spin */
    bytes = recv(fd, buf, sizeof(buf), 0);
    if (bytes == 0)
    {
    printf( "Connection Closed.\n");
    exit (1);
    }
    }

    assert((bytes >= 1) && (bytes <= 512));
    printf( "Bytes Recv: %zd\n", bytes);
    /* deal with data in buf */

    ....if you want to wait for all 512 bytes then you can change the
    0 flags argument to MSG_WAITALL, on Linux. However I'd strongly
    recommend against going this route.
    If the above even compiles in win32 I don't know or care, you
    need help from a different group.

    --
    James Antill -- james@and.org
    C String APIs use too much memory? ustr: length, ref count, size and
    read-only/fixed. Ave. 55% overhead over strdup(), for 0-20B strings
    http://www.and.org/ustr/

  11. Re: Is there a way to do asynchronous socket communication in UNIX?

    "vc6" wrote in message
    news:1181221240.079097.167560@g4g2000hsf.googlegro ups.com...
    > On Jun 7, 8:10 am, phil-news-nos...@ipal.net wrote:
    >> On Thu, 07 Jun 2007 13:13:33 +0200 Rainer Weikusat
    >> wrote:| vc6 writes:
    >> |
    >> |> When I first used recv() in the socket's default creation mode
    >> |> (blocking), I noticed that CPU use went up to 100%.
    >> |
    >> | Application bug.
    >>
    >> Programmer error.
    >>

    >
    > OK. Back to the basics: what could be wrong in the following code
    > snippet:
    >
    > while( bytesRecv == SOCKET_ERROR ) {
    > bytesRecv = recv( sd, recvbuf, 512, 0 );
    > if ( bytesRecv == 0 || bytesRecv == WSAECONNRESET ) {
    > printf( "Connection Closed.\n");
    > break;
    > }
    > printf( "Bytes Recv: %ld\n", bytesRecv );
    > }


    Since this is Windows code this is a little OT but I figured I'd
    answer anyway.
    You probably want to do something more like the following:

    int bytesRecv = SOCKET_ERROR;
    while (bytesRecv == SOCKET_ERROR) {
    bytesRecv = recv(sd, recvbuf, 512, 0);
    if (bytesRecv > 0) {
    printf("Bytes recv'd: %d\n", bytesRecv);
    } else if (bytesRecv == 0) {
    printf("Connection Closed.\n");
    break;
    } else {
    int errno = WSAGetLastError();
    if (errno != WSAEINPROGRESS) {
    printf("Read error, code %d.\n", WSAGetLastError());
    break;
    }
    }
    }

    I would guess your previous problems are due to not checking for
    errors correctly.
    Windows socket errors are accessible by calling the WSAGetLastError()
    function as above.
    In Unix, you must check the value of errno for errors after a system
    function call.
    Neither of these systems encode the precise error using the return
    value of recv, the most you can tell is there was "some error" as it
    returns -1. Try this code and see if it fixes your problem.

    HTH
    Mark



  12. Re: Is there a way to do asynchronous socket communication in UNIX?

    On Jun 7, 5:28 pm, James Antill wrote:
    > On Thu, 07 Jun 2007 06:00:40 -0700, vc6 wrote:
    > > On Jun 7, 8:10 am, phil-news-nos...@ipal.net wrote:
    > >> On Thu, 07 Jun 2007 13:13:33 +0200 Rainer Weikusat
    > >> wrote:| vc6 writes: |
    > >> |> When I first used recv() in the socket's default creation mode |>
    > >> (blocking), I noticed that CPU use went up to 100%. |
    > >> | Application bug.

    >
    > >> Programmer error.

    >
    > > OK. Back to the basics: what could be wrong in the following code
    > > snippet:

    >
    > Why are you posting obviously win32 code to a unix group? This code
    > won't compile on unix. You've also "helpfully" not shown the relevant
    > surrounding code.
    > I assume you want/meant something like:
    >
    > int fd = ... ;
    > char buf[512];
    > ssize_t bytes = -1;
    >
    > while (bytes == -1)
    > { /* assume blocking socket, or we'll busy spin */
    > bytes = recv(fd, buf, sizeof(buf), 0);
    > if (bytes == 0)
    > {
    > printf( "Connection Closed.\n");
    > exit (1);
    > }
    > }

    Are you sure you want to loop only when the first (and all other)
    calls to recv() fails?


  13. Re: Is there a way to do asynchronous socket communication in UNIX?

    On Fri, 08 Jun 2007 14:21:24 +0000, stdazi wrote:

    > On Jun 7, 5:28 pm, James Antill wrote:
    >> I assume you want/meant something like:
    >>
    >> int fd = ... ;
    >> char buf[512];
    >> ssize_t bytes = -1;
    >>
    >> while (bytes == -1)
    >> { /* assume blocking socket, or we'll busy spin */

    [...]
    >> }

    > Are you sure you want to loop only when the first (and all other) calls
    > to recv() fails?


    Damn, I'd meant that to only loop when bytes == -1, and
    errno == EINTR.

    --
    James Antill -- james@and.org
    C String APIs use too much memory? ustr: length, ref count, size and
    read-only/fixed. Ave. 55% overhead over strdup(), for 0-20B strings
    http://www.and.org/ustr/

+ Reply to Thread