Unix Stream socket -- "send()" for large amount of data - Unix

This is a discussion on Unix Stream socket -- "send()" for large amount of data - Unix ; I'm working on an application where one thread ("producer") will be adding data to a buffer and another ("consumer") will be sending it to a stream socket via "send()" as fast as the socket will take it. Is it better ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: Unix Stream socket -- "send()" for large amount of data

  1. Unix Stream socket -- "send()" for large amount of data

    I'm working on an application where one thread ("producer")
    will be adding data to a buffer and another ("consumer")
    will be sending it to a stream socket via "send()" as fast
    as the socket will take it.

    Is it better for the consumer to do a single "send()"
    call each time with all the (un-consumed) data in the
    buffer?

    Or is it better to pass no more than some
    "max_send_size" bytes with each call?
    (If the system buffer has more than "max_send_size"
    bytes available, this would mean more calls to
    send().)

    From the point of view of my application, it doesn't
    really make a difference, except that a single "send()"
    call with everything is somewhat simpler to code.
    So if there's not much advantage to sending smaller chunks,
    I'd go with the "send everything" approach.

    (In either case, of course, I will use the value returned
    by "send()" to tell how many bytes were actually
    sent.)

  2. Re: Unix Stream socket -- "send()" for large amount of data

    On Mar 21, 3:56 am, Alan McKenney wrote:

    > Is it better for the consumer to do a single "send()"
    > call each time with all the (un-consumed) data in the
    > buffer?


    Yes. (Though I think you mean the producer.)

    > Or is it better to pass no more than some
    > "max_send_size" bytes with each call?
    > (If the system buffer has more than "max_send_size"
    > bytes available, this would mean more calls to
    > send().)


    Why would that be better?

    > From the point of view of my application, it doesn't
    > really make a difference, except that a single "send()"
    > call with everything is somewhat simpler to code.
    > So if there's not much advantage to sending smaller chunks,
    > I'd go with the "send everything" approach.


    What would the advantage of sending small chunks (and therefore
    definitely requiring more 'send' calls and probably more 'recv' calls
    as well) be?

    DS

  3. Re: Unix Stream socket -- "send()" for large amount of data

    On Mar 21, 1:26 pm, David Schwartz wrote:
    > On Mar 21, 3:56 am, Alan McKenney wrote:
    >
    > > Is it better for the consumer to do a single "send()"
    > > call each time with all the (un-consumed) data in the
    > > buffer?

    >
    > Yes. (Though I think you mean the producer.)


    Using my terminology (see original post), I do, indeed
    mean "consumer".

    > > Or is it better to pass no more than some
    > > "max_send_size" bytes with each call?
    > > (If the system buffer has more than "max_send_size"
    > > bytes available, this would mean more calls to
    > > send().)

    >
    > Why would that be better?


    Short answer: if I knew, then I wouldn't need to ask.

    Long answer: It depends upon how Unix stream
    sockets are designed.

    If they don't behave well if handed more data than will
    fit in the system buffers, then it would be desirable to
    limit the size of the chunks you try to give them.

    It sounds like you're saying that (in Unix, at least), sockets
    are perfectly happy if passed chunks larger than they can
    accept at once. Am I understanding you correctly?

    > What would the advantage of sending small chunks (and therefore
    > definitely requiring more 'send' calls and probably more 'recv' calls
    > as well) be?


    It's not clear that you would end up with more send() calls -- if
    you
    try to send a chunk that is much larger than the available space
    in the system buffer, "send()" will return less than the total
    chunk
    size, and you'll have to call send() again, anyway. If you do
    "send()"
    in a loop using "select()" to wait for the system to be ready for
    more
    data, you may do the same number of send()s as if you used
    a well-tuned maximum chunk size.

    As for the number of "recv()" calls -- if you're sending as fast as
    the socket will take it (which is the point of having a dedicated
    thread to copy from the buffer to the socket) -- the number of
    recv()
    calls at the other end won't be related to the number of send()
    calls,
    anyway.




  4. Re: Unix Stream socket -- "send()" for large amount of data

    On Mar 23, 4:36 pm, Alan McKenney wrote:

    > If they don't behave well if handed more data than will
    > fit in the system buffers, then it would be desirable to
    > limit the size of the chunks you try to give them.


    It's hard to imagine a scenario in which more would be worse.

    > It sounds like you're saying that (in Unix, at least), sockets
    > are perfectly happy if passed chunks larger than they can
    > accept at once. Am I understanding you correctly?


    Yes. If they are non-blocking, they will take the amount the system
    considers ideal and will take more later. If they are blocking, they
    will block and take more as quickly as the other end sucks data out.

    > > What would the advantage of sending small chunks (and therefore
    > > definitely requiring more 'send' calls and probably more 'recv' calls
    > > as well) be?


    > It's not clear that you would end up with more send() calls -- if
    > you
    > try to send a chunk that is much larger than the available space
    > in the system buffer, "send()" will return less than the total
    > chunk
    > size, and you'll have to call send() again, anyway. If you do
    > "send()"
    > in a loop using "select()" to wait for the system to be ready for
    > more
    > data, you may do the same number of send()s as if you used
    > a well-tuned maximum chunk size.


    So things might be better and can't be worse in the non-blocking case.
    You will more often send the ideal amount, which is a win.

    > As for the number of "recv()" calls -- if you're sending as fast as
    > the socket will take it (which is the point of having a dedicated
    > thread to copy from the buffer to the socket) -- the number of
    > recv()
    > calls at the other end won't be related to the number of send()
    > calls,
    > anyway.


    If you ever send less than you have, the other end may well break out
    of 'recv' with fewer bytes than if you sent more in that call to
    'send'.

    DS

+ Reply to Thread