Shared sockets - TCP-IP

This is a discussion on Shared sockets - TCP-IP ; Hello, When a socket is shared (e.g. on Linux) by several processes, how exactly are the packets dealt out when they are received? How is the differentiation done? With Oracle for instance, server processes are spawned by the listener (which ...

+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 27

Thread: Shared sockets

  1. Shared sockets

    Hello,

    When a socket is shared (e.g. on Linux) by several processes, how
    exactly are the packets dealt out when they are received? How is the
    differentiation done?

    With Oracle for instance, server processes are spawned by the listener
    (which listens on port 1521 by defaut) - the spawned processes then
    communicate with the clients on this very port.

    TIA,
    Jerome

  2. Re: Shared sockets

    vitalisman@gmail.com writes:

    > Hello,
    >
    > When a socket is shared (e.g. on Linux) by several processes, how
    > exactly are the packets dealt out when they are received? How is the
    > differentiation done?
    >
    > With Oracle for instance, server processes are spawned by the listener
    > (which listens on port 1521 by defaut) - the spawned processes then
    > communicate with the clients on this very port.


    A port and a socket are different.

    When a client connects to a well-known-port (say 1521) the client
    machine selects a new port on its end, and makes a connection to the
    remote server. Say the port it uses is 32744. So the connection now
    has the following unique attributes

    (source address, source port=32744, +
    destination address, destination port=1521)

    And when the server replies, the source and destination pairs are
    swapped.

    In this case, each TCP socket is unique.


    --
    Posted via a free Usenet account from http://www.teranews.com


  3. Re: Shared sockets

    vitalisman@gmail.com writes:

    > When a socket is shared (e.g. on Linux) by several processes, how
    > exactly are the packets dealt out when they are received? How is the
    > differentiation done?


    When multiple processes or threads are calling accept() on the same
    socket, and a client connects, the kernel will create a new socket for
    that connection, and exactly one of the accept()ing process will get
    that connection.. As Bruce said, the kernel keeps track of which is
    which by the IP address and port of the client.

    Hope that helps,

    -----Scott.

  4. Re: Shared sockets

    Scott Gifford wrote:

    > Hope that helps,


    OK, many thanks to you and Bruce!

  5. Re: Shared sockets

    On Dec 21, 2:43 am, vitalis...@gmail.com wrote:

    > When a socket is shared (e.g. on Linux) by several processes, how
    > exactly are the packets dealt out when they are received? How is the
    > differentiation done?


    The packets go to the socket. What differentiation are you talking
    about?

    > With Oracle for instance, server processes are spawned by the listener
    > (which listens on port 1521 by defaut) - the spawned processes then
    > communicate with the clients on this very port.


    Is there a question in there somewhere?

    I don't know whether or not Oracle has multiple processes blocked in
    'accept' for the same socket, but if so, the behavior of the two
    operations (on a listening socket) is equivalent:

    1) Call 'accept' on a blocking socket.

    2A) Call 'select' to block until the socket is ready for reading.
    2B) Call 'accept' non-blocking.
    2C) If we got a 'WOULDBLOCK' error go to step 2A.
    2D) Return whatever we got from 'accept'.

    It should be obvious what process 2 does in case the descriptor is
    shared. Process 1 does the exact same thing. In fact, that's typically
    how it's implemented internally.

    DS

  6. Re: Shared sockets

    David Schwartz wrote:
    > On Dec 21, 2:43 am, vitalis...@gmail.com wrote:
    >
    >> When a socket is shared (e.g. on Linux) by several processes, how
    >> exactly are the packets dealt out when they are received? How is the
    >> differentiation done?

    >
    > The packets go to the socket. What differentiation are you talking
    > about?


    How the kernel know which packets to pass to which process. But Scott
    already answered.
    Thanks anyway for your answer.

  7. Re: Shared sockets

    On Dec 21, 3:43 pm, vitalis...@gmail.com wrote:
    > Hello,
    >
    > When a socket is shared (e.g. on Linux) by several processes, how
    > exactly are the packets dealt out when they are received? How is the
    > differentiation done?
    >
    > With Oracle for instance, server processes are spawned by the listener
    > (which listens on port 1521 by defaut) - the spawned processes then
    > communicate with the clients on this very port.
    >
    > TIA,
    > Jerome


    After i do an accept(), if i do a dup or fork, i get another socket
    descriptor pointing to the same state structure. Won't it a shared
    socket ? It is not a unique socket. ON this case, the servers(parent
    and the child) will use the same TCB (No ?) and if we have 3 k user
    data to send to the client, and if the first server send 1.5 k
    already, the other server will send the other half. Am i wrong on my
    thinking ?

  8. Re: Shared sockets

    In article
    ,
    Jijo wrote:

    > On Dec 21, 3:43 pm, vitalis...@gmail.com wrote:
    > > Hello,
    > >
    > > When a socket is shared (e.g. on Linux) by several processes, how
    > > exactly are the packets dealt out when they are received? How is the
    > > differentiation done?
    > >
    > > With Oracle for instance, server processes are spawned by the listener
    > > (which listens on port 1521 by defaut) - the spawned processes then
    > > communicate with the clients on this very port.
    > >
    > > TIA,
    > > Jerome

    >
    > After i do an accept(), if i do a dup or fork, i get another socket
    > descriptor pointing to the same state structure. Won't it a shared
    > socket ? It is not a unique socket. ON this case, the servers(parent
    > and the child) will use the same TCB (No ?) and if we have 3 k user
    > data to send to the client, and if the first server send 1.5 k
    > already, the other server will send the other half. Am i wrong on my
    > thinking ?


    This is correct. This case is treated no differently from any other
    descriptor shared between processes, e.g. if you fork and both processes
    try to read from stdin. There's nothing socket-specific going on in
    this case, so it's not really a comp.protocols.tcp-ip issue.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  9. Re: Shared sockets

    On Sun, 23 Dec 2007 22:04:59 -0800 (PST), Jijo
    wrote:

    >After i do an accept(), if i do a dup or fork, i get another socket
    >descriptor pointing to the same state structure. Won't it a shared
    >socket ?


    Yes, it will be a shared socket. The server must close the accepted
    socket after having done the fork(). Otherwise a later close() in the
    forked process will not be the last close and cause a termination of the
    connection.

    >It is not a unique socket. ON this case, the servers(parent
    >and the child) will use the same TCB (No ?) and if we have 3 k user
    >data to send to the client, and if the first server send 1.5 k
    >already, the other server will send the other half. Am i wrong on my
    >thinking ?


    Depending in the implementation of the libraries and the protocol stack
    a shared use of the socket by multiple processes will not work.

    Spawning new processes for each accept has no good performance. It is
    better to have the server processes already running and pass them the
    file descriptor.

  10. Re: Shared sockets

    In article ,
    Emil Naepflein wrote:

    >>After i do an accept(), if i do a dup or fork, i get another socket
    >>descriptor pointing to the same state structure. Won't it a shared
    >>socket ?


    >Yes, it will be a shared socket. The server must close the accepted


    >>It is not a unique socket. ON this case, the servers(parent
    >>and the child) will use the same TCB (No ?) and if we have 3 k user
    >>data to send to the client, and if the first server send 1.5 k
    >>already, the other server will send the other half. Am i wrong on my
    >>thinking ?

    >
    >Depending in the implementation of the libraries and the protocol stack
    >a shared use of the socket by multiple processes will not work.


    What is "work" in that context? If it is perfectly even 50/50 sharing
    of TCP data between the two processes as described, then I would bet
    on it rarely "working." I would expect one process to receive more or
    even all of the data. If UDP were used and if the application lacked
    inter-packet context, then the incoming packets might not be share
    perfectly, but the application might "work" fine.


    >Spawning new processes for each accept has no good performance. It is
    >better to have the server processes already running and pass them the
    >file descriptor.


    That is not "better," because it assumes a facility for passing file
    descriptors among processes that is not always available and certain
    is not portable amoung operating systems.

    To avoid fork(), I would use some brand of threads such as POSIX.

    When performance matters, it is better to avoid both the separate
    processes of fork() and separate threads. Threads are still, even at
    this late date, often illusions based on select() and in any case are
    more expensive than a single process that uses select() (or poll() when
    available) directly.


    Vernon Schryver vjs@rhyolite.com

  11. Re: Shared sockets

    On Dec 23, 10:40 pm, Emil Naepflein wrote:

    > On Sun, 23 Dec 2007 22:04:59 -0800 (PST), Jijo


    > wrote:
    > >After i do an accept(), if i do a dup or fork, i get another socket
    > >descriptor pointing to the same state structure. Won't it a shared
    > >socket ?


    > Yes, it will be a shared socket. The server must close the accepted
    > socket after having done the fork(). Otherwise a later close() in the
    > forked process will not be the last close and cause a termination of the
    > connection.


    This is a common cause of confusion. Because the last 'close' on a
    connected socket also shuts down the connection, it is common to
    associate these things beyond reason.

    The server does not need to close the accepted socket. A later close
    in the forked process won't cause the termination of the connection,
    but if the forked process wanted to shut down the connection, it
    should call 'shutdown'.

    The server should close the accepted socket if and only if it has no
    further use for that socket. If it has further use, it should leave it
    open. For example, it may wish to monitor the connection using
    'getsockopt(SO_ERROR)' or 'getpeername' to see if the socket is still
    connected.

    The child can call 'shutdown' if it wants to shut the connection down.
    It should not rely on 'close' implicitly doing a 'shutdown' if the
    connection may potentially be shared. There are many other reasons not
    to use 'close' to do a 'shutdown'. For one thing, it is impossible to
    determine the progress of the connection teardown. Typically, that is
    at least useful for logging purposes.

    Shutting down a connection and closing a reference to the connection
    endpoint are completely logically distinct operations. It is not good
    to confuse them.

    DS

  12. Re: Shared sockets

    In article , davids@webmaster.com (David Schwartz) writes:

    | The server should close the accepted socket if and only if it has no
    | further use for that socket. If it has further use, it should leave it
    | open. For example, it may wish to monitor the connection using
    | 'getsockopt(SO_ERROR)' or 'getpeername' to see if the socket is still
    | connected.

    Note that it is probably unwise to monitor the connection asynchronously
    with getsockopt(...SO_ERROR...) since this operation also clears the error
    and the child may need that information.

    Dan Lanciani
    ddl@danlan.*com

  13. Re: Shared sockets

    On Mon, 24 Dec 2007 15:08:32 +0000 (UTC), vjs@calcite.rhyolite.com
    (Vernon Schryver) wrote:

    >What is "work" in that context?


    If there is any state maintained in the library this may become
    inconsistent and cause return of errors.

    >>Spawning new processes for each accept has no good performance. It is
    >>better to have the server processes already running and pass them the
    >>file descriptor.

    >
    >That is not "better," because it assumes a facility for passing file
    >descriptors among processes that is not always available and certain
    >is not portable amoung operating systems.


    If portability of an application is an requirement then you are right.
    But if you are tight on cpu performance and optimize for a specific
    plattform then you have to look where to save cycles.

    >To avoid fork(), I would use some brand of threads such as POSIX.


    Regarding performance that will be slightly better if the OS supports
    native threads. If not then the performance may be even worse,
    especially on multiprocessing systems.

    >When performance matters, it is better to avoid both the separate
    >processes of fork() and separate threads. Threads are still, even at
    >this late date, often illusions based on select() and in any case are
    >more expensive than a single process that uses select() (or poll() when
    >available) directly.


    On todays multiprocessor systems this would limit the performance to one
    cpu. And with thousands of connections you will have to think about
    fairness regarding processing of concurrent ready file descriptors of
    different types. This is a very tricky issue.

  14. Re: Shared sockets

    On Tue, 25 Dec 2007 21:45:11 -0800 (PST), David Schwartz
    wrote:

    >The server should close the accepted socket if and only if it has no
    >further use for that socket. If it has further use, it should leave it
    >open. For example, it may wish to monitor the connection using
    >'getsockopt(SO_ERROR)' or 'getpeername' to see if the socket is still
    >connected.


    This will certainly fail on a lot of socket implementations on top of
    Streams.

    Concurrent operations of multiple Processes on the same socket may cause
    a lot of problems on certain implementations. Whenever possible this
    should be avoided.

    >The child can call 'shutdown' if it wants to shut the connection down.
    >It should not rely on 'close' implicitly doing a 'shutdown' if the
    >connection may potentially be shared.


    When the server always closes the connection after fork there is
    absolute no problem with this. BTW, most standard servers are written
    this way.

    >There are many other reasons not
    >to use 'close' to do a 'shutdown'. For one thing, it is impossible to
    >determine the progress of the connection teardown. Typically, that is
    >at least useful for logging purposes.


    Where can you get information about the connection teardown with
    shutdown()?

    Ok, you could probably poll with shutdown() whether you get ENOTCONN as
    error. But you cannot rely that this reflects the state of the
    connection in the TCP layer.

    >Shutting down a connection and closing a reference to the connection
    >endpoint are completely logically distinct operations. It is not good
    >to confuse them.


    Yes they are distinct operations. But a close() implicitely does a
    shutdown() on last call to close(). And this is very helpful especially
    when something goes wrong, i.e. a process terminates abnormally.

    Over the last 20+ years I have seen a lot of server applications that
    failed handling the close() right in some situations. The result was
    most of the time hanging connections blocking ports, not enough file
    descriptors and hanging client applications. Avoiding socket sharing
    over multiple processes prevents such problems from the start.

    All this complicated error handling is not worth the effort, and may be
    most of the time itself a major cause of problems.

  15. Re: Shared sockets

    On Dec 25, 10:13 pm, ddl@danlan.*com (Dan Lanciani) wrote:

    > Note that it is probably unwise to monitor the connection asynchronously
    > with getsockopt(...SO_ERROR...) since this operation also clears the error
    > and the child may need that information.


    Ack, you're right! How counter-intuitive for a 'get' function to
    change things.

    DS

  16. Re: Shared sockets

    On Dec 26, 12:02 am, Emil Naepflein wrote:
    > On Tue, 25 Dec 2007 21:45:11 -0800 (PST), David Schwartz


    > wrote:
    > >The server should close the accepted socket if and only if it has no
    > >further use for that socket. If it has further use, it should leave it
    > >open. For example, it may wish to monitor the connection using
    > >'getsockopt(SO_ERROR)' or 'getpeername' to see if the socket is still
    > >connected.


    > This will certainly fail on a lot of socket implementations on top of
    > Streams.


    I find that very hard to believe.

    > Concurrent operations of multiple Processes on the same socket may cause
    > a lot of problems on certain implementations. Whenever possible this
    > should be avoided.


    I find that very hard to believe. On any UNIX system which supports
    POSIX threads, which is pretty much every modern UNIX system, you'd
    have to work hard to make sockets break from concurrent processes.

    > >The child can call 'shutdown' if it wants to shut the connection down.
    > >It should not rely on 'close' implicitly doing a 'shutdown' if the
    > >connection may potentially be shared.


    > When the server always closes the connection after fork there is
    > absolute no problem with this. BTW, most standard servers are written
    > this way.


    There is definitely a problem with this. If you don't have control
    over the lifetime of every possible descriptor that might access a
    connection, you cannot assume that 'close' will imply 'shutdown'.

    > >There are many other reasons not
    > >to use 'close' to do a 'shutdown'. For one thing, it is impossible to
    > >determine the progress of the connection teardown. Typically, that is
    > >at least useful for logging purposes.


    > Where can you get information about the connection teardown with
    > shutdown()?


    Huh? I am saying you shouldn't use 'close' to do a 'shutdown'. If you
    want to close your descriptor, use 'close'. If you want to shutdown
    the connection, use 'shutdown'. These are logically unrelated
    operations.

    > Ok, you could probably poll with shutdown() whether you get ENOTCONN as
    > error. But you cannot rely that this reflects the state of the
    > connection in the TCP layer.


    I have no idea why anyone would poll with 'shutdown'. If you want to
    shutdown the connection, call 'shutdown'. If you need to track the
    connection state, you should already be doing that. Detecting a clean
    shutdown is the same whether you initiated it or the other side
    initiated it, and the other side might initiate it at any time. So if
    you don't already properly handle a connection shutdown, your code is
    already broken.

    > >Shutting down a connection and closing a reference to the connection
    > >endpoint are completely logically distinct operations. It is not good
    > >to confuse them.


    > Yes they are distinct operations. But a close() implicitely does a
    > shutdown() on last call to close(). And this is very helpful especially
    > when something goes wrong, i.e. a process terminates abnormally.


    Yes, if that process had the last reference to that connection. No if
    it didn't. You can rely on 'close' doing an implicit 'shutdown' if and
    only if you have complete control over every descriptor that might
    refer to that connection.

    > Over the last 20+ years I have seen a lot of server applications that
    > failed handling the close() right in some situations. The result was
    > most of the time hanging connections blocking ports, not enough file
    > descriptors and hanging client applications. Avoiding socket sharing
    > over multiple processes prevents such problems from the start.


    So does calling 'shutdown' when you want to shut the connection down.

    > All this complicated error handling is not worth the effort, and may be
    > most of the time itself a major cause of problems.


    It's only complicated if you don't understand it.

    DS

  17. Re: Shared sockets

    On Dec 24, 7:08 am, v...@calcite.rhyolite.com (Vernon Schryver) wrote:

    > When performance matters, it is better to avoid both the separate
    > processes of fork() and separate threads. Threads are still, even at
    > this late date, often illusions based on select() and in any case are
    > more expensive than a single process that uses select() (or poll() when
    > available) directly.


    That's wrong for so many reasons:

    1) A single process without threads cannot take advantage of multiple
    CPUs the way a multi-threaded process can. Most machines made today
    have more than one core.

    2) A single process that uses 'select' or 'poll' must be carefully
    written so that no line of code ever blocks on pain of disaster. This
    makes every single line of code performance critical, rather than the
    20% of the code that actually affects performance.

    3) If you block unexpectedly, say due to a page fault, your entire
    process stalls. This can cause severe burstiness problems.

    These are just the main reasons. That's just totally and completely
    wrong.

    DS

  18. Re: Shared sockets

    On Dec 25, 10:49 pm, Emil Naepflein wrote:

    > On todays multiprocessor systems this would limit the performance to one
    > cpu. And with thousands of connections you will have to think about
    > fairness regarding processing of concurrent ready file descriptors of
    > different types. This is a very tricky issue.


    You usually have to work to make things unfair. You will generally get
    fairness automatically.

    The problem is simply that none of your code can ever block under any
    circumstances or you are dead. (see my other post)

    DS

  19. Re: Shared sockets

    On Wed, 26 Dec 2007 22:24:12 -0800 (PST), David Schwartz
    wrote:

    >On Dec 25, 10:49 pm, Emil Naepflein wrote:
    >
    >> On todays multiprocessor systems this would limit the performance to one
    >> cpu. And with thousands of connections you will have to think about
    >> fairness regarding processing of concurrent ready file descriptors of
    >> different types. This is a very tricky issue.

    >
    >You usually have to work to make things unfair. You will generally get
    >fairness automatically.


    Just think about how you scan the ready file descriptors. If you do it
    sequentially starting at the lowest number the file descriptors at the
    beginning always get the fastest service. If you handle thousands of
    them then there will be a significant difference.

    You also will get a high variance of response times.

    To avoid this you would have to reimplement a kind of "fair" scheduling
    just like it is done for processes/threads in the OS. So it may be
    easier to just use one native process/thread to handle one connection
    and let the OS do the work. Of course, it also depends how long living
    your connections are. If you have a very high connection rate with short
    living connections then the select/poll implemention may be the way to
    go (beside the multiprocessor issue). If there are very long living
    connections then the overhead of creating processes/threads is
    amortisized over time.

  20. Re: Shared sockets

    In article ,
    vjs@calcite.rhyolite.com (Vernon Schryver) wrote:

    > When performance matters, it is better to avoid both the separate
    > processes of fork() and separate threads. Threads are still, even at
    > this late date, often illusions based on select() and in any case are
    > more expensive than a single process that uses select() (or poll() when
    > available) directly.


    Is that (emulated threading) really true on any high-performance flavor
    of Unix these days?

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

+ Reply to Thread
Page 1 of 2 1 2 LastLast