Server socket: how to limit the number of connections? - Unix

This is a discussion on Server socket: how to limit the number of connections? - Unix ; I am writing a TCP/IP (socket) server application, and I want to limit the number of connections active at any one time. Ideally, once I have MAX connections, I would like any additional attempts to get "connection refused" My first ...

+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 24

Thread: Server socket: how to limit the number of connections?

  1. Server socket: how to limit the number of connections?

    I am writing a TCP/IP (socket) server application,
    and I want to limit the number of connections active
    at any one time. Ideally, once I have MAX connections,
    I would like any additional attempts to get
    "connection refused"

    My first thought was to just have my code not
    call "accept" whenever there are MAX connections,
    but I can't find any documentation as to what happens
    to the connection attempts if "listen" has been called
    but you don't call "accept". I'd rather not have the
    connection attempt wait until it times out, and
    just doing "accept" and closing the socket won't
    make it clear to the client what is going on.

    If it makes a difference, I expect that the connections
    will remain in place for a fairly long time
    (usually O(several hours) )

    Alternatively, is there a way to change the "backlog" parameter
    on the listen socket, other than by closing and re-opening the
    listen socket?

    Or is there another way to do what I want to do?

  2. Re: Server socket: how to limit the number of connections?

    Alan McKenney writes:
    >I am writing a TCP/IP (socket) server application,
    >and I want to limit the number of connections active
    >at any one time. Ideally, once I have MAX connections,
    >I would like any additional attempts to get
    >"connection refused"
    >
    >My first thought was to just have my code not
    >call "accept" whenever there are MAX connections,
    >but I can't find any documentation as to what happens
    >to the connection attempts if "listen" has been called
    >but you don't call "accept". I'd rather not have the
    >connection attempt wait until it times out, and
    >just doing "accept" and closing the socket won't
    >make it clear to the client what is going on.
    >
    >If it makes a difference, I expect that the connections
    >will remain in place for a fairly long time
    >(usually O(several hours) )
    >
    >Alternatively, is there a way to change the "backlog" parameter
    >on the listen socket, other than by closing and re-opening the
    >listen socket?
    >
    >Or is there another way to do what I want to do?


    use listen(2) and specify MAX as the backlog parameter.

    scott

  3. Re: Server socket: how to limit the number of connections?

    In article ,
    scott@slp53.sl.home (Scott Lurndal) wrote:

    > Alan McKenney writes:
    > >I am writing a TCP/IP (socket) server application,
    > >and I want to limit the number of connections active
    > >at any one time. Ideally, once I have MAX connections,
    > >I would like any additional attempts to get
    > >"connection refused"
    > >
    > >My first thought was to just have my code not
    > >call "accept" whenever there are MAX connections,
    > >but I can't find any documentation as to what happens
    > >to the connection attempts if "listen" has been called
    > >but you don't call "accept". I'd rather not have the
    > >connection attempt wait until it times out, and
    > >just doing "accept" and closing the socket won't
    > >make it clear to the client what is going on.


    The backlog parameter of listen() specifies how many connections should
    be established awaiting calls to accept(). I'm not sure if what happens
    to additional connection attempts is specified -- I think some systems
    refuse them, while others ignore them (so the client will keep retrying
    and eventually time out). Also, I believe some implementations have a
    built-in minimum backlog -- if you specify 0, it's treated as something
    like 5.

    > >
    > >If it makes a difference, I expect that the connections
    > >will remain in place for a fairly long time
    > >(usually O(several hours) )
    > >
    > >Alternatively, is there a way to change the "backlog" parameter
    > >on the listen socket, other than by closing and re-opening the
    > >listen socket?
    > >
    > >Or is there another way to do what I want to do?

    >
    > use listen(2) and specify MAX as the backlog parameter.


    No, see above for a more accurate description of how the backlog is used.

    Anyway, I think the usual way this is done is to close the listening
    socket when you reach MAX connections. When the number of connections
    drops, you open a new listening socket. The only problem with this is
    that there's a window of error. If a connection attempt is received
    between the time that you receive the MAX'th connection and when you
    close the listening socket, it will be established. I believe closing
    the socket will then cause that extra connection to be aborted.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  4. Re: Server socket: how to limit the number of connections?

    Alan McKenney wrote:

    > I am writing a TCP/IP (socket) server application,
    > and I want to limit the number of connections active
    > at any one time. Ideally, once I have MAX connections,
    > I would like any additional attempts to get
    > "connection refused"
    > Or is there another way to do what I want to do?


    Maybe semaphores is what you need.

    sem_init (&sem, 1, MAXCONNS);

    man sem_init, sem_trywait, sem_wait, sem_post

  5. Re: Server socket: how to limit the number of connections?

    In article ,
    Ivan Chernetsky wrote:

    > Alan McKenney wrote:
    >
    > > I am writing a TCP/IP (socket) server application,
    > > and I want to limit the number of connections active
    > > at any one time. Ideally, once I have MAX connections,
    > > I would like any additional attempts to get
    > > "connection refused"
    > > Or is there another way to do what I want to do?

    >
    > Maybe semaphores is what you need.
    >
    > sem_init (&sem, 1, MAXCONNS);
    >
    > man sem_init, sem_trywait, sem_wait, sem_post


    Semaphores are only useful if the other processes you're interacting
    with also use the semaphore. But if they're network clients, they can't.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  6. Re: Server socket: how to limit the number of connections?

    On Apr 10, 3:36 pm, Alan McKenney wrote:
    > I am writing a TCP/IP (socket) server application,
    > and I want to limit the number of connections active
    > at any one time. Ideally, once I have MAX connections,
    > I would like any additional attempts to get
    > "connection refused"


    In order to get "connection refused", there must not be a socket in
    the LISTEN state. I.e., you would have to close() the listening
    socket.


    > My first thought was to just have my code not
    > call "accept" whenever there are MAX connections,
    > but I can't find any documentation as to what happens
    > to the connection attempts if "listen" has been called
    > but you don't call "accept".


    Some number of connections will proceed through the 3-packet handshake
    and will thus will appear to the client to have been accepted. There
    number of connections for which this will occur is generally a
    function of the listen() backlog parameter, but the exact relation
    depends on the OS. Connection attempts beyond that limit will simply
    be ignored; the client will generally resend the SYN some number of
    times before timing out.

    This is all described in Stevens's _UNIX_Network_Programming_
    _volume_1_. In particular, section 4.5 describes the listen function
    and how it affects what clients 'see'. It even has a table describing
    the relationship between the backlog parameter and the number of
    queued connections for various OSes.


    > I'd rather not have the
    > connection attempt wait until it times out, and
    > just doing "accept" and closing the socket won't
    > make it clear to the client what is going on.


    Well, it'll be clearer than the result of leaving the listening socket
    without calling accept(). Some internet protocols have the server
    send a banner (e.g., SMTP); for such protocols the server in an
    overload situation can send a "temp failure" banner before closing the
    connection.


    > Alternatively, is there a way to change the "backlog" parameter
    > on the listen socket, other than by closing and re-opening the
    > listen socket?


    Changing the backlog probably won't get you what you want.


    > Or is there another way to do what I want to do?


    Close the listening socket when you hit an upper limit, then reopen/
    bind/listen when the load drops below some lower limit. (Two limits
    to provide some hysteresis to reduce how often the socket is closed
    and reopened.)


    Philip Guenther

  7. Re: Server socket: how to limit the number of connections?

    On Apr 13, 7:08 pm, "guent...@gmail.com" wrote:
    > On Apr 10, 3:36 pm, Alan McKenney wrote:
    >
    > > I am writing a TCP/IP (socket) server application,
    > > and I want to limit the number of connections active
    > > at any one time. Ideally, once I have MAX connections,
    > > I would like any additional attempts to get
    > > "connection refused"

    >
    > In order to get "connection refused", there must not be a socket in
    > the LISTEN state. I.e., you would have to close() the listening
    > socket.
    >
    > > My first thought was to just have my code not
    > > call "accept" whenever there are MAX connections,
    > > but I can't find any documentation as to what happens
    > > to the connection attempts if "listen" has been called
    > > but you don't call "accept".

    >
    > Some number of connections will proceed through the 3-packet handshake
    > and will thus will appear to the client to have been accepted. There
    > number of connections for which this will occur is generally a
    > function of the listen() backlog parameter, but the exact relation
    > depends on the OS. Connection attempts beyond that limit will simply
    > be ignored; the client will generally resend the SYN some number of
    > times before timing out.
    >
    > This is all described in Stevens's _UNIX_Network_Programming_
    > _volume_1_. In particular, section 4.5 describes the listen function
    > and how it affects what clients 'see'. It even has a table describing
    > the relationship between the backlog parameter and the number of
    > queued connections for various OSes.
    >
    > > I'd rather not have the
    > > connection attempt wait until it times out, and
    > > just doing "accept" and closing the socket won't
    > > make it clear to the client what is going on.

    >
    > Well, it'll be clearer than the result of leaving the listening socket
    > without calling accept(). Some internet protocols have the server
    > send a banner (e.g., SMTP); for such protocols the server in an
    > overload situation can send a "temp failure" banner before closing the
    > connection.
    >
    > > Alternatively, is there a way to change the "backlog" parameter
    > > on the listen socket, other than by closing and re-opening the
    > > listen socket?

    >
    > Changing the backlog probably won't get you what you want.
    >
    > > Or is there another way to do what I want to do?

    >
    > Close the listening socket when you hit an upper limit, then reopen/
    > bind/listen when the load drops below some lower limit. (Two limits
    > to provide some hysteresis to reduce how often the socket is closed
    > and reopened.)
    >
    > Philip Guenther



    Test ulimit.
    Using ulimit, the unique problem is that the child process will be
    limited too.

    I have tested it in solaris and linux.

    skylazart@skylAZART:~/stuff/desenvolvimento/testes$ ulimit -n
    1024
    skylazart@skylAZART:~/stuff/desenvolvimento/testes$ ./maxfds
    i = 1021
    skylazart@skylAZART:~/stuff/desenvolvimento/testes$ ulimit -n 256
    skylazart@skylAZART:~/stuff/desenvolvimento/testes$ ./maxfds
    i = 253
    skylazart@skylAZART:~/stuff/desenvolvimento/testes$

    See also:

    setrlimit, getrlimit and sysconf (for linux)

  8. Re: Server socket: how to limit the number of connections?

    In article
    ,
    "skylazart@gmail.com" wrote:

    > Test ulimit.
    > Using ulimit, the unique problem is that the child process will be
    > limited too.


    How would this help? It would prevent the server from accessing the
    connections with accept(), but it cause the clients to get "Connection
    refused".

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  9. Re: Server socket: how to limit the number of connections?

    On Apr 10, 2:36 pm, Alan McKenney wrote:

    > Ideally, once I have MAX connections,
    > I would like any additional attempts to get
    > "connection refused"


    The short answer is, no, you cannot do this. This is considered a bad
    idea, so the API does not make it easy to do. The "connection refused"
    error is badly-named, it means "host exists but server is not running
    on it".

    DS

  10. Re: Server socket: how to limit the number of connections?

    In article
    ,
    David Schwartz wrote:

    > On Apr 10, 2:36 pm, Alan McKenney wrote:
    >
    > > Ideally, once I have MAX connections,
    > > I would like any additional attempts to get
    > > "connection refused"

    >
    > The short answer is, no, you cannot do this. This is considered a bad
    > idea, so the API does not make it easy to do. The "connection refused"
    > error is badly-named, it means "host exists but server is not running
    > on it".


    If the server is full and can't handle any more clients, it might as
    well not be running. So what's so wrong with returning that error?

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  11. Re: Server socket: how to limit the number of connections?

    On Apr 14, 10:30 pm, Barry Margolin wrote:

    > > The short answer is, no, you cannot do this. This is considered a bad
    > > idea, so the API does not make it easy to do. The "connection refused"
    > > error is badly-named, it means "host exists but server is not running
    > > on it".


    > If the server is full and can't handle any more clients, it might as
    > well not be running. So what's so wrong with returning that error?


    There are two possible sub-cases:

    1) The other side is designed with this case in mind. In which case,
    the server should accept the connection, send a protocol-specific code
    that means "this server is too full to handle you", and then drop the
    connection.

    2) The other side is not designed with this case in mind. In which
    case, it is just as likely to do the wrong thing when it gets
    "connection refused" as the right thing.

    The OP is asking for a way to teach TCP *his* protocol's quirks.
    That's almost always a mistake. The right thing to do is to design
    your protocol around TCP's quirks.

    DS

  12. Re: Server socket: how to limit the number of connections?

    Barry Margolin wrote:
    > In article
    > ,
    > David Schwartz wrote:


    > > On Apr 10, 2:36 pm, Alan McKenney wrote:
    > >
    > > > Ideally, once I have MAX connections,
    > > > I would like any additional attempts to get
    > > > "connection refused"

    > >
    > > The short answer is, no, you cannot do this. This is considered a bad
    > > idea, so the API does not make it easy to do. The "connection refused"
    > > error is badly-named, it means "host exists but server is not running
    > > on it".


    > If the server is full and can't handle any more clients, it might as
    > well not be running. So what's so wrong with returning that error?


    Isn't that what happens when a Unixy client contacts a Windows server
    with a full listen queue - Windows returns a RST and the Unixy client
    reports connection refused - unless it is tweaked to account for
    Windows' quirk.

    rick jones
    --
    oxymoron n, Hummer H2 with California Save Our Coasts and Oceans plates
    these opinions are mine, all mine; HP might not want them anyway...
    feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

  13. Re: Server socket: how to limit the number of connections?

    On Apr 15, 3:07 pm, Rick Jones wrote:

    > Isn't that what happens when a Unixy client contacts a Windows server
    > with a full listen queue - Windows returns a RST and the Unixy client
    > reports connection refused - unless it is tweaked to account for
    > Windows' quirk.


    Yeah, that's correct. And Windows clients work around the quirk by
    retrying even if a connection is refused. The net effect is that if
    you need to interoperate, you can't rely on either behavior. I have
    never heard a good explanation of why Windows does this. It's hard to
    imagine that it was intentional -- and if it was, what could the
    thinking possibly have been?

    DS

  14. Re: Server socket: how to limit the number of connections?

    On Apr 15, 7:50 pm, David Schwartz wrote:
    > On Apr 15, 3:07 pm, Rick Jones wrote:
    >
    > > Isn't that what happens when a Unixy client contacts a Windows server
    > > with a full listen queue - Windows returns a RST and the Unixy client
    > > reports connection refused - unless it is tweaked to account for
    > > Windows' quirk.

    >
    > Yeah, that's correct. And Windows clients work around the quirk by
    > retrying even if a connection is refused. The net effect is that if
    > you need to interoperate, you can't rely on either behavior. I have
    > never heard a good explanation of why Windows does this. It's hard to
    > imagine that it was intentional -- and if it was, what could the
    > thinking possibly have been?
    >
    > DS


    Another ideia:

    use a global var to count the number of connections. Increment it when
    accept a new connection and decrement when receive a signal child
    indicating that a child process died.


    Like this:

    int total_number_of_conn = 0;

    void
    handle_sigchld (int sig)
    {
    waitpid ();
    total_number_of_conn--;
    }

    int
    main (int argc, char **argv)
    {
    signal (SIGPIPE, SIG_IGN);
    signal (SIGCHLD, handle_sigchld);

    for (; {
    accept ();
    total_number_of_conn++;

    }

    return (0)
    }




  15. Re: Server socket: how to limit the number of connections?

    In article
    <4a71fa1e-10c2-4145-9af7-cb9e9770d856@s50g2000hsb.googlegroups.com>,
    "skylazart@gmail.com" wrote:

    > On Apr 15, 7:50 pm, David Schwartz wrote:
    > > On Apr 15, 3:07 pm, Rick Jones wrote:
    > >
    > > > Isn't that what happens when a Unixy client contacts a Windows server
    > > > with a full listen queue - Windows returns a RST and the Unixy client
    > > > reports connection refused - unless it is tweaked to account for
    > > > Windows' quirk.

    > >
    > > Yeah, that's correct. And Windows clients work around the quirk by
    > > retrying even if a connection is refused. The net effect is that if
    > > you need to interoperate, you can't rely on either behavior. I have
    > > never heard a good explanation of why Windows does this. It's hard to
    > > imagine that it was intentional -- and if it was, what could the
    > > thinking possibly have been?
    > >
    > > DS

    >
    > Another ideia:
    >
    > use a global var to count the number of connections. Increment it when
    > accept a new connection and decrement when receive a signal child
    > indicating that a child process died.


    What does this have to do with the question? I don't think he's
    confused about how to count how many connections are open, he wants to
    know how to prevent the stack from accepting more connections once he
    reaches the limit.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  16. Re: Server socket: how to limit the number of connections?

    On 15 abr, 19:23, Barry Margolin wrote:
    > In article
    > <4a71fa1e-10c2-4145-9af7-cb9e9770d...@s50g2000hsb.googlegroups.com>,
    >
    >
    >
    > "skylaz...@gmail.com" wrote:
    > > On Apr 15, 7:50 pm, David Schwartz wrote:
    > > > On Apr 15, 3:07 pm, Rick Jones wrote:

    >
    > > > > Isn't that what happens when a Unixy client contacts a Windows server
    > > > > with a full listen queue - Windows returns a RST and the Unixy client
    > > > > reports connection refused - unless it is tweaked to account for
    > > > > Windows' quirk.

    >
    > > > Yeah, that's correct. And Windows clients work around the quirk by
    > > > retrying even if a connection is refused. The net effect is that if
    > > > you need to interoperate, you can't rely on either behavior. I have
    > > > never heard a good explanation of why Windows does this. It's hard to
    > > > imagine that it was intentional -- and if it was, what could the
    > > > thinking possibly have been?

    >
    > > > DS

    >
    > > Another ideia:

    >
    > > use a global var to count the number of connections. Increment it when
    > > accept a new connection and decrement when receive a signal child
    > > indicating that a child process died.

    >
    > What does this have to do with the question? I don't think he's
    > confused about how to count how many connections are open, he wants to
    > know how to prevent the stack from accepting more connections once he
    > reaches the limit.
    >
    > --
    > Barry Margolin, bar...@alum.mit.edu
    > Arlington, MA
    > *** PLEASE post questions in newsgroups, not directly to me ***
    > *** PLEASE don't copy me on replies, I'll read them in the group ***


    I have just send two ideias:
    ulimit setrlimit etc
    and counting var...

    Both resolve the problem. When you get the MAX connections, close the
    socket server.


  17. Re: Server socket: how to limit the number of connections?

    On Apr 16, 1:45 pm, "skylaz...@gmail.com" wrote:

    > Both resolve the problem. When you get the MAX connections, close the
    > socket server.


    This may be the best choice depending upon the exact requirements, but
    it has a few downsides:

    1) You may be unable to re-open the port unless you set the address re-
    usable. Even if you do, another process might grab the port.
    (Especially if it's inside the range of dynamic ports.) This is
    especially bad for servers that choose a random port on startup and
    'advertise' that port.

    2) There is a performance cost associated with constantly closing and
    re-opening the listening socket.

    3) Windows clients, if you have any, will still make repeated attempts
    to connect and timeout. They will essentially ignore the RST.

    If possible, it is usually far better to accept the connection, send a
    "I cannot service your request now" message, and then close them. Or
    even just close the connection without sending anything at all,
    depending on the protocol.

    DS

  18. Re: Server socket: how to limit the number of connections?

    David Schwartz writes:

    > On Apr 10, 2:36 pm, Alan McKenney wrote:
    >
    >> Ideally, once I have MAX connections,
    >> I would like any additional attempts to get
    >> "connection refused"

    >
    > The short answer is, no, you cannot do this. This is considered a
    > bad idea, so the API does not make it easy to do. The "connection
    > refused" error is badly-named, it means "host exists but server is
    > not running on it".


    What if you close the listening socket when you have all the
    connections you want?


  19. Re: Server socket: how to limit the number of connections?

    On Apr 16, 4:30 pm, Goober wrote:
    > David Schwartz writes:
    > > On Apr 10, 2:36 pm, Alan McKenney wrote:


    > >> Ideally, once I have MAX connections,
    > >> I would like any additional attempts to get
    > >> "connection refused"


    > > The short answer is, no, you cannot do this. This is considered a
    > > bad idea, so the API does not make it easy to do. The "connection
    > > refused" error is badly-named, it means "host exists but server is
    > > not running on it".


    > What if you close the listening socket when you have all the
    > connections you want?


    That's precisely what I'm recommending against, for all the reasons
    that I explained. As soon as one of those connections closes, you need
    to re-open the listening socket.

    DS


  20. Re: Server socket: how to limit the number of connections?

    On Apr 16, 6:56 pm, David Schwartz wrote:
    > On Apr 16, 1:45 pm, "skylaz...@gmail.com" wrote:
    >
    > > Both resolve the problem. When you get the MAX connections, close the
    > > socket server.


    That's what I plan to do, unless it turns out to not work.

    > .. but it has a few downsides:
    >
    > 1) You may be unable to re-open the port unless you set the address re-
    > usable. Even if you do, another process might grab the port.
    > (Especially if it's inside the range of dynamic ports.) This is
    > especially bad for servers that choose a random port on startup and
    > 'advertise' that port.


    a. Thanks for the reminder about "reusable." I don't fully
    understand
    what it does, but I'm re-engineering existing code which has
    it all over the place, so I'm putting it in the new code.

    b. We staticly assign port numbers (and put them in /etc/services),
    so if the port does get grabbed, we go and beat someone up (well,
    verbally abuse them, anyway.)

    c. No idea whether these port numbers are in the "dynamic port"
    range. The ones I'm familiar with are in the 8,000-20,000 range.

    > 2) There is a performance cost associated with constantly closing and
    > re-opening the listening socket.


    In the situations where it happens, it will normally happen 1x per
    day
    or so, as our connections generally run for a full "business day"
    (see original post.)

    And I can't see how doing one close and re-open is worse than
    spending 8 to 24 hours doing accept and close calls every
    second or so.

    > 3) Windows clients, if you have any, will still make repeated attempts
    > to connect and timeout. They will essentially ignore the RST.


    So far, we haven't had to deal with Windows clients (as far
    as I know.) If any clients turn out to be Windows, they aren't
    under our control, and it's going to be their owners' problem.

    > If possible, it is usually far better to accept the connection, send a
    > "I cannot service your request now" message, and then close them. Or
    > even just close the connection without sending anything at all,
    > depending on the protocol.


    This is code we (will) use all over the place.
    At the level I'm working with, the protocol is TCP/IP.

    Depending on where it's used, the higher level protocol could
    be anything (mostly protocols we have no control over),
    but the most common is simply a one-way stream of data bytes.

    Hence, my preference for "connection refused."


+ Reply to Thread
Page 1 of 2 1 2 LastLast