GetQueuedCompletionStatus: when it's safe to delete the completionkey - Programmer

This is a discussion on GetQueuedCompletionStatus: when it's safe to delete the completionkey - Programmer ; Hello, I use IOCP for a socket server. When a new connection is created, the connection's session is created in memory dynamically and its address is passed to CreateIoCompletionPort as the completion key when the connection's socket is associated with ...

+ Reply to Thread
Results 1 to 13 of 13

Thread: GetQueuedCompletionStatus: when it's safe to delete the completionkey

  1. GetQueuedCompletionStatus: when it's safe to delete the completionkey

    Hello,

    I use IOCP for a socket server. When a new connection is created, the
    connection's session is created in memory dynamically and its address
    is passed to CreateIoCompletionPort as the completion key when the
    connection's socket is associated with a completion port. My question
    is - when it is 100% safe to remove the connection's session (address
    of which GetQueuedCompletionStatus returns as the completion key when
    an operation on the socket completes) from memory? In other words,
    what combination of both the return value and the output parameters of
    GetQueuedCompletionStatus is a 100% indication of that
    GetQueuedCompletionStatus will never return this same completion key
    again?

    Thanks in advance

  2. Re: GetQueuedCompletionStatus: when it's safe to delete thecompletion key

    On Jun 21, 10:05*pm, Dennis Mikhailitsky
    wrote:
    > Hello,
    >
    > I use IOCP for a socket server. When a new connection is created, the
    > connection's session is created in memory dynamically and its address
    > is passed to CreateIoCompletionPort as the completion key when the
    > connection's socket is associated with a completion port. My question
    > is - when it is 100% safe to remove the connection's session (address
    > of which GetQueuedCompletionStatus returns as the completion key when
    > an operation on the socket completes) from memory? In other words,
    > what combination of both the return value and the output parameters of
    > GetQueuedCompletionStatus is a 100% indication of that
    > GetQueuedCompletionStatus will never return this same completion key
    > again?
    >
    > Thanks in advance



    Usually, these are two ways to safe release resource associated with
    IOCP.

    The first is using refcounting. When issuing a WSARecv/WSASend,
    increase
    refcount. When GetQueuedCompletionStatus return, decrease refcount.
    It is safe to release resouece when refcount is zero.

    The second is using a hashtable. For example, pass SocketHandle to
    CreateIoCompletionPort as the completion key, and push the pair --
    (SocketHandle, Session) -- into hashtable. When
    GetQueuedCompletionStatus
    return, get the session from the hashtable by the SocketHandle.
    When it need to release the session, remove the pair from the
    hashtable first.
    After that, when GetQueuedCompletionStatus return, it cannot get the
    session
    from the hashtable any more.

  3. Re: GetQueuedCompletionStatus: when it's safe to delete thecompletion key

    On Jun 21, 7:05*am, Dennis Mikhailitsky
    wrote:

    > I use IOCP for a socket server. When a new connection is created, the
    > connection's session is created in memory dynamically and its address
    > is passed to CreateIoCompletionPort as the completion key when the
    > connection's socket is associated with a completion port. My question
    > is - when it is 100% safe to remove the connection's session (address
    > of which GetQueuedCompletionStatus returns as the completion key when
    > an operation on the socket completes) from memory? In other words,
    > what combination of both the return value and the output parameters of
    > GetQueuedCompletionStatus is a 100% indication of that
    > GetQueuedCompletionStatus will never return this same completion key
    > again?


    Every operation will, unless canceled, complete exactly once. If that
    doesn't answer your question, then I don't understand your question.

    DS

  4. Re: GetQueuedCompletionStatus: when it's safe to delete thecompletion key

    I tend to use the reference counted design that 'iuknown' suggests.

    If you want to see an example of this then take a look at my server
    framework code that's available here: http://www.lenholgate.com/archives/000637.html

    Len

  5. Re: GetQueuedCompletionStatus: when it's safe to delete thecompletion key

    On 23 , 10:05, David Schwartz wrote:
    > On Jun 21, 7:05am, Dennis Mikhailitsky
    > wrote:
    >
    >
    > Every operation will, unless canceled, complete exactly once. If that
    > doesn't answer your question, then I don't understand your question.
    >
    > DS


    Why could the following happen: a socket is closed in abortive manner
    by calling closesocket, in about 15 mins the associated overlapped
    sturcture is removed from memory, yet at some point after that
    GetQueuedCompletionStatus returns non-zero as the return value, size
    == 0, and pointer to the overlapped structure that has already been
    removed from memory? This happens very rarely, therefore I cannot
    explore the nature of the issue as I cannot reproduce it consistently,
    and this drives me crazy.

  6. Re: GetQueuedCompletionStatus: when it's safe to delete thecompletion key

    I can restructure my original question a little bit - is there a way
    to remove association between a socket and a completion port
    completely and forever? In other words, is there a way to do an action
    opposite to what CreateIoCompletionPort does - to make
    GetQueuedCompletionStatus immediately stop returning a given
    completion key once and forever, no matter what?

  7. Re: GetQueuedCompletionStatus: when it's safe to delete the completion key

    no - the only way to remove an IO completion port association is to close
    the handle.

    if this hapens in your code, it means that your reference counts are wrong
    or you are frering the OVERLAPPED somwhere that is not the IO completion
    handler

    "Dennis Mikhailitsky" wrote in message
    news:4a877f5a-5212-4470-98da-818c32740015@m45g2000hsb.googlegroups.com...
    >I can restructure my original question a little bit - is there a way
    > to remove association between a socket and a completion port
    > completely and forever? In other words, is there a way to do an action
    > opposite to what CreateIoCompletionPort does - to make
    > GetQueuedCompletionStatus immediately stop returning a given
    > completion key once and forever, no matter what?




  8. Re: GetQueuedCompletionStatus: when it's safe to delete thecompletion key

    On Jun 26, 8:17*am, Dennis Mikhailitsky
    wrote:

    > > Every operation will, unless canceled, complete exactly once. If that
    > > doesn't answer your question, then I don't understand your question.


    > Why could the following happen: a socket is closed in abortive manner
    > by calling closesocket, in about 15 mins the associated overlapped
    > sturcture is removed from memory, yet at some point after that
    > GetQueuedCompletionStatus returns non-zero as the return value, size
    > == 0, and pointer to the overlapped structure that has already been
    > removed from memory?


    For the reason I stated above -- every operation will, unless
    canceled, complete exactly once. Some operation had been started and
    hadn't complete yet, so it must complete later. There is no guaranteed
    time frame for operation completion.

    > This happens very rarely, therefore I cannot
    > explore the nature of the issue as I cannot reproduce it consistently,
    > and this drives me crazy.


    I cannot imagine why you would want to call 'closesocket' while an
    operation is in progress on the socket. I don't know for sure whether
    there's any safe way to do so -- it's not immediately obvious to me
    that it's impossible, but it seems really difficult.

    Consider:

    1) A thread calls 'GetQueuedCompletionStatus' and gets a completion
    indication for this socket, but it blocks on a page fault.

    2) Some other thread closes the socket.

    3) Lots of other stuff happens.

    4) The thread in step 1 finally pages in the page it was waiting for
    and continues, processing the completion indication it already got for
    the socket you closed in step 2.

    I would recommend not calling 'closesocket' until you are completely
    finished using the socket because it seems like an awful lot of extra
    work to do it safely. It will cancel all pending operations, they will
    complete with an error. Microsoft says:

    "Any pending overlapped send and receive operations ( WSASend/
    WSASendTo/ WSARecv/ WSARecvFrom with an overlapped socket) issued by
    any thread in this process are also canceled. Any event, completion
    routine, or completion port action specified for these overlapped
    operations is performed. The pending overlapped operations fail with
    the error status WSA_OPERATION_ABORTED."

    "An application should not assume that any outstanding I/O operations
    on a socket will all be guaranteed to completed when closesocket
    returns. The closesocket function will initiate cancellation on the
    outstanding I/O operations, but that does not mean that an application
    will receive I/O completion for these I/O operations by the time the
    closesocket function returns. Thus, an application should not cleanup
    any resources (WSAOVERLAPPED structures, for example) referenced by
    the outstanding I/O requests until the I/O requests are indeed
    completed."

    DS

  9. Re: GetQueuedCompletionStatus: when it's safe to delete thecompletion key

    On Jun 26, 9:26*am, Dennis Mikhailitsky
    wrote:

    > I can restructure my original question a little bit - is there a way
    > to remove association between a socket and a completion port
    > completely and forever? In other words, is there a way to do an action
    > opposite to what CreateIoCompletionPort does - to make
    > GetQueuedCompletionStatus immediately stop returning a given
    > completion key once and forever, no matter what?


    I don't see how that will help you, since a thread may already be in
    the process of returning from GetQueuedCompletionStatus.

    DS

  10. Re: GetQueuedCompletionStatus: when it's safe to delete thecompletion key

    > Why could the following happen: a socket is closed in abortive manner
    > by calling closesocket, in about 15 mins the associated overlapped
    > sturcture is removed from memory, yet at some point after that
    > GetQueuedCompletionStatus returns non-zero as the return value, size
    > == 0, and pointer to the overlapped structure that has already been
    > removed from memory? This happens very rarely, therefore I cannot
    > explore the nature of the issue as I cannot reproduce it consistently,
    > and this drives me crazy.


    If you are using reference counts on the overlapped structure then you
    have a bug. Each operation completes once. Closing the socket doesnt
    change that. The operations will still complete once. Increment the
    reference count when you issue the request, decrement it when the
    operation completes.

    In your example above, why is the overlapped structure removed from
    memory 15 mins after the socket is closed, what causes that removal?

    Len
    http://www.lenholgate.com

  11. Re: GetQueuedCompletionStatus: when it's safe to delete thecompletion key

    On Jun 26, 11:17*pm, Dennis Mikhailitsky
    wrote:
    > On 23 , 10:05, David Schwartz wrote:
    >
    > > On Jun 21, 7:05am, Dennis Mikhailitsky
    > > wrote:

    >
    > > Every operation will, unless canceled, complete exactly once. If that
    > > doesn't answer your question, then I don't understand your question.

    >
    > > DS

    >
    > Why could the following happen: a socket is closed in abortive manner
    > by calling closesocket, in about 15 mins the associated overlapped
    > sturcture is removed from memory, yet at some point after that
    > GetQueuedCompletionStatus returns non-zero as the return value, size
    > == 0, and pointer to the overlapped structure that has already been
    > removed from memory? This happens very rarely, therefore I cannot
    > explore the nature of the issue as I cannot reproduce it consistently,
    > and this drives me crazy.


    Does the closesocket return OK ? Or has any errno ?
    Generally, after calling closesocket is OK, the pending IO operations
    will
    return by GQCS( GetQueuedCompletionStatus ) immediately.

  12. Re: GetQueuedCompletionStatus: when it's safe to delete thecompletion key

    Hi Everyone,

    Looks like the problem is gone. I wasnt sure exactly which sends and
    receives get queued to the completion port and which dont, therefore I
    didnt not use reference counters at all and was relying on the GQCS's
    return code and combination of the out parameters when making a
    decision to destroy the overlapped structure, which obviously was a
    bad idea. Now I have implemented reference counters and the problem
    seems to be gone. Thanks a lot to all of you for helping to solve this
    problem.

    On 3 Лип, 08:59, iunknown wrote:
    > On Jun 26, 11:17*pm, Dennis Mikhailitsky
    > wrote:
    >
    >
    >
    >
    >
    > > On 23 þÅÒ, 10:05, David Schwartz wrote:

    >
    > > > On Jun 21, 7:05šam, Dennis Mikhailitsky
    > > > wrote:

    >
    > > > Every operation will, unless canceled, complete exactly once. If that
    > > > doesn't answer your question, then I don't understand your question.

    >
    > > > DS

    >
    > > Why could the following happen: a socket is closed in abortive manner
    > > by calling closesocket, in about 15 mins the associated overlapped
    > > sturcture is removed from memory, yet at some point after that
    > > GetQueuedCompletionStatus returns non-zero as the return value, size
    > > == 0, and pointer to the overlapped structure that has already been
    > > removed from memory? This happens very rarely, therefore I cannot
    > > explore the nature of the issue as I cannot reproduce it consistently,
    > > and this drives me crazy.

    >
    > Does the closesocket return OK ? Or has any errno ?
    > Generally, after calling closesocket is OK, the pending IO operations
    > will
    > return by GQCS( GetQueuedCompletionStatus ) immediately.



  13. Smile Re: GetQueuedCompletionStatus: when it's safe to delete the completionkey

    Hi..
    As you say in the end of this thread your problem is solved.
    OK its solved but how you can managed for time out of the operation.

    I mean to say if we send data to the client and client not responding than
    GetQueuedCompletionStatus will not come out from waiting state untill WsaSend
    Completed.In this case how to remove the resources of the client at serverside.
    Means how can we confirm that client is dad so remove it from CompletionPort.
    And how to remove that client resource.

    Actualy i have done some exersice i had make a thread that checking for time out
    with one variable.This variable set with gettickcount when any event occur on socket
    GetQueuedCompletionStatus come out from the loop with client completion key which is
    a class pointer and with that pointer i reset the variable with new tick count.

    In Timeout thread it will check for that variable goes above 100 sec if yes than it will close that socket and remove that class pointer but in this case some event comes after i have deleted the pointer of the class so its crash.

    So please tell me how can i detect timeout for the socket and how to remove client resources from server.

    Thanx in advance..

+ Reply to Thread