DosReleaseMutexSem is blocking??? - OS2

This is a discussion on DosReleaseMutexSem is blocking??? - OS2 ; Hi, I have a case where DosReleaseMutexSem seems to block. This causes another thread to fail with a timeout. The executed code is: [Thread 3] PID pid; TID tid; ULONG count; DosQueryMutexSem(Handle, &pid, &tid, &count); DEBUGLOG(("Mutex(%p{%p})::Release() @ %u\n", this, Handle, ...

+ Reply to Thread
Results 1 to 11 of 11

Thread: DosReleaseMutexSem is blocking???

  1. DosReleaseMutexSem is blocking???

    Hi,

    I have a case where DosReleaseMutexSem seems to block. This causes
    another thread to fail with a timeout.


    The executed code is:

    [Thread 3]
    PID pid;
    TID tid;
    ULONG count;
    DosQueryMutexSem(Handle, &pid, &tid, &count);
    DEBUGLOG(("Mutex(%p{%p})::Release() @ %u\n", this, Handle, count));
    APIRET rc = DosReleaseMutexSem(Handle);
    DEBUGLOG(("Mutex::Release() done %u\n", rc));

    [Thread 1]
    DEBUGLOG(("Mutex(%p{%p})::Request(%li)\n", this, Handle, ms));
    APIRET rc = DosRequestMutexSem(Handle, 60000);
    PID pid;
    TID tid;
    ULONG count;
    DosQueryMutexSem(Handle, &pid, &tid, &count);
    DEBUGLOG(("Mutex(%p)::Request - %u @ %u\n", this, rc, count));
    if (ms < 0 && rc == ERROR_TIMEOUT)
    { DEBUGLOG(("Mutex(%p)::Request - UNHANDLED TIMEOUT!!!\n", this));
    DosBeep(2000, 500);
    }


    Excerpt from the log:

    time[ms] PID TID stack text
    ================================================== ===================
    00035289 1998:0003 02a7fe7c Mutex(2b351b0{10114})::Release() @ 1
    00035289 1998:0001 001f5018 Mutex(e1f50{1004f})::Release() @ 1
    00035289 1998:0001 001f5018 Mutex::Release() done 0
    00035291 1998:0001 001f4ff8 Mutex(2b351b0{10114})::Request(-1)
    00095306 1998:0001 001f4ff8 Mutex(2b351b0)::Request - 640 @ 1
    00095306 1998:0001 001f4ff8 Mutex(2b351b0)::Request - UNHANDLED TIMEOUT!!!


    Thread 3 wants to release the mutex handle 10114 and thread 1 requests
    the same handle a bit later. But thread 1 blocks because thread 3
    obviously does not succeed. The 2nd debug print after DosReleaseMutexSem
    is missing from thread 3.
    After the timeout the application terminates because of an assertion.
    (This is intended.)

    Watchcat shows the thread state of thread 3 as "unknown" - whatever this
    means.


    Has anyone an idea what is going on here?


    Marcel

  2. Re: DosReleaseMutexSem is blocking???

    Hi,

    is "Handle" a global variable ? Are you sure you are not "reusing" "Handle"
    for multiple/different semaphores ? DosQueryMutexSem returns in "count" the
    number of DosRequestMutexSem minus number of DosReleaseMutexSem calls of the
    OWNING THREAD. Therefore "count" basically returns the number of nested
    DosRequestMutexSem/DosReleaseMutexSem of the OWNING and therefore SAME
    THREAD. So I would say DosQueryMutexSem works differently from what you
    think it does and cannot be used the way you intend to use it. "count"
    should be 0 if the semaphore could not be requested. Maybe it's also a good
    idea to explicitely set count=0 before calling DosQueryMutexSem. And print
    out the return code from DosQueryMutexSem and also pid and tid.

    Also, it's strange why in thread 1 it should return "count = 1" on
    DosQueryMutexSem when the preceding request to DosRequestMutexSem returned
    with rc = ERROR_TIMEOUT. My feeling is your code sample extract below does
    not give us the complete truth. Also, where do you place the calls to
    DosCreateMutexSem and DosCloseMutexSem ? Are rc,pid,tid,count allocated on
    the stack(s) ?

    And be careful with DEBUGLOG. Does that macro use printf ? Using printf is a
    tricky thing in a multithreaded program:
    a.) make sure you link to the multithreaded libraries
    b.) printf is buffered which might lead to lost output if you abort the
    program. Better to use cprintf (or VioWriteTTY) or do a fflush(stdio,..)
    after each printf.

    By the way: what is the practical scenario ? Why would you do
    DosRequestMutexSem from one thread and DosReleaseMutexSem from another ? I
    would always expect DosRequestMutexSem and DosReleaseMutexSem to show up as
    a pair in each thread as they normally bracket/protect access to a resource
    shared by those threads that make these calls (where those threads can even
    be in different processes).
    As ownership is defined for a THREAD (and not for a process), I also doubt
    that you can do a DosReleaseMutexSem without having successfully called
    DosRequestMutexSem from the SAME THREAD. Test: For exactly one semaphore,
    check if you can do a DosRequestMutexSem from one thread and successfully
    DosReleaseMutexSem from another.

    Lars

    "Marcel Müller" schrieb im Newsbeitrag
    news:4821df78$0$7555$9b4e6d93@newsspool1.arcor-online.net...
    > Hi,
    >
    > I have a case where DosReleaseMutexSem seems to block. This causes another
    > thread to fail with a timeout.
    >
    >
    > The executed code is:
    >
    > [Thread 3]
    > PID pid;
    > TID tid;
    > ULONG count;
    > DosQueryMutexSem(Handle, &pid, &tid, &count);
    > DEBUGLOG(("Mutex(%p{%p})::Release() @ %u\n", this, Handle, count));
    > APIRET rc = DosReleaseMutexSem(Handle);
    > DEBUGLOG(("Mutex::Release() done %u\n", rc));
    >
    > [Thread 1]
    > DEBUGLOG(("Mutex(%p{%p})::Request(%li)\n", this, Handle, ms));
    > APIRET rc = DosRequestMutexSem(Handle, 60000);
    > PID pid;
    > TID tid;
    > ULONG count;
    > DosQueryMutexSem(Handle, &pid, &tid, &count);
    > DEBUGLOG(("Mutex(%p)::Request - %u @ %u\n", this, rc, count));
    > if (ms < 0 && rc == ERROR_TIMEOUT)
    > { DEBUGLOG(("Mutex(%p)::Request - UNHANDLED TIMEOUT!!!\n", this));
    > DosBeep(2000, 500);
    > }
    >
    >
    > Excerpt from the log:
    >
    > time[ms] PID TID stack text
    > ================================================== ===================
    > 00035289 1998:0003 02a7fe7c Mutex(2b351b0{10114})::Release() @ 1
    > 00035289 1998:0001 001f5018 Mutex(e1f50{1004f})::Release() @ 1
    > 00035289 1998:0001 001f5018 Mutex::Release() done 0
    > 00035291 1998:0001 001f4ff8 Mutex(2b351b0{10114})::Request(-1)
    > 00095306 1998:0001 001f4ff8 Mutex(2b351b0)::Request - 640 @ 1
    > 00095306 1998:0001 001f4ff8 Mutex(2b351b0)::Request - UNHANDLED TIMEOUT!!!
    >
    >
    > Thread 3 wants to release the mutex handle 10114 and thread 1 requests the
    > same handle a bit later. But thread 1 blocks because thread 3 obviously
    > does not succeed. The 2nd debug print after DosReleaseMutexSem is missing
    > from thread 3.
    > After the timeout the application terminates because of an assertion.
    > (This is intended.)
    >
    > Watchcat shows the thread state of thread 3 as "unknown" - whatever this
    > means.
    >
    >
    > Has anyone an idea what is going on here?
    >
    >
    > Marcel




  3. Re: DosReleaseMutexSem is blocking???

    Hi,

    Lars Erdmann schrieb:
    > is "Handle" a global variable ? Are you sure you are not "reusing" "Handle"
    > for multiple/different semaphores ?


    Handle is a private member of class Mutex. Once the class is constructed
    (this calls DosCreateMutexSem) Handle is not modified anymore.

    > DosQueryMutexSem returns in "count" the
    > number of DosRequestMutexSem minus number of DosReleaseMutexSem calls of the
    > OWNING THREAD. Therefore "count" basically returns the number of nested
    > DosRequestMutexSem/DosReleaseMutexSem of the OWNING and therefore SAME
    > THREAD.


    I know. It's just a debug info. Release builds do not call
    DosQueryMutexSem at all. (But they hang too.)
    Also switching between IBM VACPP and gcc 3.3 does not make a major
    difference. Only the frequency of hangs changes slightly.

    > So I would say DosQueryMutexSem works differently from what you
    > think it does and cannot be used the way you intend to use it. "count"
    > should be 0 if the semaphore could not be requested.


    No, since the returned PID/TID need not be of the calling thread. In
    fact it is always non-zero if DosRequestMutexSem would block.

    > Maybe it's also a good
    > idea to explicitely set count=0 before calling DosQueryMutexSem. And print
    > out the return code from DosQueryMutexSem and also pid and tid.


    Maybe. But I don't expect anything special here. Since if the handle is
    not valid the next call to Dos*MutexSem would fail anyway.

    > Also, it's strange why in thread 1 it should return "count = 1" on
    > DosQueryMutexSem when the preceding request to DosRequestMutexSem returned
    > with rc = ERROR_TIMEOUT.


    That is as expected. The Mutex is held by another thread and not
    released within the given time.

    > My feeling is your code sample extract below does
    > not give us the complete truth.


    Well, of course. The real code is some megs and the log also. But I was
    able to reproduce this scenario nearly independently of other actions.

    > Also, where do you place the calls to
    > DosCreateMutexSem and DosCloseMutexSem ?


    Mutex constructor/destructor. If one of them fails the application
    abends with an assertion.

    > Are rc,pid,tid,count allocated on
    > the stack(s) ?


    Yes.

    > And be careful with DEBUGLOG. Does that macro use printf ? Using printf is a
    > tricky thing in a multithreaded program:


    DEBUGLOG uses sprintf and vsnprintf into a temporary 1k buffer on the
    stack and DosWrite to STDERR to write the output. sprintf is used for
    the heading info (up to the stack pointer). vsnprintf for the text.

    > a.) make sure you link to the multithreaded libraries


    Of course. Otherwise _beginthread would not resolve.

    > b.) printf is buffered which might lead to lost output if you abort the
    > program. Better to use cprintf (or VioWriteTTY) or do a fflush(stdio,..)
    > after each printf.


    STDERR is usually unbufferd or line buffered. Both is sufficient.


    > By the way: what is the practical scenario ? Why would you do
    > DosRequestMutexSem from one thread and DosReleaseMutexSem from another ? I
    > would always expect DosRequestMutexSem and DosReleaseMutexSem to show up as
    > a pair in each thread as they normally bracket/protect access to a resource
    > shared by those threads that make these calls (where those threads can even
    > be in different processes).


    Of course, they are paired. But I showed only one part for each thread.
    In fact the code is the same anyway, because these are the only
    locations in the application where DosRequestMutexSem and
    DosReleaseMutexSem are called.

    However, from my understanding DosReleaseMutexSem should NEVER block. It
    may unblock another thread, it may return an error, it may do whatever
    it likes, but it should not block, isn't it?

    > As ownership is defined for a THREAD (and not for a process), I also doubt
    > that you can do a DosReleaseMutexSem without having successfully called
    > DosRequestMutexSem from the SAME THREAD.


    As far as I know I would get something like ERROR_SEMAPHORE_NOT_OWNED
    (or so) in this case. But that does not happen because I don't call
    DosReleaseMutexSem from a thread that does not own the mutex. In fact
    all mutex requests are done through a small helper class:

    class Lock
    { Mutex& CS;
    bool Own;
    private: // non-copyable
    Lock(const Lock&);
    void operator=(const Lock&);
    public:
    Lock(Mutex& cs, long ms = -1) : CS(cs), Own(cs.Request(ms)) {}
    ~Lock() { if (Own) CS.Release(); }
    operator bool() const { return Own; }
    };

    class Mutex
    { HMTX Handle;
    private: // non-copyable
    Mutex(const Mutex&);
    void operator=(const Mutex&);
    public:
    Mutex(bool share = false);
    Mutex(const char* name); // Creates a _named_ mutex.
    ~Mutex();
    // Aquire Mutex object
    // If it takes more than ms milliseconds the funtion will return false.
    bool Request(long ms = -1);
    // Release the Mutex object
    // On error, e.g. if you are not owner, the funtion returns false.
    bool Release();
    };

    // usage:
    Mutex mtx;

    // request block
    { Lock lock(mtx);
    //...
    }

    It is practically impossible to do unpaired semaprore calls this way. As
    soon as the Lock instance goes out of scope the mutex is released.


    But I found another possible reason. DosReleaseMutexSem was accidently
    called within a critical section. I moved the call now behind the
    critical section and it seems to work so far. Unfortunately this is no
    sufficient evidence, since code rearrangements always influenced the
    frequency of this error. But it never disappeared completely.

    Maybe OS/2 internally uses DosEnter/ExitCritSec to handle the
    semaphores. And maybe it relies on the fact that after DosExitCritSec
    another thread may enter a critical section for handshake. While it is
    no good advise to call OS/2 APIs from within a critical section, a note
    in the documentation of DosRleaseMutexSem would be nice in this case.


    Marcel

  4. Re: DosReleaseMutexSem is blocking???

    In <482c1f42$0$6770$9b4e6d93@newsspool2.arcor-online.net>, on 05/15/2008
    at 01:31 PM, Marcel Müller said:

    Hi,

    >However, from my understanding DosReleaseMutexSem should NEVER block. It
    >may unblock another thread, it may return an error, it may do whatever
    >it likes, but it should not block, isn't it?


    DosReleaseMutexSem will not block in the sense that I think you are using
    the word, but there's no guarantee that the thread calling
    DosReleaseMutexSem will not lose control for an indefinitely long time.

    This is often caused thread priority issues or deadlocks. The later
    sounds like a possibility as you have described your problem.

    If you could force a process dump while the hang condition is active, you
    should be able to get a good picture of what the state of the other
    threads is. You might be able to do this when you detect the timeout in
    the other thread.

    >Maybe OS/2 internally uses DosEnter/ExitCritSec to handle the
    >semaphores.


    I doubt it's anything so coarse since this disables thread switching only
    for the current process.


    Steven

    --
    --------------------------------------------------------------------------------------------
    Steven Levine MR2/ICE 3.00 beta 11pre11 #10183
    eCS/Warp/DIY/14.103a_W4 www.scoug.com irc.ca.webbnet.info #scoug (Wed 7pmPST)
    --------------------------------------------------------------------------------------------


  5. Re: DosReleaseMutexSem is blocking???

    Steven Levine schrieb:
    > In <482c1f42$0$6770$9b4e6d93@newsspool2.arcor-online.net>, on 05/15/2008
    > at 01:31 PM, Marcel Müller said:
    >
    >> However, from my understanding DosReleaseMutexSem should NEVER block. It
    >> may unblock another thread, it may return an error, it may do whatever
    >> it likes, but it should not block, isn't it?

    >
    > DosReleaseMutexSem will not block in the sense that I think you are using
    > the word, but there's no guarantee that the thread calling
    > DosReleaseMutexSem will not lose control for an indefinitely long time.


    Thats true. But only if there are other threads with higher priority
    released. At 0% CPU load the calling thread should not block.

    > This is often caused thread priority issues or deadlocks. The later
    > sounds like a possibility as you have described your problem.


    It is a deadlock. But for what should the thread that want's to call
    DosReleaseMutexSem wait? And why is it's thread state unknown (not
    blocked or running)?

    > If you could force a process dump while the hang condition is active, you
    > should be able to get a good picture of what the state of the other
    > threads is. You might be able to do this when you detect the timeout in
    > the other thread.


    Hmm, I don't know how to force a process dump and if so I don't know
    what to do with the dump file. But I tried to do it in the debugger.
    Unfortunately the icsdebug crashed when it reaches the break-point.
    Maybe bcause of the thread with unknown state.


    >> Maybe OS/2 internally uses DosEnter/ExitCritSec to handle the
    >> semaphores.

    >
    > I doubt it's anything so coarse since this disables thread switching only
    > for the current process.


    Thats right. It makes no sense. Except if the non-shared semaphores are
    something different.


    Marcel

  6. Re: DosReleaseMutexSem is blocking???

    Marcel Müller wrote:
    > Steven Levine schrieb:
    >
    >> In <482c1f42$0$6770$9b4e6d93@newsspool2.arcor-online.net>, on 05/15/2008
    >> at 01:31 PM, Marcel Müller said:
    >>
    >>> However, from my understanding DosReleaseMutexSem should NEVER block.
    >>> It may unblock another thread, it may return an error, it may do
    >>> whatever it likes, but it should not block, isn't it?

    >>
    >>
    >> DosReleaseMutexSem will not block in the sense that I think you are using
    >> the word, but there's no guarantee that the thread calling
    >> DosReleaseMutexSem will not lose control for an indefinitely long time.

    >
    > Thats true. But only if there are other threads with higher priority
    > released. At 0% CPU load the calling thread should not block.
    >
    >> This is often caused thread priority issues or deadlocks. The later
    >> sounds like a possibility as you have described your problem.

    >
    > It is a deadlock. But for what should the thread that want's to call
    > DosReleaseMutexSem wait? And why is it's thread state unknown (not
    > blocked or running)?


    The thing is, your whole paradigm seems strange (or admittedly, not
    fully understood by me). The concept of the mutex is usually to protect
    data or some other resource that is accessed by multiple threads. Let's
    say for example that you have a simple structure containing a pointer
    and a size, like a resizable string:

    typedef struct
    {
    unsigned long allocSize, usedSize;
    char *stringPtr;
    } myString;

    myString string1;

    If you are reading this string in one thread, and populating it in
    another (as can be the case if, for example, a PM entry field control
    populates the string), you want to make absolutely sure that the
    allocSize and usedSize parameters are synchronized with the pointer's
    actual sizes. You need to update these 3 things atomically with respect
    to any thread using the data. To accomplish this, you would surround
    any read or write access with DosRequestMutexSem/DosReleaseMutexSem
    within the same thread. Like this:

    HMTX stringMutex = ...;

    // ... window message handler when entry field is updated
    DosRequestMutexSem( stringMutex, SEM_INDEFINITE_WAIT );
    if ( string1.allocSize < newSize )
    {
    unsigned long tmpSize = allocSize * 2;
    char *tmpPtr = realloc( string1.stringPtr, tmpSize );
    if ( !tmpPtr ) { /* Handle failure gracefully */ }
    string1.stringPtr = tmpPtr;
    string1.allocSize = tmpSize;
    }
    strncpy( string1.stringPtr, newString, newSize );
    string1.usedSize = newSize;
    DosReleaseMutexSem( stringMutex, SEM_INDEFINITE_WAIT );

    ....
    // In another thread where reading the string is necessary...
    APIRET rc;
    rc = DosRequestMutexSem( stringMutex, 5000 ); // Timeout in 5 seconds
    if ( rc == ERROR_TIMEOUT )
    {
    // Resource is temporarily unavailable
    } else if ( rc )
    {
    // Handle other errors
    } else if ( rc == NO_ERROR )
    {
    // Access string1 as needed here
    DosReleaseMutexSem( stringMutex );
    }

    So I would always expect to see the request/release paired within a
    relatively small section of code, not spanning threads. If the first
    thread which is writing to the string1 structure had to rely on yet a
    different thread to populate one of the values of the structure, I think
    it would be a better practice to store all of the partial updates in a
    temporary area, waiting for the piece of data to come back from the 3rd
    thread, and then request the mutex and populate all of the fields at the
    same time, rather than holding the mutex open while waiting on another
    thread for action. The latter technique is the recipe for deadlocks.

    --
    [Reverse the parts of the e-mail address to reply.]

  7. Re: DosReleaseMutexSem is blocking???

    On Thu, 15 May 2008 20:49:46 +0200, Marcel Müller
    wrote:

    > It is a deadlock. But for what should the thread that want's to call
    > DosReleaseMutexSem wait? And why is it's thread state unknown (not
    > blocked or running)?


    "wants" and "its". What is it with this need to put bloody apostrophes
    everywhere?

    > Hmm, I don't know how to force a process dump and if so I don't know
    > what to do with the dump file. But I tried to do it in the debugger.
    > Unfortunately the icsdebug crashed when it reaches the break-point.
    > Maybe bcause of the thread with unknown state.


    Have you tried looking with Theseus?

  8. Re: DosReleaseMutexSem is blocking???

    >> Maybe OS/2 internally uses DosEnter/ExitCritSec to handle the
    >> semaphores.

    >
    > I doubt it's anything so coarse since this disables thread switching only
    > for the current process.


    Well, I wouldn't be too sure about this. We don't know how most things
    internally work. We had so many surprises in the past .

    There is an interesting note for DosEnterCritSec() in the TK reference:

    "Threads that call DosEnterCritSec must not must not make dynamic link
    calls within these critical sections. The dynamic link procedure could
    be using semaphores to serialize a resource. If a thread entering the
    critical section blocks another thread that already owns the resource
    which the dynamic link function is about to request, a deadlock occurs.

    For example, threads of an application are serializing their access to
    a queue by means of a semaphore. A thread enters a critical section and
    makes a request to read the queue while another thread already has the
    semaphore that controls access to the queue. The thread that has the
    semaphore is now effectively blocked by DosEnterCritSec, and the thread
    that has requested the queue waits forever to access it."


    Well, I assume DosReleaseMutexSem() is in DOSCALLS1.DLL. I could
    imagine that it also needs some kind of serialization to update its
    own mutex state tables for the threads. For performance reasons there
    is a possibly that it is done in a critical section. Just guessing...

    Another theory could be that DosReleaseMutexSem() really waits until
    a suspended thread that requests the mutex as well is continued.
    But because the call is in a critical section, this will never happen
    and we have the deadlock.

  9. Re: DosReleaseMutexSem is blocking???

    In <482cbe05$0$6560$9b4e6d93@newsspool3.arcor-online.net>, on 05/16/2008
    at 12:49 AM, Heiko Nitzsche said:

    Hi,

    >Well, I assume DosReleaseMutexSem() is in DOSCALLS1.DLL.


    The entry point is defined in doscall1.dll, but I would expect the logic
    to live in the kernel.

    >Another theory could be that DosReleaseMutexSem() really waits until a
    >suspended thread that requests the mutex as well is continued.


    That makes no sense at all to me. What has to happen for thread
    priorities to work is for the kernel to check if a some other thread is
    waiting on the released mutex and if that thread has a higher scheduling
    priority. If this is not the case, there's no reason for the kernel not
    to return immediately to the calling thread. This ignores the
    complexities of timeslices expiring and such.

    >because the call is in a critical section, this will never happen and we
    >have the deadlock.


    What you are describing is classic deadlock. Thread T1 requests resource
    R1 followed by R2 and thread T2 requests resource R2 followed by R1.

    This is what is most likely happening to Marcel.

    Steven

    --
    --------------------------------------------------------------------------------------------
    Steven Levine MR2/ICE 3.00 beta 11pre11 #10183
    eCS/Warp/DIY/14.103a_W4 www.scoug.com irc.ca.webbnet.info #scoug (Wed 7pm PST)
    --------------------------------------------------------------------------------------------


  10. Re: DosReleaseMutexSem is blocking???

    On 05/15/08 02:00 pm, Paul Ratcliffe wrote:
    >
    >> It is a deadlock. But for what should the thread that want's to call
    >> DosReleaseMutexSem wait? And why is it's thread state unknown (not
    >> blocked or running)?

    >
    > "wants" and "its". What is it with this need to put bloody apostrophes
    > everywhere?
    >

    It's the new I-didn't-learn-grammar rule: When in doubt, add an apostrophe.

    --
    jmm (hyphen) list (at) sohnen-moe (dot) com
    (Remove .AXSPAMGN for email)

  11. Re: DosReleaseMutexSem is blocking???

    On Fri, 16 May 2008 04:44:27 UTC, Jim Moe wrote:

    > On 05/15/08 02:00 pm, Paul Ratcliffe wrote:
    > >
    > >> It is a deadlock. But for what should the thread that want's to call
    > >> DosReleaseMutexSem wait? And why is it's thread state unknown (not
    > >> blocked or running)?

    > >
    > > "wants" and "its". What is it with this need to put bloody apostrophes
    > > everywhere?
    > >

    > It's the new I-didn't-learn-grammar rule: When in doubt, add an apostrophe.
    >


    Or, one could get an instructive refresher course here:

    http://eatsshootsandleaves.com/

    --
    Best,
    Erick Andrews
    delete bogus to reply


+ Reply to Thread