Task goes into a Pend+I state : Semaphore issue? - VxWorks

This is a discussion on Task goes into a Pend+I state : Semaphore issue? - VxWorks ; Hi, I am running two tasks, say A and B. A communicates with a server over TCP and gets commands from the server. These commands are pushed into a queue by A and then gives control to B by doing ...

+ Reply to Thread
Results 1 to 12 of 12

Thread: Task goes into a Pend+I state : Semaphore issue?

  1. Task goes into a Pend+I state : Semaphore issue?

    Hi,

    I am running two tasks, say A and B. A communicates with a server over
    TCP and gets commands from the server. These commands are pushed into a
    queue by A and then gives control to B by doing a semGive() for a
    semaphore the task B is pended on, and then doing a sleep.

    Task B runs, processes the command from the queue, and then goes back
    to pend on the semaphore by doing a semTake.

    The task A keeps running all the time. Task A and B have the same
    priorities.

    Issue:

    After a couple of runs (not the first run), task B stops responding. An
    "i" on the consol shows B to be in Pend+I state.

    I understand that a task goes into a Pend+I state when it has a
    semaphore on which a higher priority task is pended. VxWorks thus makes
    this task's priority equal to the higher priority task, so that it can
    run, release the semaphore and allow the other higher priority task to
    run.

    However, since task A's priority is the same as B, and since A is not
    pended on any semaphore, why should this happen?

    A keeps in the pend state.

    Also, when task B goes into this state, the board hangs. Any pointers
    why this should be happening? (one idea is that B would be getting into
    an infinite loop which does nothing).
    Is there some way of measuring the CPU consumption of a task at some
    time?

    Thanks,

    Kapslock


  2. Re: Task goes into a Pend+I state : Semaphore issue?

    "kapslocked@gmail.com" wrote:

    >Hi,
    >
    >I am running two tasks, say A and B. A communicates with a server over
    >TCP and gets commands from the server. These commands are pushed into a
    >queue by A and then gives control to B by doing a semGive() for a
    >semaphore the task B is pended on, and then doing a sleep.
    >
    >Task B runs, processes the command from the queue, and then goes back
    >to pend on the semaphore by doing a semTake.
    >
    >The task A keeps running all the time. Task A and B have the same
    >priorities.


    Why have you assigned your tasks the same priority? What priority have
    you assigned those tasks?

    >Issue:
    >
    >After a couple of runs (not the first run), task B stops responding. An
    >"i" on the consol shows B to be in Pend+I state.
    >
    >I understand that a task goes into a Pend+I state when it has a
    >semaphore on which a higher priority task is pended. VxWorks thus makes
    >this task's priority equal to the higher priority task, so that it can
    >run, release the semaphore and allow the other higher priority task to
    >run.
    >
    >However, since task A's priority is the same as B, and since A is not
    >pended on any semaphore, why should this happen?


    I cannot immediately explain the behavior but the first thing I would try
    is assigning your tasks different priorities. Which to assign the higher
    priority (lower value) depends on your system's design and mission. Is
    receiving messages from the server or processing those messages most
    important?

    >A keeps in the pend state.
    >
    >Also, when task B goes into this state, the board hangs.


    What, exactly, do you mean by "hangs"?

    > Any pointers
    >why this should be happening? (one idea is that B would be getting into
    >an infinite loop which does nothing).


    This could be the problem if the priority of task B is higher (lower value)
    than the tasks that no longer run and therefore make you conclude that
    your board has hung.

    >Is there some way of measuring the CPU consumption of a task at some
    >time?


    Not directly but there is a tool called WindView that provides some such
    capabilities.

    --
    ================================================== ======================
    Michael Kesti | "And like, one and one don't make
    | two, one and one make one."
    mrkesti at comcast dot net | - The Who, Bargain

  3. Re: Task goes into a Pend+I state : Semaphore issue?

    Thanks for the reply Michael.

    Mea culpa, I didn't mention that before. The two tasks run at the same
    priorities of 175 but when the system is hung, I find that the task B
    has gone up to the priority 20. This would make sense of course, if
    there was another task with priority 20 pended on a semaphore that task
    B has taken, while B has been preempted by another task of slightly
    higher priority. However, since I create the semaphore (and so I know
    that out of the only 2 calls to semaphore, other than create, is a
    semTake in task B and a semGive in task A, so that leaves no
    possibility of another task of priority 20 pended on this semaphore).

    I tried raising the priority of B a little bit, and this "improved" the
    situation, in the sense, the system hung after about 10 repetitions,
    compared to 5-6 earlier.

    You are right that if the priority of system tasks is lower than B's,
    then it gives me a feeling that the system is hung. I tried td on most
    of the tasks with priority greater than B's. Didn't help.

    Any command to know which semaphores (and a count of) a particular task
    is pended on?

    Thanks for your help.

    Kapslock


  4. Re: Task goes into a Pend+I state : Semaphore issue?

    "kapslocked@gmail.com" wrote:

    >Thanks for the reply Michael.


    You're welcome.

    >Mea culpa, I didn't mention that before. The two tasks run at the same
    >priorities of 175 but when the system is hung, I find that the task B
    >has gone up to the priority 20. This would make sense of course, if
    >there was another task with priority 20 pended on a semaphore that task
    >B has taken, while B has been preempted by another task of slightly
    >higher priority.


    This makes sense only if your sem is a mutex created with the options
    SEM_Q_PRIORITY and SEM_INVERSION_SAFE. For your application, though, a
    binary sem is probably appropriate. Please post your sem creation code
    including the allocation of the variable in which you store the semID.

    > However, since I create the semaphore (and so I know
    >that out of the only 2 calls to semaphore, other than create, is a
    >semTake in task B and a semGive in task A, so that leaves no
    >possibility of another task of priority 20 pended on this semaphore).


    Is there a task other than B with a priority of 20? If so, what task is
    it?

    >I tried raising the priority of B a little bit, and this "improved" the
    >situation, in the sense, the system hung after about 10 repetitions,
    >compared to 5-6 earlier.


    Progress!

    >You are right that if the priority of system tasks is lower than B's,
    >then it gives me a feeling that the system is hung. I tried td on most
    >of the tasks with priority greater than B's. Didn't help.
    >
    >Any command to know which semaphores (and a count of) a particular task
    >is pended on?


    Not for a specific task but semShow() and semInfo() might help.

    >Thanks for your help.


    You're welcome.

    --
    ================================================== ======================
    Michael Kesti | "And like, one and one don't make
    | two, one and one make one."
    mrkesti at comcast dot net | - The Who, Bargain

  5. Re: Task goes into a Pend+I state : Semaphore issue?


    kapslocked@gmail.com wrote:
    > Hi,
    >
    > I am running two tasks, say A and B. A communicates with a server over
    > TCP and gets commands from the server. These commands are pushed into a
    > queue by A and then gives control to B by doing a semGive() for a
    > semaphore the task B is pended on, and then doing a sleep.
    >
    > Task B runs, processes the command from the queue, and then goes back
    > to pend on the semaphore by doing a semTake.
    >
    > The task A keeps running all the time. Task A and B have the same
    > priorities.
    >
    > Issue:
    >
    > After a couple of runs (not the first run), task B stops responding. An
    > "i" on the consol shows B to be in Pend+I state.
    >
    > I understand that a task goes into a Pend+I state when it has a
    > semaphore on which a higher priority task is pended. VxWorks thus makes
    > this task's priority equal to the higher priority task, so that it can
    > run, release the semaphore and allow the other higher priority task to
    > run.
    >
    > However, since task A's priority is the same as B, and since A is not
    > pended on any semaphore, why should this happen?
    >
    > A keeps in the pend state.
    >
    > Also, when task B goes into this state, the board hangs. Any pointers
    > why this should be happening? (one idea is that B would be getting into
    > an infinite loop which does nothing).
    > Is there some way of measuring the CPU consumption of a task at some
    > time?
    >
    > Thanks,
    >
    > Kapslock


    HI,
    As informed by Micheal,it would be appropriate for you to use a Binary
    semaphore rather then a mutex.From your problem description it seems to
    me that you are not looking for resource protection but rather trying
    to synchronise between these 2 tasks.
    If you feel like debugging further on semaphores Sem_Show command needs
    to be executed for that you have to include SEM_SHOW option in your
    code.
    Regards,
    s.subbarayan


  6. Re: Task goes into a Pend+I state : Semaphore issue?

    Why don't you use a counting semaphore in this case?

    Create it with an initial count of zero with semCCreate(). When a message is
    added to your queue, increment the semaphore with semGive(). The other task
    waits on the semaphore with semTake(,WAIT_FOREVER) and whenever it gets the
    semaphore process a single message from your queue.

    Is a VxWorks message queue not more appropriate?

    --


    Regards,


    Graham Baxter
    Freelance Software Engineer (VxWorks, Linux and pSOS BSPs)
    Graham Baxter (Software) Limited
    Europe
    http://www.gbsw.co.uk
    fromnewsgrp@NOSPAMgbsw.co.uk

    wrote in message
    news:1142029317.145745.232730@e56g2000cwe.googlegr oups.com...
    > Hi,
    >
    > I am running two tasks, say A and B. A communicates with a server over
    > TCP and gets commands from the server. These commands are pushed into a
    > queue by A and then gives control to B by doing a semGive() for a
    > semaphore the task B is pended on, and then doing a sleep.
    >
    > Task B runs, processes the command from the queue, and then goes back
    > to pend on the semaphore by doing a semTake.
    >
    > The task A keeps running all the time. Task A and B have the same
    > priorities.
    >
    > Issue:
    >
    > After a couple of runs (not the first run), task B stops responding. An
    > "i" on the consol shows B to be in Pend+I state.
    >
    > I understand that a task goes into a Pend+I state when it has a
    > semaphore on which a higher priority task is pended. VxWorks thus makes
    > this task's priority equal to the higher priority task, so that it can
    > run, release the semaphore and allow the other higher priority task to
    > run.
    >
    > However, since task A's priority is the same as B, and since A is not
    > pended on any semaphore, why should this happen?
    >
    > A keeps in the pend state.
    >
    > Also, when task B goes into this state, the board hangs. Any pointers
    > why this should be happening? (one idea is that B would be getting into
    > an infinite loop which does nothing).
    > Is there some way of measuring the CPU consumption of a task at some
    > time?
    >
    > Thanks,
    >
    > Kapslock
    >




  7. Re: Task goes into a Pend+I state : Semaphore issue?

    Hi kapslock,
    This seems to be a standard producer consumer
    problem.The common resource to be protected is the queue of commands.
    This is the problem of synchrnonization and not mutual exclusion.Hence
    opt for binary semaphore.This should fix the inheritance problem.
    This can be solved not with one but with two semaphores.That would be
    the perfect solution independent of scheduling policy and
    priorities.Soln

    Go for two semaphores: SA governed by task A,initial value =1
    SB governed by task B,initial value
    =0
    Inside the code for task A:

    semTake(SA)
    put the cmd in the queue.
    semGive(SB)

    Inside the code for task B:

    semTake(SB)
    put the cmd in the queue.
    semGive(SA)

    This would give the perfect solution avoiding all kind of race
    conditions.

    The issue in previous case 'could be' because of same semaphore ID
    used by some other task of priority 20 or temp. raising to priority 20.

    When you changed the priorities of the tasks you could get 'some
    progress'.Hence it must be the issue with priorities.This should get
    resolved when you opt for binary semaphore.Make sure the semaphore IDs
    are UNIQUE in the system.
    Hope this was useful.

    cheers-
    kaushal.


  8. Re: Task goes into a Pend+I state : Semaphore issue?

    Graham.

    Actually, I do precisely that.

    My semaphore is a counting semaphore, created with semCCreate. The task
    A calls semGive and sleeps briefly when it adds message to the queue.
    All this while the other task pends on the semaphore by doing a
    semTake with WAIT_FOREVER option.

    Michael,

    Maybe its again my fault that I held back some info (this is something
    that I am doing at my workplace, so its a little difficult for me to
    post the entire code here, or to give out all the details. I hope you'd
    understand).

    The task B, when reads the message in the message queue, replies back.
    The reply is on the standard output, which is redirected through a
    named pipe to a socket that the task A reads. (Thus the task A reads
    the messages from the server, and also from the redirected output of
    the task B).

    The trouble usually arises when the server snaps connection abruptly.
    (I now have it coded in my code, to tun that test case, that the server
    drops connection abruptly after some x lines of output received from
    task A, which in turn was the task B's output, passed through the names
    pipe). When task A recognises that it can't read from the server socket
    (because the remote server has dropped the connection), or can't write
    to it, it needs to drop the TCP connection from its own side.

    After doing that, I need to clean up things from my side. This includes
    multiple commands that might be in the message queue.

    More importantly, I need to clear up the command that might be being
    executed at that instant of time, in task B. I can either
    1. Stop the stdout redirection till the command is over, or
    2. Simply cancle the execution, and cleanup the current command in the
    queue. I am not sure if I can do the cleanup perfectly, and not leave
    memory leaks, so I prefer not taking this approach.
    3. Simply leave the command running as is, but prevent the task A from
    sending task B's output back to the server.

    I typically take approach B.

    In running this a couple of times, I find the system hung. I do have
    access to the console, but the board's other tasks don't run ( My
    device's display freezes, and its keypad also doesn't work). When I do
    an "i", I figure that the task B has gone into a "Pend+I" state.

    I am not sure if this additional scheme of things has got anything to
    do with the task going into pend + I state, but having spent quite some
    time trying to figure out what's going wrong, I was thinking of trying
    to work backwards, if I got some clue what exactly gets the task into
    pend + I state.

    Kaushal,

    I think it is a standard producer-consumer problem, but in this case,
    the producer keeps running. In fact, when the task B produces its
    reply, it keeps reading reply and sending it back to task A, but also
    in evrey iteration when it gets some data to send back to A through the
    pipe, it sleeps, allowing task A to run again to check for any more
    inputs from the server, and post task B's replies back too.

    Let me try the Mutex and see if that helps resolve the problem.

    Thanks everyone for your help. I appreciate it.


  9. Re: Task goes into a Pend+I state : Semaphore issue?

    "kapslocked@gmail.com" wrote:

    >Graham.
    >
    >Actually, I do precisely that.


    It would be very helpful for you to include quoted text that establishes
    the context that allows readers to determine just which "that" you are
    precisely doing!

    >My semaphore is a counting semaphore, created with semCCreate. The task
    >A calls semGive and sleeps briefly when it adds message to the queue.
    >All this while the other task pends on the semaphore by doing a
    >semTake with WAIT_FOREVER option.


    Why are you using a counting semphore? From the V5.5 Programmer's Guide:

    Counting semaphores are useful for guarding multiple copies of
    resources. For example, the use of five tape drives might be
    coordinated using a counting semaphore with an initial count of
    5, or a ring buffer with 256 entries might be implemented using
    a counting semaphore with an initial count of 256.

    You are not guarding multiple copies of a resource but are, instead
    performing simple task synchronization. I believe that a binary
    semaphore is most appropriate for your application.

    >Michael,
    >
    >Maybe its again my fault that I held back some info (this is something
    >that I am doing at my workplace, so its a little difficult for me to
    >post the entire code here, or to give out all the details. I hope you'd
    >understand).


    I certainly do understand. I've never known any of this newsgroup's
    participants who are not working on their employer's or client's projects
    and disclosure is therefore always an issue. At the same time, however,
    you cannot expect others to help solve your problems if you won't tell
    them about your system!

    >The task B, when reads the message in the message queue, replies back.
    >The reply is on the standard output, which is redirected through a
    >named pipe to a socket that the task A reads. (Thus the task A reads
    >the messages from the server, and also from the redirected output of
    >the task B).


    This seems overly complex. Could it be that a feedback loop occurs when
    task A receives task B's response and sends that response back to task B
    and so on? Why not have task B send its reply directly to the network
    stack using sendto()? Perhaps the answer to that question is that task
    B is not aware of recipientís address (sendto()'s "struct sockaddr * to"
    parameter). If so, this would indicate that your design may best be
    implemented using only one task. There is, after all, only one
    asynchronous event (receipt of the server's messages) to be serviced.

    >The trouble usually arises when the server snaps connection abruptly.
    >(I now have it coded in my code, to tun that test case, that the server
    >drops connection abruptly after some x lines of output received from
    >task A, which in turn was the task B's output, passed through the names
    >pipe). When task A recognises that it can't read from the server socket
    >(because the remote server has dropped the connection), or can't write
    >to it, it needs to drop the TCP connection from its own side.
    >
    >After doing that, I need to clean up things from my side. This includes
    >multiple commands that might be in the message queue.
    >
    >More importantly, I need to clear up the command that might be being
    >executed at that instant of time, in task B. I can either
    >1. Stop the stdout redirection till the command is over, or
    >2. Simply cancle the execution, and cleanup the current command in the
    >queue. I am not sure if I can do the cleanup perfectly, and not leave
    >memory leaks, so I prefer not taking this approach.
    >3. Simply leave the command running as is, but prevent the task A from
    >sending task B's output back to the server.
    >
    >I typically take approach B.
    >
    >In running this a couple of times, I find the system hung. I do have
    >access to the console, but the board's other tasks don't run ( My
    >device's display freezes, and its keypad also doesn't work). When I do
    >an "i", I figure that the task B has gone into a "Pend+I" state.


    The default priority of the shell task is very high (low value) to allow
    it to continue working is situations just such as yours!

    >I am not sure if this additional scheme of things has got anything to
    >do with the task going into pend + I state, but having spent quite some
    >time trying to figure out what's going wrong, I was thinking of trying
    >to work backwards, if I got some clue what exactly gets the task into
    >pend + I state.


    I'm almost willing to bet that it's the result of a stack overflow or
    other data corruption that is a side effect of some other problem. One
    thing to be certain of is that you examine the return value of ALL
    API calls. If there are any that returned the ERROR value but your
    program goes on as if they had succeeded, then data corruption can
    easily occur!

    >Kaushal,
    >
    >I think it is a standard producer-consumer problem, but in this case,
    >the producer keeps running. In fact, when the task B produces its
    >reply, it keeps reading reply and sending it back to task A, but also
    >in evrey iteration when it gets some data to send back to A through the
    >pipe, it sleeps, allowing task A to run again to check for any more
    >inputs from the server, and post task B's replies back too.


    Is this the feedback loop of which I am suspicious?

    >Let me try the Mutex and see if that helps resolve the problem.


    Why? Again from the V5.5 Programmer's Guide:

    The mutual-exclusion semaphore is a specialized binary semaphore
    designed to address issues inherent in mutual exclusion, including
    priority inversion, deletion safety, and recursive access to resources.
    The fundamental behavior of the mutual-exclusion semaphore is identical
    to the binary semaphore, with the following exceptions:
    [] It can be used only for mutual exclusion.
    [] It can be given only by the task that took it.
    [] It cannot be given from an ISR.
    [] The semFlush( ) operation is illegal.

    The first two exceptions indicate that a mutex is unsuitable for your
    application. Again, a binary semaphore seems most suitable for the task
    synchronization you are doing.

    >Thanks everyone for your help. I appreciate it.


    You're welcome.

    --
    ================================================== ======================
    Michael Kesti | "And like, one and one don't make
    | two, one and one make one."
    mrkesti at comcast dot net | - The Who, Bargain

  10. Re: Task goes into a Pend+I state : Semaphore issue?

    Tried semBCreate. No advantage. Still the same problem.

    However, there's something I noticed that might have a bearing on what
    you mentioned Michael. The "reply" to the messages/commands by the task
    B, if small (say 30 lines), passes by easily, with 40 repetitions.
    While the "reply" if large (say 200 lines) results in the problem
    (system hanging and Pend + I state for task B).

    This leads me into believing that there's some memory corruption
    happening. Where, is beyond me, but it sure is happening somewhere.

    Now two-three areas look a little suspicious.

    1.
    The way I read the task B's output into the pipe is like this:

    do
    {
    if (ERROR == ioctl (m_dev_fds, FIONREAD, (int)&bytes_unread))
    break;
    taskDelay(10);
    } while(bytes_unread);

    The taskDelay is to allow task A to run, if a long message is being
    read.

    My question is: Isn't it possible that I am writing into the pipe
    buffer, that I am not reading out? Could this be a buffer overflow
    problem?

    2. The problem almost always arises when the task A runs when the task
    B has just done a taskDelay. At this time, task A closes the sockets,
    and pends on another semaphore (that would signal the task A to pend on
    the server and task B's sockets again). So that implies that maybe the
    cause of this situation is that task B writes into the m_dev_fds while
    the task A which is supposed to pend on these sockets to read from them
    is just pended on the semaphore that signals it to run.

    3. The other task that has a priority of 20 is hwtk with an entry
    function of 80195004. I suspect it is some system task (something like
    hardware task???). Any clue? That might be another lead I would want to
    try.

    I appreciate your help ... I know without code its almost impossible to
    solve. I am begining to wear out from this problem, and possibly would
    post the modified (unrecognisable) code soon (I am now holding back
    because even that is a big task, modifying things, so that its no
    longer work code).

    Thanks.


  11. Re: Task goes into a Pend+I state : Semaphore issue?

    Okay here's my requirement. Maybe someone can suggest a better design:

    1. I have a server (S) that connects to my module and issues commands.
    calls functions
    init() General initialization of my module,
    start() Establishes a TCP connection to my module.

    2. S does not give me any indication that it is going to stop
    connection. It just terminates connection from its side.

    3. After connecting, S can issue commands that can be either commands
    that I implement, or can be shell commands.

    4. S can keep issuing commands. If we can get S to wait, we can
    implement the entire thing in one task. I'd have to go back to this
    design if the current one doesn't work.

    5. There's a particular format of commands, something like
    -a -b -c command

    6. Each command is identified by a unique number that (is monotonically
    increasing) helps S match its commands and replies.

    Now the way my module works is this::

    When S calls init(), I initialize
    a) A global variable called m_connected_flag to TRUE
    b) Create a named pipe
    c) Create the two tasks A and B (A is the read task and B is the
    execute task)
    c) Redirect the Task B's stdout and stderr to the socket opened with
    the pipe.

    Someone asked me this question: Why can't you get B to directly write
    to S. The reason is that the shell commands are executed in B using
    execute(). These commands don't return the output in the exact format
    that is required by S. Also, they don't know the unique number for
    identification of commands.

    There're 2 semaphores, say X and Y. At creation time, A pends on X and
    B on Y.

    Whenever some server S starts a session by calling start(), the
    function start does a semGive(X). The task A is pended on X (by doing a
    semTake()). Thus when the TCP is establishes (in function start()), the
    task A starts running and pends on a select stmt, listening to
    m_dev_fds (task B's socket) and m_sock_fds (S's socket).

    When A gets a command from S, it writes it into a command queue and
    does a semGive(Y). At this time B starts to run, and either returns a
    reply on its own, or does an "execute". The reply on stdout of B is
    read into m_dev_fds using ioctl.
    while {
    if(ERROR == ioctl (m_dev_fds, FIONREAD, &number_of_bytes_unread))
    break;
    } (number_of_bytes);

    That's about the problem and current design. Can someone suggest a
    better alternative?


  12. Re: Task goes into a Pend+I state : Semaphore issue?

    I am wondering, in task B, are there any printf, or redirect I/O cause
    issue.
    No code in taskB, not knowing what happened.


+ Reply to Thread