Hanging popen/fgets/pclose - Unix

This is a discussion on Hanging popen/fgets/pclose - Unix ; In the process of executing a tool in the background I use popen/fgets/pclose. Something I notice though is that some times fgets just hangs in this process. At the bottom is the code for this. What I'm experiencing is that: ...

+ Reply to Thread
Results 1 to 6 of 6

Thread: Hanging popen/fgets/pclose

  1. Hanging popen/fgets/pclose

    In the process of executing a tool in the background I use
    popen/fgets/pclose. Something I notice though is that some times fgets just
    hangs in this process.

    At the bottom is the code for this.

    What I'm experiencing is that:

    if (fgets(szOutput, sizeof(szOutput)-1, m_fp) == NULL)

    just hangs. I've seen this with Mac OS X 10.4 in particular as well as Linux
    on x86.

    If I run the same tool as normal from the shell it might take 15 seconds to
    excute but otherwise it completes succesfully. It dumps several hundred
    lines of text (I didn't count how many though) but I ensure that I
    fflush(NULL) before running. So the application to be executed does
    certainly flush it's streams.

    Any suggestions are welcome. Perhaps I should avoid this altogether and
    implement it using a different type of API?

    Thanks.

    -- Henrik

    -----------

    Notice that some code is missing as I've only shown the code for
    unix-synchronous processes.

    CProcess::CProcess(string strCommand, enum SyncType eSyncType, int nTimeout)
    {
    m_strCommand = strCommand;

    m_bStarted = false;
    m_eSyncType = eSyncType;
    }

    CProcess::~CProcess()
    {
    Stop();
    }

    bool CProcess::Run()
    {
    if (m_bStarted)
    return false;

    if (m_eSyncType == SYNC)
    {
    if ((m_fp = popen((char *) m_strCommand.c_str(), "r")) == NULL)
    return false;
    }

    m_bStarted = true;

    if (m_eSyncType == SYNC)
    {
    return Wait();
    }
    else
    {
    return true;
    }
    }

    void CProcess::Stop()
    {
    if (m_bStarted)
    {
    m_bStarted = false;
    }
    }

    bool CProcess::Wait()
    {
    char szOutput[1024];

    if (!m_bStarted)
    return false;

    if (m_eSyncType == SYNC)
    {
    while (true)
    {
    // Some times it hangs here
    if (fgets(szOutput, sizeof(szOutput)-1, m_fp) == NULL)
    break;

    m_strOutput += szOutput;
    }

    if (pclose(m_fp) == -1)
    return false;

    m_bStarted = false;
    }

    return true;
    }



  2. Re: Hanging popen/fgets/pclose

    In article <472f8819$0$2111$edfadb0f@dtext02.news.tele.dk>,
    "Henrik Goldman" wrote:

    > In the process of executing a tool in the background I use
    > popen/fgets/pclose. Something I notice though is that some times fgets just
    > hangs in this process.
    >
    > At the bottom is the code for this.
    >
    > What I'm experiencing is that:
    >
    > if (fgets(szOutput, sizeof(szOutput)-1, m_fp) == NULL)
    >
    > just hangs. I've seen this with Mac OS X 10.4 in particular as well as Linux
    > on x86.
    >
    > If I run the same tool as normal from the shell it might take 15 seconds to
    > excute but otherwise it completes succesfully. It dumps several hundred
    > lines of text (I didn't count how many though) but I ensure that I
    > fflush(NULL) before running. So the application to be executed does
    > certainly flush it's streams.


    Does it end its output with newline? fgets() waits until it gets a
    newline or EOF before returning.

    >
    > Any suggestions are welcome. Perhaps I should avoid this altogether and
    > implement it using a different type of API?
    >
    > Thanks.
    >
    > -- Henrik
    >
    > -----------
    >
    > Notice that some code is missing as I've only shown the code for
    > unix-synchronous processes.
    >
    > CProcess::CProcess(string strCommand, enum SyncType eSyncType, int nTimeout)
    > {
    > m_strCommand = strCommand;
    >
    > m_bStarted = false;
    > m_eSyncType = eSyncType;
    > }
    >
    > CProcess::~CProcess()
    > {
    > Stop();
    > }
    >
    > bool CProcess::Run()
    > {
    > if (m_bStarted)
    > return false;
    >
    > if (m_eSyncType == SYNC)
    > {
    > if ((m_fp = popen((char *) m_strCommand.c_str(), "r")) == NULL)
    > return false;
    > }
    >
    > m_bStarted = true;
    >
    > if (m_eSyncType == SYNC)
    > {
    > return Wait();
    > }
    > else
    > {
    > return true;
    > }
    > }
    >
    > void CProcess::Stop()
    > {
    > if (m_bStarted)
    > {
    > m_bStarted = false;
    > }
    > }
    >
    > bool CProcess::Wait()
    > {
    > char szOutput[1024];
    >
    > if (!m_bStarted)
    > return false;
    >
    > if (m_eSyncType == SYNC)
    > {
    > while (true)
    > {
    > // Some times it hangs here
    > if (fgets(szOutput, sizeof(szOutput)-1, m_fp) == NULL)
    > break;
    >
    > m_strOutput += szOutput;
    > }
    >
    > if (pclose(m_fp) == -1)
    > return false;
    >
    > m_bStarted = false;
    > }
    >
    > return true;
    > }


    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  3. Re: Hanging popen/fgets/pclose

    On Tuesday 06 Nov 2007 7:33 am Barry Margolin
    wrote in article
    :

    > In article <472f8819$0$2111$edfadb0f@dtext02.news.tele.dk>,
    > "Henrik Goldman" wrote:
    >
    >> In the process of executing a tool in the background I use
    >> popen/fgets/pclose. Something I notice though is that some times
    >> fgets just hangs in this process.
    >>
    >> At the bottom is the code for this.
    >>
    >> What I'm experiencing is that:
    >>
    >> if (fgets(szOutput, sizeof(szOutput)-1, m_fp) == NULL)
    >>
    >> just hangs. I've seen this with Mac OS X 10.4 in particular as well
    >> as Linux on x86.
    >>
    >> If I run the same tool as normal from the shell it might take 15
    >> seconds to excute but otherwise it completes succesfully. It dumps
    >> several hundred lines of text (I didn't count how many though) but I
    >> ensure that I fflush(NULL) before running. So the application to be
    >> executed does certainly flush it's streams.

    >
    > Does it end its output with newline? fgets() waits until it gets a
    > newline or EOF before returning.


    Or if the buffer becomes full.




  4. Re: Hanging popen/fgets/pclose

    >
    > Does it end its output with newline? fgets() waits until it gets a
    > newline or EOF before returning.
    >


    Yes it does. I made sure that this tool both uses printf("\n"); and
    fflush(NULL). However this does not help.
    It will still block.

    Now I replaced fgets with fgetc:

    while (true)
    {
    c = fgetc(m_fp);

    if (feof(m_fp) != 0 || ferror(m_fp) != 0)
    break;

    m_strOutput += c;
    }

    This works quite good but now I face a new problem with speed. Some
    applications can take 20 minutes longer to execute using this change.
    So it's not ideal.
    Is there any alternative to this?
    What about using fread perhaps with a length? However is there a way
    to find the pending number of chars to be read?

    Thanks.


  5. Re: Hanging popen/fgets/pclose


    > Or if the buffer becomes full.
    >


    Which buffer specifically are you thinking about?
    Even if a buffer is full... how would I empty it?

    -- Henrik


  6. Re: Hanging popen/fgets/pclose

    "Henrik Goldman" wrote in message
    news:472f8819$0$2111$edfadb0f@dtext02.news.tele.dk ...
    > In the process of executing a tool in the background I use
    > popen/fgets/pclose. Something I notice though is that some times fgets
    > just hangs in this process.
    >
    > At the bottom is the code for this.
    >
    > What I'm experiencing is that:
    >
    > if (fgets(szOutput, sizeof(szOutput)-1, m_fp) == NULL)
    >
    > just hangs.


    How do you know it hangs in fgets()?
    What happens if you use popen() to run something instead of the tool (say
    "cat file" where file resembles output from the tool)?

    > I've seen this with Mac OS X 10.4 in particular as well as Linux on x86.


    So the behaviour is more likely to be "by design" rather than some sort of
    bug.

    > If I run the same tool as normal from the shell it might take 15 seconds
    > to excute but otherwise it completes succesfully.


    What about if you do "tool args... | cat" from the shell?

    Have you tried running the application under a system call tracer (or
    attaching one before the popen()), eg strace on Linux?

    > It dumps several hundred lines of text (I didn't count how many though)
    > but I ensure that I fflush(NULL) before running. So the application to be
    > executed does certainly flush it's streams.


    Lack of flushing shouldn't cause the problem, but equally, flushing should
    do no harm either.

    > Any suggestions are welcome. Perhaps I should avoid this altogether and
    > implement it using a different type of API?


    Without knowing what the problem is, I wouldn't recommend it.

    Alex



+ Reply to Thread