newbie question about fork/pipe/stdin/stdout - Unix

This is a discussion on newbie question about fork/pipe/stdin/stdout - Unix ; Hello! I'm writing small C++ under Linux application with will be transfer email from some IMAP server and put it to local database. I have one problem: I need to pass message to anti virus filter and spam filer. When ...

+ Reply to Thread
Results 1 to 8 of 8

Thread: newbie question about fork/pipe/stdin/stdout

  1. newbie question about fork/pipe/stdin/stdout

    Hello!

    I'm writing small C++ under Linux application with will be transfer
    email from some IMAP server and put it to local database.
    I have one problem:
    I need to pass message to anti virus filter and spam filer.
    When mail exceeded some limit, I my program is stopped on "waitpid()"
    and wait when spamc script ends. but this script never ends. From
    other side, another filter (anti virus) working fine on same code.
    There is the part of code:
    -----------------------------------------------------------------------------------------------------
    pipetest = pipe(pipeStdin);
    pipe(pipeStdout);
    fcntl(pipeStdout[1], F_SETFD, fcntl(pipeStdout[1], F_GETFD) & ~1);
    if(pipetest != 0) {
    std::cerr << " unable top open pipe!!! exitting "< exit(20);
    }
    {
    if( (pid = fork()) == 0) { // child
    DEBUG ("Settign UID/GID for child process to
    "< if (setgid(gid)!=0) {
    std::cerr << "Unable to set GID "< throw;
    }
    if (setuid(uid)!=0) {
    std::cerr << "Unable to set UID "< throw;
    }

    dup2(pipeStdin[0], STDIN_FILENO);
    dup2(pipeStdout[1], STDOUT_FILENO);
    close(pipeStdin[1]);
    close(pipeStdin[0]);
    close(pipeStdout[0]);
    close(pipeStdout[1]);
    int rc;
    rc=execlp(exec.c_str(),exec.c_str(),NULL);
    close(0);
    close(1);
    _exit (rc);
    }
    close(pipeStdout[1]);

    int output;
    char c;
    DEBUG("Send "< FILE* stream;
    close (pipeStdin[0]);
    stream = fdopen (pipeStdin[1], "w");
    // for (unsigned int i = 0; i // fputc(msg[i],stream);
    // }
    fprintf( stream, msg.c_str());
    fflush (stream);
    fclose (stream);
    close(pipeStdin[1]);
    int status;
    DEBUG ("Waiting for PID");
    waitpid (pid, &status, 0);
    DEBUG ("Waiting END");
    stream = fdopen (pipeStdout[0], "r");
    std::string out;
    while(!feof(stream)) {
    if (fread(&c,1, 1,stream)==1)
    // std::cerr << c;
    out = out+ c;
    std::cout < }
    close (pipeStdout[0]);
    close (pipeStdout[1]);
    std::cout << "child end with " << WEXITSTATUS(status) <<"
    "< ---------------------------------------------------------------------------
    where I wrong?
    User (uid/gid variables) haves next limits:
    #limit
    cputime unlimited
    filesize unlimited
    datasize unlimited
    stacksize 8192 kbytes
    coredumpsize 0 kbytes
    memoryuse unlimited
    vmemoryuse unlimited
    descriptors 1024
    memorylocked 32 kbytes
    maxproc 8190


    Thank you. Vladimir.


  2. Re: newbie question about fork/pipe/stdin/stdout

    I forgot one detail:
    118kb message cannot be passed to filter.
    13kb - passed normal.


  3. Re: newbie question about fork/pipe/stdin/stdout

    wrote in message
    news:1193405143.360453.36370@57g2000hsv.googlegrou ps.com...
    > I'm writing small C++ under Linux application with will be transfer
    > email from some IMAP server and put it to local database.
    > I have one problem:
    > I need to pass message to anti virus filter and spam filer.
    > When mail exceeded some limit, I my program is stopped on "waitpid()"
    > and wait when spamc script ends. but this script never ends. From
    > other side, another filter (anti virus) working fine on same code.


    If the child consumes all input before producing any output, the parent
    should write all the input, read the output until EOF, and only then call
    wait(). Using the code you provided, if there is "too much" output, the
    child will block in write() before it can exit, and the parent is waiting
    for it to exit before calling read(): this is a deadlock.

    If the child may produce output before consuming all input, you will have to
    do something different (more complicated). You could use non-blocking IO and
    select()/poll() or equivalent in the parent to move data between the
    processes whenever possible (again, calling wait() only after you have read
    EOF from the child). Or you could use a third process with some more
    "plumbing" to create something like a shell pipeline. That is, after
    creating the first child process (in which you exec), fork() again in the
    parent to create another child which will read the output from the first
    child.

    HTH,
    Alex



  4. Re: newbie question about fork/pipe/stdin/stdout

    Alex,
    Thank you,
    put "wait" after read partially help me.
    so, one "big" message passed to child and comes back.
    But in some cases (toooooooo biiiiiiiig messages) child will output
    any data to stdout. so, my program was locked on "while(!feof())".

    vladimir
    czech republic.


  5. Re: newbie question about fork/pipe/stdin/stdout

    wrote in message
    news:1193436864.842202.78810@o38g2000hse.googlegro ups.com...
    > Alex,
    > Thank you,
    > put "wait" after read partially help me.
    > so, one "big" message passed to child and comes back.
    > But in some cases (toooooooo biiiiiiiig messages) child will output
    > any data to stdout. so, my program was locked on "while(!feof())".


    I would expect the process to block inside fprintf(stream, msg.c_str())
    (specifically, in a call to write()), if the child writes "too much" before
    you get to reading it. This is another deadlock (both processes are trying
    to write more before they read), which the second paragraph in my previous
    reply addresses.

    Alex



  6. Re: newbie question about fork/pipe/stdin/stdout

    On Oct 26, 3:14 pm, vladimir.plotni...@gmail.com wrote:

    > put "wait" after read partially help me.
    > so, one "big" message passed to child and comes back.
    > But in some cases (toooooooo biiiiiiiig messages) child will output
    > any data to stdout. so, my program was locked on "while(!feof())".


    Your proxy should make forward progress any time it can. Waiting for
    one thing to finish before starting another is a recipe for disaster
    if something else waits for the second thing to finish before
    finishing the first.

    DS


  7. Re: newbie question about fork/pipe/stdin/stdout

    hello agin,
    Thank you for replies ;-)

    I changed my code to next:
    .....
    fcntl(pipeStdout[0], F_SETFD, fcntl(pipeStdout[0], F_GETFD) |
    O_NONBLOCK);
    fcntl(pipeStdin[1], F_SETFD, fcntl(pipeStdin[1], F_GETFD) |
    O_NONBLOCK);
    ....
    stream = fdopen (pipeStdin[1], "w");
    streamIn = fdopen (pipeStdout[0], "r");
    DEBUG ("output start");
    for (unsigned int i = 0; i i++) {
    while(select(pipeStdout[0]+1,
    &rfds, NULL, NULL, &tv)) {
    DEBUG ("add input");
    if (fread(&c,1,
    1,streamIn)==1)
    out = out+ c;
    }

    if(select(pipeStdin[1]+1,
    NULL, &wfds, NULL, &tv)) {
    if (!fputc(msg[i],stream))
    {DEBUG("output failed"); break; }
    DEBUG(i);
    } else {
    DEBUG ("send data
    problem"); i--;
    }
    }
    DEBUG("output end");
    2nd "select" catch for "send data problem" when child cannot receive
    more data.
    In this time child should send me data back. but 1st "select" never
    return value other than zero.
    Where I wrong again?

    Sorry for stupid questions, I never working with select/pipes before.


  8. Re: newbie question about fork/pipe/stdin/stdout

    wrote in message
    news:1193649642.329918.170420@d55g2000hsg.googlegr oups.com...
    > hello agin,
    > Thank you for replies ;-)
    >
    > I changed my code to next:
    > ....
    > fcntl(pipeStdout[0], F_SETFD, fcntl(pipeStdout[0], F_GETFD) |
    > O_NONBLOCK);
    > fcntl(pipeStdin[1], F_SETFD, fcntl(pipeStdin[1], F_GETFD) |
    > O_NONBLOCK);


    You must use F_GETFL/F_SETFL here.

    > ....
    > stream = fdopen (pipeStdin[1], "w");
    > streamIn = fdopen (pipeStdout[0], "r");


    You can't (reliably, at least) use stdio with descriptors set to
    non-blocking mode; use read() and write().

    [snip rest of code]
    > Where I wrong again?


    I would use a loop which does something like this (I've ignored error
    conditions etc which obscure the basic logic):

    1. Call select() to wait until it reports pipeStdout[0] is readable or
    pipeStdin[1] is writeable. (The latter only if you still have data to
    write to the child.)
    2a. If pipeStdout[0] is readable, call read().
    2b. If read() returns 0 (indicating EOF), exit the loop.
    2c. Do whatever you need to with the data just read.
    3a. If you have data to write to the child and pipeStdin[1] is writeable,
    call write() to try to write some more data. For efficiency, try to
    write as much data as you have available.
    3b. If you succeed in writing the last byte, close pipeStdin[1] so the
    child will see EOF, and make it so that in #1 you no longer check the
    status of pipeStdin[1] (for obvious reasons).

    After you exit the loop, call wait() to wait for the child process to exit
    (if it hasn't already) and collect its status.

    Alex



+ Reply to Thread