How does one effect O_NONBLOCK?! - Unix

This is a discussion on How does one effect O_NONBLOCK?! - Unix ; Hello, I'm trying to create a wrapper around fork/execve to run an external process and interact with its standard streams through pipes. My method: once I've created 3 pipes (for stdout, stderr and stdin) and forked(), I close() the unneeded ...

+ Reply to Thread
Results 1 to 17 of 17

Thread: How does one effect O_NONBLOCK?!

  1. How does one effect O_NONBLOCK?!

    Hello,

    I'm trying to create a wrapper around fork/execve to run an external
    process and interact with its standard streams through pipes.

    My method: once I've created 3 pipes (for stdout, stderr and stdin)
    and forked(), I close() the unneeded ends of the pipes in both the
    child and parent processes and then dup2() the 3 descriptors into the
    corresponding ends of the pipes in the child before execve()-ing.

    In the parent I set up a loop in which select() is called to deal with
    the 3 pipes "simultaneously".

    I can read stuff that the child is writing to its stdout and stderr
    fine, but I'm having trouble detecting when I should pump stuff down
    the stdin pipe.

    The following is a sketch of my select()-loop, showing stdin related
    stuff only:

    fd_set s;
    FD_ZERO(&s); FD_SET(fd, &s); /* fd is the write end of the stdin
    pipe */

    while (select(fd + 1, NULL, &s, NULL, NULL) > 0)
    {
    if (FD_ISSET(fd, &s))
    {
    /*
    code to read from parent stdin and pump it to the child
    ...
    */
    }

    /*
    Some other checks, eventually leading to a break statement
    ...
    */

    FD_ZERO(&s); FD_SET(fd, &s);
    }

    The FD_ISSET() check inside the loop yields a positive result each
    time, rather than when the child is asking for input. I keep seeing
    the term "non-blocking" mentioned in relation to this, but I'm unsure
    how to set this up. O_NONBLOCK and fcntl() are somewhere in the mix, I
    believe. I'm going cross-eyed reading man-pages and Googling!
    Hopefully somebody can put me out of my misery with a little bit of
    example code or something?

    The full code with error checking and wot-not is rather long so I
    haven't presented it here. If needed it's available at:
    http://www.mr-edd.co.uk/files/guff/process.cpp

    I'll happily create a minimal executable if need be, but I'm hoping
    this is enough to go on...

    Kind regards,

    Edd

  2. Re: How does one effect O_NONBLOCK?!

    Edd wrote:
    > I'm trying to create a wrapper around fork/execve to run an external
    > process and interact with its standard streams through pipes.


    > My method: once I've created 3 pipes (for stdout, stderr and stdin)
    > and forked(), I close() the unneeded ends of the pipes in both the
    > child and parent processes and then dup2() the 3 descriptors into the
    > corresponding ends of the pipes in the child before execve()-ing.


    > In the parent I set up a loop in which select() is called to deal with
    > the 3 pipes "simultaneously".


    > I can read stuff that the child is writing to its stdout and stderr
    > fine, but I'm having trouble detecting when I should pump stuff down
    > the stdin pipe.


    > The following is a sketch of my select()-loop, showing stdin related
    > stuff only:


    If I understand it correctly with "stdin stuff" you mean the write
    end side of the pipe that's connected to stdin of the child process.
    Correct?

    > fd_set s;
    > FD_ZERO(&s); FD_SET(fd, &s); /* fd is the write end of the stdin
    > pipe */


    > while (select(fd + 1, NULL, &s, NULL, NULL) > 0)
    > {
    > if (FD_ISSET(fd, &s))
    > {
    > /*
    > code to read from parent stdin and pump it to the child
    > ...
    > */
    > }


    > /*
    > Some other checks, eventually leading to a break statement
    > ...
    > */


    > FD_ZERO(&s); FD_SET(fd, &s);
    > }


    > The FD_ISSET() check inside the loop yields a positive result each
    > time, rather than when the child is asking for input.


    As long as the child hasn't closed its stdin and there is still
    room in the pipe you can write without blocking, so select()
    returns true for that file descriptor. If you want to know the
    exact moment the child is doing a blocking read() (if it does
    that at all, see below) then I guess there's no way to do that.
    What the parent "sees" is a file-like pipe, into which it can
    write without blocking until the pipe's buffer is full.

    I have no idea what the child does, but it also might call
    select(), waiting for input from the parent to become ready
    for reading. In that case it would never even call read()
    until you already have written something to it's stdin.

    So I am a bit confused what exactly you want to achieve. Is
    it really a problem if you send something to the child and it
    isn't blocked in calling read()? At least I can't see that
    from your program. Can you perhaps try to describe a bit more
    about what you want to do and not just how you plan to do it?

    > I keep seeing
    > the term "non-blocking" mentioned in relation to this, but I'm unsure
    > how to set this up. O_NONBLOCK and fcntl() are somewhere in the mix, I
    > believe. I'm going cross-eyed reading man-pages and Googling!
    > Hopefully somebody can put me out of my misery with a little bit of
    > example code or something?


    I don't think setting file descriptors in the parent to non-
    blocking would help at all. You call select() before you try
    to read or write, so neither read() or write() will block.
    Thus setting the file descriptors non-blocking doesn't make
    much sense IMHO.

    As far as I can see there might be a rather different problem
    that's going to bother you. Since you obviously intend to run
    arbitrary programs as the child process you may find that
    a lot of them won't react as you may assume. By redirecting
    the stdout of the child to a pipe and if it uses the normal
    C standard output functions (printf() etc. or their C++ equi-
    valents) their output will not appear in a line-by-line
    fashion anymore like it happens when you run a program from a
    terminal but instead the output will come in in large chunks.
    That's because the output functions normally do buffering,
    storing a line-length of output when writing to a terminal
    but using the whole available buffer when writing to a file.
    And a pipe looks like a file and not like a terminal. So you
    may have to use a pseudo-terminal between the parent and the
    child process in order to make the child process believe it's
    writing to a terminal and not a file...

    Regards, Jens
    --
    \ Jens Thoms Toerring ___ jt@toerring.de
    \__________________________ http://toerring.de

  3. Re: How does one effect O_NONBLOCK?!

    On Jun 1, 10:13 pm, j...@toerring.de (Jens Thoms Toerring) wrote:
    > Edd wrote:
    > > I'm trying to create a wrapper around fork/execve to run an external
    > > process and interact with its standard streams through pipes.
    > > My method: once I've created 3 pipes (for stdout, stderr and stdin)
    > > and forked(), I close() the unneeded ends of the pipes in both the
    > > child and parent processes and then dup2() the 3 descriptors into the
    > > corresponding ends of the pipes in the child before execve()-ing.
    > > In the parent I set up a loop in which select() is called to deal with
    > > the 3 pipes "simultaneously".
    > > I can read stuff that the child is writing to its stdout and stderr
    > > fine, but I'm having trouble detecting when I should pump stuff down
    > > the stdin pipe.
    > > The following is a sketch of my select()-loop, showing stdin related
    > > stuff only:

    >
    > If I understand it correctly with "stdin stuff" you mean the write
    > end side of the pipe that's connected to stdin of the child process.
    > Correct?


    Yes, exactly.

    > > fd_set s;
    > > FD_ZERO(&s); FD_SET(fd, &s); /* fd is the write end of the stdin
    > > pipe */
    > > while (select(fd + 1, NULL, &s, NULL, NULL) > 0)
    > > {
    > > if (FD_ISSET(fd, &s))
    > > {
    > > /*
    > > code to read from parent stdin and pump it to the child
    > > ...
    > > */
    > > }
    > > /*
    > > Some other checks, eventually leading to a break statement
    > > ...
    > > */
    > > FD_ZERO(&s); FD_SET(fd, &s);
    > > }
    > > The FD_ISSET() check inside the loop yields a positive result each
    > > time, rather than when the child is asking for input.

    >
    > As long as the child hasn't closed its stdin and there is still
    > room in the pipe you can write without blocking, so select()
    > returns true for that file descriptor. If you want to know the
    > exact moment the child is doing a blocking read() (if it does
    > that at all, see below) then I guess there's no way to do that.


    Ah, for some reason, this is what I was expecting select() to do.

    > I have no idea what the child does, but it also might call
    > select(), waiting for input from the parent to become ready
    > for reading. In that case it would never even call read()
    > until you already have written something to it's stdin.
    >
    > So I am a bit confused what exactly you want to achieve. Is
    > it really a problem if you send something to the child and it
    > isn't blocked in calling read()?


    No, it's no problem. I was under the impression that I would be told
    by select() when to start sending. But as you've explained, that's not
    going to happen.

    > > I keep seeing
    > > the term "non-blocking" mentioned in relation to this, but I'm unsure
    > > how to set this up. O_NONBLOCK and fcntl() are somewhere in the mix, I
    > > believe. I'm going cross-eyed reading man-pages and Googling!
    > > Hopefully somebody can put me out of my misery with a little bit of
    > > example code or something?

    >
    > I don't think setting file descriptors in the parent to non-
    > blocking would help at all. You call select() before you try
    > to read or write, so neither read() or write() will block.
    > Thus setting the file descriptors non-blocking doesn't make
    > much sense IMHO.


    I see that now, thanks.

    > As far as I can see there might be a rather different problem
    > that's going to bother you. Since you obviously intend to run
    > arbitrary programs as the child process you may find that
    > a lot of them won't react as you may assume. By redirecting
    > the stdout of the child to a pipe and if it uses the normal
    > C standard output functions (printf() etc. or their C++ equi-
    > valents) their output will not appear in a line-by-line
    > fashion anymore like it happens when you run a program from a
    > terminal but instead the output will come in in large chunks.
    > That's because the output functions normally do buffering,
    > storing a line-length of output when writing to a terminal
    > but using the whole available buffer when writing to a file.
    > And a pipe looks like a file and not like a terminal. So you
    > may have to use a pseudo-terminal between the parent and the
    > child process in order to make the child process believe it's
    > writing to a terminal and not a file...


    I see. But what negative effects could this have? For a child process
    that doesn't explicitly an regularly flush its output, I should expect
    data to be sent through in larger chunks. But the output buffer size
    isn't something that's typically considered when writing a "console
    application", so should this matter?

    I can see that it would perhaps be the case that the data written to
    the child's stdout and stderr might appear in the parent interleaved
    in a different way than it was originally sent. Would pseudo-terminals
    help with this particular issue?

    Thanks for your patient comments. They have been very enlightening.

    Kind regards,

    Edd

  4. Re: How does one effect O_NONBLOCK?!

    Edd wrote:
    > > As far as I can see there might be a rather different problem
    > > that's going to bother you. Since you obviously intend to run
    > > arbitrary programs as the child process you may find that
    > > a lot of them won't react as you may assume. By redirecting
    > > the stdout of the child to a pipe and if it uses the normal
    > > C standard output functions (printf() etc. or their C++ equi-
    > > valents) their output will not appear in a line-by-line
    > > fashion anymore like it happens when you run a program from a
    > > terminal but instead the output will come in in large chunks.
    > > That's because the output functions normally do buffering,
    > > storing a line-length of output when writing to a terminal
    > > but using the whole available buffer when writing to a file.
    > > And a pipe looks like a file and not like a terminal. So you
    > > may have to use a pseudo-terminal between the parent and the
    > > child process in order to make the child process believe it's
    > > writing to a terminal and not a file...


    > I see. But what negative effects could this have? For a child process
    > that doesn't explicitly an regularly flush its output, I should expect
    > data to be sent through in larger chunks. But the output buffer size
    > isn't something that's typically considered when writing a "console
    > application", so should this matter?


    Since I don't know what exactly you want to achieve with your
    program I wrote "might". It becomes a problem if you wait for
    some output of the child process in the parent before you send
    it new data. It's not an uncommon case in console applications
    that you send the application a line, it reacts with another
    query that you then answer etc. Now, if the parent process
    waits for a query from the child process before it sends the
    next line both processes might hanging indefinitely since the
    query from the child process got stuck in its output buffer.

    > I can see that it would perhaps be the case that the data written to
    > the child's stdout and stderr might appear in the parent interleaved
    > in a different way than it was originally sent.


    Yes, that could happen since stderr isn't buffered (even if its
    redirected to a file or pipe) while stdout is. So you could get
    output via stderr earlier than stuff that had been written to
    stderr.

    Citing Stevens: ANSI C requires that both stdin and stdout are
    (per deault) fully buffered unless they are connected to an
    interactive device (e.g. a terminal), stderr is never fully
    buffered.

    Under UNIX the normal behaviour is that stdin and stdout are
    line buffered when they are connected to a terminal and other-
    wise fully buffered, while stderr is always unbuffered.

    Of course, an application can change that via setbuf() or
    set setvbuf(), but most don't.

    > Would pseudo-terminals help with this particular issue?


    Yes, it would since using by a pseudo-terminal in between you
    would get the child process to react exactly the same way as
    it would if its stdout would be connected to a terminal (that's
    why it's called a pseudo-terminal), i.e. it's stdout would be
    line buffered (of course assuming that the application hasn't
    changed the default buffering mode).

    Regards, Jens
    --
    \ Jens Thoms Toerring ___ jt@toerring.de
    \__________________________ http://toerring.de

  5. Re: How does one effect O_NONBLOCK?!

    Edd wrote:
    > > As long as the child hasn't closed its stdin and there is still
    > > room in the pipe you can write without blocking, so select()
    > > returns true for that file descriptor. If you want to know the
    > > exact moment the child is doing a blocking read() (if it does
    > > that at all, see below) then I guess there's no way to do that.


    > Ah, for some reason, this is what I was expecting select() to do.


    > > I have no idea what the child does, but it also might call
    > > select(), waiting for input from the parent to become ready
    > > for reading. In that case it would never even call read()
    > > until you already have written something to it's stdin.
    > >
    > > So I am a bit confused what exactly you want to achieve. Is
    > > it really a problem if you send something to the child and it
    > > isn't blocked in calling read()?


    > No, it's no problem. I was under the impression that I would be told
    > by select() when to start sending. But as you've explained, that's not
    > going to happen.


    Perhaps a few more remarks on select(2) since there often
    seem to be a number of misconceptions about what it does.
    All select() does tell you is that you can call read() or
    write() without your process getting blocked, i.e. get-
    ting put to sleep until data are available for reading
    or can be written by your process, and other processes
    getting scheduled instead. And "not getting blocked" also
    includes the possibility that, when you try, you can't
    read() or write() even a single byte, e.g. because the
    other side has closed its side of a pipe or socket etc.
    (in that case, when you try to read() or write(), the
    call will also return immediately, returning a value
    indicating failure). So it just tells you that you can
    call read() or write() without having to wait for any-
    thing - there is really nothing more select() tells you.
    No guarantees for success or anything else, just "You
    won't be put to sleep" when you read() or write() on
    the file descriptors flagged as ready for that opera-
    tions. That may seem of very limited usefulness, but it
    is actually very useful if employed according to these
    limitations, like not expecting it to do some "magic"
    like determining the state of another process you're
    trying to communicate with.

    Regards, Jens
    --
    \ Jens Thoms Toerring ___ jt@toerring.de
    \__________________________ http://toerring.de

  6. Re: How does one effect O_NONBLOCK?!

    Edd wrote:
    > I'm trying to create a wrapper around fork/execve to run an external
    > process and interact with its standard streams through pipes.
    >
    > My method: once I've created 3 pipes (for stdout, stderr and stdin)
    > and forked(), I close() the unneeded ends of the pipes in both the
    > child and parent processes and then dup2() the 3 descriptors into the
    > corresponding ends of the pipes in the child before execve()-ing.


    This all sounds fine. But you must set the pipes non-blocking else you
    can still end up blocking, even if you only call read() or write() after
    select() says the descriptor is, respectively, readable or writeable:

    If there is an error condition, read() and write() will return
    immediately indicating the error. No problem there.

    If select() says a descriptor is readable or writeable, that might not
    be the case by the time you call read() or write(). Usually not a
    problem in practice, but not guaranteed.

    A blocking write() normally blocks until all data have been written. If
    select() says a descriptor is writeable, you don't know how many bytes
    you can write without blocking.

    > In the parent I set up a loop in which select() is called to deal with
    > the 3 pipes "simultaneously".
    >
    > I can read stuff that the child is writing to its stdout and stderr
    > fine, but I'm having trouble detecting when I should pump stuff down
    > the stdin pipe.


    Depending on the application, something like the following may suit well
    (ignoring error conditions for clarity):

    Create a buffer for the pipe, initially empty.

    When you call select(), only check for writeability if the buffer is not
    empty.

    After select() indicates the pipe is writeable, try to write() the
    buffer's contents and update the buffer accordingly.

    When you have data to write and the buffer is empty, call write()
    immediately; if write() returns less than the size argument, copy the
    remainder to the buffer.

    When you have data to write and the buffer is not empty, append the data
    to the buffer.

    Alex

  7. Re: How does one effect O_NONBLOCK?!

    Hi Alex,

    On Jun 2, 8:32 pm, Alex Fraser wrote:
    > Edd wrote:
    > > I'm trying to create a wrapper around fork/execve to run an external
    > > process and interact with its standard streams through pipes.

    >
    > > My method: once I've created 3 pipes (for stdout, stderr and stdin)
    > > and forked(), I close() the unneeded ends of the pipes in both the
    > > child and parent processes and then dup2() the 3 descriptors into the
    > > corresponding ends of the pipes in the child before execve()-ing.

    >
    > This all sounds fine. But you must set the pipes non-blocking else you
    > can still end up blocking, even if you only call read() or write() after
    > select() says the descriptor is, respectively, readable or writeable:
    >
    > If there is an error condition, read() and write() will return
    > immediately indicating the error. No problem there.
    >
    > If select() says a descriptor is readable or writeable, that might not
    > be the case by the time you call read() or write(). Usually not a
    > problem in practice, but not guaranteed.
    >
    > A blocking write() normally blocks until all data have been written. If
    > select() says a descriptor is writeable, you don't know how many bytes
    > you can write without blocking.


    I was just about to post a question asking about this! Thanks. I'll
    endeavor to use non-blocking I/O using the buffering method you
    described.

    So, coming kind of full-circle, how do I set up non-blocking I/O.
    Something like this?

    /* For the business end of each pipe in the parent: */
    fcntl(fd, F_SETFL | fds_current_flags, O_NONBLOCK);

    Many thanks,

    Edd

  8. Re: How does one effect O_NONBLOCK?!

    Hi again Jens,

    On Jun 2, 12:27 am, j...@toerring.de (Jens Thoms Toerring) wrote:
    > Edd wrote:


    > > Would pseudo-terminals help with this particular issue?


    > Yes, it would since using by a pseudo-terminal in between you
    > would get the child process to react exactly the same way as
    > it would if its stdout would be connected to a terminal (that's
    > why it's called a pseudo-terminal), i.e. it's stdout would be
    > line buffered (of course assuming that the application hasn't
    > changed the default buffering mode).


    Ok great. Looks like I have some more reading to do, then!

    But I'm curious, now; if using a pseudo-terminal allows you to create
    a perfect proxy, what reason is there to use pipes? I guess larger
    buffers make them more efficient for "noisy" programs?

    For the record, I will be using this function in a piece of software
    used for automation of tasks. A bit like make in some ways. From the
    sounds of things, pipes will probably suffice 99% of the time.
    Occasionally though, I will want to capture and process child output
    e.g. from pkg-config. I'm sure I won't need pseudo-terminals for pkg-
    config either, but the system is scriptable, so I have to assume as
    little as possible about the children that will be created.

    Thanks again, you've been a great help.

    Edd

  9. Re: How does one effect O_NONBLOCK?!

    Edd wrote:
    > Hi again Jens,


    > On Jun 2, 12:27 am, j...@toerring.de (Jens Thoms Toerring) wrote:
    > > Edd wrote:


    > > > Would pseudo-terminals help with this particular issue?


    > > Yes, it would since using by a pseudo-terminal in between you
    > > would get the child process to react exactly the same way as
    > > it would if its stdout would be connected to a terminal (that's
    > > why it's called a pseudo-terminal), i.e. it's stdout would be
    > > line buffered (of course assuming that the application hasn't
    > > changed the default buffering mode).


    > Ok great. Looks like I have some more reading to do, then!


    ;-)

    > But I'm curious, now; if using a pseudo-terminal allows you to create
    > a perfect proxy, what reason is there to use pipes? I guess larger
    > buffers make them more efficient for "noisy" programs?


    Pipes are often used between a parent and a child process that
    the parent spawned, but which didn't call exec(). So the author
    wrote both the parents and the childs code and could avoid any
    such issues. Then adding another layer of complexity completely
    unneeded don't make sense. And it's not really a problem due
    to the design of pipes as such, it's more a problem introduced
    by the output buffering of the C standard output functions.
    They may simply not flush the buffer to the pipe. And that's
    where pseudo-terminals come to the rescue.

    So normally things only get "interesting" if the childs exec()'s
    a program that hasn't been written with the possiblilty of being
    connected to a pipe in mind by using e.g. printf() and not flushing
    the output buffer at the end of a line (or using setvbuf() etc) and
    expects a kind of dialog with the user.

    > For the record, I will be using this function in a piece of software
    > used for automation of tasks. A bit like make in some ways. From the
    > sounds of things, pipes will probably suffice 99% of the time.
    > Occasionally though, I will want to capture and process child output
    > e.g. from pkg-config. I'm sure I won't need pseudo-terminals for pkg-
    > config either, but the system is scriptable, so I have to assume as
    > little as possible about the children that will be created.


    I guess pkg-config shouldn't pose a problem since it doesn't seem
    to do a "dialog" kind of thing (at least as far as I know) but just
    outputs data and exits. And when it exits its buffers get flushed
    automatically. So a simple pipe should be sufficient with that.

    Regards, Jens
    --
    \ Jens Thoms Toerring ___ jt@toerring.de
    \__________________________ http://toerring.de

  10. Re: How does one effect O_NONBLOCK?!

    Edd wrote:
    > Hi Alex,
    >
    > On Jun 2, 8:32 pm, Alex Fraser wrote:
    >> Edd wrote:
    >>> I'm trying to create a wrapper around fork/execve to run an external
    >>> process and interact with its standard streams through pipes.
    >>> My method: once I've created 3 pipes (for stdout, stderr and stdin)
    >>> and forked(), I close() the unneeded ends of the pipes in both the
    >>> child and parent processes and then dup2() the 3 descriptors into the
    >>> corresponding ends of the pipes in the child before execve()-ing.

    >> This all sounds fine. But you must set the pipes non-blocking else you
    >> can still end up blocking, even if you only call read() or write() after
    >> select() says the descriptor is, respectively, readable or writeable:
    >>
    >> If there is an error condition, read() and write() will return
    >> immediately indicating the error. No problem there.
    >>
    >> If select() says a descriptor is readable or writeable, that might not
    >> be the case by the time you call read() or write(). Usually not a
    >> problem in practice, but not guaranteed.
    >>
    >> A blocking write() normally blocks until all data have been written. If
    >> select() says a descriptor is writeable, you don't know how many bytes
    >> you can write without blocking.

    >
    > I was just about to post a question asking about this! Thanks. I'll
    > endeavor to use non-blocking I/O using the buffering method you
    > described.


    I meant to point out that I agree with the comments Jens made about
    using a pty, and that all the above applies in that case just the same.
    If you want to keep the child's stderr separate, using a pipe for that
    will not normally cause any ill effects due to buffering because (as
    Jens explained) stderr is unbuffered regardless of what it is connected to.

    > So, coming kind of full-circle, how do I set up non-blocking I/O.
    > Something like this?
    >
    > /* For the business end of each pipe in the parent: */
    > fcntl(fd, F_SETFL | fds_current_flags, O_NONBLOCK);


    Not sure if it was a typo, but:
    fcntl(fd, F_SETFL, fds_current_flags | O_NONBLOCK);

    You can get the current flags with fcntl(F_GETFL) - although in this
    case you know what they are anyway.

    Alex

  11. Re: How does one effect O_NONBLOCK?!

    On Jun 2, 11:59 pm, j...@toerring.de (Jens Thoms Toerring) wrote:
    > Edd wrote:
    > > Hi again Jens,
    > > On Jun 2, 12:27 am, j...@toerring.de (Jens Thoms Toerring) wrote:
    > > > Edd wrote:
    > > > > Would pseudo-terminals help with this particular issue?
    > > > Yes, it would since using by a pseudo-terminal in between you
    > > > would get the child process to react exactly the same way as
    > > > it would if its stdout would be connected to a terminal (that's
    > > > why it's called a pseudo-terminal), i.e. it's stdout would be
    > > > line buffered (of course assuming that the application hasn't
    > > > changed the default buffering mode).

    > > Ok great. Looks like I have some more reading to do, then!

    >
    > ;-)
    >
    > > But I'm curious, now; if using a pseudo-terminal allows you to create
    > > a perfect proxy, what reason is there to use pipes? I guess larger
    > > buffers make them more efficient for "noisy" programs?

    >
    > Pipes are often used between a parent and a child process that
    > the parent spawned, but which didn't call exec(). So the author
    > wrote both the parents and the childs code and could avoid any
    > such issues. Then adding another layer of complexity completely
    > unneeded don't make sense.


    I see, thanks.

    So I have one final question on the matter. Even with pseudo
    terminals, can I rely on select() indicating to me that I should read
    from the child's stdout and stderr independently? In other words will
    a select()-filled fd_set ever tell me that I can read from both the
    child's stdout and sterr and if so, how do I know which to read and
    forward first?

    Kind regards,

    Edd

  12. Re: How does one effect O_NONBLOCK?!

    Edd wrote:
    [snip]
    > So I have one final question on the matter. Even with pseudo
    > terminals, can I rely on select() indicating to me that I should read
    > from the child's stdout and stderr independently? In other words will
    > a select()-filled fd_set ever tell me that I can read from both the
    > child's stdout and sterr and if so, how do I know which to read and
    > forward first?


    It may, and you can't tell, because the timing of the writes is
    decoupled from select() indicating the readability. That is, there is an
    indeterminate delay between the two events, and stdout and stderr have
    independent delays. In practice, the delay is typically very small such
    that it may rarely happen.

    If you will ultimately merge the two streams, it would be best to never
    have them separated: dup2(1, 2) in the child before you exec.

    Alex

  13. Re: How does one effect O_NONBLOCK?!

    On Jun 4, 12:25 am, Alex Fraser wrote:
    > Edd wrote:
    >
    > [snip]
    >
    > > So I have one final question on the matter. Even with pseudo
    > > terminals, can I rely on select() indicating to me that I should read
    > > from the child's stdout and stderr independently? In other words will
    > > a select()-filled fd_set ever tell me that I can read from both the
    > > child's stdout and sterr and if so, how do I know which to read and
    > > forward first?

    >
    > It may, and you can't tell, because the timing of the writes is
    > decoupled from select() indicating the readability. That is, there is an
    > indeterminate delay between the two events, and stdout and stderr have
    > independent delays. In practice, the delay is typically very small such
    > that it may rarely happen.
    >
    > If you will ultimately merge the two streams, it would be best to never
    > have them separated: dup2(1, 2) in the child before you exec.


    Gah! I was afraid of that

    In the general case it will be rather hard to tell if the two streams
    are to be merged. I'll have to fudge my interface a little bit. C'est
    la guerre!

    Thanks for you help. I have non-blocking I/O working well now,
    seemingly.

    Edd

  14. Re: How does one effect O_NONBLOCK?!

    Edd wrote:
    > Hello,
    >
    > I'm trying to create a wrapper around fork/execve to run an external
    > process and interact with its standard streams through pipes.
    >


    Here is a GPL'd program which does that:

    http://saf.bio.caltech.edu/pub/softw...e-0.2.0.tar.gz

    Most of what you are looking for is in the nio.c and nio.h files. See
    in particular the examples which are shown in the nettee_cmd man page,
    which I think you can see here:

    http://saf.bio.caltech.edu/cgi-bin/c.../?q=nettee_cmd

    Regards,

    David Mathog


  15. Re: How does one effect O_NONBLOCK?!

    Hi David,

    On Jun 4, 6:00 pm, David Mathog wrote:
    > Edd wrote:
    > > Hello,

    >
    > > I'm trying to create a wrapper around fork/execve to run an external
    > > process and interact with its standard streams through pipes.

    >
    > Here is a GPL'd program which does that:
    >
    > http://saf.bio.caltech.edu/pub/softw...ools/beta-nett...
    >
    > Most of what you are looking for is in the nio.c and nio.h files. See
    > in particular the examples which are shown in the nettee_cmd man page,
    > which I think you can see here:
    >
    > http://saf.bio.caltech.edu/cgi-bin/c.../?q=nettee_cmd


    That's very kind of you, but I dare not look as the license I'd like
    to use is incompatible with the GPL. I'd like to use the Boost
    Software License v1 (http://www.boost.org/LICENSE_1_0.txt). I don't
    know if you'd be willing to let me look at your code under different
    terms?

    Kind regards,

    Edd

  16. Re: How does one effect O_NONBLOCK?!

    Edd wrote:
    > Hi David,
    >
    > On Jun 4, 6:00 pm, David Mathog wrote:
    >> Edd wrote:
    >>> Hello,
    >>> I'm trying to create a wrapper around fork/execve to run an external
    >>> process and interact with its standard streams through pipes.

    >> Here is a GPL'd program which does that:
    >>
    >> http://saf.bio.caltech.edu/pub/softw...ools/beta-nett...
    >>
    >> Most of what you are looking for is in the nio.c and nio.h files. See
    >> in particular the examples which are shown in the nettee_cmd man page,
    >> which I think you can see here:
    >>
    >> http://saf.bio.caltech.edu/cgi-bin/c.../?q=nettee_cmd

    >
    > That's very kind of you, but I dare not look as the license I'd like
    > to use is incompatible with the GPL.


    Ugh. we're not talking a lot of code here. The relevant sections are
    quite small. Most of it is just calls to standard routines, I'm not
    even sure that the parts you asked about are large enough to be
    "protected" by a license. In any case, you need legal cover, so here is
    an additional "educational license" you (or anybody else who just wants
    to see an example) may use:

    You are hereby granted permission to read, run, and modify the
    sections of this code in which O_NONBLOCK is implemented. You may use
    whatever you learn from this exercise in your own software, under
    whatever license you wish. However, you may not cut from the example
    code and paste it into nonGPL code.

    In other words, once you understand how I did it, you are free to
    do it the same way in your own code, if that's what you want. But
    you have to write your own code.

    Good enough?

    Regards,

    David Mathog


  17. Re: How does one effect O_NONBLOCK?!

    Hi David,

    Thanks very much for this. I'll give it a thorough going-over in due
    course.

    Edd

    On Jun 6, 6:13 pm, David Mathog wrote:
    > Edd wrote:
    > > Hi David,

    >
    > > On Jun 4, 6:00 pm, David Mathog wrote:
    > >> Edd wrote:
    > >>> Hello,
    > >>> I'm trying to create a wrapper around fork/execve to run an external
    > >>> process and interact with its standard streams through pipes.
    > >> Here is a GPL'd program which does that:

    >
    > >>http://saf.bio.caltech.edu/pub/softw...ools/beta-nett...

    >
    > >> Most of what you are looking for is in the nio.c and nio.h files. See
    > >> in particular the examples which are shown in the nettee_cmd man page,
    > >> which I think you can see here:

    >
    > >>http://saf.bio.caltech.edu/cgi-bin/c.../?q=nettee_cmd

    >
    > > That's very kind of you, but I dare not look as the license I'd like
    > > to use is incompatible with the GPL.

    >
    > Ugh. we're not talking a lot of code here. The relevant sections are
    > quite small. Most of it is just calls to standard routines, I'm not
    > even sure that the parts you asked about are large enough to be
    > "protected" by a license. In any case, you need legal cover, so here is
    > an additional "educational license" you (or anybody else who just wants
    > to see an example) may use:
    >
    > You are hereby granted permission to read, run, and modify the
    > sections of this code in which O_NONBLOCK is implemented. You may use
    > whatever you learn from this exercise in your own software, under
    > whatever license you wish. However, you may not cut from the example
    > code and paste it into nonGPL code.
    >
    > In other words, once you understand how I did it, you are free to
    > do it the same way in your own code, if that's what you want. But
    > you have to write your own code.
    >
    > Good enough?
    >
    > Regards,
    >
    > David Mathog



+ Reply to Thread