Creating child processes - Unix

This is a discussion on Creating child processes - Unix ; hi, i'm quite new to programming for various unix flavours, so excuse me if i ask something dumb. i have done some research on my own, but haven't come to a solution, so i hope someone here can help me ...

+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 22

Thread: Creating child processes

  1. Creating child processes

    hi,
    i'm quite new to programming for various unix flavours, so excuse me
    if i ask something dumb. i have done some research on my own, but
    haven't come to a solution, so i hope someone here can help me out

    i'm using C++ and, if possible, i'd like to stick to the C or C++
    standard libraries or some widely supported libraries and standards,
    such as POSIX, so i don't have too much trouble porting to other
    platforms or compilers.
    so, what i need to do is create child processes that are not
    necessarily the same executable as the parent process. for what i have
    found out, i need to fork another instance of the callee and use
    execvp or something similar. this is where i have my first question:
    fork copies nearly every resources of the callee, such as file
    descriptors, doesn't it? will these resources be closed automatically
    when i call a function of the exec* family? does the child process
    write to stdout and stderr and read from stdin just as the parent
    process? how do i wait for the child to complete and get its return
    code?

    somewhere else in my code, i also need to create a child process
    similar as above, but there are additional requirements: i need to set
    the environment of the child process and redirect the stdin, stdout
    and stderr streams in the child so i can read from/write to them from
    the parent. i guess i have to do something with pipes, but i haven't
    figured out how to do it exactly.


    thank you in advance

  2. Re: Creating child processes

    On Apr 3, 1:49 am, plee...@gmail.com wrote:

    > fork copies nearly every resources of the callee, such as file
    > descriptors, doesn't it?


    Yes.

    > will these resources be closed automatically
    > when i call a function of the exec* family?


    It depends on exactly what resources you are talking about and what
    you mean by "closed". The short answer is that almost all resources
    will except file descriptors that have the FD_CLOEXEC flag cleared.

    > does the child process
    > write to stdout and stderr and read from stdin just as the parent
    > process?


    Typically, these descriptors are set to remain open across an 'exec',
    so the new process will have the sane stdout, stderr, and stdin as the
    parent. However, you can certainly change them *after* the 'fork' and
    *before* the 'exec' so that the other executable has whatever stdout,
    stderr, and stdin you want it to have.

    > how do i wait for the child to complete and get its return
    > code?


    'man wait', 'man waitpid'

    > somewhere else in my code, i also need to create a child process
    > similar as above, but there are additional requirements: i need to set
    > the environment of the child process and redirect the stdin, stdout
    > and stderr streams in the child so i can read from/write to them from
    > the parent. i guess i have to do something with pipes, but i haven't
    > figured out how to do it exactly.


    After the 'fork' but before the 'exec' you can manipulate the
    environment, change stdin, stdout, and stderr, and otherwise monkey
    around with the context the new process gets. You can replace the
    standard I/O descriptors with pipes if you like and then talk to the
    child process from the parent using the other ends of those pipes.

    DS

  3. Re: Creating child processes

    pleexed@gmail.com wrote in news:757bf237-28f3-4411-954c-
    bca64942194a@z38g2000hsc.googlegroups.com:

    > hi,
    > i'm quite new to programming for various unix flavours, so excuse me
    > if i ask something dumb. i have done some research on my own, but
    > haven't come to a solution, so i hope someone here can help me out




    What UNIX text are your reading that doesn't cover fork() and exec()?

  4. Re: Creating child processes

    In article ,
    Lloyd Bonafide wrote:

    > pleexed@gmail.com wrote in news:757bf237-28f3-4411-954c-
    > bca64942194a@z38g2000hsc.googlegroups.com:
    >
    > > hi,
    > > i'm quite new to programming for various unix flavours, so excuse me
    > > if i ask something dumb. i have done some research on my own, but
    > > haven't come to a solution, so i hope someone here can help me out

    >
    >
    >
    > What UNIX text are your reading that doesn't cover fork() and exec()?


    Or putting it another way, if you want to learn how to do this kind of
    programming, get the book "Advanced Programming in the Unix Environment"
    or something similar.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  5. Re: Creating child processes

    first of all, thanks for the help.

    i have been reading the manpages only, but i understood that a proper
    book might be the better the way to go, as Lloyd Bonafide said. so i
    got me the unix book Barry Margolin recommended and read the parts
    relevant to my problem (the rest is going to follow), and i find
    everything quite logical now.

    however, i haven't found any way to make sure all file descriptors are
    closed when i exec. i'd be happy both with a function that closes
    every file descriptor, with one that specifies that files should be
    opened with the FD_CLOEXEC flag set by default or with any other
    solution. is there a practicable thing i can do about it?

    thank you in advance

  6. Re: Creating child processes

    In article
    ,
    pleexed@gmail.com wrote:

    > however, i haven't found any way to make sure all file descriptors are
    > closed when i exec. i'd be happy both with a function that closes
    > every file descriptor, with one that specifies that files should be
    > opened with the FD_CLOEXEC flag set by default or with any other
    > solution. is there a practicable thing i can do about it?


    The only way to do this is to call close() yourself on all the FD's.

    What problem are you trying to solve? In many cases, there's no problem
    with FD's being inherited into the exec'ed program, as it will just
    ignore them. In cases where you need to ensure that an FD doesn't get
    inherited (e.g. sockets, where the network connection doesn't actually
    close until the last FD is closed), set the FD_CLOEXEC flag.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  7. Re: Creating child processes


    > > however, i haven't found any way to make sure all file descriptors are
    > > closed when i exec. i'd be happy both with a function that closes
    > > every file descriptor, with one that specifies that files should be
    > > opened with the FD_CLOEXEC flag set by default or with any other
    > > solution. is there a practicable thing i can do about it?

    >
    > The only way to do this is to call close() yourself on all the FD's.
    >
    > What problem are you trying to solve? *In many cases, there's no problem
    > with FD's being inherited into the exec'ed program, as it will just
    > ignore them. *In cases where you need to ensure that an FD doesn't get
    > inherited (e.g. sockets, where the network connection doesn't actually
    > close until the last FD is closed), set the FD_CLOEXEC flag.


    well, i'm porting a number of libraries of mine from windows. i'm well
    aware that there are libraries out there that do the tasks mine do,
    and do them lot better, but since i'm neither ignoring these libraries
    nor producing some kind of code that is of importance, that shouldn't
    be a problem. what i do is for educational purposes only. therefore,
    the "problems" i have may be no problems in real world situations at
    all and i might sound rather pedantic. excuse me if this is the
    case

    whatever, so one of these libraries is centered around processes and
    threads. one function is designed to spawn a new process that should
    be decoupled from its parent, that is, sharing nothing at all. so, the
    open FDs are kind of disturbing that idea. this might not actually be
    a problem in most cases, but i imagine that there could be, at least
    in theory, nasty effects with open FDs. for example if the parent
    opens a file, spawns a long-running process and closes its handle
    afterwards, the file will be open as long as the child process runs,
    won't it? that would mean that the file could not be delted (iirc, it
    can be unlinked, but it won't actually be removed from the filesystem)
    until that child process terminates.

    furthermore (please correct me if i'm wrong), FDs are property of a
    process, not a thread. so, when i fork() in a multithreaded
    environment, the calling thread gets copied and all of its FDs get
    copied, too. so, even if i ensure that each call that opens a FD that
    gets inherited by default is followed by a call setting the FD_CLOEXEX
    flag, the following situation could arise in theory:
    Thread 1 calls open(), socket() or something similar, gets a new FD
    Thread 2 interrupts, calls fork(), the new process inherits the FD
    from the previous operation in T1
    Thread 2 calls exec*(), the child process still has that handle open
    Thread 1 resumes, calls fcntl() for the new FD and sets FD_CLOEXEC,
    which however has no effect on the recently started child process

    thank you for your answers so far

  8. Re: Creating child processes

    >> > however, i haven't found any way to make sure all file descriptors are
    >> > closed when i exec. i'd be happy both with a function that closes
    >> > every file descriptor, with one that specifies that files should be
    >> > opened with the FD_CLOEXEC flag set by default or with any other
    >> > solution. is there a practicable thing i can do about it?

    >>
    >> The only way to do this is to call close() yourself on all the FD's.
    >>
    >> What problem are you trying to solve? *In many cases, there's no problem
    >> with FD's being inherited into the exec'ed program, as it will just
    >> ignore them.


    Until it tries to open "too many" file descriptors and fails.

    >> *In cases where you need to ensure that an FD doesn't get
    >> inherited (e.g. sockets, where the network connection doesn't actually
    >> close until the last FD is closed), set the FD_CLOEXEC flag.


    >whatever, so one of these libraries is centered around processes and
    >threads. one function is designed to spawn a new process that should
    >be decoupled from its parent, that is, sharing nothing at all. so, the
    >open FDs are kind of disturbing that idea. this might not actually be
    >a problem in most cases, but i imagine that there could be, at least
    >in theory, nasty effects with open FDs. for example if the parent
    >opens a file, spawns a long-running process and closes its handle
    >afterwards, the file will be open as long as the child process runs,


    There are nastier effects. For example, if there is an "extra" fd
    for the write end of a pipe, the reader won't see EOF until it's
    closed. If there is an "extra" fd for the read end of a pipe, a
    writer won't get SIGPIPE. An extra open socket might prevent binding
    a port address already in use in a restarted copy of the program.
    A "rewind on close" tape device won't rewind if there's an extra fd
    open for it (used tape lately?).


    >won't it? that would mean that the file could not be delted (iirc, it
    >can be unlinked, but it won't actually be removed from the filesystem)
    >until that child process terminates.


    >furthermore (please correct me if i'm wrong), FDs are property of a
    >process, not a thread. so, when i fork() in a multithreaded
    >environment, the calling thread gets copied and all of its FDs get
    >copied, too. so, even if i ensure that each call that opens a FD that
    >gets inherited by default is followed by a call setting the FD_CLOEXEX
    >flag, the following situation could arise in theory:
    >Thread 1 calls open(), socket() or something similar, gets a new FD
    >Thread 2 interrupts, calls fork(), the new process inherits the FD
    >from the previous operation in T1
    >Thread 2 calls exec*(), the child process still has that handle open
    >Thread 1 resumes, calls fcntl() for the new FD and sets FD_CLOEXEC,
    >which however has no effect on the recently started child process


    One way to deal with this, without FD_CLOEXEC everywhere, is between
    the fork() and exec() calls. Close everything you don't need. A
    reasonable guess might be to close(3) and continuing up until 20
    consecutive close()s fail. The assumption here is that you won't
    have large gaps between file descriptors. This might fail if you've
    got a routine that normally opens 40 file descriptors, opens another
    one, then closes the 40, and keeps the other one open. Otherwise
    you keep going until the maximum file descriptor, if you've got a
    way to find that.

  9. Re: Creating child processes

    On Apr 6, 2:49 pm, gordonb.4i...@burditt.org (Gordon Burditt) wrote:

    > One way to deal with this, without FD_CLOEXEC everywhere, is between
    > the fork() and exec() calls. Close everything you don't need. A
    > reasonable guess might be to close(3) and continuing up until 20
    > consecutive close()s fail. The assumption here is that you won't
    > have large gaps between file descriptors. This might fail if you've
    > got a routine that normally opens 40 file descriptors, opens another
    > one, then closes the 40, and keeps the other one open. Otherwise
    > you keep going until the maximum file descriptor, if you've got a
    > way to find that.


    Use 'getdtablesize'. You may want to skip descriptors zero, one, and
    two. If you don't want to share those, you should probably open '/dev/
    null' onto them.

    DS

  10. Re: Creating child processes

    David Schwartz wrote:
    > Use 'getdtablesize'. You may want to skip descriptors zero, one, and
    > two. If you don't want to share those, you should probably open '/dev/
    > null' onto them.


    FWIW I could swear that some Unix version/s (Solaris?) have a function
    whose purpose is to close all open file descriptors in one fell swoop.
    But I can't find it now.

    RM

  11. Re: Creating child processes

    > >won't it? that would mean that the file could not be delted (iirc, it
    > >can be unlinked, but it won't actually be removed from the filesystem)
    > >until that child process terminates.
    > >furthermore (please correct me if i'm wrong), FDs are property of a
    > >process, not a thread. so, when i fork() in a multithreaded
    > >environment, the calling thread gets copied and all of its FDs get
    > >copied, too. so, even if i ensure that each call that opens a FD that
    > >gets inherited by default is followed by a call setting the FD_CLOEXEX
    > >flag, the following situation could arise in theory:
    > >Thread 1 calls open(), socket() or something similar, gets a new FD
    > >Thread 2 interrupts, calls fork(), the new process inherits the FD
    > >from the previous operation in T1
    > >Thread 2 calls exec*(), the child process still has that handle open
    > >Thread 1 resumes, calls fcntl() for the new FD and sets FD_CLOEXEC,
    > >which however has no effect on the recently started child process

    >
    > One way to deal with this, without FD_CLOEXEC everywhere, is between
    > the fork() and exec() calls. *Close everything you don't need. *A
    > reasonable guess might be to close(3) and continuing up until 20
    > consecutive close()s fail. *The assumption here is that you won't
    > have large gaps between file descriptors. *This might fail if you've
    > got a routine that normally opens 40 file descriptors, opens another
    > one, then closes the 40, and keeps the other one open. *Otherwise
    > you keep going until the maximum file descriptor, if you've got a
    > way to find that.


    my unix book is not too clear about it (the first edition of the
    chapter in question is available online by chance:
    http://www.informit.com/articles/art...9706&seqNum=15 ), but
    i guess the /dev/fd directory is different for each process, that is,
    is showing each process which file descriptors are open. although this
    directory is nonstandard, the second edition of the book says that it
    is supported by at least FreeBSD 5.2.1, Linux 2.4.22, Mac OS X 10.3
    and Solaris 9; i think that is enough to claim some portability
    so, if my assumptions are right, i could list the contents of /dev/fd
    between fork() and exec(), and close every item listes except for 0, 1
    and 2. would that be a reasonable solution?

    > Use 'getdtablesize'. You may want to skip descriptors zero, one, and
    > two. If you don't want to share those, you should probably open '/dev/
    > null' onto them.

    i have checked the manpage, but i don't quite get how it could help
    me. could you explain what you mean in more detail?

  12. Re: Creating child processes

    In article
    ,
    pleexed@gmail.com wrote:

    > > Use 'getdtablesize'. You may want to skip descriptors zero, one, and
    > > two. If you don't want to share those, you should probably open '/dev/
    > > null' onto them.

    > i have checked the manpage, but i don't quite get how it could help
    > me. could you explain what you mean in more detail?


    maxfd = getdtablesize();
    for (fd = 3; fd<=maxfd; fd++)
    close(fd);

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  13. Re: Creating child processes

    Barry Margolin wrote:

    >> > Use 'getdtablesize'. You may want to skip descriptors zero, one, and
    >> > two. If you don't want to share those, you should probably open '/dev/
    >> > null' onto them.

    >> i have checked the manpage, but i don't quite get how it could help
    >> me. could you explain what you mean in more detail?

    >
    > maxfd = getdtablesize();
    > for (fd = 3; fd<=maxfd; fd++)
    > close(fd);


    The portable solution would be using sysconf(_SC_OPEN_MAX), instead of
    getdtablesize()

    Regards.


  14. Re: Creating child processes

    Jaap Bruinsma writes:
    > Barry Margolin wrote:
    >
    >>> > Use 'getdtablesize'. You may want to skip descriptors zero, one, and
    >>> > two. If you don't want to share those, you should probably open '/dev/
    >>> > null' onto them.
    >>> i have checked the manpage, but i don't quite get how it could help
    >>> me. could you explain what you mean in more detail?

    >>
    >> maxfd = getdtablesize();
    >> for (fd = 3; fd<=maxfd; fd++)
    >> close(fd);

    >
    > The portable solution would be using sysconf(_SC_OPEN_MAX), instead of
    > getdtablesize()


    This isn't necessarily true: sysconf(_SC_OPEN_MAX) may indicate that
    there is no such limit and/ or may return a number smaller than the
    highest-numbered currenlty open file descriptor if an rlimit has been
    set after that descriptor was created (reportedly true for Solaris).

  15. Re: Creating child processes

    On Apr 9, 10:07 am, Rainer Weikusat wrote:

    > This isn't necessarily true: sysconf(_SC_OPEN_MAX) may indicate that
    > there is no such limit and/ or may return a number smaller than the
    > highest-numbered currenlty open file descriptor if an rlimit has been
    > set after that descriptor was created (reportedly true for Solaris).


    On Linux, such a descriptor is permanently inaccessible and any
    attempt to use it returns 'EBADF'. The descriptor remains open though.

    DS

  16. Re: Creating child processes

    Rainer Weikusat wrote:

    > Jaap Bruinsma writes:
    >> Barry Margolin wrote:
    >>
    >>> maxfd = getdtablesize();
    >>> for (fd = 3; fd<=maxfd; fd++)
    >>> close(fd);

    >>
    >> The portable solution would be using sysconf(_SC_OPEN_MAX), instead of
    >> getdtablesize()

    >
    > This isn't necessarily true: sysconf(_SC_OPEN_MAX) may indicate that
    > there is no such limit


    On a system where sysconf(_SC_OPEN_MAX) indicates that there is no
    limit, what do you expect getdtablesize() to do (if it exists)?
    It would either have to return -1 to indicate no limit, or return
    INT_MAX. Neither is any more helpful than what sysconf() does.

    > or may return a number smaller than the
    > highest-numbered currenlty open file descriptor if an rlimit has been
    > set after that descriptor was created (reportedly true for Solaris).


    So can getdtablesize() - on Solaris it returns "the current soft
    limit as if obtained from a call to getrlimit() with the RLIMIT_NOFILE
    option". (If the limit is RLIM_INFINITY it returns INT_MAX.)

    --
    Geoff Clare

  17. Re: Creating child processes

    Geoff Clare writes:
    > Rainer Weikusat wrote:
    >> Jaap Bruinsma writes:
    >>> Barry Margolin wrote:
    >>>
    >>>> maxfd = getdtablesize();
    >>>> for (fd = 3; fd<=maxfd; fd++)
    >>>> close(fd);
    >>>
    >>> The portable solution would be using sysconf(_SC_OPEN_MAX), instead of
    >>> getdtablesize()

    >>
    >> This isn't necessarily true: sysconf(_SC_OPEN_MAX) may indicate that
    >> there is no such limit

    >
    > On a system where sysconf(_SC_OPEN_MAX) indicates that there is no
    > limit, what do you expect getdtablesize() to do (if it exists)?
    > It would either have to return -1 to indicate no limit, or return
    > INT_MAX. Neither is any more helpful than what sysconf() does.


    So, how does "another subroutine call which doesn't help to solve the
    problem, either" make the first subroutine call, which already didn't
    help, a more portable way of solving the problem?

    >> or may return a number smaller than the highest-numbered currenlty
    >> open file descriptor if an rlimit has been set after that
    >> descriptor was created (reportedly true for Solaris).

    >
    > So can getdtablesize() - on Solaris it returns "the current soft
    > limit as if obtained from a call to getrlimit() with the RLIMIT_NOFILE
    > option". (If the limit is RLIM_INFINITY it returns INT_MAX.)


    See above.

  18. Re: Creating child processes

    Rainer Weikusat writes:
    > Geoff Clare writes:
    >> Rainer Weikusat wrote:
    >>> Jaap Bruinsma writes:
    >>>> Barry Margolin wrote:
    >>>>
    >>>>> maxfd = getdtablesize();
    >>>>> for (fd = 3; fd<=maxfd; fd++)
    >>>>> close(fd);
    >>>>
    >>>> The portable solution would be using sysconf(_SC_OPEN_MAX), instead of
    >>>> getdtablesize()
    >>>
    >>> This isn't necessarily true: sysconf(_SC_OPEN_MAX) may indicate that
    >>> there is no such limit

    >>
    >> On a system where sysconf(_SC_OPEN_MAX) indicates that there is no
    >> limit, what do you expect getdtablesize() to do (if it exists)?
    >> It would either have to return -1 to indicate no limit, or return
    >> INT_MAX. Neither is any more helpful than what sysconf() does.

    >
    > So, how does "another subroutine call which doesn't help to solve the
    > problem, either" make the first subroutine call, which already didn't
    > help, a more portable way of solving the problem?


    Additionally, how does 'sysconf' returning a value not useful to solve
    the problem on a system without getdtable size make the former a more
    portable way to solve the problem?

  19. Re: Creating child processes

    On Apr 10, 7:15 am, Rainer Weikusat wrote:

    > So, how does "another subroutine call which doesn't help to solve the
    > problem, either" make the first subroutine call, which already didn't
    > help, a more portable way of solving the problem?


    If the second option works in almost every case where the first one
    does and a few cases where it doesn't, and is also part of a better-
    known standard, then it is a more portable way of solving the problem.

    The 'getdtablesize' is only part of informal standards. The 'sysconf'
    function is part of POSIX.1. It is almost axiomatic that it is more
    portable.

    DS

  20. Re: Creating child processes

    David Schwartz writes:
    >On Apr 10, 7:15 am, Rainer Weikusat wrote:
    >
    >> So, how does "another subroutine call which doesn't help to solve the
    >> problem, either" make the first subroutine call, which already didn't
    >> help, a more portable way of solving the problem?

    >
    >If the second option works in almost every case where the first one
    >does and a few cases where it doesn't, and is also part of a better-
    >known standard, then it is a more portable way of solving the problem.
    >
    >The 'getdtablesize' is only part of informal standards. The 'sysconf'
    >function is part of POSIX.1. It is almost axiomatic that it is more
    >portable.
    >
    >DS


    I've always prefered 'getrlimit' for this as being most portable over
    time (SVR4 systems didn't have the BSD-derived getdtablesize(), for those that
    did, it was implemented in the BSD compatability library which had broken
    signal semantics). On at least two UNIX-based systems that I worked on, we added
    a closeall() system call due to the overhead of calling close 2k times[*]
    before every exec in the bourne shell. Linux system call overhead isn't as
    bad as the traditional unix, but a closeall() equiv would still
    perform better than 2k close() calls.

    scott
    [*] Several common applications required RLIMIT_NOFILE > 2048.

+ Reply to Thread
Page 1 of 2 1 2 LastLast