weird fork problem - Unix

This is a discussion on weird fork problem - Unix ; Hi all, I am having a strange issue with a fork call. After some runs, I cannot call fork from my program. This is my code: pid_t pID=-1; while (pID { cout pID = fork (); if (pID { cerr ...

+ Reply to Thread
Results 1 to 10 of 10

Thread: weird fork problem

  1. weird fork problem

    Hi all,

    I am having a strange issue with a fork call. After some runs, I
    cannot call fork from my program. This is my code:

    pid_t pID=-1;
    while (pID<0)
    {
    cout << "trying to fork ... " << endl;
    pID = fork ();
    if (pID < 0) // failed to fork
    {
    cerr << "errno: " << strerror(errno) << endl;
    cerr << "Failed to fork" << endl;
    sleep(5);
    }
    }

    After a number of runs that varies between 20 and 30, I have the
    following error output:

    trying to fork ...
    errno: Cannot allocate memory
    Failed to fork
    trying to fork ...
    errno: Cannot allocate memory
    Failed to fork

    And so on. But a look to free gives enough memory:

    $ free -m
    total used free shared
    buffers cached
    Mem: 1011 656 355 0 30
    353
    -/+ buffers/cache: 271 739
    Swap: 1176 0 1176

    I have about 355 MB of physical and more than 1GB of swap, which
    should be enough.
    I also had a look to the number of processes running:
    $ ps aux | wc -l
    111

    And the system limitations:
    $ ulimit -a
    core file size (blocks, -c) 0
    data seg size (kbytes, -d) unlimited
    max nice (-e) 0
    file size (blocks, -f) unlimited
    pending signals (-i) 8183
    max locked memory (kbytes, -l) 32
    max memory size (kbytes, -m) unlimited
    open files (-n) 1024
    pipe size (512 bytes, -p) 8
    POSIX message queues (bytes, -q) 819200
    max rt priority (-r) 0
    stack size (kbytes, -s) 8192
    cpu time (seconds, -t) unlimited
    max user processes (-u) 8183
    virtual memory (kbytes, -v) unlimited
    file locks (-x) unlimited


    So I have no idea where the problem is ... any help please?

    Thanks


  2. Re: weird fork problem

    On Oct 28, 2:22 pm, bel...@gmail.com wrote:
    > Hi all,
    >
    > I am having a strange issue with a fork call. After some runs, I
    > cannot call fork from my program. This is my code:
    >
    > pid_t pID=-1;
    > while (pID<0)
    > {
    > cout << "trying to fork ... " << endl;
    > pID = fork ();
    > if (pID < 0) // failed to fork
    > {
    > cerr << "errno: " << strerror(errno) << endl;
    > cerr << "Failed to fork" << endl;
    > sleep(5);
    > }
    > }
    >
    > After a number of runs that varies between 20 and 30, I have the
    > following error output:
    >
    > trying to fork ...
    > errno: Cannot allocate memory
    > Failed to fork
    > trying to fork ...
    > errno: Cannot allocate memory
    > Failed to fork


    "Cannot allocate memory" is presumably ENOMEM. It's kind of a
    catchall. Some systems will fail with ENOMEM if some other resource
    is exhausted; and calling fork() duplicates a lot of resources. File
    descriptors, memory mappings, etc.

    What system is this on? Someone might know in what specific
    situations it returns ENOMEM. Also, does the process do anything
    noteworthy that might consume a lot of some resource?


  3. Re: weird fork problem

    > What system is this on? Someone might know in what specific
    > situations it returns ENOMEM. Also, does the process do anything
    > noteworthy that might consume a lot of some resource?

    I see no reason for consuming a lot of resources. After doing the
    fork, the child launches a external program using execlp(). The father
    monitors this program, and after a timeout it kills it:

    kill( pID, SIGKILL); // Kill child process
    int childExitStatus;
    waitpid (pID, &childExitStatus, 0);

    And then just repeat.






  4. Re: weird fork problem

    On Oct 29, 2:22 am, bel...@gmail.com wrote:
    > Hi all,
    >
    > I am having a strange issue with a fork call. After some runs, I
    > cannot call fork from my program. This is my code:
    >
    > pid_t pID=-1;
    > while (pID<0)
    > {
    > cout << "trying to fork ... " << endl;
    > pID = fork ();
    > if (pID < 0) // failed to fork
    > {
    > cerr << "errno: " << strerror(errno) << endl;
    > cerr << "Failed to fork" << endl;
    > sleep(5);
    > }
    > }
    >
    > After a number of runs that varies between 20 and 30, I have the
    > following error output:
    >
    > trying to fork ...
    > errno: Cannot allocate memory
    > Failed to fork
    > trying to fork ...
    > errno: Cannot allocate memory
    > Failed to fork
    >
    > And so on. But a look to free gives enough memory:
    >
    > $ free -m
    > total used free shared
    > buffers cached
    > Mem: 1011 656 355 0 30
    > 353
    > -/+ buffers/cache: 271 739
    > Swap: 1176 0 1176
    >
    > I have about 355 MB of physical and more than 1GB of swap, which
    > should be enough.
    > I also had a look to the number of processes running:
    > $ ps aux | wc -l
    > 111
    >
    > And the system limitations:
    > $ ulimit -a
    > core file size (blocks, -c) 0
    > data seg size (kbytes, -d) unlimited
    > max nice (-e) 0
    > file size (blocks, -f) unlimited
    > pending signals (-i) 8183
    > max locked memory (kbytes, -l) 32
    > max memory size (kbytes, -m) unlimited
    > open files (-n) 1024
    > pipe size (512 bytes, -p) 8
    > POSIX message queues (bytes, -q) 819200
    > max rt priority (-r) 0
    > stack size (kbytes, -s) 8192
    > cpu time (seconds, -t) unlimited
    > max user processes (-u) 8183
    > virtual memory (kbytes, -v) unlimited
    > file locks (-x) unlimited
    >
    > So I have no idea where the problem is ... any help please?
    >
    > Thanks


    I understand that you are running your program as a non-root user?
    If that assumption is right, then does the number of child processes
    that the parent process can spawn change if you are logged in as root
    (or if the parent process is running on behalf of root)?
    If the answer to that is yes, then as is mentioned in other comments
    to this messae, you are probably reaching some resource limit while
    running the parent process as non-root.
    Try changing open files limit etc.
    HTH.


  5. Re: weird fork problem

    Finally it resulted to be a memory leak ... but it was funny because
    neither the "free" command or "top" did not report excesive memory
    usage.

    Searching in ulimit, the only limitation that I may be breaking is the
    stack size, which is limited to 8192 kB.

    On 29 oct, 08:16, Amol Vaikar wrote:
    > On Oct 29, 2:22 am, bel...@gmail.com wrote:
    >
    >
    >
    > > Hi all,

    >
    > > I am having a strange issue with a fork call. After some runs, I
    > > cannot call fork from my program. This is my code:

    >
    > > pid_t pID=-1;
    > > while (pID<0)
    > > {
    > > cout << "trying to fork ... " << endl;
    > > pID = fork ();
    > > if (pID < 0) // failed to fork
    > > {
    > > cerr << "errno: " << strerror(errno) << endl;
    > > cerr << "Failed to fork" << endl;
    > > sleep(5);
    > > }
    > > }

    >
    > > After a number of runs that varies between 20 and 30, I have the
    > > following error output:

    >
    > > trying to fork ...
    > > errno: Cannot allocate memory
    > > Failed to fork
    > > trying to fork ...
    > > errno: Cannot allocate memory
    > > Failed to fork

    >
    > > And so on. But a look to free gives enough memory:

    >
    > > $ free -m
    > > total used free shared
    > > buffers cached
    > > Mem: 1011 656 355 0 30
    > > 353
    > > -/+ buffers/cache: 271 739
    > > Swap: 1176 0 1176

    >
    > > I have about 355 MB of physical and more than 1GB of swap, which
    > > should be enough.
    > > I also had a look to the number of processes running:
    > > $ ps aux | wc -l
    > > 111

    >
    > > And the system limitations:
    > > $ ulimit -a
    > > core file size (blocks, -c) 0
    > > data seg size (kbytes, -d) unlimited
    > > max nice (-e) 0
    > > file size (blocks, -f) unlimited
    > > pending signals (-i) 8183
    > > max locked memory (kbytes, -l) 32
    > > max memory size (kbytes, -m) unlimited
    > > open files (-n) 1024
    > > pipe size (512 bytes, -p) 8
    > > POSIX message queues (bytes, -q) 819200
    > > max rt priority (-r) 0
    > > stack size (kbytes, -s) 8192
    > > cpu time (seconds, -t) unlimited
    > > max user processes (-u) 8183
    > > virtual memory (kbytes, -v) unlimited
    > > file locks (-x) unlimited

    >
    > > So I have no idea where the problem is ... any help please?

    >
    > > Thanks

    >
    > I understand that you are running your program as a non-root user?
    > If that assumption is right, then does the number of child processes
    > that the parent process can spawn change if you are logged in as root
    > (or if the parent process is running on behalf of root)?
    > If the answer to that is yes, then as is mentioned in other comments
    > to this messae, you are probably reaching some resource limit while
    > running the parent process as non-root.
    > Try changing open files limit etc.
    > HTH.




  6. Re: weird fork problem

    On Oct 31, 8:01 am, bel...@gmail.com wrote:
    > Finally it resulted to be a memory leak ... but it was funny because
    > neither the "free" command or "top" did not report excesive memory
    > usage.
    >
    > Searching in ulimit, the only limitation that I may be breaking is the
    > stack size, which is limited to 8192 kB.
    >
    > On 29 oct, 08:16, Amol Vaikar wrote:
    >
    > > On Oct 29, 2:22 am, bel...@gmail.com wrote:

    >
    > > > Hi all,

    >
    > > > I am having a strange issue with a fork call. After some runs, I
    > > > cannot call fork from my program. This is my code:

    >
    > > > pid_t pID=-1;
    > > > while (pID<0)
    > > > {
    > > > cout << "trying to fork ... " << endl;
    > > > pID = fork ();
    > > > if (pID < 0) // failed to fork
    > > > {
    > > > cerr << "errno: " << strerror(errno) << endl;
    > > > cerr << "Failed to fork" << endl;
    > > > sleep(5);
    > > > }
    > > > }

    >
    > > > After a number of runs that varies between 20 and 30, I have the
    > > > following error output:

    >
    > > > trying to fork ...
    > > > errno: Cannot allocate memory
    > > > Failed to fork
    > > > trying to fork ...
    > > > errno: Cannot allocate memory
    > > > Failed to fork

    >
    > > > And so on. But a look to free gives enough memory:

    >
    > > > $ free -m
    > > > total used free shared
    > > > buffers cached
    > > > Mem: 1011 656 355 0 30
    > > > 353
    > > > -/+ buffers/cache: 271 739
    > > > Swap: 1176 0 1176

    >
    > > > I have about 355 MB of physical and more than 1GB of swap, which
    > > > should be enough.
    > > > I also had a look to the number of processes running:
    > > > $ ps aux | wc -l
    > > > 111

    >
    > > > And the system limitations:
    > > > $ ulimit -a
    > > > core file size (blocks, -c) 0
    > > > data seg size (kbytes, -d) unlimited
    > > > max nice (-e) 0
    > > > file size (blocks, -f) unlimited
    > > > pending signals (-i) 8183
    > > > max locked memory (kbytes, -l) 32
    > > > max memory size (kbytes, -m) unlimited
    > > > open files (-n) 1024
    > > > pipe size (512 bytes, -p) 8
    > > > POSIX message queues (bytes, -q) 819200
    > > > max rt priority (-r) 0
    > > > stack size (kbytes, -s) 8192
    > > > cpu time (seconds, -t) unlimited
    > > > max user processes (-u) 8183
    > > > virtual memory (kbytes, -v) unlimited
    > > > file locks (-x) unlimited

    >
    > > > So I have no idea where the problem is ... any help please?

    >
    > > > Thanks

    >
    > > I understand that you are running your program as a non-root user?
    > > If that assumption is right, then does the number of child processes
    > > that the parent process can spawn change if you are logged in as root
    > > (or if the parent process is running on behalf of root)?
    > > If the answer to that is yes, then as is mentioned in other comments
    > > to this messae, you are probably reaching some resource limit while
    > > running the parent process as non-root.
    > > Try changing open files limit etc.
    > > HTH.


    A memory leak where? Concerning the fork() function or elsewhere in
    your program?
    It is a sad fact that malloc will almost never return NULL as a sign
    there's no memory left,
    nor will C++'s new operator, but instead they usually just block the
    whole system, slow
    it down or kill the process. Or is it just from my experience?


  7. Re: weird fork problem

    On Oct 28, 5:22 pm, bel...@gmail.com wrote:
    > Hi all,
    >
    > I am having a strange issue with a fork call. After some runs, I
    > cannot call fork from my program. This is my code:
    >
    > pid_t pID=-1;
    > while (pID<0)
    > {
    > cout << "trying to fork ... " << endl;
    > pID = fork ();
    > if (pID < 0) // failed to fork
    > {
    > cerr << "errno: " << strerror(errno) << endl;
    > cerr << "Failed to fork" << endl;
    > sleep(5);
    > }
    > }
    >
    > After a number of runs that varies between 20 and 30, I have the
    > following error output:
    >
    > trying to fork ...
    > errno: Cannot allocate memory
    > Failed to fork
    > trying to fork ...
    > errno: Cannot allocate memory
    > Failed to fork
    >
    > And so on. But a look to free gives enough memory:


    You've come close to reinventing the fork bomb, where you take up all
    the available process table entries with your forking program, and
    leave no entries for any other new processes. Fortunately, you have
    some system limits (see below) that prevent you from taking /all/ the
    entries.

    [snip irrelevant free stats]

    > And the system limitations:
    > $ ulimit -a

    [snip]
    > max user processes (-u) 8183


    OK, so there cannot be any more than 8183 processes running with your
    UID. That's the limit to your fork bomb.

    > virtual memory (kbytes, -v) unlimited
    > file locks (-x) unlimited
    >
    > So I have no idea where the problem is ... any help please?



    Take a look at the fork(2) documentation ("man 2 fork"), and read the
    ERRORS section. For Linux, fork(2) says

    ERRORS
    EAGAIN fork() cannot allocate sufficient memory

    EAGAIN It was not possible to create a new process

    ENOMEM fork() failed to allocate the necessary kernel
    structures because memory is tight.

    So, if you are running on Linux, you ran into a kernel structure out-
    of-memory condition before you ran into the ulimit process limit
    condition.

    To remedy the situation, either get more real memory for your kernel
    to use, or stop forking so many child processes.

    HTH
    --
    Lew



  8. Re: weird fork problem

    Lew Pitcher wrote:
    > On Oct 28, 5:22 pm, bel...@gmail.com wrote:
    >> Hi all,
    >>
    >> I am having a strange issue with a fork call. After some runs, I
    >> cannot call fork from my program. This is my code:
    >>
    >> pid_t pID=-1;
    >> while (pID<0)
    >> {
    >> cout << "trying to fork ... " << endl;
    >> pID = fork ();
    >> if (pID < 0) // failed to fork
    >> {
    >> cerr << "errno: " << strerror(errno) << endl;
    >> cerr << "Failed to fork" << endl;
    >> sleep(5);
    >> }
    >> }


    > You've come close to reinventing the fork bomb


    I thought so too until I read carefully. The condition on the
    while() loop is only true if the fork() fails. So this loop
    will only result in one *successful* call to fork(). It's not
    a fork() bomb; it's fork() with error handling and a retry when
    it fails (and until it succeeds). Interestingly, when it does
    succeed, both parent and child break out of the loop because
    the pid is nonnegative for both.

    Were I to write it, I would probably write something like this
    instead:

    pid_t pid;
    while (1) {
    pid = fork();
    if (pid != -1) {
    break;
    }

    printf("fork failed; will try again\n");
    sleep(5);
    }

    I believe that's equivalent, but to me it's easier to read.

    - Logan

  9. Re: weird fork problem

    Fork problems are always weird.

    --
    The most powerful Usenet tool you have ever heard of.
    NewsMaestro v. 4.0.6 - Dictionary Update/Expert Mode has been released.

    * Significant improvement in symbol substitution mechanism
    for verb tense and plurals.
    * Expert mode.
    * Miscellaneous improvements and bug fixes.
    * Templates generator improvements.
    * Multi-job support.

    Note: In some previous releases some class files were missing.
    As a result, the program would not run.
    Sorry for the inconvenience.

    Web page:
    http://newsmaestro.sourceforge.net/

    Download page:
    http://newsmaestro.sourceforge.net/D...nformation.htm

    Send any feedback, ideas, suggestions, test results to
    newsmaestroinfo \at/ mail.ru.

    Your personal info will not be released and your privacy
    will be honored.

  10. Re: weird fork problem

    On 2007-10-31, belion@gmail.com wrote:
    > Finally it resulted to be a memory leak ... but it was funny because
    > neither the "free" command or "top" did not report excesive memory
    > usage.

    [...]

    Were you looking at the right place? In "top" the VIRT column was supposed
    to give you increasing size of your virtual memory. In "free" you should
    have seen the increased amount of used swap space.


    --
    Minds, like parachutes, function best when open

+ Reply to Thread