setrlimit() extension suggestion - Linux

This is a discussion on setrlimit() extension suggestion - Linux ; I'd like to suggest an extension to setrlimit() and getrlimit(). The resource symbol would be RLIMIT_TIME (or RLIMIT_LINUX_TIME to designate an extension in case a future POSIX change uses RLIMIT_TIME in an incompatible way). What it should do is set ...

+ Reply to Thread
Results 1 to 13 of 13

Thread: setrlimit() extension suggestion

  1. setrlimit() extension suggestion

    I'd like to suggest an extension to setrlimit() and getrlimit(). The resource
    symbol would be RLIMIT_TIME (or RLIMIT_LINUX_TIME to designate an extension in
    case a future POSIX change uses RLIMIT_TIME in an incompatible way). What it
    should do is set a limit on "wallclock" time.

    Is that something the kernel could do? It can already do RLIMIT_CPU but that
    is not exactly the same.

    Unreliable means exist now. One of them is a parent process that keeps time
    and kills the child. But a child could "run away" fairly easily (fork and
    change process group). A hard RLIMIT_TIME would be inescapable.

    Discussion?

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  2. Re: setrlimit() extension suggestion

    On Apr 20, 10:43 pm, phil-news-nos...@ipal.net wrote:

    > I'd like to suggest an extension to setrlimit() and getrlimit(). The resource
    > symbol would be RLIMIT_TIME (or RLIMIT_LINUX_TIME to designate an extension in
    > case a future POSIX change uses RLIMIT_TIME in an incompatible way). What it
    > should do is set a limit on "wallclock" time.


    I can't think of a use for it.

    > Unreliable means exist now. One of them is a parent process that keeps time
    > and kills the child. But a child could "run away" fairly easily (fork and
    > change process group). A hard RLIMIT_TIME would be inescapable.


    I don't see how that would be useful. You would face an insoluable
    "three bears" problem. Minutes are too low for long-lived working
    processes, but seconds are too long for fork bombs.

    If that's the job, the proposed RLIMIT_TIME is the wrong tool.

    DS

  3. Re: setrlimit() extension suggestion

    On Mon, 21 Apr 2008 04:58:53 -0700 (PDT) David Schwartz wrote:
    | On Apr 20, 10:43 pm, phil-news-nos...@ipal.net wrote:
    |
    |> I'd like to suggest an extension to setrlimit() and getrlimit(). The resource
    |> symbol would be RLIMIT_TIME (or RLIMIT_LINUX_TIME to designate an extension in
    |> case a future POSIX change uses RLIMIT_TIME in an incompatible way). What it
    |> should do is set a limit on "wallclock" time.
    |
    | I can't think of a use for it.

    I'd like to suggest that all the features in Linux that you can't think of
    a use for be immediately removed.

    I had a use for it yesterday. I ended up using the "timelimit" program in
    the netpipes package.


    |> Unreliable means exist now. One of them is a parent process that keeps time
    |> and kills the child. But a child could "run away" fairly easily (fork and
    |> change process group). A hard RLIMIT_TIME would be inescapable.
    |
    | I don't see how that would be useful. You would face an insoluable
    | "three bears" problem. Minutes are too low for long-lived working
    | processes, but seconds are too long for fork bombs.

    Uh ... you can specify whatever time limit you want. Minutes and seconds are
    just a resolution. If 15 seconds is just right, then use that. If you think
    we need a finer resolution than 1 second, what is in your mind? Milliseconds?


    | If that's the job, the proposed RLIMIT_TIME is the wrong tool.

    What do you think is the right tool to make sure a process be run for no more
    than 15 seconds wallclock time (it doesn't use much CPU time)?

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  4. Re: setrlimit() extension suggestion

    On Apr 21, 11:36 am, phil-news-nos...@ipal.net wrote:
    > On Mon, 21 Apr 2008 04:58:53 -0700 (PDT) David Schwartz wrote:
    > | On Apr 20, 10:43 pm, phil-news-nos...@ipal.net wrote:


    > |> I'd like to suggest an extension to setrlimit() and getrlimit(). The resource
    > |> symbol would be RLIMIT_TIME (or RLIMIT_LINUX_TIME to designate an extension in
    > |> case a future POSIX change uses RLIMIT_TIME in an incompatible way). What it
    > |> should do is set a limit on "wallclock" time.
    > |
    > | I can't think of a use for it.


    > I'd like to suggest that all the features in Linux that you can't think of
    > a use for be immediately removed.


    The criteria for *removing* a feature are not the same as the criteria
    for *adding* a feature.

    > I had a use for it yesterday. I ended up using the "timelimit" program in
    > the netpipes package.


    Well then explain the use! If you're going to suggest that a feature
    be added, you should at least explain what it might be used for.

    > |> Unreliable means exist now. One of them is a parent process that keeps time
    > |> and kills the child. But a child could "run away" fairly easily (fork and
    > |> change process group). A hard RLIMIT_TIME would be inescapable.
    > |
    > | I don't see how that would be useful. You would face an insoluable
    > | "three bears" problem. Minutes are too low for long-lived working
    > | processes, but seconds are too long for fork bombs.


    > Uh ... you can specify whatever time limit you want. Minutes and seconds are
    > just a resolution. If 15 seconds is just right, then use that. If you think
    > we need a finer resolution than 1 second, what is in your mind? Milliseconds?


    I guess you missed my point. You are advocating a new feature. You
    claim it might be useful to prevent a child from running away. I'm
    saying it couldn't be used for that purpose. If you set the limit to a
    time measured in minutes, you can still allow the child to run away
    for a minute, which is way too much. If you set the limit to a time
    measured in seconds, you will damage long-running processes that don't
    hurt the system at all.

    > | If that's the job, the proposed RLIMIT_TIME is the wrong tool.


    > What do you think is the right tool to make sure a process be run for no more
    > than 15 seconds wallclock time (it doesn't use much CPU time)?


    I don't know. What's the actual problem? I understand that limiting
    the wall clock time is your proposed solution, but solution TO WHAT?
    Why don't you want it to run more than 15 seconds? (The solution to
    your problem as stated is probably to write a trivial 'container'
    program that launches a program in a new process group and kills that
    process group in 15 seconds. But that might be the wrong solution to
    your actual problem. I don't know.)

    DS

  5. Re: setrlimit() extension suggestion

    David Schwartz wrote:
    > [...]
    > I don't know. What's the actual problem? I understand that limiting
    > the wall clock time is your proposed solution, but solution TO WHAT?
    > Why don't you want it to run more than 15 seconds? (The solution to
    > your problem as stated is probably to write a trivial 'container'
    > program that launches a program in a new process group and kills that
    > process group in 15 seconds. But that might be the wrong solution to
    > your actual problem. I don't know.)


    How about: "There is an all-campus computer for the students to
    run their email and compile and run simple c++ programs. This is
    a small server machine, but there are hundreds of students using
    it. Some students test their MPI programs, when these deadlock,
    they leave tens of sleeping processes each. They do not utilize
    the CPU, but clutter outputs of 'top', 'ps' and the like with,
    in time, thousands of processes nobody needs. It would be good
    to automatically kill such deadlocked processes in, say, 4 hours
    rather than waiting for the next reboot."

    Admittedly, one could do that with a simple script.

    - J.

  6. Re: setrlimit() extension suggestion

    On Apr 24, 1:46 am, Jacek Dziedzic spa...@gmail.com> wrote:

    > How about: "There is an all-campus computer for the students to
    > run their email and compile and run simple c++ programs. This is
    > a small server machine, but there are hundreds of students using
    > it. Some students test their MPI programs, when these deadlock,
    > they leave tens of sleeping processes each. They do not utilize
    > the CPU, but clutter outputs of 'top', 'ps' and the like with,
    > in time, thousands of processes nobody needs. It would be good
    > to automatically kill such deadlocked processes in, say, 4 hours
    > rather than waiting for the next reboot."
    >
    > Admittedly, one could do that with a simple script.


    Yep, and such a script would be much more flexible and powerful. This
    seems like a solution in search of a problem.

    DS

  7. Re: setrlimit() extension suggestion

    On Tue, 22 Apr 2008 08:16:42 -0700 (PDT) David Schwartz wrote:
    | On Apr 21, 11:36 am, phil-news-nos...@ipal.net wrote:
    |> On Mon, 21 Apr 2008 04:58:53 -0700 (PDT) David Schwartz wrote:
    |> | On Apr 20, 10:43 pm, phil-news-nos...@ipal.net wrote:
    |
    |> |> I'd like to suggest an extension to setrlimit() and getrlimit(). The resource
    |> |> symbol would be RLIMIT_TIME (or RLIMIT_LINUX_TIME to designate an extension in
    |> |> case a future POSIX change uses RLIMIT_TIME in an incompatible way). What it
    |> |> should do is set a limit on "wallclock" time.
    |> |
    |> | I can't think of a use for it.
    |
    |> I'd like to suggest that all the features in Linux that you can't think of
    |> a use for be immediately removed.
    |
    | The criteria for *removing* a feature are not the same as the criteria
    | for *adding* a feature.
    |
    |> I had a use for it yesterday. I ended up using the "timelimit" program in
    |> the netpipes package.
    |
    | Well then explain the use! If you're going to suggest that a feature
    | be added, you should at least explain what it might be used for.

    I did. It would be used as a _reliable_ means to ensure a process not be
    run beyond a certain point in time.

    I've since thought of different way to do it, which would be broader and
    enable even more variety of uses. But some more details need to be worked
    in the thought design. I'll post that idea if/when I work up those details.


    |> |> Unreliable means exist now. One of them is a parent process that keeps time
    |> |> and kills the child. But a child could "run away" fairly easily (fork and
    |> |> change process group). A hard RLIMIT_TIME would be inescapable.
    |> |
    |> | I don't see how that would be useful. You would face an insoluable
    |> | "three bears" problem. Minutes are too low for long-lived working
    |> | processes, but seconds are too long for fork bombs.
    |
    |> Uh ... you can specify whatever time limit you want. Minutes and seconds are
    |> just a resolution. If 15 seconds is just right, then use that. If you think
    |> we need a finer resolution than 1 second, what is in your mind? Milliseconds?
    |
    | I guess you missed my point. You are advocating a new feature. You
    | claim it might be useful to prevent a child from running away. I'm
    | saying it couldn't be used for that purpose. If you set the limit to a
    | time measured in minutes, you can still allow the child to run away
    | for a minute, which is way too much. If you set the limit to a time
    | measured in seconds, you will damage long-running processes that don't
    | hurt the system at all.

    You are making a judgement about a minute of time. The judgement should be
    left up to whoever is making use of the facility. No one is required to use
    it. And that's why the facility has a time setting. It is NOT a proposal
    that specifies exactly one minute, no more no less.


    |> | If that's the job, the proposed RLIMIT_TIME is the wrong tool.
    |
    |> What do you think is the right tool to make sure a process be run for no more
    |> than 15 seconds wallclock time (it doesn't use much CPU time)?
    |
    | I don't know. What's the actual problem? I understand that limiting
    | the wall clock time is your proposed solution, but solution TO WHAT?
    | Why don't you want it to run more than 15 seconds? (The solution to
    | your problem as stated is probably to write a trivial 'container'
    | program that launches a program in a new process group and kills that
    | process group in 15 seconds. But that might be the wrong solution to
    | your actual problem. I don't know.)

    There have been many times I've needed to limit runaway processes. This
    is really an application issue and I do not believe it is appropriate for
    discussion in this system oriented group.

    I have been entrapped by requests like this before, where people then in turn
    try to come up with many ways to do something similar, but is never the same
    thing. Those suggestions also tend to be application specific and not a general
    reusable solution. Even if I was willing to describe one application, I do not
    want to keep repeating this over and over with each other application. So I
    have limited what I propose to being a general purpose facility that can be
    used by any number of applications that have a need for what it does.

    If you can understand the effect as described by the suggestion, maybe you can
    suggest a way to do it that does not involve a kernel change. I won't think
    less of you if you cannot, because I cannot think of such a way.

    If you cannot understand the effect as described, then I don't see any purpose
    in continuing the discussion with you. But I highly doubt this is the issue.
    I suspect it is more likely you do work more at the system level and less at
    the application level and have simply never encounted the need because of the
    lesser exposure. You and I frequently do not see things the same way, and
    this might be at least one of the reasons why.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  8. Re: setrlimit() extension suggestion

    On Thu, 24 Apr 2008 10:46:27 +0200 Jacek Dziedzic wrote:
    | David Schwartz wrote:
    | > [...]
    |> I don't know. What's the actual problem? I understand that limiting
    |> the wall clock time is your proposed solution, but solution TO WHAT?
    |> Why don't you want it to run more than 15 seconds? (The solution to
    |> your problem as stated is probably to write a trivial 'container'
    |> program that launches a program in a new process group and kills that
    |> process group in 15 seconds. But that might be the wrong solution to
    |> your actual problem. I don't know.)
    |
    | How about: "There is an all-campus computer for the students to
    | run their email and compile and run simple c++ programs. This is
    | a small server machine, but there are hundreds of students using
    | it. Some students test their MPI programs, when these deadlock,
    | they leave tens of sleeping processes each. They do not utilize
    | the CPU, but clutter outputs of 'top', 'ps' and the like with,
    | in time, thousands of processes nobody needs. It would be good
    | to automatically kill such deadlocked processes in, say, 4 hours
    | rather than waiting for the next reboot."
    |
    | Admittedly, one could do that with a simple script.

    These would generally be independent processes, and with no need for a means
    to kill them off based on wallclock time. In their case, merely idleness
    time might be appropriate. But the accuracy of the time is far from critical
    here, and they can be killed off by the script in a somewhat reliable way.
    This is not the kind of thing that inspired my suggestion.

    Note that such a script would have a very slight exposure to killing the wrong
    process. If it acquires a process ID of a process that has been present for
    quite a long time doing nothing, then before it can kill that process, that
    process exits just as the process ID roller reaches the same number, and a new
    process is forked with the same PID, that new process gets killed.

    The script might have a tough time of killing off a process that is forking
    new instances of itself every few milliseconds, with the parent exiting as
    each new child is created.

    In the case of what I'm doing, the timelimit program basically does the job.
    It just has certain reliability exposures that I do not trust. This is the
    kind of thing I believe the kernel should be doing. But I am thinking about
    a more flexible way to do so than the way I previously posted.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  9. Re: setrlimit() extension suggestion

    On Apr 27, 2:14 pm, phil-news-nos...@ipal.net wrote:

    > If you can understand the effect as described by the suggestion, maybe you can
    > suggest a way to do it that does not involve a kernel change. I won't think
    > less of you if you cannot, because I cannot think of such a way.


    It's impossible to solve a problem if you don't say what the problem
    is. All you've told me is how you propose to solve the problem, and
    then you've asked me how I would solve it. Well, I don't know. The
    reason I don't know is because I don't know what the problem is.

    > If you cannot understand the effect as described, then I don't see any purpose
    > in continuing the discussion with you. But I highly doubt this is the issue.
    > I suspect it is more likely you do work more at the system level and less at
    > the application level and have simply never encounted the need because of the
    > lesser exposure. You and I frequently do not see things the same way, and
    > this might be at least one of the reasons why.


    I understand the effect, I just don't understand the reason you would
    want that effect. It seems to me that such an effect would be useless
    because of the three bears problem I already explained. Settings of,
    say, 30 seconds would be too long because a process running away for
    30 seconds would make an interactive machine useless. Settings of,
    say, 4 minutes would be too short because it would be too hard to make
    long-running processes. These two settings overlap in the wrong way.

    So I don't understand why you would ever want the effect this setting
    gives you. There may or may not be other good ways to get this same
    effect, but what I don't see is why it matters.

    DS

  10. Re: setrlimit() extension suggestion

    On Mon, 28 Apr 2008 04:32:10 -0700 (PDT) David Schwartz wrote:
    | On Apr 27, 2:14 pm, phil-news-nos...@ipal.net wrote:
    |
    |> If you can understand the effect as described by the suggestion, maybe you can
    |> suggest a way to do it that does not involve a kernel change. I won't think
    |> less of you if you cannot, because I cannot think of such a way.
    |
    | It's impossible to solve a problem if you don't say what the problem
    | is. All you've told me is how you propose to solve the problem, and
    | then you've asked me how I would solve it. Well, I don't know. The
    | reason I don't know is because I don't know what the problem is.

    The problem is: how to make sure a process, including all its children,
    even if any of these processes change controlling TTY and/or process group,
    shall not exceed a specified resource limit, where that resource is a
    wallclock time frame.


    |> If you cannot understand the effect as described, then I don't see any purpose
    |> in continuing the discussion with you. But I highly doubt this is the issue.
    |> I suspect it is more likely you do work more at the system level and less at
    |> the application level and have simply never encounted the need because of the
    |> lesser exposure. You and I frequently do not see things the same way, and
    |> this might be at least one of the reasons why.
    |
    | I understand the effect, I just don't understand the reason you would
    | want that effect. It seems to me that such an effect would be useless
    | because of the three bears problem I already explained. Settings of,
    | say, 30 seconds would be too long because a process running away for
    | 30 seconds would make an interactive machine useless. Settings of,
    | say, 4 minutes would be too short because it would be too hard to make
    | long-running processes. These two settings overlap in the wrong way.

    I think maybe you don't understand the effect. I conclude this because your
    of your assertion that setting a limit of 30 seconds would "make an interactive
    machine useless". You seem to be assuming that the setting would be applied to
    every process running in the entire system. I've never suggsted such a thing.
    And setrlimit() doesn't do such a thing. What setrlimit() does is set the limit
    value for the current process (limited to only the soft limit if the process
    calling setrlimit() does not have root permission). Your discussion about a
    4 minute limit seems to also indicate a mis-understanding of the effect. The
    settings would not overlap. The settings would not be applied universally to
    all processes. The settings would only apply to the specific processes (and
    their children inherit it) that call setrlimit() to set it.

    I'll describe the effect in more detail so that maybe you can understand it
    correctly.

    For each process, a (new) resource limit shall exist with a soft and hard
    value. This resource limit specified in the members of struct rlimit shall
    be the number of seconds from the current time afterwhich the process shall
    not be allowed to continue to execute. An implementation shall behave in
    such a manner that is equivalent to an implementation that stores an internal
    value of an absolute wallclock time beyond what the process shall not be
    allowed to continue to execute. When a process forks a child, the child
    inherits the same wallclock time limit as the parent such that of both try
    to continue to run, both shall not be allowed to continue beyond the same
    point in time, unless that limit is changed. Like other resources used in
    setrlimit(), the soft limit is not permitted to exceed the hard limit, and
    only a process with root permissions may set the hard limit. If setrlimit()
    has never been called for this resource, or if setrlimit() is called with
    values of RLIM_INFINITY, then this resource shall not cause a process to be
    stopped.

    My interest is whether or not any issues in the design of the kernel would
    make an implementation of the above feature/effect difficult or impossible.
    I would think it should be entirely possible. Somewhere, other resources
    are checked to be sure a process does not exceed them. This would be one
    more resource to check. Checking it would be a matter of comparing the
    internally stored wallclock limits (derived from adding the limit values
    given in struct rlimit with the wallclock time when setrlimit() is called)
    with the current wallclock value.

    What signal to send a process that exceeds the soft limit, but has not yet
    exceeded the hard limit, I have not specified. Suggestions? For a process
    reaching the hard limit, RLIMIT_CPU uses SIGKILL and that seems appropriate
    for this proposed resource. But I'm not so sure that SIGXCPU would be for
    the soft limit, even though that is what is used for RLIMIT_CPU.


    | So I don't understand why you would ever want the effect this setting
    | gives you. There may or may not be other good ways to get this same
    | effect, but what I don't see is why it matters.

    You don't need to understand the why. You can choose to dismiss the idea
    because you can't see any reason _you_ would want to use it. Maybe someone
    else won't.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  11. Re: setrlimit() extension suggestion

    On Apr 29, 11:24 am, phil-news-nos...@ipal.net wrote:

    You: Wouldn't it be nice if cars had a feature where they stopped
    running at 3AM every Thursday?

    Me: No. Why would that be good?

    > You don't need to understand the why. You can choose to dismiss the idea
    > because you can't see any reason _you_ would want to use it. Maybe someone
    > else won't.


    Just in case you don't understand the rules of feature suggestion, it
    goes like this: The person suggesting a feature must state at least
    one use case. If the person suggesting the feature cannot defend that
    use case, they have to stop asking for the feature.

    DS

  12. Re: setrlimit() extension suggestion

    On Wed, 30 Apr 2008 01:22:16 -0700 (PDT) David Schwartz wrote:

    | You: Wouldn't it be nice if cars had a feature where they stopped
    | running at 3AM every Thursday?
    |
    | Me: No. Why would that be good?

    I would never propose a feature that would make a car stop only at 3AM on
    Thursdays. OTOH, I might, if the feature can be programmed to stop at any
    time the owner specifies for it to stop (and has a safety feature to prevent
    this when doing so would be dangerous ... e.g. not while actually rolling
    along on the highway, but only when it has stopped rolling for some minimum
    amount of time).

    This still makes it seem that you think the feature I suggest for setrlimit()
    is only to allow the caller to enable it but not be able to select the amount
    of time.


    |> You don't need to understand the why. You can choose to dismiss the idea
    |> because you can't see any reason _you_ would want to use it. Maybe someone
    |> else won't.
    |
    | Just in case you don't understand the rules of feature suggestion, it
    | goes like this: The person suggesting a feature must state at least
    | one use case. If the person suggesting the feature cannot defend that
    | use case, they have to stop asking for the feature.

    To implement the "timelimit" command correctly so it operates reliably.
    FYI, this command is part of the netpipes package.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  13. Re: setrlimit() extension suggestion

    On Apr 30, 8:53 pm, phil-news-nos...@ipal.net wrote:

    > | Just in case you don't understand the rules of feature suggestion, it
    > | goes like this: The person suggesting a feature must state at least
    > | one use case. If the person suggesting the feature cannot defend that
    > | use case, they have to stop asking for the feature.


    > To implement the "timelimit" command correctly so it operates reliably.
    > FYI, this command is part of the netpipes package.


    Do you understand what a "use case" is? You have to state an actual
    problem, not a proposed means of solving it. What problem would be
    solved by implementing the "timelimit" command correctly? And don't
    cite bugs in "timelimit", cite the actual problem you are trying to
    solve -- using "timelimit" is a solution to some problem, what?

    Give us enough information so that we can determine what the best
    solution is. If it's the resource limit you suggest, then you've done
    it -- you've defended a use case. You have an actual, bona fide
    problem best solved with the feature you suggested.

    DS

+ Reply to Thread