Re: [9fans] plan 9 overcommits memory? - Plan9

This is a discussion on Re: [9fans] plan 9 overcommits memory? - Plan9 ; erik quanstrom wrote: > > That's why many OSes have a "spawn" primitive that combines fork-and-exec. > the problem with spawn is it requires a mechanism to replace that little > block of code between the fork and exec. that ...

+ Reply to Thread
Page 3 of 3 FirstFirst 1 2 3
Results 41 to 52 of 52

Thread: Re: [9fans] plan 9 overcommits memory?

  1. Re: [9fans] plan 9 overcommits memory?

    erik quanstrom wrote:
    > > That's why many OSes have a "spawn" primitive that combines fork-and-exec.

    > the problem with spawn is it requires a mechanism to replace that little
    > block of code between the fork and exec. that code is hardly ever the
    > same so spawn keeps growing arguments.


    Yes, on the other hand I bet a spawn interface could be devised that
    is sufficiently programmable. (May require some extra dup()s etc. to
    really handle all the common usage patterns.)

  2. Re: [9fans] plan 9 overcommits memory?

    > > > That's why many OSes have a "spawn" primitive that combines fork-and-exec.
    > >
    > > the problem with spawn is it requires a mechanism to replace that little
    > > block of code between the fork and exec. that code is hardly ever the
    > > same so spawn keeps growing arguments.

    >
    > Yes, on the other hand I bet a spawn interface could be devised that
    > is sufficiently programmable. (May require some extra dup()s etc. to
    > really handle all the common usage patterns.)


    What could accomplish this are hooks into the process-creation bits of
    the kernel. I've lost the links to this, but some toy OS accomplished
    process creation with the equivalent of 'cp /bin/ls /proc/clone'.
    This isn't useful on its own, but it could be an interesting exercise
    to construct a process outside the kernel and kick-start it somehow.

    The one somewhat useful application of this idea would be to mitigate
    the bug documented in notify(2):
    Since exec(2) discards the notification handler, there
    is a window of vulnerability to notes in a new process.
    In a hand-rolled spawn(), one could conceivably install a notification
    handler before the process starts executing or accepting notes.

    Possibly some form of dynld() could be built on this too.

    I can't imagine that either of these uses are nearly compelling enough
    to open this can of worms.... Has anyone truly felt confined by Plan
    9's fork+exec model?

    --Joel

  3. Re: [9fans] plan 9 overcommits memory?

    On 9/6/07, Joel C. Salomon wrote:

    > I can't imagine that either of these uses are nearly compelling enough
    > to open this can of worms.... Has anyone truly felt confined by Plan
    > 9's fork+exec model?


    yes, because exec takes a pathname. that's a pull model. That is
    pretty awful in a large machine. Define awful: ok, it's the difference
    between startup times of 3+ minutes, 400 nodes, vs. 3 seconds. That's
    awful.

    it's why we started doing xcpu in the first place: push the binary to
    a ram disk, then at least xcpu is pulling from a local place, not a
    network. But xcpu was a compromise: I really wanted to do a process
    creation device.

    ron

  4. Re: [9fans] plan 9 overcommits memory?

    > yes, because exec takes a pathname. that's a pull model. That is

    9p can remove your pain


  5. Re: [9fans] plan 9 overcommits memory?

    I wonder if this is interesting but the user mode processes creation device is,
    in effect, what I and brucee have been working on (seperately) for plan9 binaries
    on windows and what russ did for linux binaries on plan9.

    BTW the latter has seen some development recently - somone managed to get dynamic libs
    working with russ's code (no major work, more a hassle I beleive) and got as far as
    getting dillo to run. The downside is dillo has to render its output on an X11 display...

    still, makes you think...

    -Steve

  6. Re: [9fans] plan 9 overcommits memory?

    On Thu, 2007-09-06 at 12:38 -0700, ron minnich wrote:
    > On 9/6/07, Joel C. Salomon wrote:
    >
    > > I can't imagine that either of these uses are nearly compelling enough
    > > to open this can of worms.... Has anyone truly felt confined by Plan
    > > 9's fork+exec model?

    >
    > yes, because exec takes a pathname. that's a pull model. That is
    > pretty awful in a large machine. Define awful: ok, it's the difference
    > between startup times of 3+ minutes, 400 nodes, vs. 3 seconds. That's
    > awful.
    >
    > it's why we started doing xcpu in the first place: push the binary to
    > a ram disk, then at least xcpu is pulling from a local place, not a
    > network. But xcpu was a compromise: I really wanted to do a process
    > creation device.


    Exposing an interface for process manipulation to userspace would be
    quite cool. Any prototypes on Plan9 so far?

    Thanks,
    Roman.


  7. Re: [9fans] plan 9 overcommits memory?

    this thread is now twice as big as the swap and proc code.
    is it really that much fun to do research this way?
    yes, it has been done. no, my current work has nothing
    to do with this issue.

    someone written a line of relevant code during this "discussion"?

    brucee

    On 9/7/07, Roman Shaposhnik wrote:
    > On Thu, 2007-09-06 at 12:38 -0700, ron minnich wrote:
    > > On 9/6/07, Joel C. Salomon wrote:
    > >
    > > > I can't imagine that either of these uses are nearly compelling enough
    > > > to open this can of worms.... Has anyone truly felt confined by Plan
    > > > 9's fork+exec model?

    > >
    > > yes, because exec takes a pathname. that's a pull model. That is
    > > pretty awful in a large machine. Define awful: ok, it's the difference
    > > between startup times of 3+ minutes, 400 nodes, vs. 3 seconds. That's
    > > awful.
    > >
    > > it's why we started doing xcpu in the first place: push the binary to
    > > a ram disk, then at least xcpu is pulling from a local place, not a
    > > network. But xcpu was a compromise: I really wanted to do a process
    > > creation device.

    >
    > Exposing an interface for process manipulation to userspace would be
    > quite cool. Any prototypes on Plan9 so far?
    >
    > Thanks,
    > Roman.
    >
    >


  8. Re: [9fans] plan 9 overcommits memory?


    On 2007-Sep-6, at 21:09 , Bruce Ellis wrote:

    > someone written a line of relevant code during this "discussion"?


    Be careful. Coding to mailing list discussions results in Linux.

    FWIW, the discussion here has made more sense than any and all
    arguments/conversations I've had with UNIX vendors over the last
    decade-and-a-half.

    And I'm talking about both malloc and spawn.

    The concept of assembling processes in user space is intriguing, and
    I'd like to hear more from the people who have thought/played with
    the idea.

    I'm convinced there is no solution to brk; it's an untenable
    implementation (although it made perfect sense at the time). As long
    as our programming languages insist on direct memory pointers I don't
    see a way out. But so far all the alternatives just lead to madness ...

    --lyndon

  9. Re: [9fans] plan 9 overcommits memory?

    coding at home and getting something significant working
    is what i meant. but by all means write to the list instead.

    brucee

    On 9/7/07, Lyndon Nerenberg wrote:
    >
    > On 2007-Sep-6, at 21:09 , Bruce Ellis wrote:
    >
    > > someone written a line of relevant code during this "discussion"?

    >
    > Be careful. Coding to mailing list discussions results in Linux.
    >
    > FWIW, the discussion here has made more sense than any and all
    > arguments/conversations I've had with UNIX vendors over the last
    > decade-and-a-half.
    >
    > And I'm talking about both malloc and spawn.
    >
    > The concept of assembling processes in user space is intriguing, and
    > I'd like to hear more from the people who have thought/played with
    > the idea.
    >
    > I'm convinced there is no solution to brk; it's an untenable
    > implementation (although it made perfect sense at the time). As long
    > as our programming languages insist on direct memory pointers I don't
    > see a way out. But so far all the alternatives just lead to madness ...
    >
    > --lyndon
    >


  10. Re: [9fans] plan 9 overcommits memory?


    On 2007-Sep-6, at 21:37 , Bruce Ellis wrote:

    > coding at home and getting something significant working
    > is what i meant. but by all means write to the list instead.


    So how do you see the way out of a brk() implementation while still
    coding in a pointer based environment? Code only follows engineering.

    If I even Thought About Mentioning GC, well, ... let me put up the
    window shutters.

  11. Re: [9fans] plan 9 overcommits memory?

    > this thread is now twice as big as the swap and proc code.
    > is it really that much fun to do research this way?
    > yes, it has been done. no, my current work has nothing
    > to do with this issue.
    >
    > someone written a line of relevant code during this "discussion"?


    would you care to give references to this being done
    in plan 9 or similar operating system?

    i would hope the discussion is much bigger than the
    resulting code. this discussion has boiled down the
    issues for me. this is what i remember

    plan 9 overcommits memory three ways
    1. when a process forks, the cow pages are counted once.
    2. plan 9 hands out brk'ed pages it doesn't have.
    3. the stack is potentially very large (16mb), grown
    implictly and not reserved.

    by the way, what are the scare quote quotes for?
    participants might take that in the wrong way.

    - erik


  12. Re: [9fans] plan 9 overcommits memory?

    brucee wrote:
    > coding at home and getting something significant working
    > is what i meant. but by all means write to the list instead.


    i agree totally that it's a Good Thing to go and actually code something
    up, but i also think that it's a positive thing to have technical discussions
    out in the open. not necessarily because it's an efficient way to do things
    (it probably isn't), but because it keeps a useful record that people
    can later use to learn from.

    i heard someone on another list say it nice and succinctly:

    : FWIW, I strongly prefer to have as much of technical discussions on-list
    : as possible: that way they benefit future generations of SBCL hackers as
    : well.

+ Reply to Thread
Page 3 of 3 FirstFirst 1 2 3