paging - Minix

This is a discussion on paging - Minix ; I note a comment by Andrew Tanenbaum re the topic of segmentation versus paging, where hs states: "However, plain old paging was invented in a time of 1-MB VAXes and 20-MB programs. With physical memory on PCs now at 512 ...

+ Reply to Thread
Results 1 to 11 of 11

Thread: paging

  1. paging

    I note a comment by Andrew Tanenbaum re the topic of segmentation
    versus paging, where hs states: "However, plain old paging was invented
    in a time of 1-MB VAXes and 20-MB programs. With physical memory on
    PCs now at 512 MB and more, it is far from clear
    that demand paging is a good idea." This comment seems to suggest that
    the only or main use of paging is to allow programs larger than
    available memory to run (e.g. paging programs out to secondary storage
    such as hard disk).

    Isnt one of the other main benefits of paging to allow many different
    pieces of code that reference the same physical addresses to share a
    memory space (e.g. be relocatable) without having to do address
    translation during loading?

    In other words, is not paging still a useful technique even though most
    computers now have so much memory that programs are rarely if ever
    "paged out" to secondary storage?


  2. Re: paging

    ptay1685@bigpond.net.au wrote:
    > I note a comment by Andrew Tanenbaum re the topic of segmentation
    > versus paging, where hs states: "However, plain old paging was invented
    > in a time of 1-MB VAXes and 20-MB programs. With physical memory on
    > PCs now at 512 MB and more, it is far from clear
    > that demand paging is a good idea." This comment seems to suggest that
    > the only or main use of paging is to allow programs larger than
    > available memory to run (e.g. paging programs out to secondary storage
    > such as hard disk).


    > Isnt one of the other main benefits of paging to allow many different
    > pieces of code that reference the same physical addresses to share a
    > memory space (e.g. be relocatable) without having to do address
    > translation during loading?


    If I understand you correctly, what you are saying is a benefit of
    virtual memory. Each process is linked at the same virtual address
    region, and during the run of each process, only the physical pages of
    that process are translated to the fixed virtual process address
    region. This is the process part of things.

    If a process shares a dynamic library with another, then each process
    maps the library at a different virtual address, and because the
    library is compiled position independent, there's no need to do address
    translation during loading.

    >
    > In other words, is not paging still a useful technique even though most
    > computers now have so much memory that programs are rarely if ever
    > "paged out" to secondary storage?


    Paging is surely a useful technique still. Programs are never fully
    mapped in memory. Only the used parts of them are mapped in pages which
    gives better utilisation of main memory. It isn't rare that there are
    many processes running on the system at one moment and main memory
    becomes precious.

    Thanks,
    Bahadir


  3. Re: paging


    wrote in message
    news:1152842710.058771.179010@75g2000cwc.googlegro ups.com...
    >I note a comment by Andrew Tanenbaum re the topic of
    > segmentation versus paging, where hs states: "However, plain
    > old paging was invented in a time of 1-MB VAXes and 20-MB
    > programs. With physical memory on PCs now at 512 MB and
    > more, it is far from clear that demand paging is a good idea."
    > This comment seems to suggest that the only or main use of
    > paging is to allow programs larger than available memory to
    > run (e.g. paging programs out to secondary storage such as
    > hard disk).
    >
    > Isnt one of the other main benefits of paging to allow many
    > different pieces of code that reference the same physical
    > addresses to share a memory space (e.g. be relocatable)
    > without having to do address translation during loading?
    >
    > In other words, is not paging still a useful technique even
    > though most computers now have so much memory that programs
    > are rarely if ever "paged out" to secondary storage?


    I think your concerns arise out of the semantics around 'policy'
    vs. 'mechanism'. Professor Tanenbaum appears to be using the
    term 'paging' in the above passage to refer to the _policy_ of
    dynamic paging. Your observations seem to be more directed at
    the general usefulness available from a hardware paging
    _mechanism_.

    I think such a distinction is important on the context of O.S.
    design. It is certainly true that the memory mapping available
    via segments can also be accomplished by the appropriate use of
    a hardware paging mechanism. Similarly, when the segmentation
    or paging _mechanisms_ will support it, one is free to choose
    from a range of memory management _policies_.

    In-between the policies of static memory allocation and dynamic
    paging of virtual memory, there are choices that might be called
    'dynamic mapping' or 'partial paging'. These intermediate
    policies allow the stack and heap regions to grow from an
    initially truncated allocation of physical memory, but can handle
    over-all physical memory demands by (for example) process swapping.
    This allows one to avoid most of the housekeeping and the page
    replacement decisions of dynamic paging.

    It is also possible to use a physical paging mechanism to support
    shared memory segments, without implementing dynamic paging.
    Without the dynamic paging of virtual memory the O.S. can
    more easily coalesce physical memory chunks (to reduce its
    bookkeeping load).

    Based on your and professor Tanenbaum's observations, and on
    the frequency of C.O.M. posting concerning problems from the
    need to manually set program memory allocations, the current
    development of Minix 3 might do well to incorporate an
    intermediate memory policy before turning to virtual memory.



  4. Re: paging

    On Thu, 13 Jul 2006 19:05:10 -0700, ptay1685 wrote:


    >
    > In other words, is not paging still a useful technique even though most
    > computers now have so much memory that programs are rarely if ever
    > "paged out" to secondary storage?


    Actually there is another advantage, that not all pages of
    instruction or data are needed, paging in only the referenced
    pages can be a big win for really large images. For instance the
    X server seems to need more that 256MB of real memory to just
    start up. I have no idea why, I used to run X on an 8MB 386.


  5. Re: paging

    In article ,
    Brian Beattie wrote:
    >For instance the
    >X server seems to need more that 256MB of real memory to just
    >start up. I have no idea why, I used to run X on an 8MB 386.


    The reason for that is the lack of VM.

    X needs enough address space to be able to map the video memory into
    it's own address space (the is a small hack in the kernel that allows
    memory mapped hardware to be mapped into a process' address space).

    Currently, the only way to allocate address space, it to request enough
    memory when the process is started.


    --
    That was it. Done. The faulty Monk was turned out into the desert where it
    could believe what it liked, including the idea that it had been hard done
    by. It was allowed to keep its horse, since horses were so cheap to make.
    -- Douglas Adams in Dirk Gently's Holistic Detective Agency

  6. Re: paging


    philip@ue.aioy.eu (Philip Homburg) writes:

    > In article ,
    > Brian Beattie wrote:
    > >For instance the
    > >X server seems to need more that 256MB of real memory to just
    > >start up. I have no idea why, I used to run X on an 8MB 386.

    >
    > The reason for that is the lack of VM.
    >
    > X needs enough address space to be able to map the video memory into
    > it's own address space (the is a small hack in the kernel that allows
    > memory mapped hardware to be mapped into a process' address space).
    >
    > Currently, the only way to allocate address space, it to request enough
    > memory when the process is started.


    Interesting. That means, that 256M or something are actually wasted
    mostly since they are shadowed by the video memory later?

    Regards -- Markus

  7. Re: paging

    In article ,
    M E Leypold wrote:
    >philip@ue.aioy.eu (Philip Homburg) writes:
    >> Currently, the only way to allocate address space, it to request enough
    >> memory when the process is started.

    >
    >Interesting. That means, that 256M or something are actually wasted
    >mostly since they are shadowed by the video memory later?


    Yes. The assumption was that a virtual memory implementation would arrive
    soon.


    --
    That was it. Done. The faulty Monk was turned out into the desert where it
    could believe what it liked, including the idea that it had been hard done
    by. It was allowed to keep its horse, since horses were so cheap to make.
    -- Douglas Adams in Dirk Gently's Holistic Detective Agency

  8. Re: paging


    philip@ue.aioy.eu (Philip Homburg) writes:

    > In article ,
    > M E Leypold wrote:
    > >philip@ue.aioy.eu (Philip Homburg) writes:
    > >> Currently, the only way to allocate address space, it to request enough
    > >> memory when the process is started.

    > >
    > >Interesting. That means, that 256M or something are actually wasted
    > >mostly since they are shadowed by the video memory later?

    >
    > Yes. The assumption was that a virtual memory implementation would arrive
    > soon.


    Both things -- that the 256K were actually only shadowed memory and
    the soon arriving VM implementation are good news. But especially the
    information about why the 256K are actually needed: I've already had
    very dark thoughts about the state of the X implementation since my
    suspicion had been that that was memory really malloc()ed for later
    use and that X is relying overly much on copy-on-write. (I hope I made
    understandable what I suspected). It's nice to hear that this was
    wrong, i.e. the 256K are "just" a Minix issue not a bad implementation
    of X that is covered up by the VM in other OSes.

    Thanks -- Markus



  9. Re: paging

    In article ,
    M E Leypold wrote:
    >It's nice to hear that this was
    >wrong, i.e. the 256K are "just" a Minix issue not a bad implementation
    >of X that is covered up by the VM in other OSes.


    Actually, it is worse. Some X application (I think it is googleearth,
    but I'm not sure) manages to get the X server on a Linux box to become in
    the order of 1300 MB. On a system with just 1GB of memory, you don't
    get much work done when that happens. Restarting the X server seems to
    be the only solution.

    The size of the X server (on a platform with a reasonable VM implementation)
    depends a lot on what you do with it. Just a couple of xterms and the
    X server remains small. Large amounts of high resolution graphics and
    the X server may grow without bounds.


    --
    That was it. Done. The faulty Monk was turned out into the desert where it
    could believe what it liked, including the idea that it had been hard done
    by. It was allowed to keep its horse, since horses were so cheap to make.
    -- Douglas Adams in Dirk Gently's Holistic Detective Agency

  10. Re: paging


    philip@ue.aioy.eu (Philip Homburg) writes:

    > In article ,
    > M E Leypold wrote:
    > >It's nice to hear that this was
    > >wrong, i.e. the 256K are "just" a Minix issue not a bad implementation
    > >of X that is covered up by the VM in other OSes.

    >
    > Actually, it is worse. Some X application (I think it is googleearth,
    > but I'm not sure) manages to get the X server on a Linux box to become in
    > the order of 1300 MB. On a system with just 1GB of memory, you don't
    > get much work done when that happens. Restarting the X server seems to


    Arrgs.

    > be the only solution.


    So the server stays that size when the application / client terminates?

    > The size of the X server (on a platform with a reasonable VM implementation)
    > depends a lot on what you do with it. Just a couple of xterms and the
    > X server remains small. Large amounts of high resolution graphics and
    > the X server may grow without bounds.


    Yes, I've already seen that effect. Since I haven't debugged it, I
    cannot say anything for sure, but I always assumed that this is not a
    memory leak in the server itself, but server side resources (font,
    bitmaps) slowly accumulating. Do you per chance know (a) wether in the
    cases you're refering to the size of the server shrinks again after
    closing the program which gave rise to this grow in memory size (like
    Googlke earth) and (b) wether bitmaps etc stored in the server are
    supposed to be freed automatically when the corresponding clients
    terminate or wether they must explicitly freed by the client?

    (I hope I'm not talking total nonsense. Again I haven't researched
    anything, but my assumption was that there are resources like bitmaps
    and fonts that can be loaded into the server by the client thus
    avoiding the need to retransmitting them all the time when a window
    using that bitmap has to be redrawn. My question is stimulated (??) by
    the idea that this apparent limitless growing of an X-Server is due to
    badly behaved clients which don't free those ressources when they
    aren't needed. If that is what happens then the fault is really with
    the X protocol (not the quality of the server implementation) which
    might not make enough allowance for unreliable and misbehaved
    clients.)

    Regards -- Markus




  11. Re: paging

    In article ,
    M E Leypold wrote:
    >So the server stays that size when the application / client terminates?


    Yes.

    >> The size of the X server (on a platform with a reasonable VM implementation)
    >> depends a lot on what you do with it. Just a couple of xterms and the
    >> X server remains small. Large amounts of high resolution graphics and
    >> the X server may grow without bounds.

    >
    >Yes, I've already seen that effect. Since I haven't debugged it, I
    >cannot say anything for sure, but I always assumed that this is not a
    >memory leak in the server itself, but server side resources (font,
    >bitmaps) slowly accumulating. Do you per chance know (a) wether in the
    >cases you're refering to the size of the server shrinks again after
    >closing the program which gave rise to this grow in memory size (like
    >Googlke earth) and (b) wether bitmaps etc stored in the server are
    >supposed to be freed automatically when the corresponding clients
    >terminate or wether they must explicitly freed by the client?


    I think most X resources are associated with the client, so they
    should be freed when the connection is closed.

    >If that is what happens then the fault is really with
    >the X protocol (not the quality of the server implementation) which
    >might not make enough allowance for unreliable and misbehaved
    >clients.)


    I think the X protocol has plenty of room for refusing service to
    bad clients.

    It is even possible that the X server already has tunables for that but
    that they hidden well enough that you don't find them by accident.


    --
    That was it. Done. The faulty Monk was turned out into the desert where it
    could believe what it liked, including the idea that it had been hard done
    by. It was allowed to keep its horse, since horses were so cheap to make.
    -- Douglas Adams in Dirk Gently's Holistic Detective Agency

+ Reply to Thread