Reclaiming freed memory back from running process - Linux

This is a discussion on Reclaiming freed memory back from running process - Linux ; Scenario: 1. Process have allocated a big chunk of memory 2. Process freed that memory (and it's still running) 3. Freed memory still available to the process for future re-use. So, we don't see any memory usage reduction for that ...

+ Reply to Thread
Results 1 to 10 of 10

Thread: Reclaiming freed memory back from running process

  1. Reclaiming freed memory back from running process

    Scenario:
    1. Process have allocated a big chunk of memory
    2. Process freed that memory (and it's still running)
    3. Freed memory still available to the process for future re-use. So,
    we don't see any memory usage reduction for that process - OK.
    The question is: If system start getting out of memory will kernel
    reclaim free memory that available for process use? Or kernel will take
    it back only after process exits?

    Thank you,

    Dimitry


  2. Re: Reclaiming freed memory back from running process

    "Dimitry" writes:

    > Scenario:
    > 1. Process have allocated a big chunk of memory
    > 2. Process freed that memory (and it's still running)
    > 3. Freed memory still available to the process for future re-use. So,
    > we don't see any memory usage reduction for that process - OK.
    > The question is: If system start getting out of memory will kernel
    > reclaim free memory that available for process use? Or kernel will take
    > it back only after process exits?


    If a memory page is not used by a process, the kernel will swap it out
    to the swap files or partitions, so the physical RAM can be used by
    other (or the same) process.

    If the swap space becomes full, then the kernel indeed can reclaim the
    memory from a process. It does so by killing a process, at random (or
    following some heuristics).


    --
    __Pascal Bourguignon__ http://www.informatimago.com/

    COMPONENT EQUIVALENCY NOTICE: The subatomic particles (electrons,
    protons, etc.) comprising this product are exactly the same in every
    measurable respect as those used in the products of other
    manufacturers, and no claim to the contrary may legitimately be
    expressed or implied.

  3. Re: Reclaiming freed memory back from running process

    "Dimitry" writes:

    > Scenario:
    > 1. Process have allocated a big chunk of memory


    How big was the chunk?

    Glibc malloc will use mmap() instead of sbrk() to allocate really
    large chunks (128K by default, settable with mallopt()). If such a
    chunk is later free()ed, the chunk is munmap()ed, returning it to
    the kernel immediately.

    Cheers,
    --
    In order to understand recursion you must first understand recursion.
    Remove /-nsp/ for email.

  4. Re: Reclaiming freed memory back from running process



    On 1月8日, 上午9时09分, Pascal Bourguignon
    wrote:
    > "Dimitry" writes:
    > > Scenario:
    > > 1. Process have allocated a big chunk of memory
    > > 2. Process freed that memory (and it's still running)
    > > 3. Freed memory still available to the process for future re-use. So,
    > > we don't see any memory usage reduction for that process - OK.
    > > The question is: If system start getting out of memory will kernel
    > > reclaim free memory that available for process use? Or kernel will take
    > > it back only after process exits?If a memory page is not used by a process, the kernel will swap it out

    > to the swap files or partitions, so the physical RAM can be used by
    > other (or the same) process.
    >
    > If the swap space becomes full, then the kernel indeed can reclaim the
    > memory from a process. It does so by killing a process, at random (or
    > following some heuristics).


    I think what he said is regarding the glibc cache for freed chunk.
    >
    > --
    > __Pascal Bourguignon__ http://www.informatimago.com/
    >
    > COMPONENT EQUIVALENCY NOTICE: The subatomic particles (electrons,
    > protons, etc.) comprising this product are exactly the same in every
    > measurable respect as those used in the products of other
    > manufacturers, and no claim to the contrary may legitimately be
    > expressed or implied.



  5. Re: Reclaiming freed memory back from running process

    "Dimitry" writes:

    > Scenario:
    > 1. Process have allocated a big chunk of memory
    > 2. Process freed that memory (and it's still running)
    > 3. Freed memory still available to the process for future re-use. So,
    > we don't see any memory usage reduction for that process - OK.
    > The question is: If system start getting out of memory will kernel
    > reclaim free memory that available for process use? Or kernel will take
    > it back only after process exits?


    It's not a problem. The memory that was allocated to the process is
    virtual; each process has its own virtual space. Physical memory is
    used and reclaimed as needed, so once the process quits using it, it's
    available to other processes.

    The real only issue is swap space; you could conceivably run short on
    swap space if your system doesn't have enough.
    --
    Joseph J. Pfeiffer, Jr., Ph.D. Phone -- (505) 646-1605
    Department of Computer Science FAX -- (505) 646-1002
    New Mexico State University http://www.cs.nmsu.edu/~pfeiffer

  6. Re: Reclaiming freed memory back from running process

    > It's not a problem. The memory that was allocated to the process is
    > virtual; each process has its own virtual space. Physical memory is
    > used and reclaimed as needed, so once the process quits using it, it's
    > available to other processes.
    >

    Thanks guys,

    The only thing I am worried about is that process is running - it will
    not quit.
    It was one time allocation of a lot of memory by small chunks (i.e.
    4KB) - so all together it's quite a bit of memory taken (about 20MB). I
    can see that after freeing, memory still part of "process used memory"
    - so it's not reclaimed back. Process itself is running but how do I
    make kernel to take this memory back when it becomes low in memory. We
    are talking about embedded application with no swap.

    Thanks


  7. Re: Reclaiming freed memory back from running process

    "Dimitry" writes:

    >> It's not a problem. The memory that was allocated to the process is
    >> virtual; each process has its own virtual space. Physical memory is
    >> used and reclaimed as needed, so once the process quits using it, it's
    >> available to other processes.
    >>

    > Thanks guys,
    >
    > The only thing I am worried about is that process is running - it will
    > not quit.
    > It was one time allocation of a lot of memory by small chunks (i.e.
    > 4KB) - so all together it's quite a bit of memory taken (about 20MB). I
    > can see that after freeing, memory still part of "process used memory"
    > - so it's not reclaimed back. Process itself is running but how do I
    > make kernel to take this memory back when it becomes low in memory. We
    > are talking about embedded application with no swap.


    The application can ask the kernel memory with mmap and can use munmap
    to give it back. Instead of using an unspecific malloc, you can use
    one that explicitely use mmap and will munmap freed memory (preferably
    using high and low water marks).

    --
    __Pascal Bourguignon__ http://www.informatimago.com/

    This is a signature virus. Add me to your signature and help me to live.

  8. Re: Reclaiming freed memory back from running process

    "Dimitry" writes:

    > > It's not a problem. The memory that was allocated to the process is
    > > virtual; each process has its own virtual space. Physical memory is
    > > used and reclaimed as needed, so once the process quits using it, it's
    > > available to other processes.
    > >

    > Thanks guys,
    >
    > The only thing I am worried about is that process is running - it will
    > not quit.


    That's a whole different question. What state is it stuck in?

    > It was one time allocation of a lot of memory by small chunks (i.e.
    > 4KB) - so all together it's quite a bit of memory taken (about
    > 20MB). I
    > can see that after freeing, memory still part of "process used memory"
    > - so it's not reclaimed back. Process itself is running but how do I
    > make kernel to take this memory back when it becomes low in memory. We
    > are talking about embedded application with no swap.


    If you were in trouble because of the lack of swap, the symptoms would
    be processes dying at random.
    --
    Joseph J. Pfeiffer, Jr., Ph.D. Phone -- (505) 646-1605
    Department of Computer Science FAX -- (505) 646-1002
    New Mexico State University http://www.cs.nmsu.edu/~pfeiffer

  9. Re: Reclaiming freed memory back from running process

    Joe Pfeiffer writes:
    > "Dimitry" writes:
    >> Scenario:
    >> 1. Process have allocated a big chunk of memory
    >> 2. Process freed that memory (and it's still running)
    >> 3. Freed memory still available to the process for future re-use. So,
    >> we don't see any memory usage reduction for that process - OK.
    >> The question is: If system start getting out of memory will kernel
    >> reclaim free memory that available for process use? Or kernel will take
    >> it back only after process exits?

    >
    > It's not a problem. The memory that was allocated to the process is
    > virtual; each process has its own virtual space. Physical memory is
    > used and reclaimed as needed, so once the process quits using it, it's
    > available to other processes.
    >
    > The real only issue is swap space; you could conceivably run short on
    > swap space if your system doesn't have enough.


    If the page is dirty, the kernel must write it to some paging area
    before the memory can be reclaimed. This needs space inside the paging
    area (pointless for unused memory), needs time for the I/O (pointless
    for unused memory) and will, depending on the timing, force another
    process that needs memory to block until the unused dirty page has
    been written to the paging area (bad for latency) or may even cause
    kernel failures (eg GFP_ATOMIC allocation fails because it would be
    necessary to wait for memory to become available).

    The rationale for keeping the page nevertheless is that future
    allocations done by the same process will be more efficient if a page
    the processs has already acquired can be reused, instead of needing
    to get another one from the kernel, which, of course, assumes that it
    hasn't been paged out in the meantime -- otherwise, it would be a hell
    lot more efficient to get a fresh page instead of waiting until the
    non-interesting old content has been read back from disk into a fresh
    page.

    Putting this together leads to the conclusion that keeping the page is
    only sensible if it can reasonably be expected that the system has
    more physical memory than it will ever need together at any given
    point in time.

  10. Re: Reclaiming freed memory back from running process

    On 2007-01-08, Dimitry wrote:
    > Scenario:
    > 1. Process have allocated a big chunk of memory
    > 2. Process freed that memory (and it's still running)
    > 3. Freed memory still available to the process for future re-use. So,
    > we don't see any memory usage reduction for that process - OK.
    > The question is: If system start getting out of memory will kernel
    > reclaim free memory that available for process use? Or kernel will take
    > it back only after process exits?


    kernel will swap it to disk, and reallocate the RAM to the process that
    needs it now. if the first process never uses it it'll sit in the swap until
    the process exits.



    --

    Bye.
    Jasen

+ Reply to Thread