Weird memory usage - SUN

This is a discussion on Weird memory usage - SUN ; Hi all, I have an Ultra 2, dual 400 MHz with 1280 MB RAM, running Solaris 9. Typically it uses 400 MB to 500 MB of physical memory and 200 MB to 250 MB of swap. So my questions are: ...

+ Reply to Thread
Results 1 to 13 of 13

Thread: Weird memory usage

  1. Weird memory usage

    Hi all,

    I have an Ultra 2, dual 400 MHz with 1280 MB RAM, running Solaris 9.
    Typically it uses 400 MB to 500 MB of physical memory and 200 MB to
    250 MB of swap.

    So my questions are:

    1. Why does it use any swap when it has an enormous amount of physical
    memory free?

    2. Is there any way to make it use physical memory instead of swap,
    and would this be a wise thing to do?

    3. Am I right in thinking that if the machine is never using all its
    physical memory, it's pointless to add more RAM?

    Any suggestions would be much appreciated.

    Best wishes,

    Chris Tidy


  2. Re: Weird memory usage


    "Christopher Tidy" wrote in message
    news:1169927691.037368.41460@v45g2000cwv.googlegro ups.com...
    > Hi all,
    >
    > I have an Ultra 2, dual 400 MHz with 1280 MB RAM, running Solaris 9.
    > Typically it uses 400 MB to 500 MB of physical memory and 200 MB to
    > 250 MB of swap.
    >
    > So my questions are:
    >
    > 1. Why does it use any swap when it has an enormous amount of physical
    > memory free?


    Unused memory pages and blocks, as well as shared libraries can be placed in
    swap in advance. So if the system immediately needs memory, it can just
    discard (fast) in memory pages knowing that when they are needed it can go
    to swap to get them. This is an optimization feature as time is saved in
    not having to write (slow) it out on demand as the last moment in time.

    > 2. Is there any way to make it use physical memory instead of swap,
    > and would this be a wise thing to do?


    If it is Solaris, it probably already is - sort of. /tmp on a default
    install is special in that it is /tmp and swap at the same time. But it has
    it's own cache allocations between the application and the disk.

    Unless your really familiar with performance tuning and analysis, your
    almost always best advised just to leave this alone. For the most part,
    Solaris will adapt itself nicely to workloads all on it's own. There are
    kernel parameters but it is best left to those that know how to set them.

    > 3. Am I right in thinking that if the machine is never using all its
    > physical memory, it's pointless to add more RAM?


    If a system never uses all its memory, adding more would be of little value.
    If your having performance problems, find someone who knows how to read
    "vmstat" and "sar" reports. Be careful, of 10 Unix Admins, maybe one knows
    how to do this correctly.

    For example, "vmstat" and a commercial performance monitoring said the
    system was using all it's memory. I was called in and reviewed the sar logs
    and ran vmstat while they had the issue. I said no, it was disk I/O bound.
    The saleperson was real mad at me, they wanted to dump $50+k of memory on my
    client. Management didn't know what to do, conflicting results. So the OEM
    vendor was good enough to take in the application to a lab and run it and
    see what it needed.

    When the results came back, $50K of memory increased performance by only
    1.8%. Putting a cached disk array w. RAID 0+1 as I recomended made the app
    go 3 times faster!! Just understanding wait I/O and kernel behavior gave me
    that answer in 1 hour of analysis.

    If your not sure of vendor recomandations with performance issues, take
    their recomendation on consignment, if it doesn't fix the issue leave
    yourself an out in the contrract that you don't pay for it.

    If you want to get into doing this yourself, start by reading up on vmstat,
    sar, iostat, nfsstat, trace and others.

    > Any suggestions would be much appreciated.
    >
    > Best wishes,
    >
    > Chris Tidy
    >




  3. Re: Weird memory usage

    Canuck57 wrote:

    > Unused memory pages and blocks, as well as shared libraries can be placed in
    > swap in advance. So if the system immediately needs memory, it can just
    > discard (fast) in memory pages knowing that when they are needed it can go
    > to swap to get them. This is an optimization feature as time is saved in
    > not having to write (slow) it out on demand as the last moment in time.


    Thanks. That makes good sense.

    > If your not sure of vendor recomandations with performance issues, take
    > their recomendation on consignment, if it doesn't fix the issue leave
    > yourself an out in the contrract that you don't pay for it.


    The performance of my system is fine. I was just wondering why it was
    using any swap at all, and if storing that information in RAM instead
    would make the system faster. It looks like the information is
    probably already stored in RAM, then.

    > If you want to get into doing this yourself, start by reading up on vmstat,
    > sar, iostat, nfsstat, trace and others.


    I'll do that out of interest.

    Many thanks,

    Chris


  4. Re: Weird memory usage

    On 27 Jan 2007 11:54:51 -0800, "Christopher Tidy" wrote:
    > Hi all,
    >
    > I have an Ultra 2, dual 400 MHz with 1280 MB RAM, running Solaris 9.
    > Typically it uses 400 MB to 500 MB of physical memory and 200 MB to
    > 250 MB of swap.
    >
    > So my questions are:
    >
    > 1. Why does it use any swap when it has an enormous amount of physical
    > memory free?


    Swap space is not only used for swapping out stuff which doesn't
    currently fit into physical memory. It is also used as the backing
    store for virtual memory pages which are 'anonymous', i.e. they don't
    directly map through a vnode/offset pair to an existing, physical file,
    stored on a file system.

    More over, the page daemon will try to aggressively move pages out of
    physical memory, freeing the faster, easier to access physical pages for
    the "current working set". How aggressively this happens depends on a
    lot of things, but rest assured that if you are not really putting too
    much strain on your Ultra 2, and it can handle the load, you don't have
    to worry about swap usage; it's not abnormal to see swap space being
    used, even if you are not currently running out of physical memory.

    > 2. Is there any way to make it use physical memory instead of swap,
    > and would this be a wise thing to do?


    It's probably not as wise as it may seem. Swap space is used in order
    to free physical pages for other things, which may need the physical
    memory sooner than the swapped out stuff. By moving some of the unused
    virtual memory pages out of physical memory, Solaris tries to make more
    physical memory available for other things (like file system cache, and
    the processes which use their virtual memory pages more often), so you
    are actually *gaining* in speed by moving unused stuff out of the
    physical memory space.

    > 3. Am I right in thinking that if the machine is never using all its
    > physical memory, it's pointless to add more RAM?


    AFAIK, all of physical memory *is* used, but it may not be as obvious as
    it initially seems _where_ it is used and _how_ it is used. For
    instance, 100% of physical memory is not available to user processes;
    some of it has to be used for internal kernel data and house-keeping
    information.

    - Giorgos


  5. Re: Weird memory usage

    Giorgos Keramidas wrote:



    > AFAIK, all of physical memory *is* used, but it may not be as obvious as
    > it initially seems _where_ it is used and _how_ it is used. For
    > instance, 100% of physical memory is not available to user processes;
    > some of it has to be used for internal kernel data and house-keeping
    > information.


    Thanks Giorgos. So if I'm seeing 400 MB to 500 MB of physical memory
    used in Solaris Management Console, what does that number mean, and
    what is the rest of my 1280 MB being used for?

    Do you think I would see much increase in speed if I upgraded to 2048
    MB physical memory?

    Best wishes,

    Chris


  6. Re: Weird memory usage

    On 27 Jan 2007 20:56:16 -0800, "Christopher Tidy" wrote:
    > Giorgos Keramidas wrote:
    >
    >> AFAIK, all of physical memory *is* used, but it may not be as obvious as
    >> it initially seems _where_ it is used and _how_ it is used. For
    >> instance, 100% of physical memory is not available to user processes;
    >> some of it has to be used for internal kernel data and house-keeping
    >> information.

    >
    > Thanks Giorgos. So if I'm seeing 400 MB to 500 MB of physical memory
    > used in Solaris Management Console, what does that number mean, and
    > what is the rest of my 1280 MB being used for?


    I'm usually looking at memory usage with prtconf/vmstat, so I can't tell
    for sure what you are seeing in Solaris Management Console.

    For example, in a relatively old Celeron system I have at home, which
    runs Solaris 10 Update 3, I can look at memory usage with:

    % root@fingon:/root# prtconf | head -2
    % System Configuration: Sun Microsystems i86pc
    % Memory size: 384 Megabytes
    % root@fingon:/root# vmstat 1 3
    % kthr memory page disk faults cpu
    % r b w swap free re mf pi po fr de sr cd f0 -- -- in sy cs us sy id
    % 0 0 0 2251640 193956 7 24 12 0 0 0 10 2 0 0 0 835 175 454 1 2 96
    % 0 0 0 2252016 189000 7 44 0 0 0 0 0 0 0 0 0 352 133 161 0 2 98
    % 0 0 0 2252016 189000 0 0 0 0 0 0 0 0 0 0 0 375 111 157 1 1 98
    % root@fingon:/root#

    The amount of physical memory installed is printed by prtconf, and the
    amount of free memory is printed by vmstat (in the memory/free column).

    > Do you think I would see much increase in speed if I upgraded to 2048
    > MB physical memory?


    This depends a lot on what your typical workload is. If you frequently
    run memory intensive programs, it's very likely to help a lot

    Best regards,
    Giorgos


  7. Re: Weird memory usage

    "Christopher Tidy" writes:

    >1. Why does it use any swap when it has an enormous amount of physical
    >memory free?


    What are you basing this observation on?

    Swap space in Solaris is seen as the combination of physical memory
    and physical swap; it will report swap used, even if no physical
    disk space is touched.

    "swap -s" reports about virtual swap
    "swap -l" reports physical swap use

    >2. Is there any way to make it use physical memory instead of swap,
    >and would this be a wise thing to do?


    It will use physical swap only when it needs to.

    >3. Am I right in thinking that if the machine is never using all its
    >physical memory, it's pointless to add more RAM?


    If it never sees a physical memory shortfal, then it may be pointless.

    But the kernel evicts memory long before it is gone (it needs memory
    to evict memory, so it tries to keep a certain percentage free)

    Casper
    --
    Expressed in this posting are my opinions. They are in no way related
    to opinions held by my employer, Sun Microsystems.
    Statements on Sun products included here are not gospel and may
    be fiction rather than truth.

  8. Re: Weird memory usage

    On 2007-01-28 04:56:16 +0000, "Christopher Tidy"
    said:

    >
    > Thanks Giorgos. So if I'm seeing 400 MB to 500 MB of physical memory
    > used in Solaris Management Console, what does that number mean, and
    > what is the rest of my 1280 MB being used for?
    >
    > Do you think I would see much increase in speed if I upgraded to 2048
    > MB physical memory?


    I think this is something that is hard to answer without looking at the
    system in detail. VM management is a complicated business. One thing
    to look at is the sr ('scan rate') field from vmstat: this measures how
    fast the page scanner is looking for pages to reclaim. If it is zero
    (remember to ignore the first batch of output from vmstat) then I think
    it's reasonably safe to say that the machine has enough memory.
    However this is just one figure.

    (Although actually, this may not be true: if the machine is doing a lot
    amount of filesystem I/O then depending on the characteristics of that
    I/O it can benefit from really a lot of physical memory I think.)

    --tim



  9. Re: Weird memory usage


    "Giorgos Keramidas" wrote in message
    news:87zm83iovn.fsf@kobe.laptop...
    > On 27 Jan 2007 20:56:16 -0800, "Christopher Tidy"
    > wrote:
    >> Giorgos Keramidas wrote:
    >>
    >>> AFAIK, all of physical memory *is* used, but it may not be as obvious as
    >>> it initially seems _where_ it is used and _how_ it is used. For
    >>> instance, 100% of physical memory is not available to user processes;
    >>> some of it has to be used for internal kernel data and house-keeping
    >>> information.

    >>
    >> Thanks Giorgos. So if I'm seeing 400 MB to 500 MB of physical memory
    >> used in Solaris Management Console, what does that number mean, and
    >> what is the rest of my 1280 MB being used for?

    >
    > I'm usually looking at memory usage with prtconf/vmstat, so I can't tell
    > for sure what you are seeing in Solaris Management Console.
    >
    > For example, in a relatively old Celeron system I have at home, which
    > runs Solaris 10 Update 3, I can look at memory usage with:
    >
    > % root@fingon:/root# prtconf | head -2
    > % System Configuration: Sun Microsystems i86pc
    > % Memory size: 384 Megabytes
    > % root@fingon:/root# vmstat 1 3
    > % kthr memory page disk faults
    > cpu
    > % r b w swap free re mf pi po fr de sr cd f0 -- -- in sy cs us
    > sy id
    > % 0 0 0 2251640 193956 7 24 12 0 0 0 10 2 0 0 0 835 175 454 1
    > 2 96
    > % 0 0 0 2252016 189000 7 44 0 0 0 0 0 0 0 0 0 352 133 161 0
    > 2 98
    > % 0 0 0 2252016 189000 0 0 0 0 0 0 0 0 0 0 0 375 111 157 1
    > 1 98
    > % root@fingon:/root#
    >
    > The amount of physical memory installed is printed by prtconf, and the
    > amount of free memory is printed by vmstat (in the memory/free column).
    >
    >> Do you think I would see much increase in speed if I upgraded to 2048
    >> MB physical memory?

    >
    > This depends a lot on what your typical workload is. If you frequently
    > run memory intensive programs, it's very likely to help a lot
    >
    > Best regards,
    > Giorgos


    The memory/free column is what is deceptive here to newbie types. By
    itself, it is a near useless number. Solaris will usually make this low
    over time as disk files are read and unused memory is allocated to disk
    cache - eventually consuming most of it. Solaris uses ALL memory as
    effectively as it can... but that does not mean your our of memory. If you
    used memory/free as an indicator you would almost always be out of memory.

    "vmstat 2 10" and "vmstat -S 2 10" while under load. If pi/po goes nuts,
    then run the one with the -S option checking swap. If so/si show large
    amounts of activity, THEN and only then combined with performance issues
    will memory have a good impact on performance issues. Also, skip the first
    row of output on vmstat --- it often has cumulative results and takes a row
    to get stable.

    Some pi/po is quite normal. In fact, where memory is "right sized" you will
    often see moderate to low activity. si/so is the one you really need to
    look at.

    memory/free is only an indicator of how much ram is available for immediate
    use without page discards or swap. It is NOT an indicator if the system is
    starved for memory. If your on a long running system under load and this is
    always high, you bought a lot more RAM than you needed. Mind you, sales
    types like that one. See my original reply.




  10. Re: Weird memory usage


    "Christopher Tidy" wrote in message
    news:1169932511.224785.250210@k78g2000cwa.googlegr oups.com...
    > Canuck57 wrote:
    >
    >> Unused memory pages and blocks, as well as shared libraries can be placed
    >> in
    >> swap in advance. So if the system immediately needs memory, it can just
    >> discard (fast) in memory pages knowing that when they are needed it can
    >> go
    >> to swap to get them. This is an optimization feature as time is saved in
    >> not having to write (slow) it out on demand as the last moment in time.

    >
    > Thanks. That makes good sense.
    >
    >> If your not sure of vendor recomandations with performance issues, take
    >> their recomendation on consignment, if it doesn't fix the issue leave
    >> yourself an out in the contrract that you don't pay for it.

    >
    > The performance of my system is fine. I was just wondering why it was
    > using any swap at all, and if storing that information in RAM instead
    > would make the system faster. It looks like the information is
    > probably already stored in RAM, then.


    If your just looking purely on swap use, "vmstat -S 2 100", if you see a lot
    of activity under the si/so columns, then swap is being used. You may see
    some activity, especially "so", however high si count on a persistant basis
    does indicate a low memory condition. On systems that start a lot of
    processes, so can be more active as least frequently used pages are swapped
    out in advance. This is normal.

    Myth: paging and swap are bad
    Truth: Paging and swap are normal
    Eval: If swap "in" is excessive and performance poor, add RAM.

    >> If you want to get into doing this yourself, start by reading up on
    >> vmstat,
    >> sar, iostat, nfsstat, trace and others.

    >
    > I'll do that out of interest.
    >
    > Many thanks,
    >
    > Chris
    >




  11. Re: Weird memory usage

    In comp.unix.solaris Canuck57 wrote:
    >> % root@fingon:/root# vmstat 1 3
    >> % kthr memory page disk faults
    >> cpu
    >> % r b w swap free re mf pi po fr de sr cd f0 -- -- in sy cs us
    >> sy id
    >> % 0 0 0 2251640 193956 7 24 12 0 0 0 10 2 0 0 0 835 175 454 1
    >> 2 96
    >> % 0 0 0 2252016 189000 7 44 0 0 0 0 0 0 0 0 0 352 133 161 0
    >> 2 98
    >> % 0 0 0 2252016 189000 0 0 0 0 0 0 0 0 0 0 0 375 111 157 1
    >> 1 98
    >> % root@fingon:/root#
    >>
    >> The amount of physical memory installed is printed by prtconf, and the
    >> amount of free memory is printed by vmstat (in the memory/free column).


    > The memory/free column is what is deceptive here to newbie types. By
    > itself, it is a near useless number. Solaris will usually make this low
    > over time as disk files are read and unused memory is allocated to disk
    > cache - eventually consuming most of it.


    Well, it depends on the OS you're talking about, but generally it is
    useful.

    The situation you mention existed up through Solaris 7. With Solaris 8
    and higher, the free memory does not include the UFS cache, so it should
    be quite representative of actual memory usage by the kernel and
    applications. It should not lower over time simply due to I/O.

    > Solaris uses ALL memory as
    > effectively as it can... but that does not mean your our of memory. If you
    > used memory/free as an indicator you would almost always be out of
    > memory.


    Not any more. The UFS cache does not lower free memory. A low free
    memory in Solaris 8 and higher should be investigated.

    --
    Darren Dunham ddunham@taos.com
    Senior Technical Consultant TAOS http://www.taos.com/
    Got some Dr Pepper? San Francisco, CA bay area
    < This line left intentionally blank to confuse you. >

  12. Re: Weird memory usage

    In comp.unix.solaris Christopher Tidy wrote:
    > Hi all,


    > I have an Ultra 2, dual 400 MHz with 1280 MB RAM, running Solaris 9.
    > Typically it uses 400 MB to 500 MB of physical memory and 200 MB to
    > 250 MB of swap.


    > So my questions are:


    > 1. Why does it use any swap when it has an enormous amount of physical
    > memory free?


    I think you'd need be more explicit with how you're measuring this.

    Can you show the output of 'swap -l' and 'swap -s'?

    > 2. Is there any way to make it use physical memory instead of swap,
    > and would this be a wise thing to do?


    It's probably doing the right thing already.

    > 3. Am I right in thinking that if the machine is never using all its
    > physical memory, it's pointless to add more RAM?


    If you could guarantee that, then yes. However it probably is using all
    the physical memory for disk caches. In some cases, increasing them
    might give a performance benefit.

    --
    Darren Dunham ddunham@taos.com
    Senior Technical Consultant TAOS http://www.taos.com/
    Got some Dr Pepper? San Francisco, CA bay area
    < This line left intentionally blank to confuse you. >

  13. Re: Weird memory usage

    In comp.unix.solaris Canuck57 wrote:
    >> I have an Ultra 2, dual 400 MHz with 1280 MB RAM, running Solaris 9.
    >> Typically it uses 400 MB to 500 MB of physical memory and 200 MB to
    >> 250 MB of swap.
    >>
    >> So my questions are:
    >>
    >> 1. Why does it use any swap when it has an enormous amount of physical
    >> memory free?


    > Unused memory pages and blocks, as well as shared libraries can be placed in
    > swap in advance. So if the system immediately needs memory, it can just
    > discard (fast) in memory pages knowing that when they are needed it can go
    > to swap to get them. This is an optimization feature as time is saved in
    > not having to write (slow) it out on demand as the last moment in time.


    Shared libraries already exist on disk. They would never be paged out
    in the first place. That only happens with anonymous memory. Because
    they can be read from disk, they are always available for purging.

    Even so, pages are not normally paged out much in advance. If there is
    no memory pressure, then the page scanner will not run (sr field in
    vmstat), and nothing moves.

    The only thing that should appear immediately in the swap backing store
    is reserved memory (memory that a process says it will need, but is not
    actually using yet). Because it's possible that the memory will never
    be used, the reservation is made against the backing store.

    --
    Darren Dunham ddunham@taos.com
    Senior Technical Consultant TAOS http://www.taos.com/
    Got some Dr Pepper? San Francisco, CA bay area
    < This line left intentionally blank to confuse you. >

+ Reply to Thread