Regarding calculation of free memory - VxWorks

This is a discussion on Regarding calculation of free memory - VxWorks ; Gurus, I was just wondering what could be the best possible way to calculate free memory while our code is running in SDRAM.I have used vxworks and it uses a approach to calculate the free memory as follows: Fill up ...

+ Reply to Thread
Page 1 of 4 1 2 3 ... LastLast
Results 1 to 20 of 63

Thread: Regarding calculation of free memory

  1. Regarding calculation of free memory

    Gurus,
    I was just wondering what could be the best possible way to calculate
    free memory while our code is running in SDRAM.I have used vxworks and
    it uses a approach to calculate the free memory as follows:
    Fill up the entire stack with 0xeeeee and then when you start using the
    stack,if at all a particular address in memory is used,the value of
    0xeeeee in that location should be overwritten with real data.So if we
    are able to track the first occurance of the pattern 0xeeeee we should
    be able to subtract it from the entire size of memory and make a fair
    assesment of the available free memory.

    How ever I beleive such a algorithm has drawbacks.First drawback which
    I can think is
    1)What is the guarentee that the real data is not ur pattern chosen(In
    vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
    out real size!)?

    2)What should you look for to identify unique pattern?Or other way how
    would you arrive at the pattern word to be used?

    3)Are there any other good techniques for calculating free memory?

    4)We are making software for a consumer electronics appliance.It uses a
    premitive propreitory RTOS which does not give enough tools to find out
    free memory.We are in a forced situation to reduce memory
    consumption,so we have to get a way to calculate free available memory
    first.To add to the complexity,our application uses
    heap,partitions,partition with in partitions.At any point of time we
    have to give the user memory utilised like most RTOS tools
    provide.Since Heap is dynamic how do u arrive at a fair judgment?
    If we wanted to get total free space,we need to calculate free space in
    heap,free space in partition and then arrive at it.

    Are any one aware of any good way to achieve it in the given case of
    RTOS vendor not providing tools for memory calculations?

    Looking farward for all your replys and Advanced thanks for the same.

    Regards,
    s.subbarayan


  2. Re: Regarding calculation of free memory

    > I was just wondering what could be the best possible way to calculate
    > free memory while our code is running in SDRAM.I have used vxworks and
    > it uses a approach to calculate the free memory as follows:
    > Fill up the entire stack with 0xeeeee and then when you start using the
    > stack,if at all a particular address in memory is used,the value of
    > 0xeeeee in that location should be overwritten with real data.So if we
    > are able to track the first occurance of the pattern 0xeeeee we should
    > be able to subtract it from the entire size of memory and make a fair
    > assesment of the available free memory.


    It sounds like your 0xeeeee method you outline is perfectly suited for
    your purposes. Even if 0xeeeee is actually part of your real data that
    happens to be in the stack, consider that it's highly unlikely that
    0xeeeee will fill your stack to the extent that your stack usage figure
    is way off. The point is, even if 0xeeeee is real stack data, the
    information you get from hunting for unused stack will be close enough.
    Get your usage total and add some margin for safety and you're done.

    > How ever I beleive such a algorithm has drawbacks.First drawback which
    > I can think is
    > 1)What is the guarentee that the real data is not ur pattern chosen(In
    > vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
    > out real size!)?


    That's application specific. Look at your operating scenarios. Ask
    yourself if the pattern you've selected is likely to be real data or
    not.

    > 2)What should you look for to identify unique pattern?Or other way how
    > would you arrive at the pattern word to be used?


    See #1 reply.

    > 3)Are there any other good techniques for calculating free memory?


    Don't know but... This method is commonly used and usually works very
    well. A good white-box test designed to stress the stack usage will
    give you the real info you need to accurately size your RAM.

    > 4)We are making software for a consumer electronics appliance.It uses a
    > premitive propreitory RTOS which does not give enough tools to find out
    > free memory.We are in a forced situation to reduce memory
    > consumption,so we have to get a way to calculate free available memory
    > first.To add to the complexity,our application uses
    > heap,partitions,partition with in partitions.At any point of time we
    > have to give the user memory utilised like most RTOS tools
    > provide.Since Heap is dynamic how do u arrive at a fair judgment?
    > If we wanted to get total free space,we need to calculate free space in
    > heap,free space in partition and then arrive at it.


    IMO, it's like gambling when you size how much stack margin you want to
    add. If you believe your SW tests are "NASA grade" (nearly
    exhaustive), you will not need to add much stack margin. If you don't
    do SW testing, it's a roll of the dice. You probably fit in the middle
    of the two extremes.

    > Are any one aware of any good way to achieve it in the given case of
    > RTOS vendor not providing tools for memory calculations?


    Good way is the way you outlined for run-time stack checking. Maybe
    some CPUs feature some sort of built-in high water mark stack pointer
    register. That would be nice but I don't know if it exists.

    JJS


  3. Re: Regarding calculation of free memory

    "ssubbarayan" writes:

    > Gurus,
    > I was just wondering what could be the best possible way to calculate
    > free memory while our code is running in SDRAM.I have used vxworks and
    > it uses a approach to calculate the free memory as follows:
    > Fill up the entire stack with 0xeeeee and then when you start using the
    > stack,if at all a particular address in memory is used,the value of
    > 0xeeeee in that location should be overwritten with real data.So if we
    > are able to track the first occurance of the pattern 0xeeeee we should
    > be able to subtract it from the entire size of memory and make a fair
    > assesment of the available free memory.
    >
    > How ever I beleive such a algorithm has drawbacks.First drawback which
    > I can think is
    > 1)What is the guarentee that the real data is not ur pattern chosen(In
    > vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
    > out real size!)?
    >
    > 2)What should you look for to identify unique pattern?Or other way how
    > would you arrive at the pattern word to be used?




    I believe the traditional value is "0xDEADBEEF"


    --

    John Devereux

  4. Re: Regarding calculation of free memory

    "John Devereux" wrote in message
    news:87d5mtb06d.fsf@cordelia.devereux.me.uk...
    > "ssubbarayan" writes:
    >
    > > Gurus,
    > > I was just wondering what could be the best possible way to calculate
    > > free memory while our code is running in SDRAM.I have used vxworks and
    > > it uses a approach to calculate the free memory as follows:
    > > Fill up the entire stack with 0xeeeee and then when you start using the
    > > stack,if at all a particular address in memory is used,the value of
    > > 0xeeeee in that location should be overwritten with real data.So if we
    > > are able to track the first occurance of the pattern 0xeeeee we should
    > > be able to subtract it from the entire size of memory and make a fair
    > > assesment of the available free memory.
    > >
    > > How ever I beleive such a algorithm has drawbacks.First drawback which
    > > I can think is
    > > 1)What is the guarentee that the real data is not ur pattern chosen(In
    > > vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
    > > out real size!)?
    > >
    > > 2)What should you look for to identify unique pattern?Or other way how
    > > would you arrive at the pattern word to be used?

    >
    >
    >
    > I believe the traditional value is "0xDEADBEEF"
    >


    There is also a slight advantage to using this value because it is odd. On
    many processors using an odd address for larger than byte size data will
    thow an exception. As a result uninitialised data on the stack used as
    pointers can be caught more often.

    The bigest problem I have though is that I am a vegetarian so I prefer
    0xFACEFEED.

    Peter



  5. Re: Regarding calculation of free memory

    John Devereux writes:

    > "ssubbarayan" writes:
    >
    > > Gurus,
    > > I was just wondering what could be the best possible way to calculate
    > > free memory while our code is running in SDRAM.I have used vxworks and
    > > it uses a approach to calculate the free memory as follows:
    > > Fill up the entire stack with 0xeeeee and then when you start using the
    > > stack,if at all a particular address in memory is used,the value of
    > > 0xeeeee in that location should be overwritten with real data.So if we
    > > are able to track the first occurance of the pattern 0xeeeee we should
    > > be able to subtract it from the entire size of memory and make a fair
    > > assesment of the available free memory.
    > >
    > > How ever I beleive such a algorithm has drawbacks.First drawback which
    > > I can think is
    > > 1)What is the guarentee that the real data is not ur pattern chosen(In
    > > vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
    > > out real size!)?
    > >
    > > 2)What should you look for to identify unique pattern?Or other way how
    > > would you arrive at the pattern word to be used?

    >
    >
    >
    > I believe the traditional value is "0xDEADBEEF"


    Which has the same problem. But really, if you pick a pattern that's
    unlikely to turn up by coincidence, you'll come close enough to the
    right answer for all practical purposes. Don't use, say, 00000000,
    but beyond that it doesn't much matter.
    --
    Joseph J. Pfeiffer, Jr., Ph.D. Phone -- (505) 646-1605
    Department of Computer Science FAX -- (505) 646-1002
    New Mexico State University http://www.cs.nmsu.edu/~pfeiffer
    skype: jjpfeifferjr

  6. Re: Regarding calculation of free memory

    "John Devereux" wrote in message
    news:87d5mtb06d.fsf@cordelia.devereux.me.uk...
    > "ssubbarayan" writes:
    >
    >> Gurus,
    >> I was just wondering what could be the best possible way to calculate
    >> free memory while our code is running in SDRAM.I have used vxworks and
    >> it uses a approach to calculate the free memory as follows:
    >> Fill up the entire stack with 0xeeeee and then when you start using the
    >> stack,if at all a particular address in memory is used,the value of
    >> 0xeeeee in that location should be overwritten with real data.So if we
    >> are able to track the first occurance of the pattern 0xeeeee we should
    >> be able to subtract it from the entire size of memory and make a fair
    >> assesment of the available free memory.
    >>
    >> How ever I beleive such a algorithm has drawbacks.First drawback which
    >> I can think is
    >> 1)What is the guarentee that the real data is not ur pattern chosen(In
    >> vxworks case,its 0xeeeee,if 0xeeeee itself is real data,we cant find
    >> out real size!)?
    >>
    >> 2)What should you look for to identify unique pattern?Or other way how
    >> would you arrive at the pattern word to be used?

    >
    >
    >
    > I believe the traditional value is "0xDEADBEEF"



    Seen in various 16 bit codes:
    #define STACKPAT 0x55aa // Stack fill value for high water mark
    checking.

    --

    ... Hank

    http://home.earthlink.net/~horedson
    http://home.earthlink.net/~w0rli



  7. Re: Regarding calculation of free memory

    "ssubbarayan" wrote in message
    news:1127886917.777311.119870@z14g2000cwz.googlegr oups.com...
    > Gurus,
    > I was just wondering what could be the best possible way to calculate
    > free memory while our code is running in SDRAM.




    > 4)We are making software for a consumer electronics appliance.It uses a
    > premitive propreitory RTOS which does not give enough tools to find out
    > free memory.We are in a forced situation to reduce memory
    > consumption,so we have to get a way to calculate free available memory
    > first.To add to the complexity,our application uses
    > heap,partitions,partition with in partitions.At any point of time we
    > have to give the user memory utilised like most RTOS tools
    > provide.Since Heap is dynamic how do u arrive at a fair judgment?




    If you want your appliance to be totally reliable, avoid using a heap at
    all. Free memory calculation then becomes rather simple [1]. Logic: heaps
    are (usually) non-deterministic and *will* cause eventual malloc failures
    due to fragmentation [2].

    [1] Excluding stack use, but that's not too hard either.
    [2] Unless you allocate all memory at startup; a common trick. But then the
    linker could do the same...



    Steve
    http://www.fivetrees.com



  8. Re: Regarding calculation of free memory

    In article ,
    Steve at fivetrees wrote:
    >
    >
    >
    >If you want your appliance to be totally reliable, avoid using a heap at
    >all. Free memory calculation then becomes rather simple [1]. Logic: heaps
    >are (usually) non-deterministic and *will* cause eventual malloc failures
    >due to fragmentation [2].
    >
    >[1] Excluding stack use, but that's not too hard either.
    >[2] Unless you allocate all memory at startup; a common trick. But then the
    >linker could do the same...


    < extreme_pedant_mode=ON >

    Not so. In Fortran 77, it was possible to allocate all memory statically,
    but it isn't generally possible. It can't be done in Fortran 90 or C.

    Also, of the memory management schemes that are lumped under the term of
    'heap' ones, there are several that have deterministic behaviour and do
    not suffer from fragmentation. The simplest one is if all allocations
    are of the same size, and chains are used instead of arrays.

    The whole area of memory management schemes is sadly neglected, and
    much of garbage collection work is based on unrealistic assumptions,
    so much of what is currently believed isn't so.

    < extreme_pedant_mode=OFF >

    >



    Regards,
    Nick Maclaren.

  9. Re: Regarding calculation of free memory

    "Nick Maclaren" wrote in message
    news:dhobjl$e2v$1@gemini.csx.cam.ac.uk...
    > < extreme_pedant_mode=ON >
    >
    > Not so. In Fortran 77, it was possible to allocate all memory statically,
    > but it isn't generally possible. It can't be done in Fortran 90 or C.




    It *can* be done in C - by avoiding malloc .



    > Also, of the memory management schemes that are lumped under the term of
    > 'heap' ones, there are several that have deterministic behaviour and do
    > not suffer from fragmentation. The simplest one is if all allocations
    > are of the same size, and chains are used instead of arrays.


    Agreed. I've used techniques such as these (effectively an application-level
    memory manager) in certain situations where extreme reliability was
    required, but the platform insisted on the use of its memory manager...
    ...

    > The whole area of memory management schemes is sadly neglected, and
    > much of garbage collection work is based on unrealistic assumptions,
    > so much of what is currently believed isn't so.


    Completely agreed. I've often fantasized about a hardware memory manager
    that remaps blocks in such a way that fragmentation cannot occur. OTOH, I've
    also fantasized that one day, a majority will write good code that doesn't
    leak memory and traps malloc errors properly...

    Steve
    http://www.fivetrees.com



  10. Re: Regarding calculation of free memory

    Nick Maclaren wrote:
    > Also, of the memory management schemes that are lumped under the term of
    > 'heap' ones, there are several that have deterministic behaviour and do
    > not suffer from fragmentation. The simplest one is if all allocations
    > are of the same size, and chains are used instead of arrays.
    >
    > The whole area of memory management schemes is sadly neglected, and
    > much of garbage collection work is based on unrealistic assumptions,
    > so much of what is currently believed isn't so.


    That's a worthlessly vague statement. As far as I can see, the problem isn't
    lack of research; it's lack of application of that research in the most
    commonly used systems.

    --
    David Hopwood

  11. Re: Regarding calculation of free memory

    In article <5pCdnWbp1rsoKaLeRVnyuw@pipex.net>,
    Steve at fivetrees wrote:
    >"Nick Maclaren" wrote in message
    >news:dhobjl$e2v$1@gemini.csx.cam.ac.uk...
    >> < extreme_pedant_mode=ON >
    >>
    >> Not so. In Fortran 77, it was possible to allocate all memory statically,
    >> but it isn't generally possible. It can't be done in Fortran 90 or C.

    >
    >
    >
    >It *can* be done in C - by avoiding malloc .
    >
    >


    I will skip the mode nesting, as things are getting ridiculous :-)

    Problem one: in C, there is no memory management beyond stack scoped
    except by using malloc. Any algorithm that requires more than that
    can't be done if you don't use malloc.

    Problem two: there are many library facilities that use malloc either
    explicitly or implicitly - including almost all I/O - you would have
    to avoid them, too.


    Regards,
    Nick Maclaren.

  12. Re: Regarding calculation of free memory

    In article ,
    David Hopwood wrote:
    >Nick Maclaren wrote:
    >> Also, of the memory management schemes that are lumped under the term of
    >> 'heap' ones, there are several that have deterministic behaviour and do
    >> not suffer from fragmentation. The simplest one is if all allocations
    >> are of the same size, and chains are used instead of arrays.
    >>
    >> The whole area of memory management schemes is sadly neglected, and
    >> much of garbage collection work is based on unrealistic assumptions,
    >> so much of what is currently believed isn't so.

    >
    >That's a worthlessly vague statement. As far as I can see, the problem isn't
    >lack of research; it's lack of application of that research in the most
    >commonly used systems.


    On the contrary - one of the main reasons that it hasn't been applied
    is the impracticality of the research. That was the case in 1965,
    and is the case today. A fairly good rule when you find that real
    engineers (and there are some real software engineers, even today,
    and used to be more) are ignoring the wonderful results that come
    out of 'scientific' research is to ask whether the research is
    actually addressing the whole of the problem that the engineers
    are faced with.


    Regards,
    Nick Maclaren.

  13. Re: Regarding calculation of free memory

    "Nick Maclaren" wrote in message
    news:dhp6mi$7u3$1@gemini.csx.cam.ac.uk...
    >
    > Problem two: there are many library facilities that use malloc either
    > explicitly or implicitly - including almost all I/O - you would have
    > to avoid them, too.


    I'm posting this from c.a.e - embedded (or OS-less) is my main area. One
    tends not to need such I/O functions therein.

    You're right, though - it certainly pays to be aware of the platform
    requirements of any such library function.

    Steve
    http://www.fivetrees.com



  14. Re: Regarding calculation of free memory

    Nick Maclaren wrote:
    > David Hopwood wrote:
    >>Nick Maclaren wrote:
    >>
    >>>Also, of the memory management schemes that are lumped under the term of
    >>>'heap' ones, there are several that have deterministic behaviour and do
    >>>not suffer from fragmentation. The simplest one is if all allocations
    >>>are of the same size, and chains are used instead of arrays.
    >>>
    >>>The whole area of memory management schemes is sadly neglected, and
    >>>much of garbage collection work is based on unrealistic assumptions,
    >>>so much of what is currently believed isn't so.

    >>
    >>That's a worthlessly vague statement. As far as I can see, the problem isn't
    >>lack of research; it's lack of application of that research in the most
    >>commonly used systems.

    >
    > On the contrary - one of the main reasons that it hasn't been applied
    > is the impracticality of the research. That was the case in 1965,
    > and is the case today. A fairly good rule when you find that real
    > engineers (and there are some real software engineers, even today,
    > and used to be more) are ignoring the wonderful results that come
    > out of 'scientific' research is to ask whether the research is
    > actually addressing the whole of the problem that the engineers
    > are faced with.


    Does every research paper on memory management describe something practical? No.
    Does the research that has been done on memory management collectively address
    the problems that arise in real systems in a practical way, for those engineers
    that are prepared to do a bit of searching to find the papers relevant to those
    problems? Yes, it does. If you think otherwise, then be specific about the
    problems you think are not addressed.

    --
    David Hopwood

  15. Re: Regarding calculation of free memory

    "Nick Maclaren" wrote in message
    news:dhp6mi$7u3$1@gemini.csx.cam.ac.uk...
    > In article <5pCdnWbp1rsoKaLeRVnyuw@pipex.net>,
    > Steve at fivetrees wrote:
    >>"Nick Maclaren" wrote in message
    >>news:dhobjl$e2v$1@gemini.csx.cam.ac.uk...
    >>> < extreme_pedant_mode=ON >
    >>>
    >>> Not so. In Fortran 77, it was possible to allocate all memory
    >>> statically,
    >>> but it isn't generally possible. It can't be done in Fortran 90 or C.

    >>
    >>
    >>
    >>It *can* be done in C - by avoiding malloc .
    >>
    >>

    >
    > I will skip the mode nesting, as things are getting ridiculous :-)
    >
    > Problem one: in C, there is no memory management beyond stack scoped
    > except by using malloc. Any algorithm that requires more than that
    > can't be done if you don't use malloc.


    sbrk()?

    > Problem two: there are many library facilities that use malloc either
    > explicitly or implicitly - including almost all I/O - you would have
    > to avoid them, too.


    Provide your own malloc() so the I/O that needs it will get it
    from your own memory pool. Since you are doing the I/O,
    you can provide optimal buffering.

    --

    ... Hank

    http://home.earthlink.net/~horedson
    http://home.earthlink.net/~w0rli



  16. Re: Regarding calculation of free memory

    In article ,
    Steve at fivetrees wrote:
    >"Nick Maclaren" wrote in message
    >news:dhp6mi$7u3$1@gemini.csx.cam.ac.uk...
    >>
    >> Problem two: there are many library facilities that use malloc either
    >> explicitly or implicitly - including almost all I/O - you would have
    >> to avoid them, too.

    >
    >I'm posting this from c.a.e - embedded (or OS-less) is my main area. One
    >tends not to need such I/O functions therein.


    And still less the internationalisation and wide character
    facilities :-)

    My experience of that area is that the worst problem is finding out
    when a compiler has chosen to implement some basic facility (e.g.
    a mathematical function) in a way that drags in completely
    unrelated junk from the library. But I have been usually trying
    to used hosted compilers in an embedded fashion - I would expect
    compilers designed for embedded use to be better.


    Regards,
    Nick Maclaren.

  17. Re: Regarding calculation of free memory

    In article ,
    David Hopwood wrote:
    >
    >Does every research paper on memory management describe something practical? No.
    >Does the research that has been done on memory management collectively address
    >the problems that arise in real systems in a practical way, for those engineers
    >that are prepared to do a bit of searching to find the papers relevant to those
    >problems? Yes, it does. If you think otherwise, then be specific about the
    >problems you think are not addressed.


    I have, many times. You ignored the issues every time.


    Regards,
    Nick Maclaren.

  18. Re: Regarding calculation of free memory

    In article ,
    Hank Oredson wrote:
    >>
    >> Problem one: in C, there is no memory management beyond stack scoped
    >> except by using malloc. Any algorithm that requires more than that
    >> can't be done if you don't use malloc.

    >
    >sbrk()?


    Not in C - in fact, I can't think of any standard that it IS in,
    because it makes such extremely horrible assumptions about the
    implementation.

    >> Problem two: there are many library facilities that use malloc either
    >> explicitly or implicitly - including almost all I/O - you would have
    >> to avoid them, too.

    >
    >Provide your own malloc() so the I/O that needs it will get it
    >from your own memory pool. Since you are doing the I/O,
    >you can provide optimal buffering.


    No. Replacing malloc is not supported in C, and often breaks
    the nastier vendor and third-party libraries. The reason is that
    the suite of malloc-related functions is not independent, and they
    often use non-standard interfaces.

    Yes, I have done it, over many years - but it isn't something that
    I recommend to users. Or can :-(


    Regards,
    Nick Maclaren.

  19. Re: Regarding calculation of free memory

    In comp.arch Nick Maclaren wrote:
    >
    > No. Replacing malloc is not supported in C, and often breaks
    > the nastier vendor and third-party libraries. The reason is that
    > the suite of malloc-related functions is not independent, and they
    > often use non-standard interfaces.


    There are still systems where you can reasonably do so - shipping
    more than one possible malloc to use is a good hint for example.

    >
    > Yes, I have done it, over many years - but it isn't something that
    > I recommend to users. Or can :-(
    >
    >
    > Regards,
    > Nick Maclaren.


    --
    Sander

    +++ Out of cheese error +++

  20. Re: Regarding calculation of free memory


    In article <1128335888.327425@haldjas.folklore.ee>,
    Sander Vesik writes:
    |> In comp.arch Nick Maclaren wrote:
    |> >
    |> > No. Replacing malloc is not supported in C, and often breaks
    |> > the nastier vendor and third-party libraries. The reason is that
    |> > the suite of malloc-related functions is not independent, and they
    |> > often use non-standard interfaces.
    |>
    |> There are still systems where you can reasonably do so - shipping
    |> more than one possible malloc to use is a good hint for example.

    It's a hint that the VENDOR can do it - but not that the application
    programmer can. Yes, an experienced hacker can, which was the basis
    of my remark that I can, but I can't advise users to.

    Also, are you aware of how often third-party libraries demand a
    particular one of those mallocs, either in their release notes or
    because they fall over with others?


    Regards,
    Nick Maclaren.

+ Reply to Thread
Page 1 of 4 1 2 3 ... LastLast