how to access the memory information using c++ on unix - Unix

This is a discussion on how to access the memory information using c++ on unix - Unix ; hi, i have written an sorting application which uses lot of memory to sort huge files. On window we i can GlobalMemoryStatusEx() to get the available free memory status and according to that i am use the memory in the ...

+ Reply to Thread
Results 1 to 13 of 13

Thread: how to access the memory information using c++ on unix

  1. how to access the memory information using c++ on unix

    hi,

    i have written an sorting application which uses lot of memory to sort
    huge files.
    On window we i can GlobalMemoryStatusEx() to get the available free
    memory status and according to that i am use the memory in the program

    but on UNIX i have no idea how to get the memory status..ie available
    free RAM/swap.

    Can we use BOOST library?

    Can somebody please help me?

    thanks in advance.

  2. Re: how to access the memory information using c++ on unix

    pp writes:
    > i have written an sorting application which uses lot of memory to sort
    > huge files. On window we i can GlobalMemoryStatusEx() to get the
    > available free memory status and according to that i am use the
    > memory in the program


    Even Windows supports multiprogramming these days, making the
    assumption that whatever memory may be available is available solely
    for the benefit of a single process running on the system without any
    knowledge of its other activities as wrong as it always[*] was for
    UNIX(*).

    [...]

    > Can somebody please help me?




  3. Re: how to access the memory information using c++ on unix

    pp writes:
    > i have written an sorting application which uses lot of memory to sort
    > huge files. On window we i can GlobalMemoryStatusEx() to get the
    > available free memory status and according to that i am use the
    > memory in the program


    Even Windows supports multiprogramming these days, making the
    assumption that whatever memory may be available is available solely
    for the benefit of a single process running on the system without any
    knowledge of its other activities as wrong as it always[*] was for
    UNIX(*).
    [*] Since the early PDP-7 days.

    [...]

    > Can somebody please help me?




  4. Re: how to access the memory information using c++ on unix

    "pp" wrote in message
    news:9cf45c62-254a-4f3a-ba8c-5a74a894af51@w34g2000hsg.googlegroups.com...
    > i have written an sorting application which uses lot of memory to sort
    > huge files.
    > On window we i can GlobalMemoryStatusEx() to get the available free
    > memory status and according to that i am use the memory in the program
    > but on UNIX i have no idea how to get the memory status..ie available
    > free RAM/swap.


    There is no standard method, probably because it is largely meaningless for
    an application for the reason Rainer gave. However, there are
    platform-specific methods, eg parsing /proc/meminfo on Linux.

    The best solution IMHO is to allow the amount of memory to be specified, and
    also take into account getrlimit(RLIMIT_DATA). This information could be
    supplemented with a value derived from using a platform-specific method as
    above.

    Alex



  5. Re: how to access the memory information using c++ on unix

    "Alex Fraser" writes:
    > "pp" wrote in message
    > news:9cf45c62-254a-4f3a-ba8c-5a74a894af51@w34g2000hsg.googlegroups.com...
    >> i have written an sorting application which uses lot of memory to sort
    >> huge files.
    >> On window we i can GlobalMemoryStatusEx() to get the available free
    >> memory status and according to that i am use the memory in the program
    >> but on UNIX i have no idea how to get the memory status..ie available
    >> free RAM/swap.

    >
    > There is no standard method, probably because it is largely meaningless for
    > an application for the reason Rainer gave. However, there are
    > platform-specific methods, eg parsing /proc/meminfo on Linux.


    The problem with any of these methods is that this information was
    accurate at the time the kernel updated the corresponding variable for
    the last time and the actual value at the time the application is
    making a descision can (and very likely, will) be completely different
    because this is a parameter of the system which constantly changes
    while it is in use.

    THIS CANNOT BE DONE. PERIOD.

    I would like to have used capital capitals for cannot. Accomplishing
    this task is technically impossible from userspace. That Windows
    provides an API-call for the benefit of people who cannot understand
    this or simply refuse to accept it does not make any difference.

  6. Re: how to access the memory information using c++ on unix

    On Nov 21, 3:13 am, pp wrote:
    > hi,
    >
    > i have written an sorting application which uses lot of memory to sort
    > huge files.
    > On window we i can GlobalMemoryStatusEx() to get the available free
    > memory status and according to that i am use the memory in the program
    >
    > but on UNIX i have no idea how to get the memory status..ie available
    > free RAM/swap.


    For process or O/S? For process you can parse /proc/self/statm (see
    'man proc').

    >
    > Can we use BOOST library?
    >
    > Can somebody please help me?
    >
    > thanks in advance.



  7. Re: how to access the memory information using c++ on unix

    On Nov 21, 6:08 pm, toby wrote:
    > On Nov 21, 3:13 am, pp wrote:
    >
    > > hi,

    >
    > > i have written an sorting application which uses lot of memory to sort
    > > huge files.
    > > On window we i can GlobalMemoryStatusEx() to get the available free
    > > memory status and according to that i am use the memory in the program

    >
    > > but on UNIX i have no idea how to get the memory status..ie available
    > > free RAM/swap.

    >
    > For process or O/S? For process you can parse /proc/self/statm (see
    > 'man proc').
    >
    >
    >
    > > Can we use BOOST library?

    >
    > > Can somebody please help me?

    >
    > > thanks in advance.


    Thank you all for your suggestions.

    I am trying to get the available free memory (just ram) which in
    currently not used by any processes,.
    thats ok if its runtime dependent.

    the API also clearly says that the very next call may give different
    results.
    but still this info is useful because if we know approximate amount of
    available memory than we can at least run our applications faster by
    utilizing some 30-40% of the memory.

    many popular application like Eclipe can be customized like this...so
    that they can utilize up to n% of available memory.

    i am able to get all the info on windows. but i am facing problem on
    unix.
    so from this discussion can we conclude that there is no reliable
    method?

    thank you.

  8. Re: how to access the memory information using c++ on unix

    pp writes:

    [...]

    > I am trying to get the available free memory (just ram) which in
    > currently not used by any processes,.


    There is not way to determine this information except stopping the
    system completely, including interrupt processing, and examining the
    state of the respective kernel variables using a second computer.

    > the API also clearly says that the very next call may give different
    > results.


    But so far, your have managed to avoid understanding it.

    > but still this info is useful


    It is useful to display statistics about past system ressource usage.

    > because if we know approximate amount of available memory than we
    > can at least run our applications faster by utilizing some 30-40% of
    > the memory.


    But you'll never know the amount of available memory at the time you
    would need this information.

    > many popular application like Eclipe can be customized like this...


    Understanding systems which dynamically change over the course of time
    is beyond the mental capabilities of a lot of
    people. Right. Ocassionally, if they drive bicycles, they run over
    pedestrians because they could not imagine that these may be moving
    indepedently ...

    > i am able to get all the info on windows.


    No, you are not able to get this information because this information
    cannot be provided.

    > but i am facing problem on unix.


    You are facing the problem whenever you look into a mirror.

  9. Re: how to access the memory information using c++ on unix

    pp wrote:
    > On Nov 21, 6:08 pm, toby wrote:
    >> On Nov 21, 3:13 am, pp wrote:
    >>
    >>> hi,
    >>> i have written an sorting application which uses lot of memory to sort
    >>> huge files.
    >>> On window we i can GlobalMemoryStatusEx() to get the available free
    >>> memory status and according to that i am use the memory in the program
    >>> but on UNIX i have no idea how to get the memory status..ie available
    >>> free RAM/swap.

    >> For process or O/S? For process you can parse /proc/self/statm (see
    >> 'man proc').
    >>
    >>
    >>
    >>> Can we use BOOST library?
    >>> Can somebody please help me?
    >>> thanks in advance.

    >
    > Thank you all for your suggestions.
    >
    > I am trying to get the available free memory (just ram) which in
    > currently not used by any processes,.
    > thats ok if its runtime dependent.
    >
    > the API also clearly says that the very next call may give different
    > results.
    > but still this info is useful because if we know approximate amount of
    > available memory than we can at least run our applications faster by
    > utilizing some 30-40% of the memory.
    >
    > many popular application like Eclipe can be customized like this...so
    > that they can utilize up to n% of available memory.
    >
    > i am able to get all the info on windows. but i am facing problem on
    > unix.
    > so from this discussion can we conclude that there is no reliable
    > method?


    For Solaris you can query _SC_AVPHYS_PAGES with sysconf(),
    and it wouldn't surprise me to learn that other Unices have
    something quite similar. But in addition to the transitory
    nature of the returned value, note also that different Unix
    kernels manage memory differently and have different notions
    of what "free" means; you may need to interpret the result
    differently on different systems. Also, the fact that some
    amount of memory is "free" does not mean that your program
    will actually be allowed to use that much.

    --
    Eric Sosman
    esosman@ieee-dot-org.invalid

  10. Re: how to access the memory information using c++ on unix

    On Nov 23, 7:02 am, pp wrote:

    > Thank you all for your suggestions.
    >
    > I am trying to get the available free memory (just ram) which in
    > currently not used by any processes,.
    > thats ok if its runtime dependent.


    That's the wrong question. What you want to know is how much memory
    the user would like to use that you are not yet using. The system
    doesn't know this directly, but the user does.

    > the API also clearly says that the very next call may give different
    > results.
    > but still this info is useful because if we know approximate amount of
    > available memory than we can at least run our applications faster by
    > utilizing some 30-40% of the memory.


    Unless the person is about to run some other application that needs
    90% of that memory. Again, you are asking the wrong question of the
    wrong thing.

    > many popular application like Eclipe can be customized like this...so
    > that they can utilize up to n% of available memory.


    I don't think that's true, because as stated it makes no sense. For
    one thing, the definition of "available" is highly problemmatic. For
    one thing, when do you measure that? For another thing, what happens
    when your own memory usage reduces the available memory? How does that
    figure into your percentage?

    > i am able to get all the info on windows. but i am facing problem on
    > unix.
    > so from this discussion can we conclude that there is no reliable
    > method?


    There are plenty of reliable methods. A very, very good one was
    already suggested in this thread -- have the user set a resource limit
    on the process, and have your program check its resource limits.

    You can read the soft limit, then raise it to the hard limit. Keep
    track of the original soft limit and try to use only that much memory.

    DS

  11. Re: how to access the memory information using c++ on unix

    >I am trying to get the available free memory (just ram) which in
    >currently not used by any processes,.
    >thats ok if its runtime dependent.
    >
    >the API also clearly says that the very next call may give different
    >results.
    >but still this info is useful because if we know approximate amount of
    >available memory than we can at least run our applications faster by
    >utilizing some 30-40% of the memory.


    It is exactly this sort of behavior that causes Unix systems to
    have resource limits (and causes administrators to ban some
    applications). And what makes you think allocating 30-40% of
    available (whatever the hell that means) memory will speed up your
    application (especially without slowing down the rest of the system)?
    You could be causing it to page, page, page.

    >many popular application like Eclipe can be customized like this...so
    >that they can utilize up to n% of available memory.


    From the point of view of some operating systems, if a block of
    memory has the contents of a file block in it, it's not unused,
    it's cache, just in case someone wants that file block again. Hardly
    any memory is "free". But if you try to allocate it, you might get
    that block anyway. So what does "available" mean?

    >i am able to get all the info on windows. but i am facing problem on
    >unix.
    >so from this discussion can we conclude that there is no reliable
    >method?


    You shouldn't be using that method even if there was a reliable way
    to get the info. I also think you shouldn't be using that method
    on Windows if the system in question is used as multi-user (remote
    desktop).


  12. Re: how to access the memory information using c++ on unix

    Dnia Fri, 23 Nov 2007 07:02:37 -0800 (PST), pp napisał(a):

    > I am trying to get the available free memory (just ram) which in
    > currently not used by any processes,.
    > thats ok if its runtime dependent.


    On Linux you can use libproc:

    -----freemem.cpp------
    #include
    #include

    int main(void)
    {
    meminfo();
    cout << kb_main_free << endl;
    return 0;
    }
    -------------

    $ g++ freemem.cpp /lib/libproc-3.2.7.so
    $ ./a.out
    604836
    $

    The size is given in kilobytes.

    Even if it's what you are searching for, I bet you don't have
    libproc header files on your system. You can get them from procps
    package (in my case it's procps-3.2.7; just refer to your libproc
    version). IIRC, you'll find procps here: http://procps.sf.net

    Regards,
    Wojtek Walczak

  13. Re: how to access the memory information using c++ on unix

    Wojtek Walczak writes:
    > Dnia Fri, 23 Nov 2007 07:02:37 -0800 (PST), pp napisał(a):
    >
    >> I am trying to get the available free memory (just ram) which in
    >> currently not used by any processes,.
    >> thats ok if its runtime dependent.

    >
    > On Linux you can use libproc:


    No, you cannot 'use libproc on Linux' for that.

    The code below is part of a network driver for Intel IX420
    NPEs I have written for Linux 2.4:

    static void do_refill(struct mac_device *mdev)
    {
    struct qm_queue *q;
    struct sk_buff *skb;
    unsigned want, was_empty;

    q = mdev->skb_q;
    do {
    down(&mdev->skb_lock);
    want = queue_size(q) - queue_len(q);
    p_debug("%s: want %u", __func__, want);

    while (want) {
    up(&mdev->skb_lock);
    --want;

    while (!(skb = do_alloc_skb())) {
    set_current_state(TASK_INTERRUPTIBLE);
    schedule_timeout(1);
    }
    p_debug("%s: fresh skb @ %p (data %p)", __func__,
    skb, skb->data);

    down(&mdev->skb_lock);
    was_empty = queue_empty(q);
    queue_put_entry(q, (u32)skb);
    if (was_empty) wake_up(&mdev->refill_wq);
    }

    up(&mdev->skb_lock);
    } while (queue_nearly_empty(q));

    queue_ack_irq(q);
    queue_enable_irq(q);
    }
    [as such, it is GPL'ed, of course]

    This dynamically allocates struct sk_buffs from 'free memory' and is
    scheduled (ultimatively) in response to RX interrupts coming from the
    hardware, ie depending on packets arriving from the network. It runs
    from an own kernel thread, which means that the amount of 'free
    memory' can (for this example) change dramatically while your toy
    program, which has been interrupted by the RX interrupt, has not yet
    gotten around to flush the I/O buffer associated with cout which contains
    the now incorrect 'free memory' value formatted as ASCII string.

    And it is not just evil me doing memory allocations in response to
    device interrupts but ALL Linux network drivers and a large amount of
    other drivers do so, too.

    And have you per chance ever seen the output of ps faux on a system
    where ps has a BSD-compatible syntax? These are all other processes
    running on the same system, happily allocating memory as they see fit
    and not caring the least about poor programmers trying to make a stone
    swim because their simply MUST be a way and The World just cannot be
    THAT mean, can it?

    "My second remark is that our intellectual powers are rather geared to
    master static relations and that our powers to visualize processes
    evolving in time are relatively poorly developed."

    For a lot of people, this apparently means that they simply don't
    believe that anything could ever 'evolve over time' ...

+ Reply to Thread