Dynamic memory allocation and mmap - Linux

This is a discussion on Dynamic memory allocation and mmap - Linux ; My program runs under linux Kernel 2.6.21.5 and use glibc 2.5. My computer has a RAM of 12 Gbytes. I believed that, using a 32 bits OS, I could use as far as 4 Gbytes. When it is launched, my ...

+ Reply to Thread
Results 1 to 3 of 3

Thread: Dynamic memory allocation and mmap

  1. Dynamic memory allocation and mmap

    My program runs under linux Kernel 2.6.21.5 and use glibc 2.5.
    My computer has a RAM of 12 Gbytes.
    I believed that, using a 32 bits OS, I could use as far as 4 Gbytes.
    When it is launched, my program allocates dynamic memory.
    But, when a little more thean 2GBytes are allocated,
    my malloc invocation fails.

    Of course, I know that the total memory comprises
    the data segment and the stack.
    Maybe, the stack size + the data segement size alltogether reaches 4 GB.
    But that would mean that the stack size is about 2GBytes.
    That seems huge indeed.


    In order to investigate for other explanations,
    I used the mallinfo function that gives statistics about the heap.
    During the execution of my programm, i logged regularly
    the values returned by this function.

    Here is what I got just before the malloc default :
    Statistiques heap : arena 2349433528 ordblks 714 hblks 3 hblkhd 954368
    Statistiques heap : arena 2349433528 ordblks 776 hblks 3 hblkhd 954368
    Statistiques heap : arena 2349433528 ordblks 3102 hblks 17 hblkhd 2617344
    Statistiques heap : arena 2349433528 ordblks 1667 hblks 10 hblkhd 1785856
    Statistiques heap : arena 2349433528 ordblks 449 hblks 39 hblkhd 5230592

    The last line is just after the malloc default.
    I noticed something strange with the hblks value
    that gives the number of chunks allocated by mmap.
    Since the beginning of the execution, hblks remains at 3 chunks.
    And just before the memory default, it started to increase.
    I read that large chunks of more than 128KBytes are allocated using mmap.

    So, at this point, I have 2 explanations :
    1 - My program allocates chunks of data with a growing size over the time.
    So, it is possible that when the size goes beyond 128KB,
    mmap starts to be invocated and, for an unknown reason,
    it soon fails to allocate memory.

    2 - When the maximum size of the data segement is reached,
    memory allocation automatically tries to use mmap
    even if the requested chunk size is less than 128KB.
    That would mean that memory allocated by mmap is not in the data segment
    (I m not sure of that).

    Thank you for any hints...

    Arnaud




  2. Re: Dynamic memory allocation and mmap

    > I believed that, using a 32 bits OS, I could use as far as 4 Gbytes.

    There must be communication between the user space and the kernel space,
    and most applications favor fast communication over any other choice.
    The fastest communication requires that the kernel and the user share
    a single 4GB address space. The style of sharing is chosen by compile-
    time parameters of the Linux kernel itself. The most common choice is
    3GB user and 1GB kernel, although some servers having much RAM, and running
    many hundreds of simultaneous small- and medium-sized processes, choose 2GB
    user and 2GB kernel. The kernel must use its space to keep track of physical
    RAM; 12GB of RAM requires lots of space for bookkeeping (3M entries
    to track the pages of 4KB, etc.)

    In the last few years there has been a "4G/4G" option which allows
    something like (4GB - 16MB) of user space and (4GB - 16MB) of kernel space,
    but it slows down system calls. Depending on load, the slowdown is often
    10% to 25%. Most users who say that they want a full 4GB of user space
    have been unwilling to pay that much in speed, and the 4G/4G split
    is not used often.

    Anyone who is really serious about processes running with more than
    about 2.8GB of address space should use a 64-bit machine such as x86_64.
    If you absolutely must use every last byte of address space on a 32-bit
    machine, then you must pay close attention to fragmentation of the address
    space. Consult /proc//maps to see what happening, and
    http://BitWagon.com/tub/tub.html for how to control it.

    --

  3. Re: Dynamic memory allocation and mmap

    On Mar 3, 9:25 am, Arnaud Megret wrote:

    > Of course, I know that the total memory comprises
    > the data segment and the stack.
    > Maybe, the stack size + the data segement size alltogether reaches 4 GB.
    > But that would mean that the stack size is about 2GBytes.
    > That seems huge indeed.


    You are probably using a kernel with a 2GB user limit. The other 2GB
    of addressable space are reserved for kernel memory. This allows the
    kernel to easily copy data from kernel space into user space and vice
    versa.

    I believe you can tweak this to a 3GB/1GB split, allowing 3GB of user
    space memory and just 1GB for the kernel. This may cause you issues
    with the kernel having too little address space.

    There once was a 4GB/4GB option, but I don't think it exists anymore.
    It forced the kernel to use a more complex and expensive mechanism to
    switch to kernel space and to access user space memory from kernel
    space.

    I haven't keep up-to-date on this issue. It may help to google things
    like '"4GB/4GB" linux'.

    DS

+ Reply to Thread