C libraries: where loaded on 64-bit applications - Linux

This is a discussion on C libraries: where loaded on 64-bit applications - Linux ; My colleagues and I are running a 64-bit application on 64-bit Red Hat Enterprise Linux, version 4, and have noticed a puzzling phenomenon. On different machines, the C libraries are loaded at slightly different addresses. The starting location tends to ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: C libraries: where loaded on 64-bit applications

  1. C libraries: where loaded on 64-bit applications

    My colleagues and I are running a 64-bit application on 64-bit Red Hat
    Enterprise Linux, version 4, and have noticed a puzzling phenomenon.
    On different machines, the C libraries are loaded at slightly different
    addresses. The starting location tends to be in the range of
    0x3000000000 to 0x3c00000000 (just below the 256-GB level), but is
    sytematically different. Examples from three different machines, with
    the same executable, appear below.

    The ranges do not seem to be random, as they are always identical on a
    given machine. In this regard, exec-shield-randomize, a feature that
    caused random loading of libraries in the past, is turned off.

    We are curious as to how the the addresses for the C libraries are
    determined. Note that application libraries are loaded in the same
    location on each machine, as is the executable itself.

    -- Jay


    Machine 1:

    00400000-00b91000 r-xp 00000000 08:02 7225490
    /home/braun/v184-b39/exe/cmr39.exe
    00c91000-00e4f000 rw-p 00791000 08:02 7225490
    /home/braun/v184-b39/exe/cmr39.exe
    00e4f000-015fd000 rwxp 00e4f000 00:00 0
    2a95556000-2a95557000 rw-p 2a95556000 00:00 0
    2a95557000-2a970d5000 r-xp 00000000 08:02 7225489
    /home/braun/v184-b39/exe/dll39.so_mod00
    2a970d5000-2a971d4000 ---p 01b7e000 08:02 7225489
    /home/braun/v184-b39/exe/dll39.so_mod00
    2a971d4000-2a97655000 rw-p 01b7d000 08:02 7225489
    /home/braun/v184-b39/exe/dll39.so_mod00
    2a97655000-2a9769f000 rw-p 2a97655000 00:00 0
    2a9769f000-2a976b5000 r-xp 00000000 08:02 7225497
    /home/braun/v184-b39/exe/geepc.so
    2a976b5000-2a977b4000 ---p 00016000 08:02 7225497
    /home/braun/v184-b39/exe/geepc.so
    2a977b4000-2a977be000 rw-p 00015000 08:02 7225497
    /home/braun/v184-b39/exe/geepc.so
    2a977be000-2a977bf000 rw-p 2a977be000 00:00 0
    2a977bf000-2a977fe000 r-xp 00000000 08:02 6752329
    /home/braun/v184-b39/sim64/lib/libsims.so
    2a977fe000-2a978fd000 ---p 0003f000 08:02 6752329
    /home/braun/v184-b39/sim64/lib/libsims.so
    2a978fd000-2a9790f000 rw-p 0003e000 08:02 6752329
    /home/braun/v184-b39/sim64/lib/libsims.so
    2a9790f000-2a97913000 rw-p 2a9790f000 00:00 0
    2a9793f000-2a97941000 rw-p 2a9793f000 00:00 0
    3ba2e00000-3ba2e15000 r-xp 00000000 08:0c 357276
    /lib64/ld-2.3.4.so
    3ba2f14000-3ba2f16000 rw-p 00014000 08:0c 357276
    /lib64/ld-2.3.4.so
    3ba3200000-3ba332b000 r-xp 00000000 08:0c 357277
    /lib64/tls/libc-2.3.4.so
    3ba332b000-3ba342a000 ---p 0012b000 08:0c 357277
    /lib64/tls/libc-2.3.4.so
    3ba342a000-3ba342d000 r--p 0012a000 08:0c 357277
    /lib64/tls/libc-2.3.4.so
    3ba342d000-3ba3430000 rw-p 0012d000 08:0c 357277
    /lib64/tls/libc-2.3.4.so
    3ba3430000-3ba3434000 rw-p 3ba3430000 00:00 0
    3ba3500000-3ba3585000 r-xp 00000000 08:0c 357278
    /lib64/tls/libm-2.3.4.so
    3ba3585000-3ba3684000 ---p 00085000 08:0c 357278
    /lib64/tls/libm-2.3.4.so
    3ba3684000-3ba3686000 rw-p 00084000 08:0c 357278
    /lib64/tls/libm-2.3.4.so
    7fbfffb000-7fc0000000 rw-p 7fbfffb000 00:00 0
    ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0

    Machine 2:

    00400000-00b91000 r-xp 00000000 08:15 2687146
    /home/cbs/v184-b39/exe/cmr39.exe
    00c91000-00e4f000 rw-p 00791000 08:15 2687146
    /home/cbs/v184-b39/exe/cmr39.exe
    00e4f000-015fd000 rwxp 00e4f000 00:00 0
    2a95556000-2a95557000 rw-p 2a95556000 00:00 0
    2a95557000-2a970d5000 r-xp 00000000 08:15 2687145
    /home/cbs/v184-b39/exe/dll39.so_mod00
    2a970d5000-2a971d4000 ---p 01b7e000 08:15 2687145
    /home/cbs/v184-b39/exe/dll39.so_mod00
    2a971d4000-2a97655000 rw-p 01b7d000 08:15 2687145
    /home/cbs/v184-b39/exe/dll39.so_mod00
    2a97655000-2a9769f000 rw-p 2a97655000 00:00 0
    2a9769f000-2a976b5000 r-xp 00000000 08:15 2687154
    /home/cbs/v184-b39/exe/geepc.so
    2a976b5000-2a977b4000 ---p 00016000 08:15 2687154
    /home/cbs/v184-b39/exe/geepc.so
    2a977b4000-2a977be000 rw-p 00015000 08:15 2687154
    /home/cbs/v184-b39/exe/geepc.so
    2a977be000-2a977bf000 rw-p 2a977be000 00:00 0
    2a977bf000-2a977fe000 r-xp 00000000 08:15 2719943
    /home/cbs/v184-b39/sim64/lib/libsims.so
    2a977fe000-2a978fd000 ---p 0003f000 08:15 2719943
    /home/cbs/v184-b39/sim64/lib/libsims.so
    2a978fd000-2a9790f000 rw-p 0003e000 08:15 2719943
    /home/cbs/v184-b39/sim64/lib/libsims.so
    2a9790f000-2a97913000 rw-p 2a9790f000 00:00 0
    2a9793f000-2a97941000 rw-p 2a9793f000 00:00 0
    35c9e00000-35c9e15000 r-xp 00000000 08:09 1087997
    /lib64/ld-2.3.4.so
    35c9f14000-35c9f16000 rw-p 00014000 08:09 1087997
    /lib64/ld-2.3.4.so
    35ca200000-35ca32b000 r-xp 00000000 08:09 1087998
    /lib64/tls/libc-2.3.4.so
    35ca32b000-35ca42a000 ---p 0012b000 08:09 1087998
    /lib64/tls/libc-2.3.4.so
    35ca42a000-35ca42d000 r--p 0012a000 08:09 1087998
    /lib64/tls/libc-2.3.4.so
    35ca42d000-35ca430000 rw-p 0012d000 08:09 1087998
    /lib64/tls/libc-2.3.4.so
    35ca430000-35ca434000 rw-p 35ca430000 00:00 0
    35ca500000-35ca585000 r-xp 00000000 08:09 1087999
    /lib64/tls/libm-2.3.4.so
    35ca585000-35ca684000 ---p 00085000 08:09 1087999
    /lib64/tls/libm-2.3.4.so
    35ca684000-35ca686000 rw-p 00084000 08:09 1087999
    /lib64/tls/libm-2.3.4.so
    7fbfffb000-7fc0000000 rw-p 7fbfffb000 00:00 0
    ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0

    Machine 3:

    00400000-00b91000 r-xp 00000000 08:15 3506354
    /home/cbs/v184-b39/exe/cmr39.exe
    00c91000-00e4f000 rw-p 00791000 08:15 3506354
    /home/cbs/v184-b39/exe/cmr39.exe
    00e4f000-015fd000 rwxp 00e4f000 00:00 0
    2a95556000-2a95557000 rw-p 2a95556000 00:00 0
    2a95557000-2a970d5000 r-xp 00000000 08:15 3506435
    /home/cbs/v184-b39/exe/dll39.so_mod00
    2a970d5000-2a971d4000 ---p 01b7e000 08:15 3506435
    /home/cbs/v184-b39/exe/dll39.so_mod00
    2a971d4000-2a97655000 rw-p 01b7d000 08:15 3506435
    /home/cbs/v184-b39/exe/dll39.so_mod00
    2a97655000-2a9769f000 rw-p 2a97655000 00:00 0
    2a9769f000-2a976b5000 r-xp 00000000 08:15 3506361
    /home/cbs/v184-b39/exe/geepc.so
    2a976b5000-2a977b4000 ---p 00016000 08:15 3506361
    /home/cbs/v184-b39/exe/geepc.so
    2a977b4000-2a977be000 rw-p 00015000 08:15 3506361
    /home/cbs/v184-b39/exe/geepc.so
    2a977be000-2a977bf000 rw-p 2a977be000 00:00 0
    2a977bf000-2a977fe000 r-xp 00000000 08:15 3506692
    /home/cbs/v184-b39/sim64/lib/libsims.so
    2a977fe000-2a978fd000 ---p 0003f000 08:15 3506692
    /home/cbs/v184-b39/sim64/lib/libsims.so
    2a978fd000-2a9790f000 rw-p 0003e000 08:15 3506692
    /home/cbs/v184-b39/sim64/lib/libsims.so
    2a9790f000-2a97913000 rw-p 2a9790f000 00:00 0
    2a9793f000-2a97941000 rw-p 2a9793f000 00:00 0
    3a7e400000-3a7e415000 r-xp 00000000 08:09 878517
    /lib64/ld-2.3.4.so
    3a7e514000-3a7e516000 rw-p 00014000 08:09 878517
    /lib64/ld-2.3.4.so
    3a7e800000-3a7e92b000 r-xp 00000000 08:09 878518
    /lib64/tls/libc-2.3.4.so
    3a7e92b000-3a7ea2a000 ---p 0012b000 08:09 878518
    /lib64/tls/libc-2.3.4.so
    3a7ea2a000-3a7ea2d000 r--p 0012a000 08:09 878518
    /lib64/tls/libc-2.3.4.so
    3a7ea2d000-3a7ea30000 rw-p 0012d000 08:09 878518
    /lib64/tls/libc-2.3.4.so
    3a7ea30000-3a7ea34000 rw-p 3a7ea30000 00:00 0
    3a7eb00000-3a7eb85000 r-xp 00000000 08:09 878519
    /lib64/tls/libm-2.3.4.so
    3a7eb85000-3a7ec84000 ---p 00085000 08:09 878519
    /lib64/tls/libm-2.3.4.so
    3a7ec84000-3a7ec86000 rw-p 00084000 08:09 878519
    /lib64/tls/libm-2.3.4.so
    7fbfffb000-7fc0000000 rw-p 7fbfffb000 00:00 0
    ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0


  2. Re: C libraries: where loaded on 64-bit applications

    braunsimulation@aol.com writes:

    > My colleagues and I are running a 64-bit application on 64-bit Red Hat
    > Enterprise Linux, version 4, and have noticed a puzzling phenomenon.


    Do "man prelink", then look at your /etc/sysconfig/prelink,
    which likely contains:

    # Options to pass to prelink
    # -m Try to conserve virtual memory by allowing overlapping
    # assigned virtual memory slots for libraries which
    # never appear together in one binary
    NB> # -R Randomize virtual memory slot assignments for libraries.
    # This makes it slightly harder for various buffer overflow
    # attacks, since library addresses will be different on each
    # host using -R.
    PRELINK_OPTS=-mR

    > The ranges do not seem to be random, as they are always identical on a
    > given machine.


    They are only identical until the next time prelink is run, which
    is controlled by the same config file.

    Cheers,
    --
    In order to understand recursion you must first understand recursion.
    Remove /-nsp/ for email.

  3. Re: C libraries: where loaded on 64-bit applications

    Paul Pluzhnikov wrote:

    > # Options to pass to prelink
    > # -m Try to conserve virtual memory by allowing overlapping
    > # assigned virtual memory slots for libraries which
    > # never appear together in one binary
    > NB> # -R Randomize virtual memory slot assignments for libraries.
    > # This makes it slightly harder for various buffer overflow
    > # attacks, since library addresses will be different on each
    > # host using -R.
    > PRELINK_OPTS=-mR


    Thank you, Paul.

    I edited that line in /etc/sysconfig/prelink on all three machines so
    that it reads:

    PRELINK_OPTS=-m

    and rebooted. But the libraries still load differently on the 3
    machines and, in fact, load at addresses identical to those that I
    posted previously. Were there additional steps that I should have
    taken?

    j


  4. Re: C libraries: where loaded on 64-bit applications

    braunsimulation@aol.com wrote:

    > I edited that line in /etc/sysconfig/prelink on all three machines so
    > that it reads:
    >
    > PRELINK_OPTS=-m
    >
    > and rebooted. But the libraries still load differently on the 3
    > machines and, in fact, load at addresses identical to those that I
    > posted previously. Were there additional steps that I should have
    > taken?


    Re-run prelink, as Paul's message suggested.

    -n

  5. Re: C libraries: where loaded on 64-bit applications

    "Nikolaos D. Bougalis" writes:

    > braunsimulation@aol.com wrote:
    >
    >> I edited that line in /etc/sysconfig/prelink on all three machines so
    >> that it reads:
    >> PRELINK_OPTS=-m
    >> and rebooted.


    Rebooting isn't necessary and doesn't help (as you discovered).

    >> Were there additional steps that I should have taken?

    >
    > Re-run prelink, as Paul's message suggested.


    Correct.
    I believe all you need to do is run "/usr/sbin/prelink -m" as root.

    Cheers,
    --
    In order to understand recursion you must first understand recursion.
    Remove /-nsp/ for email.

  6. Re: C libraries: where loaded on 64-bit applications


    Paul Pluzhnikov wrote:
    > >> PRELINK_OPTS=-m


    > I believe all you need to do is run "/usr/sbin/prelink -m" as root.
    >


    Thanks. When I turn off the randomization, the C shared libraries load
    at 0x3000000000 on a variety of systems. This appears to be a
    "standard" setting. What determines this value?

    Jay


  7. Re: C libraries: where loaded on 64-bit applications

    braunsimulation@aol.com writes:

    > When I turn off the randomization, the C shared libraries load
    > at 0x3000000000 on a variety of systems. This appears to be a
    > "standard" setting. What determines this value?


    Good question.

    Apparently that's the default start address for prelink.
    On FC6, in /usr/src/redhat/BUILD/prelink/src/arch-x86_64.c:

    PL_ARCH = {
    .name = "x86-64",
    .class = ELFCLASS64,
    .machine = EM_X86_64,
    ....
    /* Although TASK_UNMAPPED_BASE is 0x2a95555555, we leave some
    area so that mmap of /etc/ld.so.cache and ld.so's malloc
    does not take some library's VA slot.
    Also, if this guard area isn't too small, typically
    even dlopened libraries will get the slots they desire. */
    .mmap_base = 0x3000000000LL,
    .mmap_end = 0x4000000000LL,
    .max_page_size = 0x200000,
    .page_size = 0x1000
    };

    This value appears to be changeable with --mmap-region-start
    (undocumented) prelink option.

    Cheers,
    --
    In order to understand recursion you must first understand recursion.
    Remove /-nsp/ for email.

+ Reply to Thread