Dynamic library loader search path - Linux

This is a discussion on Dynamic library loader search path - Linux ; Loader searches for dynamic libraries in directories specified in $LD_LIBRARY_PATH environment variables. However it's not all, loader also searches in directories in $LD_LIBRARY_PATH augmented with many sub-paths like lib/i686/mmx, tls, mmx, i686 and so on. Where do these sub-paths come ...

+ Reply to Thread
Results 1 to 11 of 11

Thread: Dynamic library loader search path

  1. Dynamic library loader search path

    Loader searches for dynamic libraries in directories specified in
    $LD_LIBRARY_PATH environment variables. However it's not all, loader
    also searches in directories in $LD_LIBRARY_PATH augmented with many
    sub-paths like lib/i686/mmx, tls, mmx, i686 and so on. Where do these
    sub-paths come from ? Is it possible to configure them or even cancel
    them ? The problem I encountered is that loader first searches within
    directories suffixed with all these sub-passes and only then in the
    directory itself. This highly increases dynamic library search time.


  2. Re: Dynamic library loader search path

    Gregory wrote:
    > Loader searches for dynamic libraries in directories specified in
    > $LD_LIBRARY_PATH environment variables. However it's not all, [...]
    > Where do these sub-paths come from?


    Read
    man ld.so
    man ldconfig

    ...and the ones referenced from there.

    Uli


  3. Re: Dynamic library loader search path

    I read them before posting here but did not find anything that answers
    my question. Can
    you be more specific ?
    Thanks,

    Gregory


  4. Re: Dynamic library loader search path

    "Gregory" writes:

    > Loader searches for dynamic libraries in directories specified in
    > $LD_LIBRARY_PATH environment variables. However it's not all, loader
    > also searches in directories in $LD_LIBRARY_PATH augmented with many
    > sub-paths like lib/i686/mmx, tls, mmx, i686 and so on. Where do these
    > sub-paths come from ?


    These are compiled into ld-linux.so.2 -- it goes over all the search
    directories, and for each appends each "capability string". Look
    in elf/dl-load.c open_path().

    The capability strings are filled by _dl_important_hwcaps() in
    sysdeps/generic/dl-sysdep.c and (AFAICT) you can't modify any of
    this without rebuilding glibc.

    > Is it possible to configure them or even cancel them ?


    No (AFAICT).

    > The problem I encountered is that loader first searches within
    > directories suffixed with all these sub-passes and only then in the
    > directory itself. This highly increases dynamic library search time.


    How many directories *are* you searching?

    Unless your LD_LIBRARY_PATH contains 1000s of directories (or you
    are searching directories on a slow NFS server), I don't see how
    searcing 4 times as many would "highly increase search time".

    If you are searching many directories, a better solution might be
    to either dlopen() with absolute path, or collect all your libraries
    into a single directory (with symlinks).

    Cheers,
    --
    In order to understand recursion you must first understand recursion.
    Remove /-nsp/ for email.

  5. Re: Dynamic library loader search path

    Ulrich Eckhardt writes:

    > Gregory wrote:
    >> Where do these sub-paths come from?

    >
    > Read
    > man ld.so
    > man ldconfig


    Neither answers the question that OP asked.

    > ..and the ones referenced from there.


    ldconfig doesn't reference any other man pages,
    ld.so refers back to ldconfig and ldd, and ldd refers back to ld.so
    and ldconfig.

    I am afraid your answer was not very useful :-(

    Cheers,
    --
    In order to understand recursion you must first understand recursion.
    Remove /-nsp/ for email.

  6. Re: Dynamic library loader search path

    Paul,

    Thanks a lot for your comprehensive answer !

    > The capability strings are filled by _dl_important_hwcaps() in
    > sysdeps/generic/dl-sysdep.c and (AFAICT) you can't modify any of
    > this without rebuilding glibc.


    As I understand dl-sysdeps.c is glibc source file. However it seems
    that ld-linux.so links statically with glibc. So I also need to rebuild
    ld-linux.so.
    Correct ? Where can I get ld-linux.so sources for RHEL 3 ? Is it gnu
    application ?

    > How many directories *are* you searching?


    > Unless your LD_LIBRARY_PATH contains 1000s of directories (or you
    > are searching directories on a slow NFS server), I don't see how
    > searching 4 times as many would "highly increase search time".


    Well, may be I was too pessimistic. Lets reword my claim as "increase
    search time".
    By the way I checked the strace output and found that in case the
    directories augmented with "capacity strings" do not exist (this is
    almost always the case) they are searched only once and then checked
    for existence with stat64() and if they do not exist they are no longer

    searched. This is much better situation than I thought.

    Regards,

    Gregory


  7. Re: Dynamic library loader search path

    Paul,

    I found a way to mask out all "capacity substrings" but 'tls' and
    platform ('i686') that are hardcoded in dl-sysdeps.c. Just setenv
    LD_HWCAP_MASK environment variable to 0.
    Not bad :-)

    Regards,

    Gregory


  8. Re: Dynamic library loader search path

    "Gregory" writes:

    > As I understand dl-sysdeps.c is glibc source file. However it seems
    > that ld-linux.so links statically with glibc.


    Rather /lib/ld-linux.so.2 is one of many (hundreds) of glibc
    "components", and shares source and object code with libc.so.6

    > So I also need to rebuild ld-linux.so. Correct ?


    If you want to get rid of "capability strings", you must rebuild
    and install entire glibc (ld-linux.so.6 and libc.so.6 *must* be
    updated simultaneously and from the same build).

    Be warned -- a small mistake and your system will be un-bootable.
    For a safewr way to experiment with glibc builds, see this:
    http://bitwagon.com/rtldi/rtldi.html

    > Where can I get ld-linux.so sources for RHEL 3 ? Is it gnu
    > application ?


    It's (inseparable) part of glibc.

    > Well, may be I was too pessimistic. Lets reword my claim as "increase
    > search time".


    Let's reword some more: "increase search time by a fraction of percent".
    This is probably a case of "premature optimization"; you should
    measure the time spent there ('strace -T ./a.out') and only optimize
    if the "wasted" time accounts for significant percentage of all
    syscall time (which it likely will not).

    > By the way I checked the strace output and found that in case the
    > directories augmented with "capacity strings" do not exist (this is
    > almost always the case) they are searched only once


    Yes, this is one of many glibc optimizations ...

    Cheers,
    --
    In order to understand recursion you must first understand recursion.
    Remove /-nsp/ for email.

  9. Re: Dynamic library loader search path

    "Gregory" writes:

    > I found a way to mask out all "capacity substrings"


    They are "capability strings" (they depend on whether the processor
    is capable of MMX, SSE, etc).

    > but 'tls' and platform ('i686') that are hardcoded in dl-sysdeps.c.
    > Just setenv LD_HWCAP_MASK environment variable to 0.


    Right, I missed that ...

    Cheers,
    --
    In order to understand recursion you must first understand recursion.
    Remove /-nsp/ for email.

  10. Re: Dynamic library loader search path

    > Rather /lib/ld-linux.so.2 is one of many (hundreds) of glibc
    > "components", and shares source and object code with libc.so.


    I see.

    > Be warned -- a small mistake and your system will be un-bootable.
    > For a safewr way to experiment with glibc builds, see this:
    > http://bitwagon.com/rtldi/rtldi.html


    Thanks,

    Gregory


  11. Re: Dynamic library loader search path

    Paul Pluzhnikov wrote:
    > "Gregory" writes:
    >
    > > I found a way to mask out all "capacity substrings"

    >
    > They are "capability strings" (they depend on whether the processor
    > is capable of MMX, SSE, etc).


    Nice to know !
    I meant substrings which combinations comprise search path suffixes
    appended to the
    $LD_LIBRARY_PATH directories like 'tls/i686/mmx'.

    Again, thank you very much for your answers !

    Gregory

    С новым 2007 годом дорогие товарищи !!!


+ Reply to Thread