Dynamic library loader search path - Linux
This is a discussion on Dynamic library loader search path - Linux ; Loader searches for dynamic libraries in directories specified in
$LD_LIBRARY_PATH environment variables. However it's not all, loader
also searches in directories in $LD_LIBRARY_PATH augmented with many
sub-paths like lib/i686/mmx, tls, mmx, i686 and so on. Where do these
sub-paths come ...
-
Dynamic library loader search path
Loader searches for dynamic libraries in directories specified in
$LD_LIBRARY_PATH environment variables. However it's not all, loader
also searches in directories in $LD_LIBRARY_PATH augmented with many
sub-paths like lib/i686/mmx, tls, mmx, i686 and so on. Where do these
sub-paths come from ? Is it possible to configure them or even cancel
them ? The problem I encountered is that loader first searches within
directories suffixed with all these sub-passes and only then in the
directory itself. This highly increases dynamic library search time.
-
Re: Dynamic library loader search path
Gregory wrote:
> Loader searches for dynamic libraries in directories specified in
> $LD_LIBRARY_PATH environment variables. However it's not all, [...]
> Where do these sub-paths come from?
Read
man ld.so
man ldconfig
...and the ones referenced from there.
Uli
-
Re: Dynamic library loader search path
I read them before posting here but did not find anything that answers
my question. Can
you be more specific ?
Thanks,
Gregory
-
Re: Dynamic library loader search path
"Gregory" writes:
> Loader searches for dynamic libraries in directories specified in
> $LD_LIBRARY_PATH environment variables. However it's not all, loader
> also searches in directories in $LD_LIBRARY_PATH augmented with many
> sub-paths like lib/i686/mmx, tls, mmx, i686 and so on. Where do these
> sub-paths come from ?
These are compiled into ld-linux.so.2 -- it goes over all the search
directories, and for each appends each "capability string". Look
in elf/dl-load.c open_path().
The capability strings are filled by _dl_important_hwcaps() in
sysdeps/generic/dl-sysdep.c and (AFAICT) you can't modify any of
this without rebuilding glibc.
> Is it possible to configure them or even cancel them ?
No (AFAICT).
> The problem I encountered is that loader first searches within
> directories suffixed with all these sub-passes and only then in the
> directory itself. This highly increases dynamic library search time.
How many directories *are* you searching?
Unless your LD_LIBRARY_PATH contains 1000s of directories (or you
are searching directories on a slow NFS server), I don't see how
searcing 4 times as many would "highly increase search time".
If you are searching many directories, a better solution might be
to either dlopen() with absolute path, or collect all your libraries
into a single directory (with symlinks).
Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.
-
Re: Dynamic library loader search path
Ulrich Eckhardt writes:
> Gregory wrote:
>> Where do these sub-paths come from?
>
> Read
> man ld.so
> man ldconfig
Neither answers the question that OP asked.
> ..and the ones referenced from there.
ldconfig doesn't reference any other man pages,
ld.so refers back to ldconfig and ldd, and ldd refers back to ld.so
and ldconfig.
I am afraid your answer was not very useful :-(
Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.
-
Re: Dynamic library loader search path
Paul,
Thanks a lot for your comprehensive answer !
> The capability strings are filled by _dl_important_hwcaps() in
> sysdeps/generic/dl-sysdep.c and (AFAICT) you can't modify any of
> this without rebuilding glibc.
As I understand dl-sysdeps.c is glibc source file. However it seems
that ld-linux.so links statically with glibc. So I also need to rebuild
ld-linux.so.
Correct ? Where can I get ld-linux.so sources for RHEL 3 ? Is it gnu
application ?
> How many directories *are* you searching?
> Unless your LD_LIBRARY_PATH contains 1000s of directories (or you
> are searching directories on a slow NFS server), I don't see how
> searching 4 times as many would "highly increase search time".
Well, may be I was too pessimistic. Lets reword my claim as "increase
search time".
By the way I checked the strace output and found that in case the
directories augmented with "capacity strings" do not exist (this is
almost always the case) they are searched only once and then checked
for existence with stat64() and if they do not exist they are no longer
searched. This is much better situation than I thought.
Regards,
Gregory
-
Re: Dynamic library loader search path
Paul,
I found a way to mask out all "capacity substrings" but 'tls' and
platform ('i686') that are hardcoded in dl-sysdeps.c. Just setenv
LD_HWCAP_MASK environment variable to 0.
Not bad :-)
Regards,
Gregory
-
Re: Dynamic library loader search path
"Gregory" writes:
> As I understand dl-sysdeps.c is glibc source file. However it seems
> that ld-linux.so links statically with glibc.
Rather /lib/ld-linux.so.2 is one of many (hundreds) of glibc
"components", and shares source and object code with libc.so.6
> So I also need to rebuild ld-linux.so. Correct ?
If you want to get rid of "capability strings", you must rebuild
and install entire glibc (ld-linux.so.6 and libc.so.6 *must* be
updated simultaneously and from the same build).
Be warned -- a small mistake and your system will be un-bootable.
For a safewr way to experiment with glibc builds, see this:
http://bitwagon.com/rtldi/rtldi.html
> Where can I get ld-linux.so sources for RHEL 3 ? Is it gnu
> application ?
It's (inseparable) part of glibc.
> Well, may be I was too pessimistic. Lets reword my claim as "increase
> search time".
Let's reword some more: "increase search time by a fraction of percent".
This is probably a case of "premature optimization"; you should
measure the time spent there ('strace -T ./a.out') and only optimize
if the "wasted" time accounts for significant percentage of all
syscall time (which it likely will not).
> By the way I checked the strace output and found that in case the
> directories augmented with "capacity strings" do not exist (this is
> almost always the case) they are searched only once
Yes, this is one of many glibc optimizations ...
Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.
-
Re: Dynamic library loader search path
"Gregory" writes:
> I found a way to mask out all "capacity substrings"
They are "capability strings" (they depend on whether the processor
is capable of MMX, SSE, etc).
> but 'tls' and platform ('i686') that are hardcoded in dl-sysdeps.c.
> Just setenv LD_HWCAP_MASK environment variable to 0.
Right, I missed that ...
Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.
-
Re: Dynamic library loader search path
> Rather /lib/ld-linux.so.2 is one of many (hundreds) of glibc
> "components", and shares source and object code with libc.so.
I see.
> Be warned -- a small mistake and your system will be un-bootable.
> For a safewr way to experiment with glibc builds, see this:
> http://bitwagon.com/rtldi/rtldi.html
Thanks,
Gregory
-
Re: Dynamic library loader search path
Paul Pluzhnikov wrote:
> "Gregory" writes:
>
> > I found a way to mask out all "capacity substrings"
>
> They are "capability strings" (they depend on whether the processor
> is capable of MMX, SSE, etc).
Nice to know !
I meant substrings which combinations comprise search path suffixes
appended to the
$LD_LIBRARY_PATH directories like 'tls/i686/mmx'.
Again, thank you very much for your answers !
Gregory
С новым 2007 годом дорогие товарищи !!!