This is a multi-part message in MIME format.
--------------010207090909060607090400
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Setting a built-in shared library search path should be optional.
If you build a shared library based openssl dist for inclusion in a
software
distribution packet that can be installed in another location on
the target machine you don't want a built-in path to the
install location on the build machine.
I know that on some platforms it can be very hard to avoid
that the linker automatically includes a search path to the
location of the shared libraries at link time, but when it is
possible I think it should be made optional.

/leif

kabe@sra-tohoku.co.jp via RT wrote:
> Platform: NetBSD 1.6.1 x86 (full ./testlog attached below)
> OpenSSL: 0.9.8b
>
> openssl command fails to load, when built with
> libssl and libprypto as both shared, and
> the OS tries a good job on finding shared library dependencies on runtime.
>
>
>>> kabe@vega% /usr/ssl/bin/openssl
>>> Shared object "libcrypto.so.0.9.8" not found
>>>

>
> Standard problems, such as missing RPATH in the executable, is
> already eliminated.
>
> What is happening:
> 1) openssl executable is built with RPATH=/usr/ssl/lib, and
> NEEDED libssl.so.0.9.8, libcrypto.so.0.9.8, libc.so.12 .
> 2) The dynamic loader first pulls in libssl.so.0.9.8 and libcrypto.so.0.9.8,
> using the executable's RPATH=/usr/ssl/lib as the search path,
> which succeeds in loading
> /usr/ssl/lib/libssl.so.0.9.8 and /usr/ssl/lib/libcrypto.so.0.9.8 .
> 3) The pulled in libssl.so.0.9.8 is built with
> NEEDED libcrypto.so.0.9.8, and no RPATH.
> 4) The dynamic loader thus tries to load libcrypto.so.0.9.8,
> this time using the default search path only,
> which tries for only /usr/lib/libcrypto.so.0.9.8,
> and fails.
> (This particular loader seems not try to compare filename or signature
> for already loaded library)
>
> Setting LD_DEBUG=1 will reveal the steps:
>
>
>>> kabe@vega% env LD_DEBUG=1 /usr/ssl/bin/openssl version
>>> processing mapping "libm.so.0 machdep.fpu_present 1:libm387.so.0,libm.so.0
>>> "
>>> library "libm.so.0"
>>> sysctl "machdep.fpu_present"
>>> sysctl 7, 1
>>> sysctl 7, 2
>>> library "libm387.so.0"
>>> library "libm.so.0"
>>> key "1"
>>> /usr/libexec/ld.elf_so is initialized, base address = 0x480a2000
>>> processing main program's program header
>>> added path "/usr/ssl/lib" **** RPATH of openssl
>>> preloading objects
>>> loading needed objects
>>> load by name libssl.so.0.9.8 0x480af000
>>> Searching for "libssl.so.0.9.8" (0x480af200) **** request by openssl
>>> Trying "/usr/ssl/lib/libssl.so.0.9.8"
>>> 0x480b5000 .. 0x480eefff: /usr/ssl/lib/libssl.so.0.9.8
>>> load by name libcrypto.so.0.9.8 0x480af000
>>> Searching for "libcrypto.so.0.9.8" (0x480af200) **** request by openssl
>>> Trying "/usr/ssl/lib/libcrypto.so.0.9.8"
>>> 0x480ef000 .. 0x48212fff: /usr/ssl/lib/libcrypto.so.0.9.8
>>> load by name libc.so.12 0x480af000
>>> Searching for "libc.so.12" (0x480af200) **** request by openssl
>>> Trying "/usr/ssl/lib/libc.so.12"
>>> Trying "/usr/lib/libc.so.12"
>>> added path "/usr/lib"
>>> 0x48213000 .. 0x482b2fff: /usr/lib/libc.so.12
>>> load by name libcrypto.so.0.9.8 0x480af000
>>> Searching for "libcrypto.so.0.9.8" (0x480af400) **** request by libssl.so
>>> Trying "/usr/lib/libcrypto.so.0.9.8" **** tries only default path, as RPATH of libssl.so is unset
>>> Shared object "libcrypto.so.0.9.8" not found
>>>

>
>
> The attached patch will always add the $(LIBRPATH) to RPATH,
> also when building a shared library.
>
>
> Workaround without the patch:
> Build with "make SHARED_LDFLAGS=-Wl,-rpath,/usr/ssl/lib"
>
>
> Note that the method of recursive searching the dependent shared library
> is different on each OS;
> - some os just ignores RPATH on *.so and only use the executable's RPATH
> (Solaris)
> - some will use RPATH on the *.so individually for each loaded library (NetBSD)
> - some will MERGE the current RPATH with the loaded *.so (IRIX)
> so some platforms are, as the result, unaffected whether there's
> RPATH on *.so or not.
>
>
> There seems a debate many times on the list about setting RPATH
> on the shared library, but it should NOT be a problem using ELF platform
> on normal circumstances.
>
> The Normal Circumstances:
> - When compiling yourselves, you set the correct --prefix,
> install where it should and off you go.
> - When the Distributor wants the package to go to different directory,
> he should first compile with --prefix with that directory.
> - When the Administrator wants to install the package into directory
> different from which the package defaults, he should setup
> LD_LIBRARY_PATH himself. That's what the environ is for.
>
> It could be a problem when the admin wants a twisted setup like
> - the OS has /usr/lib/libz, /usr/lib/libssl and /usr/lib/libcrypto
> - admin compiled the latest OpenSSL on /usr/local/lib
> - he has another libz on /usr/local/lib/libz, which may or may not be new
> (not unrealistic when you had built X11R* yourself)
> - he wants the executables to use
> /usr/local/lib/libssl, /usr/local/lib/libcrypto with /usr/lib/libz
> but the admin already needs to grok with Makefile to properly link
> with libs he wants. OpenSSL does't have to support that far.
>
>
> ====
> OpenSSL self-test report:
>
> OpenSSL version: 0.9.8b
> Last change: When applying a cipher rule check to see if string matc...
> Options: --prefix=/usr/ssl --openssldir=/usr/ssl enable-shared enable-zlib-dynamic no-asm no-gmp no-idea no-krb5 no-mdc2 no-rc5
> OS (uname): NetBSD vega.sra-tohoku.jp 1.6.1 NetBSD 1.6.1 (QQQ) #10: Thu Jun 19 18:21:56 JST 2003 kabe@vega.sra-tohoku.co.jp:/export/NetBSD/sys/arch/i386/compile/QQQ i386
> OS (config): i686-whatever-netbsd
> Target (default): BSD-x86-elf
> Target: BSD-x86-elf
> Compiler: Using builtin specs.
> gcc version 2.95.3 20010315 (release) (NetBSD nb3)
>
> Test passed.
>
> ==== PATCH
> diff -6 -ur openssl-0.9.8b-orig/Makefile.shared openssl-0.9.8b/Makefile.shared
> --- openssl-0.9.8b-orig/Makefile.shared Sat Dec 17 05:52:38 2005
> +++ openssl-0.9.8b/Makefile.shared Thu Aug 31 17:25:14 2006
> @@ -148,13 +148,13 @@
>
> DO_GNU_SO=$(CALC_VERSIONS); \
> SHLIB=lib$(LIBNAME).so; \
> SHLIB_SUFFIX=; \
> ALLSYMSFLAGS='-Wl,--whole-archive'; \
> NOALLSYMSFLAGS='-Wl,--no-whole-archive'; \
> - SHAREDFLAGS="$(CFLAGS) $(SHARED_LDFLAGS) -shared -Wl,-Bsymbolic -Wl,-soname=$$SHLIB$$SHLIB_SOVER$$SHLIB_SUFFIX"
> + SHAREDFLAGS="$(CFLAGS) $(SHARED_LDFLAGS) -shared -Wl,-Bsymbolic -Wl,-soname=$$SHLIB$$SHLIB_SOVER$$SHLIB_SUFFIX -Wl,-rpath,$(LIBRPATH)"
>
> DO_GNU_APP=LDFLAGS="$(CFLAGS) -Wl,-rpath,$(LIBRPATH)"
>
> #This is rather special. It's a special target with which one can link
> #applications without bothering with any features that have anything to
> #do with shared libraries, for example when linking against static
>
> __________________________________________________ ____________________
> OpenSSL Project http://www.openssl.org
> Development Mailing List openssl-dev@openssl.org
> Automated List Manager majordomo@openssl.org
>



--------------010207090909060607090400
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit







Setting a built-in shared library search path should be optional.

If you build a shared library based openssl dist for inclusion in a
software

distribution packet that can be installed in another location on

the target machine you don't want a built-in path to the

install location on the build machine.

I know that on some platforms it can be very hard to avoid

that the linker automatically includes a search path to the

location of the shared libraries at link time, but when it is

possible I think it should be made optional.



/leif



kabe@sra-tohoku.co.jp via RT wrote:
type="cite">
Platform: NetBSD 1.6.1 x86 (full ./testlog attached below)
OpenSSL: 0.9.8b

openssl command fails to load, when built with
libssl and libprypto as both shared, and
the OS tries a good job on finding shared library dependencies on runtime.




kabe@vega% /usr/ssl/bin/openssl
Shared object "libcrypto.so.0.9.8" not found




Standard problems, such as missing RPATH in the executable, is
already eliminated.

What is happening:
1) openssl executable is built with RPATH=/usr/ssl/lib, and
NEEDED libssl.so.0.9.8, libcrypto.so.0.9.8, libc.so.12 .
2) The dynamic loader first pulls in libssl.so.0.9.8 and libcrypto.so.0.9.8,
using the executable's RPATH=/usr/ssl/lib as the search path,
which succeeds in loading
/usr/ssl/lib/libssl.so.0.9.8 and /usr/ssl/lib/libcrypto.so.0.9.8 .
3) The pulled in libssl.so.0.9.8 is built with
NEEDED libcrypto.so.0.9.8, and no RPATH.
4) The dynamic loader thus tries to load libcrypto.so.0.9.8,
this time using the default search path only,
which tries for only /usr/lib/libcrypto.so.0.9.8,
and fails.
(This particular loader seems not try to compare filename or signature
for already loaded library)

Setting LD_DEBUG=1 will reveal the steps:




kabe@vega% env LD_DEBUG=1 /usr/ssl/bin/openssl version
processing mapping "libm.so.0 machdep.fpu_present 1:libm387.so.0,libm.so.0
"
library "libm.so.0"
sysctl "machdep.fpu_present"
sysctl 7, 1
sysctl 7, 2
library "libm387.so.0"
library "libm.so.0"
key "1"
/usr/libexec/ld.elf_so is initialized, base address = 0x480a2000
processing main program's program header
added path "/usr/ssl/lib" **** RPATH of openssl
preloading objects
loading needed objects
load by name libssl.so.0.9.8 0x480af000
Searching for "libssl.so.0.9.8" (0x480af200) **** request by openssl
Trying "/usr/ssl/lib/libssl.so.0.9.8"
0x480b5000 .. 0x480eefff: /usr/ssl/lib/libssl.so.0.9.8
load by name libcrypto.so.0.9.8 0x480af000
Searching for "libcrypto.so.0.9.8" (0x480af200) **** request by openssl
Trying "/usr/ssl/lib/libcrypto.so.0.9.8"
0x480ef000 .. 0x48212fff: /usr/ssl/lib/libcrypto.so.0.9.8
load by name libc.so.12 0x480af000
Searching for "libc.so.12" (0x480af200) **** request by openssl
Trying "/usr/ssl/lib/libc.so.12"
Trying "/usr/lib/libc.so.12"
added path "/usr/lib"
0x48213000 .. 0x482b2fff: /usr/lib/libc.so.12
load by name libcrypto.so.0.9.8 0x480af000
Searching for "libcrypto.so.0.9.8" (0x480af400) **** request by libssl.so
Trying "/usr/lib/libcrypto.so.0.9.8" **** tries only default path, as RPATH of libssl.so is unset
Shared object "libcrypto.so.0.9.8" not found





The attached patch will always add the $(LIBRPATH) to RPATH,
also when building a shared library.


Workaround without the patch:
Build with "make SHARED_LDFLAGS=-Wl,-rpath,/usr/ssl/lib"


Note that the method of recursive searching the dependent shared library
is different on each OS;
- some os just ignores RPATH on *.so and only use the executable's RPATH
(Solaris)
- some will use RPATH on the *.so individually for each loaded library (NetBSD)
- some will MERGE the current RPATH with the loaded *.so (IRIX)
so some platforms are, as the result, unaffected whether there's
RPATH on *.so or not.


There seems a debate many times on the list about setting RPATH
on the shared library, but it should NOT be a problem using ELF platform
on normal circumstances.

The Normal Circumstances:
- When compiling yourselves, you set the correct --prefix,
install where it should and off you go.
- When the Distributor wants the package to go to different directory,
he should first compile with --prefix with that directory.
- When the Administrator wants to install the package into directory
different from which the package defaults, he should setup
LD_LIBRARY_PATH himself. That's what the environ is for.

It could be a problem when the admin wants a twisted setup like
- the OS has /usr/lib/libz, /usr/lib/libssl and /usr/lib/libcrypto
- admin compiled the latest OpenSSL on /usr/local/lib
- he has another libz on /usr/local/lib/libz, which may or may not be new
(not unrealistic when you had built X11R* yourself)
- he wants the executables to use
/usr/local/lib/libssl, /usr/local/lib/libcrypto with /usr/lib/libz
but the admin already needs to grok with Makefile to properly link
with libs he wants. OpenSSL does't have to support that far.


====
OpenSSL self-test report:

OpenSSL version: 0.9.8b
Last change: When applying a cipher rule check to see if string matc...
Options: --prefix=/usr/ssl --openssldir=/usr/ssl enable-shared enable-zlib-dynamic no-asm no-gmp no-idea no-krb5 no-mdc2 no-rc5
OS (uname): NetBSD vega.sra-tohoku.jp 1.6.1 NetBSD 1.6.1 (QQQ) #10: Thu Jun 19 18:21:56 JST 2003 kabe@vega.sra-tohoku.co.jp:/export/NetBSD/sys/arch/i386/compile/QQQ i386
OS (config): i686-whatever-netbsd
Target (default): BSD-x86-elf
Target: BSD-x86-elf
Compiler: Using builtin specs.
gcc version 2.95.3 20010315 (release) (NetBSD nb3)

Test passed.

==== PATCH
diff -6 -ur openssl-0.9.8b-orig/Makefile.shared openssl-0.9.8b/Makefile.shared
--- openssl-0.9.8b-orig/Makefile.shared Sat Dec 17 05:52:38 2005
+++ openssl-0.9.8b/Makefile.shared Thu Aug 31 17:25:14 2006
@@ -148,13 +148,13 @@

DO_GNU_SO=$(CALC_VERSIONS); \
SHLIB=lib$(LIBNAME).so; \
SHLIB_SUFFIX=; \
ALLSYMSFLAGS='-Wl,--whole-archive'; \
NOALLSYMSFLAGS='-Wl,--no-whole-archive'; \
- SHAREDFLAGS="$(CFLAGS) $(SHARED_LDFLAGS) -shared -Wl,-Bsymbolic -Wl,-soname=$$SHLIB$$SHLIB_SOVER$$SHLIB_SUFFIX"
+ SHAREDFLAGS="$(CFLAGS) $(SHARED_LDFLAGS) -shared -Wl,-Bsymbolic -Wl,-soname=$$SHLIB$$SHLIB_SOVER$$SHLIB_SUFFIX -Wl,-rpath,$(LIBRPATH)"

DO_GNU_APP=LDFLAGS="$(CFLAGS) -Wl,-rpath,$(LIBRPATH)"

#This is rather special. It's a special target with which one can link
#applications without bothering with any features that have anything to
#do with shared libraries, for example when linking against static

__________________________________________________ ____________________
OpenSSL Project http://www.openssl.org
Development Mailing List penssl-dev@openssl.org">openssl-dev@openssl.org
Automated List Manager majordomo@openssl.org







--------------010207090909060607090400--
__________________________________________________ ____________________
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager majordomo@openssl.org