shared libraries ? - Programmer

This is a discussion on shared libraries ? - Programmer ; s> It seems to me, version traceablility is a problem. Say appM requires s> version 3, good luck. Not really. One can solve this by embedding the version number in the DLL basename (e.g. "apps\common\dll\steve.1.2.11.DLL", "apps\common\dll \steve.1.3.15.DLL") and either augmenting ...

+ Reply to Thread
Results 1 to 12 of 12

Thread: shared libraries ?

  1. shared libraries ?

    s> It seems to me, version traceablility is a problem. Say appM
    requires
    s> version <3, and appN requires version >3, good luck.

    Not really. One can solve this by embedding the version number in the
    DLL basename (e.g. "apps\common\dll\steve.1.2.11.DLL", "apps\common\dll
    \steve.1.3.15.DLL") and either augmenting the process loader to be
    version aware or using symbolic links to map version numbers. Shared
    library loaders do this. (The Linux manual page for "ldconfig" is
    your friend.) One can solve it by embedding the version number in the
    DLL pathname (e.g. "apps\steve\steveprog1.2\dll\steve.DLL", "apps\steve
    \steveprog1.3\dll\steve.DLL"), and ensuring that one's system allows
    multiple DLLs with the same module name to be loaded in parallel (in
    different processes) and allows users to reconfigure DLL search paths
    on the fly. Both OS/2 and Windows NT do this (albeit with
    idiosyncracies in both).


  2. Re: shared libraries ?

    J de Boyne Pollard wrote:

    > s> It seems to me, version traceablility is a problem. Say appM
    > requires
    > s> version <3, and appN requires version >3, good luck.
    >
    > Not really. One can solve this by embedding the version number in the
    > DLL basename (e.g. "apps\common\dll\steve.1.2.11.DLL", "apps\common\dll
    > \steve.1.3.15.DLL") and either augmenting the process loader to be
    > version aware or using symbolic links to map version numbers. Shared
    > library loaders do this. (The Linux manual page for "ldconfig" is
    > your friend.) One can solve it by embedding the version number in the
    > DLL pathname (e.g. "apps\steve\steveprog1.2\dll\steve.DLL", "apps\steve
    > \steveprog1.3\dll\steve.DLL"), and ensuring that one's system allows
    > multiple DLLs with the same module name to be loaded in parallel (in
    > different processes) and allows users to reconfigure DLL search paths
    > on the fly. Both OS/2 and Windows NT do this (albeit with
    > idiosyncracies in both).
    >


    No, no, no. This is a reaction to the mess M$ made of shared libraries.
    The appropriate way to do it is to insure that the libraries are
    always upward-compatible. Even if you need to fix a bug, maintain
    compatibility by creating a new entry. Then you will be guaranteed that
    you can always update to the newest version without breaking anything.


  3. Re: shared libraries ?

    On Sep 12, 6:21 am, J de Boyne Pollard
    wrote:
    > s> It seems to me, version traceablility is a problem. Say appM
    > requires
    > s> version <3, and appN requires version >3, good luck.
    >
    > Not really. One can solve this by embedding the version number in the
    > DLL basename (e.g. "apps\common\dll\steve.1.2.11.DLL", "apps\common\dll
    > \steve.1.3.15.DLL") and either augmenting the process loader to be
    > version aware or using symbolic links to map version numbers. Shared
    > library loaders do this. (The Linux manual page for "ldconfig" is
    > your friend.) One can solve it by embedding the version number in the
    > DLL pathname (e.g. "apps\steve\steveprog1.2\dll\steve.DLL", "apps\steve
    > \steveprog1.3\dll\steve.DLL"), and ensuring that one's system allows
    > multiple DLLs with the same module name to be loaded in parallel (in
    > different processes) and allows users to reconfigure DLL search paths
    > on the fly. Both OS/2 and Windows NT do this (albeit with
    > idiosyncracies in both).


    Those are clever approaches, thanks.

    An obverse point to make is that version specific DLL's don't auger
    well as shareable code, especially for apps which end up requiring a
    version. In the above, one has, potentiallly, 2 or more non-shareable
    DLL versions rattling around the enviornment.

    One thing worse than an app which requires a version specific DLL...is
    an app that doesn't know it requires a version specific DLL. If not
    'good luck' programmer, then certainly 'good luck' user.

    ;-) .. Steve


  4. shared libraries ?

    JdeBP> Not really. One can solve this by embedding the
    JdeBP> version number in the DLL basename (e.g.
    JdeBP> "apps\common\dll\steve.1.2.11.DLL",
    JdeBP> "apps\common\dll\steve.1.3.15.DLL") and either augmenting
    JdeBP> the process loader to be version aware or using symbolic
    JdeBP> links to map version numbers. Shared library loaders do
    JdeBP> this. (The Linux manual page for "ldconfig" is your friend.)
    JdeBP> One can solve it by embedding the version number in the
    JdeBP> DLL pathname (e.g. "apps\steve\steveprog1.2\dll\steve.DLL",
    JdeBP> "apps\steve\steveprog1.3\dll\steve.DLL"), and ensuring that
    JdeBP> one's system allows multiple DLLs with the same module
    JdeBP> name to be loaded in parallel (indifferent processes) and
    JdeBP> allows users to reconfigure DLL search paths on the fly.
    JdeBP> Both OS/2 and Windows NT do this (albeit with
    JdeBP> idiosyncracies in both).

    PF> No, no, no. This is a reaction to the mess M$ made of shared
    libraries.

    Wrong. If you lose the knee-jerk "Microsoft did it so it must by
    definition be wrong." thinking, you'll get the right answers more
    often. Without such mental blinkers one is able to read the rest of
    what is written _surrounding_ words such as "Microsoft" and
    "Windows". Reading the above without wearing such mental blinkers,
    for example, reveals that these approaches are not necessarily
    Microsoft's, let alone Microsoft-specific. (Hint: What is your
    friend?)

    You are also erroneously conflating DLLs and (Unix/Linux) shared
    libraries. The two are not the same. There are significant design
    differences.

    PF> The appropriate way to do it is to insure that the libraries are
    PF> always upward-compatible.

    This is not always possible, and not necessarily the correct thing to
    do even where possible. In practice, sometimes an implementation is a
    new architecture that simply doesn't fit older interfaces. In
    practice, sometimes old and new interfaces are irretrievably
    irreconcilable. Sometimes, for example, the bug is in the interface
    itself.


  5. Re: shared libraries ?

    J de Boyne Pollard wrote:
    > JdeBP> Not really. One can solve this by embedding the
    > JdeBP> version number in the DLL basename (e.g.
    > JdeBP> "apps\common\dll\steve.1.2.11.DLL",
    > JdeBP> "apps\common\dll\steve.1.3.15.DLL") and either augmenting
    > JdeBP> the process loader to be version aware or using symbolic
    > JdeBP> links to map version numbers. Shared library loaders do
    > JdeBP> this. (The Linux manual page for "ldconfig" is your friend.)
    > JdeBP> One can solve it by embedding the version number in the
    > JdeBP> DLL pathname (e.g. "apps\steve\steveprog1.2\dll\steve.DLL",
    > JdeBP> "apps\steve\steveprog1.3\dll\steve.DLL"), and ensuring that
    > JdeBP> one's system allows multiple DLLs with the same module
    > JdeBP> name to be loaded in parallel (indifferent processes) and
    > JdeBP> allows users to reconfigure DLL search paths on the fly.
    > JdeBP> Both OS/2 and Windows NT do this (albeit with
    > JdeBP> idiosyncracies in both).
    >
    > PF> No, no, no. This is a reaction to the mess M$ made of shared
    > libraries.
    >
    > Wrong. If you lose the knee-jerk "Microsoft did it so it must by
    > definition be wrong." thinking, you'll get the right answers more
    > often. Without such mental blinkers one is able to read the rest of
    > what is written _surrounding_ words such as "Microsoft" and
    > "Windows". Reading the above without wearing such mental blinkers,
    > for example, reveals that these approaches are not necessarily
    > Microsoft's, let alone Microsoft-specific. (Hint: What is your
    > friend?)


    I realize this, I just hardly ever miss an opportunity to take a jab a
    microsoft.
    >
    > You are also erroneously conflating DLLs and (Unix/Linux) shared
    > libraries. The two are not the same. There are significant design
    > differences.
    >
    > PF> The appropriate way to do it is to insure that the libraries are
    > PF> always upward-compatible.
    >
    > This is not always possible, and not necessarily the correct thing to
    > do even where possible. In practice, sometimes an implementation is a
    > new architecture that simply doesn't fit older interfaces. In
    > practice, sometimes old and new interfaces are irretrievably
    > irreconcilable. Sometimes, for example, the bug is in the interface
    > itself.
    >


    Sometimes. Occasionally. I argue, however, that versioning is overused
    and incompatible changes are introduced unnecessarily with the thought
    that older applications can just use the older version. As someone
    said, this can result in lots of copies of the library on your system,
    and perhaps occupying address space, when only one should be needed.

    I'm not clear on what a "bug in the interface" would be.


  6. shared libraries ?

    s> Those are clever approaches, thanks.

    I take no credit. They have been around for over a decade.

    s> An obverse point to make is that version specific DLL's don't auger
    s> well as shareable code, especially for apps which end up requiring
    a
    s> version.

    You're making an erroneous assumption about what applications should
    require here, from which you are drawing an erroneous conclusion.

    s> One thing worse than an app which requires a version specific DLL
    [...]

    The assumption that you are making is that this is actually the case,
    and that applications always import from exact versions of DLLs. If
    you look at the shared library loader mechanism that I outlined
    before, you'll find that in general whilst the shared library may be
    named "steve.1.2.11.DLL", that exact version isn't necessarily what
    the application requires. The application will require simply "steve.
    1.2.DLL", or even "steve.1.DLL", and a symbolic link resolves that to
    "steve.1.2.11.DLL", "steve.1.2.13.DLL", or whatever revision is
    actually installed. (A library upgrade thus involves adding a file
    and adjusting a symbolic link in this scheme.)

    To use a real-world example: On one system that I have access to, "/
    bin/sh" requires "/lib/libc.so.6", which is simply a symbolic link to
    the _specific_ version of that library that is installed on the system
    that has a binary compatible interface. On that system it happens to
    be "/lib/libc-2.3.2.so". On another system, it is "/lib/libc-2.5.so".

    Going back to the original example, then: That "steve.1.2.11.DLL" and
    "steve.1.3.15.DLL" are both present doesn't necessarily mean that both
    will be loaded when two separate applications are run, and that code
    is thus duplicated. Applications could all be simply importing "steve.
    1.DLL", resolved via a symbolic link to either the one or the other,
    and thus all loading the same library. The symbolic link could be
    pointing to "steve.1.3.15.DLL", for example, with the presence of the
    other DLL being a leftover from an upgrade.

    In the other part of the original example, applications always import
    from simply "steve.DLL", of course. Whether they thereby share a
    single DLL depends from their LIBPATHs at invocation. If the DLLs had
    binary compatible interfaces, one might choose to replace "apps\steve
    \steveprog1.2\dll\" in a default LIBPATH with "apps\steve
    \steveprog1.3\dll\" when doing a library upgrade, for example.


  7. Re: shared libraries ?

    J de Boyne Pollard wrote:

    > s> Those are clever approaches, thanks.
    >
    > I take no credit. They have been around for over a decade.
    >
    > s> An obverse point to make is that version specific DLL's don't auger
    > s> well as shareable code, especially for apps which end up requiring
    > a
    > s> version.
    >
    > You're making an erroneous assumption about what applications should
    > require here, from which you are drawing an erroneous conclusion.
    >
    > s> One thing worse than an app which requires a version specific DLL
    > [...]
    >
    > The assumption that you are making is that this is actually the case,
    > and that applications always import from exact versions of DLLs.


    Yes, I understand that. My complaint is that, especially in the windoze
    world, but also sometimes in unix, not enough emphasis is put on
    upward-compatibility of libraries, leading to "DLL-hell", or
    applications that will only run with a specific version of a library.

    I come from a mainframe background. IBM's take on a "common language
    runtime" or the C standard library is called "Language Environment"
    (LE). We've been running LE for around 10 years (IIRC), through several
    major versions and various minor ones. We *expect* to be able to just
    replace the old library with the new one, independent of compiler
    versions, etc., and just have them work. If they didn't, we'd have a
    legitimate bug report. We've never had an application break. There's
    no way we'd keep multiple versions of libraries around more than
    temporarily to work-around problems.


  8. Re: shared libraries ?


    > (e.g. "apps\steve\steveprog1.2\dll\steve.DLL"


    Possible disadvantages: the OS and/or file system isn't really aimed
    at such solutions, and less demanding users are confronted with some
    overcomplicated structure. I cannot recall any *ix-like distribution
    of an app which offers a choice between Apps\App-1_2a4\bin\dll\Steve
    and just Apps\Steve. If Apps\Steve is enough for you, you probably
    won't care about "this allows you to use more than one version, while
    maintaining the beloved "Apps\" and "\bin\dll\Steve"-parts. FWIW.



    ---

  9. shared libraries ?

    PF> The appropriate way to do it is to insure that the libraries are
    PF> always upward-compatible.

    JdeBP> This is not always possible, and not necessarily the
    JdeBP> correct thing to do even where possible. In practice,
    JdeBP> sometimes an implementation is a new architecture
    JdeBP> that simply doesn't fit older interfaces. In practice,
    JdeBP> sometimes old and new interfaces are irretrievably
    JdeBP> irreconcilable. Sometimes, for example, the bug is in
    JdeBP> the interface itself.

    PF> Sometimes. Occasionally. I argue, however, that versioning is
    PF> overused and incompatible changes are introduced unnecessarily
    PF> with the thought that older applications can just use the older
    PF> version.

    Well since one of the newsgroups here is comp.os.os2.programmer.misc
    we should note that it is almost _never_ used on OS/2, let alone
    overused.

    PF> this can result in lots of copies of the library on your system,
    PF> and perhaps occupying address space, when only one should be
    needed.

    As I pointed out elsewhere, this depends from the needs of the
    application. Sometimes the application needs a highly specific
    version. Sometimes it simply needs something that is binary
    compatible. So sometimes there will be multiple DLLs loaded, because
    that is _required_, and sometimes, if the library upgrade has been
    done properly and the application was appropriately linked, there will
    only be the one DLL loaded.

    PF> I'm not clear on what a "bug in the interface" would be.

    The classic example from Win16 is exporting the Windows Exit
    Procedure. But there are many others. A more general case is
    exporting some implementation detail that should not have been part of
    the interface in the first place, which often happens when the
    programmer flips the compiler switches that automatically export all
    non-static variables and functions. One example from IBM OS/2 is the
    DNS client library in one of the networking DLLs, which exports a
    whole raft of internal variables and functions that are in fact
    internal details of the BIND DNS client and not parts of the API at
    all. That error makes it impossible to write a different
    implementation that is wholly compatible with the entire interface.
    (Indeed, because those internal details - naturally - change with
    different versions of the BIND DNS client library, not even new
    versions of the BIND library itself will support the entire
    interface.) Another case is getting the ABI wrong. An example of
    this, again from IBM OS/2, is the networking DLL that exports
    functions using the calling convention that is specific to IBM's own C/
    C++ compiler, rather than using the system calling convention. Other
    interface bugs include building unwarranted assumptions, tied to a
    specific design, into the interface, causing the interface to break
    when those assuptions are later invalidated.

    http://homepages.tesco.net./J.deBoyn...GA/dll-design-
    mistakes-to-avoid.html>


  10. shared libraries ?

    PF> My complaint is that, especially in the windoze world, but
    PF> also sometimes in unix, not enough emphasis is put on
    PF> upward-compatibility of libraries, leading to "DLL-hell", or
    PF> applications that will only run with a specific version of a
    library.

    Actually, in the Windows world an _enormous_ amount of emphasis is
    placed upon library and API compatibility, albeit backwards
    compatibility not forwards compatibility, by Microsoft. Win32
    applications can still, even on Windows NT version 6, call
    GlobalLock() and _lcreat() functions, for examples, even though in
    Win32 there is no notion of global memory shared across all processes
    and the latter function is defined as using "compatibility" sharing
    mode which has no meaning outside of DOS-based Windows.


  11. shared libraries ?

    JdeBP> [...] (e.g. "apps\steve\steveprog1.2\dll\steve.DLL" [...])
    [...]

    s> Possible disadvantages: the OS and/or file system isn't
    s> really aimed at such solutions, and less demanding users
    s> are confronted with some overcomplicated structure. I cannot
    s> recall any *ix-like distribution of an app which offers a choice
    s> between Apps\App-1_2a4\bin\dll\Steve and just Apps\Steve.

    You need to read .
    You also need to consider "\Program Files\Microsoft Office\Office11\"
    and "\Program Files\Microsoft Office\Office12\"; and also read
    http://homepages.tesco.net./~J.deBoy...dont-put-your-
    dll-in-the-system32-directory.html>, http://msdn2.microsoft.com/
    en-us/library/s2esdf4x(VS.80).aspx>, http://msdn2.microsoft.com/
    en-us/library/ms182186(vs.80).aspx>, and msdn2.microsoft.com/en-us/library/ms649274.aspx>.

    And there is no real world situation in which the structure is
    overcomplicated. Thinking that simpler structures will suffice is
    exactly the design error that has led to the problems that people have
    had over the years. _Learn_ from past mistakes. Don't repeat them.

    "bin" and "dll" are by convention mutually exclusive, by the way. One
    doesn't use both in a single pathname.


  12. Re: shared libraries ?


    > You also need to consider "\Program Files\Microsoft Office\Office11\"
    > and "\Program Files\Microsoft Office\Office12\"; and also read


    I also consider being a stupid user just needing a copy of Office, not
    wanting to lither (limited in size, sorry) PATHs with both copies.

    > And there is no real world situation in which the structure is
    > overcomplicated.


    The latest *ix-like distributions I saw all were overcomplicated in a
    certain way. All basicly required a reboot to run, and all assumed a
    situation of a need for (having) multiple working versions. That can
    typically be read in more than one readme.*.

    > Thinking that simpler structures will suffice is exactly the design
    > error that has led to the problems that people have had over the
    > years.


    That doesn't mean that the complicated structure is the solution. It
    just is "a" solution, covering a limited number of situations. I can
    expect a power-user to (perhaps) care and to think about that, but I
    cannot recall any of those file distribution keeping in mind that an
    user may not have nor care about any of the possible problems. I have
    to say that a complicated structure typically has a clean/empty root
    directory, but one may not care about that too.

    > _Learn_ from past mistakes. Don't repeat them.


    The same is true for simple structures, the mistake then being some
    assumption that all users (could) have a need for a more complicated
    structure and are faced with that. The installer of Office11 may not
    even check the existence of Office10-2_a2, in a typical situation it
    assumes one could have multiple Offices instead. There's no need nor
    requirement to assume anything here.

    > "bin" and "dll" are by convention mutually exclusive, by the way.
    > One doesn't use both in a single pathname.


    Another BTW is that OS/2 has no file system structure like original
    *ix'es have. We cannot really use \Progra~1\Micros~1\Office11=\Off11,
    so the choosen solution doesn't match the OS that good.

    I don't want to argue with anything you said, I actually just spent
    some time deleting a second, fatal instance of a DLL in \ECS\DLL (a
    WarpIN-problem). But that doesn't mean (over)complicated structures
    are "the" solution. It's just a way to possibly avoid very specific
    problems, while making installing such apps a PITA. If you don't have
    such a situation, you're still faced with the complicated structure
    which doesn't really match the OS and OS file system. Maybe it saves
    the developer some time, which is okay with me, but I just wanted to
    make the remark that (over)complicated structures aren't the one and
    only solution. Because not everybody needs multiple instances of the
    same app nor cares about that, but also because OS'es are different
    and we don't have true "links". With a possible result that you can
    develop a cool app with an adopted complicated structure, while an
    user may forget about it because alternative solutions are less
    complicated to apply and use. Let's put it this way: I don't like to
    spent over 10% of limited CONFIG.SYS-resources to apps used seldomly
    while such *ix-like file distribution like to use the opposite. Yes,
    it may technically be perfect. No, technicians don't rule the world
    and a simple user may just be happy with one working Office without
    ever having a need for more than one installed, ready-to-run Office.
    Complicated structures don't keep that in mind, I'ld say, and there's
    no need to "assume" multiple instances. Better safe than sorry is a
    possible choice here, but that choice still does assume a "possibly
    unsafe" situation.

    My WarpIN perhaps may not have crashed using \WarpIN\1-0-14_a2_r1, a
    possible solution. OTOH it could have detected the error (WPIRTL.DLL
    in \ECS\DLL), avoiding the need to add every bloody instance of this
    app to system setup files, and avoiding the need to use a complicated
    structure. You're not wrong, but (over)complicated structures are "a"
    possible solution for a possible problem. I'ld like (over)complicated
    structures slightly better if I could always use a tiny link instead
    of the real, underlying directory structure, but I can't. And all I
    really want is to just being able to use one working copy of WarpIN,
    that'll do for me.



    ---

+ Reply to Thread