[RFC] Adding udebs in shlibs files for glibc - Debian

This is a discussion on [RFC] Adding udebs in shlibs files for glibc - Debian ; As you may know, dependencies on glibc udebs are the only category that is still incorrect [1]. This is an attempt to fix that. Normally we would add an option '--udeb ' to the dh_makeshlibs call in debian/rules. However, in ...

+ Reply to Thread
Results 1 to 17 of 17

Thread: [RFC] Adding udebs in shlibs files for glibc

  1. [RFC] Adding udebs in shlibs files for glibc

    As you may know, dependencies on glibc udebs are the only category that is
    still incorrect [1]. This is an attempt to fix that.

    Normally we would add an option '--udeb ' to the dh_makeshlibs
    call in debian/rules. However, in the case of glibc this does not work.
    Reason is that two libraries (libnss-dns and libnss-files) that are both in
    the regular libc6 package, have been split out into separate udebs.
    Another (minor) reason is that not all libraries are copied into udebs, so
    adding udeb: lines in the shlibs files for those seems redundant.

    Solution I have come up with is to write a small script that will generate
    the udeb: lines _after_ dh_makeshlibs has generated the basic shlibs file.

    As an example, the output of the script on amd64 is:
    udeb: ld-linux-x86-64 2 libc6-udeb (>= 2.7-1)
    udeb: libm 6 libc6-udeb (>= 2.7-1)
    udeb: libdl 2 libc6-udeb (>= 2.7-1)
    udeb: libresolv 2 libc6-udeb (>= 2.7-1)
    udeb: libc 6 libc6-udeb (>= 2.7-1)
    udeb: libutil 1 libc6-udeb (>= 2.7-1)
    udeb: libcrypt 1 libc6-udeb (>= 2.7-1)
    udeb: librt 1 libc6-udeb (>= 2.7-1)
    udeb: libpthread 0 libc6-udeb (>= 2.7-1)
    udeb: libnss_dns 2 libnss-dns-udeb (>= 2.7-1)
    udeb: libnss_files 2 libnss-files-udeb (>= 2.7-1)

    Any comments on this approach and the patch before I proceed [2] with this?

    Joey: do you see any chance to incorporate this in debhelper, or is adding
    it glibc the better option?

    Cheers,
    FJP

    [1] Except maybe the deps of some components we currently don't use.
    See also: http://wiki.debian.org/DebianInstaller/LibraryUdebs
    [2] Before I submit the patch I would at least like to do some build and
    run tests with this to see if there'd be any transition issues.


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQBHrnddgm/Kwh6ICoQRAjjqAJ972I2dg0kBU3guXWo8zyA3XBW7xACfXJLx
    6If2plXK2LdjbSS4ZAEP8DM=
    =TyX6
    -----END PGP SIGNATURE-----


  2. Re: [RFC] Adding udebs in shlibs files for glibc

    On Sunday 10 February 2008, Frans Pop wrote:
    > Solution I have come up with is to write a small script that will
    > generate the udeb: lines _after_ dh_makeshlibs has generated the basic
    > shlibs file.


    One thing I'm unsure of is whether or not I was correct in excluding shlibs
    files for library packages like:
    - on amd64: libc6-i386
    - on i386: libc6-amd64, libc6-i686, libc6-xen

    Can other udebs be built using those or not?
    If they can, then the solution is probably to just chop off the second part
    of the package name when determining the udeb name.

    Cheers,
    FJP

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQBHrwDrgm/Kwh6ICoQRAjpXAKC+56TG0CCpB6JtYVe1dd6MOBs3UACgvGbr
    /rwzgdDCOg41mcqymUotDzI=
    =EKvi
    -----END PGP SIGNATURE-----


  3. Dependencies on libnewt0.52 (was: Adding udebs in shlibs files for glibc)

    On Sunday 10 February 2008, Frans Pop wrote:
    > As you may know, dependencies on glibc udebs are the only category that
    > is still incorrect.


    Well, there is actually another category. We have one library (libnewt0.52)
    that is only included in D-I initrds through library reduction and for
    which no udeb exists.
    Udebs that depend on it are: cdebconf-newt-udeb and cdebconf-newt-entropy.

    I see two solutions for this.

    Option 1
    --------
    Give libnewt0.52 a proper shlibs file changing the dependencies to
    'libnewt-udeb' and add an empty libnewt-udeb package in the newt source
    package so that dependencies can be satisfied.

    Option 2
    --------
    Hack cdebconf-newt-udeb and cdebconf-newt-entropy so that their dependencies
    on libnewt0.52 get dropped.

    The first option seems cleaner and therefore has my preference.

    The reason I think we should fix this is that we want britney to check
    dependencies before migrating udebs to testing and it seems to me that all
    dependencies of udebs should be satisfiable by udebs and crosschecks
    against regular packages should not be needed.

    If there are no objections, I will prepare a patch against newt to implement
    the first option.

    Cheers,
    FJP

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQBHrwUCgm/Kwh6ICoQRAkTdAJ9KtMK72ok4t8QgpSZKSilhbNJWjwCgvM49
    onjoTa3RBWdxBq2/T4k2X30=
    =vx4O
    -----END PGP SIGNATURE-----


  4. Re: Dependencies on libnewt0.52 (was: Adding udebs in shlibs files for glibc)

    On Sun, Feb 10, 2008 at 03:06:57PM +0100, Frans Pop wrote:
    > Well, there is actually another category. We have one library (libnewt0.52)
    > that is only included in D-I initrds through library reduction and for
    > which no udeb exists.
    > Udebs that depend on it are: cdebconf-newt-udeb and cdebconf-newt-entropy.
    >
    > I see two solutions for this.
    >
    > Option 1
    > --------
    > Give libnewt0.52 a proper shlibs file changing the dependencies to
    > 'libnewt-udeb' and add an empty libnewt-udeb package in the newt source
    > package so that dependencies can be satisfied.
    >
    > Option 2
    > --------
    > Hack cdebconf-newt-udeb and cdebconf-newt-entropy so that their dependencies
    > on libnewt0.52 get dropped.
    >
    > The first option seems cleaner and therefore has my preference.


    The first option seems better, indeed, but I still find strange to have
    an empty package in the archive.

    Would it be possible to put libnewt.so* in the libnewt-udeb package and
    make the library reduction step use it instead of the system library?
    I would find such behaviour easier to understand; and we could tailor a
    special version of newt closer to our needs if that is needed in the
    future.

    Cheers,
    --
    Jérémy Bobbio .''`.
    lunar@debian.org : :Ⓐ : # apt-get install anarchism
    `. `'`
    `-

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFHsCbX2PUjs9fQ72URAiokAJ0Up9aXxb9A9MSxqjYApy aluohiRACgqHdG
    BwBD87cy9WHLEk+PVuvJf8U=
    =voNs
    -----END PGP SIGNATURE-----


  5. Re: Dependencies on libnewt0.52 (was: Adding udebs in shlibs files for glibc)

    On Monday 11 February 2008, Jérémy Bobbio wrote:
    > Would it be possible to put libnewt.so* in the libnewt-udeb package and
    > make the library reduction step use it instead of the system library?
    > I would find such behaviour easier to understand; and we could tailor a
    > special version of newt closer to our needs if that is needed in the
    > future.


    No, that is not possible, just as it is not possible for libc and other
    libraries that get reduced. Or at least, it's not possible to do reduction
    _solely_ on the basis of a udeb (you need the pic files).

    OTOH, just adding the lib in the udeb cannot do any harm I guess, except
    that it could result in the full library getting loaded during anna, which
    would be a waste of memory. I'll look at that option though.

    Thanks.

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQBHsFMIgm/Kwh6ICoQRAhIhAJ9gi19eWGdU1q/9ZkhUhSNRykOWhQCgra7o
    er5jUF7XBdPa1gJ9Le7gjY8=
    =9gU5
    -----END PGP SIGNATURE-----


  6. Re: Dependencies on libnewt0.52

    Frans Pop writes:

    <...>
    > OTOH, just adding the lib in the udeb cannot do any harm I guess, except
    > that it could result in the full library getting loaded during anna, which
    > would be a waste of memory. I'll look at that option though.


    It makes sense. Even we waste some memory it would end up being a
    clearer solution IMO.

    --
    O T A V I O S A L V A D O R
    ---------------------------------------------
    E-mail: otavio@debian.org UIN: 5906116
    GNU/Linux User: 239058 GPG ID: 49A5F855
    Home Page: http://otavio.ossystems.com.br
    ---------------------------------------------
    "Microsoft sells you Windows ... Linux gives
    you the whole house."


    --
    To UNSUBSCRIBE, email to debian-boot-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  7. Re: Dependencies on libnewt0.52 (was: Adding udebs in shlibs files for glibc)

    Frans Pop wrote:
    > OTOH, just adding the lib in the udeb cannot do any harm I guess, except
    > that it could result in the full library getting loaded during anna, which
    > would be a waste of memory. I'll look at that option though.


    Well, that won't happen for most images, since the udeb would be
    pre-installed by the build process.

    --
    see shy jo

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFHsL9od8HHehbQuO8RAiRrAKDMjKzOK63piynhKiBY/XUJbH5qKwCg3qqc
    r3c4T36UcNCxaKD0Fy6ZYec=
    =D47o
    -----END PGP SIGNATURE-----


  8. Re: [RFC] Adding udebs in shlibs files for glibc

    Frans Pop wrote:
    > As you may know, dependencies on glibc udebs are the only category that is
    > still incorrect [1]. This is an attempt to fix that.
    >
    > Normally we would add an option '--udeb ' to the dh_makeshlibs
    > call in debian/rules. However, in the case of glibc this does not work.
    > Reason is that two libraries (libnss-dns and libnss-files) that are both in
    > the regular libc6 package, have been split out into separate udebs.
    > Another (minor) reason is that not all libraries are copied into udebs, so
    > adding udeb: lines in the shlibs files for those seems redundant.
    >
    > Solution I have come up with is to write a small script that will generate
    > the udeb: lines _after_ dh_makeshlibs has generated the basic shlibs file.
    >
    > As an example, the output of the script on amd64 is:
    > udeb: ld-linux-x86-64 2 libc6-udeb (>= 2.7-1)
    > udeb: libm 6 libc6-udeb (>= 2.7-1)
    > udeb: libdl 2 libc6-udeb (>= 2.7-1)
    > udeb: libresolv 2 libc6-udeb (>= 2.7-1)
    > udeb: libc 6 libc6-udeb (>= 2.7-1)
    > udeb: libutil 1 libc6-udeb (>= 2.7-1)
    > udeb: libcrypt 1 libc6-udeb (>= 2.7-1)
    > udeb: librt 1 libc6-udeb (>= 2.7-1)
    > udeb: libpthread 0 libc6-udeb (>= 2.7-1)
    > udeb: libnss_dns 2 libnss-dns-udeb (>= 2.7-1)
    > udeb: libnss_files 2 libnss-files-udeb (>= 2.7-1)
    >
    > Any comments on this approach and the patch before I proceed [2] with this?
    >
    > Joey: do you see any chance to incorporate this in debhelper, or is adding
    > it glibc the better option?


    Since splitting udebs differently than the debs are split is a special
    case that seems likely to affect other libraries than glibc, I'd prefer
    to keep the special case in glibc, rather than in dehelper.

    BTW, has anyone looked at how library symbols files interact with udeb:
    lines in shlibs files? The symbols file tend to override the shlibs
    files, and I wonder if this won't result in bad dependencies in udebs as
    the libraries they depend on are converted to have symbols files.

    --
    see shy jo

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFHsMNmd8HHehbQuO8RAjXVAKCDlRVueU0+RDLgnSud+c uX9jmXGACdFhpb
    dQ1WR2DwmGsiXZmaQLid/Vk=
    =+pVm
    -----END PGP SIGNATURE-----


  9. Re: [RFC] Adding udebs in shlibs files for glibc

    On Monday 11 February 2008, Joey Hess wrote:
    > Since splitting udebs differently than the debs are split is a special
    > case that seems likely to affect other libraries than glibc, I'd prefer
    > to keep the special case in glibc, rather than in dehelper.


    s/likely/unlikely/ I guess. Yes, I thought as much.

    > BTW, has anyone looked at how library symbols files interact with udeb:
    > lines in shlibs files? The symbols file tend to override the shlibs
    > files, and I wonder if this won't result in bad dependencies in udebs as
    > the libraries they depend on are converted to have symbols files.


    I haven't and doubt anyone else has. As this seems to be my area now, I'll
    contact the initiators of that scheme.

    Cheers,
    FJP

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQBHsMsYgm/Kwh6ICoQRAjMWAJoDvUPE3R/IuRHuRbjy80QqXxeD8QCg0IKB
    16+9oTOtI2ByI5CCH6kUJg0=
    =H6xE
    -----END PGP SIGNATURE-----


  10. Re: Dependencies on libnewt0.52 (was: Adding udebs in shlibs files for glibc)

    On Monday 11 February 2008, Joey Hess wrote:
    > Frans Pop wrote:
    > > OTOH, just adding the lib in the udeb cannot do any harm I guess,
    > > except that it could result in the full library getting loaded during
    > > anna, which would be a waste of memory. I'll look at that option
    > > though.

    >
    > Well, that won't happen for most images, since the udeb would be
    > pre-installed by the build process.


    Don't think that is always true in this case.
    Because cdebconf-newt-entropy depends on it and because that is not included
    in the initrs (and thus installed later), I think that will pull in the
    full library (just as the full glibc udeb is pulled in during anna).

    Of course, that would not happen if udpkg can be made to think that it is
    already installed, which would mean removing it from pkg-lists/exclude.

    Is it strictly necessary that libslang2-udeb and libnewt0.52 are excluded?
    AFAIK mklibs should replace the lib from the udeb by the reduced version, so
    I think it should be safe to not exclude them as long as we ensure no
    needed symbols are reduced out (which we do in the build process).


    BTW, could it be that this comment in pkg-lists/exclude is obsolete?
    # some arches and versions of glibc link against libgcc1,
    # it will be pulled in via library reduction
    libgcc1 -

    If I scan all current udebs I find no dependencies on libgcc*.

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQBHsMqKgm/Kwh6ICoQRAtiDAJ0ch9ReqWCu99/KMupqHJ0Q5kzUsQCgu4Ih
    rRMIzZNFsS0V13uKNiP1+lw=
    =3Zhi
    -----END PGP SIGNATURE-----


  11. Re: Dependencies on libnewt0.52 (was: Adding udebs in shlibs files for glibc)

    On Sunday 10 February 2008, Frans Pop wrote:
    > Give libnewt0.52 a proper shlibs file changing the dependencies to
    > 'libnewt-udeb' and add an empty libnewt-udeb package in the newt source
    > package so that dependencies can be satisfied.


    Just to make sure that it's archived somewhere, here's a patch for newt that
    adds a udeb (with the library) and generates a correct shlibs file.

    I need to do some testing with it before submitting the patch.


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQBHsNmOgm/Kwh6ICoQRAtKiAJ9GyHNRSLG/fUa5mGYv0OAVjFg+0QCfXsC8
    UoO37qumAcmytBDN9lSizm4=
    =udKd
    -----END PGP SIGNATURE-----


  12. Re: Dependencies on libnewt0.52 (was: Adding udebs in shlibs files for glibc)

    Frans Pop wrote:
    > Don't think that is always true in this case.
    > Because cdebconf-newt-entropy depends on it and because that is not included
    > in the initrs (and thus installed later), I think that will pull in the
    > full library (just as the full glibc udeb is pulled in during anna).


    libc6-udeb is pulled in during anna because the initrds do not have a
    udeb by that name installed on them. That would not be the case with a
    real newt udeb that was really installed and then overwritten by the
    reduced library.

    > BTW, could it be that this comment in pkg-lists/exclude is obsolete?
    > # some arches and versions of glibc link against libgcc1,
    > # it will be pulled in via library reduction
    > libgcc1 -
    >
    > If I scan all current udebs I find no dependencies on libgcc*.


    Have you ldd's the libc on all arches? AFAIK libgcc1 is used on some 64 bit
    arches or such.

    --
    see shy jo

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFHsOZAd8HHehbQuO8RAqDgAKDlbHxi3nXmYFuA5c0M8p c/tC+nXwCfbuLo
    6/BGDvrRpU5ZuvhrIj3pAPQ=
    =k67c
    -----END PGP SIGNATURE-----


  13. Re: Dependencies on libnewt0.52 (was: Adding udebs in shlibs files for glibc)

    On Tuesday 12 February 2008, Joey Hess wrote:
    > Frans Pop wrote:
    > > BTW, could it be that this comment in pkg-lists/exclude is obsolete?
    > > # some arches and versions of glibc link against libgcc1,
    > > # it will be pulled in via library reduction
    > > libgcc1 -
    > >
    > > If I scan all current udebs I find no dependencies on libgcc*.

    >
    > Have you ldd's the libc on all arches? AFAIK libgcc1 is used on some 64
    > bit arches or such.


    No. I only checked the dependencies of all udebs. Shouldn't it show up as a
    dependency of libc*-udeb for those arches?
    [...]
    Ah, no. I see that the glibc control file for libc*-udebs does not contain
    any Depends: lines (and neither does libc6 itself).

    Oh, well. There's probably a good reason for that.

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQBHsOzBgm/Kwh6ICoQRAvuUAJ9wIoTuxhSKchFvE72kkBA3Nm/axwCfYWvb
    cvbtRVF/yjNQmkVUeh0yK28=
    =ZST1
    -----END PGP SIGNATURE-----


  14. Re: [RFC] Adding udebs in shlibs files for glibc

    On Mon, 11 Feb 2008, Frans Pop wrote:
    > > BTW, has anyone looked at how library symbols files interact with udeb:
    > > lines in shlibs files? The symbols file tend to override the shlibs
    > > files, and I wonder if this won't result in bad dependencies in udebs as
    > > the libraries they depend on are converted to have symbols files.

    >
    > I haven't and doubt anyone else has. As this seems to be my area now, I'll
    > contact the initiators of that scheme.


    Symbols files are not used for udebs. So they won't affect dependencies of
    udebs.

    Cheers,
    --
    Raphal Hertzog

    Le best-seller franais mis jour pour Debian Etch :
    http://www.ouaza.com/livre/admin-debian/


    --
    To UNSUBSCRIBE, email to debian-boot-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  15. Re: [RFC] Adding udebs in shlibs files for glibc

    Hi Raphael,

    Thanks for the preemptive response :-)

    On Wednesday 13 February 2008, Raphael Hertzog wrote:
    > On Mon, 11 Feb 2008, Frans Pop wrote:
    > > > BTW, has anyone looked at how library symbols files interact with
    > > > udeb: lines in shlibs files? The symbols file tend to override the
    > > > shlibs files, and I wonder if this won't result in bad dependencies
    > > > in udebs as the libraries they depend on are converted to have
    > > > symbols files.

    > >
    > > I haven't and doubt anyone else has. As this seems to be my area now,
    > > I'll contact the initiators of that scheme.

    >
    > Symbols files are not used for udebs. So they won't affect dependencies
    > of udebs.


    Not that I don't trust you about this, but just to make sure...

    Do symbol files affect the way shlibs files are generated, or will those
    remain unchanged?
    If shlibs files are changed, then I think the impact on udebs should be
    verified. Or have you already tested this specifically?

    Cheers,
    FJP

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQBHssUzgm/Kwh6ICoQRAj0CAKDEcN+S3tMrCYAIkHcW/NcV66nCSACgxrjP
    7dwwwO0DhTwhYFsISFXg6QI=
    =Wubb
    -----END PGP SIGNATURE-----


  16. Re: [RFC] Adding udebs in shlibs files for glibc

    On Wed, 13 Feb 2008, Frans Pop wrote:
    > > Symbols files are not used for udebs. So they won't affect dependencies
    > > of udebs.

    >
    > Not that I don't trust you about this, but just to make sure...
    >
    > Do symbol files affect the way shlibs files are generated, or will those
    > remain unchanged?


    Those remain unchanged (unless the maintainer decide to change the way
    they are generated...). It's possible that some maintainers care less about
    shlibs once they added support of symbols files... they could decide to
    use dh_makeshlibs -V instead of providing the minimal version that should
    be needed.

    > If shlibs files are changed, then I think the impact on udebs should be
    > verified. Or have you already tested this specifically?


    I haven't made any extra-test for udebs, however in the dpkg-shlibdeps
    code, symbols files are only looked up if ($package_type eq "deb"). So
    if dpkg-shlibdeps has the "-tudeb" parameter, the dependency generation
    should remain unchanged.

    Cheers,
    --
    Raphal Hertzog

    Le best-seller franais mis jour pour Debian Etch :
    http://www.ouaza.com/livre/admin-debian/


    --
    To UNSUBSCRIBE, email to debian-boot-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  17. Re: [RFC] Adding udebs in shlibs files for glibc

    On Sunday 10 February 2008, Frans Pop wrote:
    > As you may know, dependencies on glibc udebs are the only category that
    > is still incorrect [1]. This is an attempt to fix that.


    JFYI.
    After some more testing and minor changes in the patch, I have just filed
    the BR against glibc to request this:http://bugs.debian.org/474293.

    Cheers,
    FJP

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQBH9omwgm/Kwh6ICoQRApeqAKC79smf/aNKs24YGIQCBJdA+26gcQCeKURm
    t9vEopGfWv1D+uCnoWQ5pRM=
    =j19f
    -----END PGP SIGNATURE-----


+ Reply to Thread