net-snmp build system anomalities - SNMP

This is a discussion on net-snmp build system anomalities - SNMP ; Hi all, This email is an attempt at constructive criticism. I have just started to work with net-snmp, and have ran across several things having to do with the build system that I believe could be done better. Personally, I ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: net-snmp build system anomalities

  1. net-snmp build system anomalities

    Hi all,

    This email is an attempt at constructive criticism. I have just started
    to work with net-snmp, and have ran across several things having to do
    with the build system that I believe could be done better. Personally, I
    tend to use automake where applicable, and for me it provides a
    convenient way of resolving most of the following issues with very
    little work on my part (the rest are issues that should have been
    handled by autoconf, and I'm not sure why they were not).

    Of course, not everyone are a fan of automake, and I can perfectly
    understand if you choose not to. Still, I think it would be constructive
    to look at the issues themselves.

    Semantic and minor problems:

    * Non standard compiler/linker etc. specification - usually
    performed by the command line environment, the configure help
    directs to using command line
    * Partly as a result of the above - the help strings take a huge
    area instead of using "AS_HELP_STRING"
    * Non-standard compiler choice when cross compiling. For most
    projects, merely having (in my case) "arm-unknown-linux-gnu-gcc"
    in my path while running configure with
    "--host=arm-unkown-linux-gnu" will cause configure to pick that as
    my compiler. This does not work with your build environment (I'm
    not even sure why - maybe it's just an old version of autoconf).

    More functional issues:

    * Incorrect use of AC_C_BIGENDIAN. Instead of merely using it, and
    letting it figure out the endianity on its own (which it usually
    can), you only invoke it if we are not cross compiling. I'm trying
    to cross compile for arm. I have confirmed that AC_C_BIGENDIAN
    manages to figure out the endianity without any help from me, but
    your build script never gives it a chance, so when building
    net-snmp I need to manually specify the endianity for no good reason.
    * Incorrect DESTDIR behavior. While running "make install DESTDIR="
    does install the files to the right place, it runs "ldconfig" over
    the base path (i.e. - $prefix/lib rather than
    $DESTDIR/$prefix/lib), which causes problems if you are not root,
    and even more serious problems if you are.
    * Missing semi-standard targets. Automake generated makefiles have
    the "install-strip" target, which installs the files and strips
    them in one go. This is extremely convenient, especially when
    cross-compiling a non-standard strip is used (namely -
    arm-unkown-linux-gnu-strip, automatically determined by autoconf).
    * Incorrect concurrent build behavior. At the moment, net-snmp is
    the project in my environment which takes the longest to build. I
    would love to be able to do "make -j2" to speed things up, but
    doing so causes the build to fail. Apparently, it tries to build
    something that needs a library before that library is ready.
    Again, automake usually makes sure that these things are correctly
    handled.

    These are the issues I found so far. I'd love to offer any help I can in
    resolving them (including sending patches), but I would like to first
    know that you also think these are problems and whether the suggested
    solution is an acceptable one to you.

    Thanks,
    Shachar

    -------------------------------------------------------------------------
    This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
    Build the coolest Linux based applications with Moblin SDK & win great prizes
    Grand prize is a trip for two to an Open Source event anywhere in the world
    http://moblin-contest.org/redirect.p...r_id=100&url=/
    _______________________________________________
    Net-snmp-coders mailing list
    Net-snmp-coders@lists.sourceforge.net
    https://lists.sourceforge.net/lists/...et-snmp-coders


  2. Re: net-snmp build system anomalities

    2008/9/18 Shachar Shemesh :
    > This email is an attempt at constructive criticism.


    Noted - and taken as such.

    > I have just started to work with net-snmp, and have ran
    > across several things having to do with the build system
    > that I believe could be done better.


    Almost certainly true.
    The Net-SNMP build configuration mechanism has
    grown incrementally over the years, to a point where
    it strikes fear into the bones of even the most resilient
    programmer! We've been wanting to improve things
    for some time, but have never had the strength of
    purpose to actually address this herculean task.

    However, over the last few weeks, we have finally
    started to do something about this nightmare. The
    initial work has concentrated on simplifying and
    clarifying the organisation of configure script. If you
    have a look at the main development branch of the
    code, you can see where we've got to so far.

    If you are interested in improving the build environment,
    then this is probably the best starting point. I don't think
    any of the core developers would claim to be autoconf
    experts, so we would be delighted to take advantage of
    (and shamelessly exploit) anyone with a fuller understanding
    of these issues.


    Dave

    -------------------------------------------------------------------------
    This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
    Build the coolest Linux based applications with Moblin SDK & win great prizes
    Grand prize is a trip for two to an Open Source event anywhere in the world
    http://moblin-contest.org/redirect.p...r_id=100&url=/
    _______________________________________________
    Net-snmp-coders mailing list
    Net-snmp-coders@lists.sourceforge.net
    https://lists.sourceforge.net/lists/...et-snmp-coders


  3. Re: net-snmp build system anomalities

    On Thu, 2008-09-18 at 16:03 +0300, Shachar Shemesh wrote:
    > Hi all,
    >
    > This email is an attempt at constructive criticism. I have just started
    > to work with net-snmp, and have ran across several things having to do
    > with the build system that I believe could be done better. Personally, I
    > tend to use automake where applicable, and for me it provides a
    > convenient way of resolving most of the following issues with very
    > little work on my part (the rest are issues that should have been
    > handled by autoconf, and I'm not sure why they were not).
    >
    > Of course, not everyone are a fan of automake, and I can perfectly
    > understand if you choose not to. Still, I think it would be constructive
    > to look at the issues themselves.
    >
    > Semantic and minor problems:
    >
    > * Non standard compiler/linker etc. specification - usually
    > performed by the command line environment, the configure help
    > directs to using command line


    I think the compiler-choice patch is what you are asking for?
    For the rest of you, do you think it is a good thing?

    > * Non-standard compiler choice when cross compiling. For most
    > projects, merely having (in my case) "arm-unknown-linux-gnu-gcc"
    > in my path while running configure with
    > "--host=arm-unkown-linux-gnu" will cause configure to pick that as
    > my compiler. This does not work with your build environment (I'm
    > not even sure why - maybe it's just an old version of autoconf).


    We are using 2.59 (on branches) and 2.61 (on trunk). I do not think they
    are extremely old.

    Does the compiler-choice patch help for this issue as well?

    > More functional issues:
    >
    > * Incorrect use of AC_C_BIGENDIAN. Instead of merely using it, and
    > letting it figure out the endianity on its own (which it usually
    > can), you only invoke it if we are not cross compiling. I'm trying
    > to cross compile for arm. I have confirmed that AC_C_BIGENDIAN
    > manages to figure out the endianity without any help from me, but
    > your build script never gives it a chance, so when building
    > net-snmp I need to manually specify the endianity for no good reason.


    I think the big-endian patch is the way to handle this.
    One other thing one could do is to make the endianess forcing depend on
    cross_compiling.
    This one I am inclined to apply right away.

    /MF

    -------------------------------------------------------------------------
    This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
    Build the coolest Linux based applications with Moblin SDK & win great prizes
    Grand prize is a trip for two to an Open Source event anywhere in the world
    http://moblin-contest.org/redirect.p...r_id=100&url=/
    _______________________________________________
    Net-snmp-coders mailing list
    Net-snmp-coders@lists.sourceforge.net
    https://lists.sourceforge.net/lists/...et-snmp-coders


  4. Re: net-snmp build system anomalities

    Magnus Fromreide wrote:
    > I think the compiler-choice patch is what you are asking for?
    > For the rest of you, do you think it is a good thing?


    What problems does it promise to solve by just dropping support for
    --with-{cc,cflags,ld,ldflags,ar,libs}?

    > I think the big-endian patch is the way to handle this.
    > One other thing one could do is to make the endianess forcing depend on
    > cross_compiling.
    > This one I am inclined to apply right away.


    +1.


    +Thomas

    -------------------------------------------------------------------------
    This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
    Build the coolest Linux based applications with Moblin SDK & win great prizes
    Grand prize is a trip for two to an Open Source event anywhere in the world
    http://moblin-contest.org/redirect.p...r_id=100&url=/
    _______________________________________________
    Net-snmp-coders mailing list
    Net-snmp-coders@lists.sourceforge.net
    https://lists.sourceforge.net/lists/...et-snmp-coders


  5. Re: net-snmp build system anomalities

    On Sat, 2008-09-20 at 07:26 +0200, Thomas Anders wrote:
    > Magnus Fromreide wrote:
    > > I think the compiler-choice patch is what you are asking for?
    > > For the rest of you, do you think it is a good thing?

    >
    > What problems does it promise to solve by just dropping support for
    > --with-{cc,cflags,ld,ldflags,ar,libs}?


    None, really.
    The problem was listed as semantic or minor so this just takes us a
    small step in the direction of a more standard autoconf interface.

    Note that you still have the ability to specify the variables on the
    command line but the syntax becomes

    ../configure CFLAGS=-qodd-compiler-flag

    /MF


    -------------------------------------------------------------------------
    This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
    Build the coolest Linux based applications with Moblin SDK & win great prizes
    Grand prize is a trip for two to an Open Source event anywhere in the world
    http://moblin-contest.org/redirect.p...r_id=100&url=/
    _______________________________________________
    Net-snmp-coders mailing list
    Net-snmp-coders@lists.sourceforge.net
    https://lists.sourceforge.net/lists/...et-snmp-coders


  6. Re: net-snmp build system anomalities

    I've just subscribed to this list, and not all messages in this thread
    appear in the web archives, so forgive me if I'm answering a little out
    of context.

    Thomas Anders wrote:
    > Magnus Fromreide wrote:
    >
    >> I think the compiler-choice patch is what you are asking for?
    >> For the rest of you, do you think it is a good thing?
    >>

    >
    > What problems does it promise to solve by just dropping support for
    > --with-{cc,cflags,ld,ldflags,ar,libs}?
    >

    The problem with the duplicate configuration methods is that things
    break, and it's not clear why. For example (given at the start of this
    thread), for a simple project compiled with autoconf, merely passing
    "--host=arm-unknown-linux-unknown" will automatically cause the
    resulting makefile to search for "arm-unknown-linux-unknown-gcc" as the
    compiler tool, "arm-unknown-linux-unknown-ldd" as the linker tool,
    "arm-unknown-linux-unknown-strip" as the strip tool etc. We know that
    this is not the case for net-snmp, and we're not sure why. The explicit
    (read - redundant) compiler setting is a likely culprit.

    I've started looking at what it would take to move the project over to
    automake. From the preliminary work I've done it seems that automake
    will reduce the build system source code size by about 80%(1), plus
    solve many (if not all) of the problems mentioned in my original email.
    While alternate mechanism for specifying build tools is not exactly
    contradicting to using automake, it does make life more complicated (and
    enhances the chances for bugs, as we may have already seen happen)

    The main question, as I see it, is this: Environment passing to
    configure(2) is the way its done for virtually any other project out
    there. Is there a reason why net-snmp needs to be different in that regard?

    Shachar

    1 - I've only started to work on it, and it obviously is going to take a
    while. So far, I haven't seen anything that cannot fit, neatly, into the
    models suggested by automake, but I cannot claim to know the net-snmp
    build system well enough to be sure. In any case, nuances will have to
    be checked by someone better versed with what net-snmp needs to do than
    me. I'll send a patch when there is something to show.

    2 - Just in case that is not understood - the environment variables need
    to be set when running "configure". The environment need not be set when
    running "make", as configure embeds the values into the generated make file.

    -------------------------------------------------------------------------
    This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
    Build the coolest Linux based applications with Moblin SDK & win great prizes
    Grand prize is a trip for two to an Open Source event anywhere in the world
    http://moblin-contest.org/redirect.p...r_id=100&url=/
    _______________________________________________
    Net-snmp-coders mailing list
    Net-snmp-coders@lists.sourceforge.net
    https://lists.sourceforge.net/lists/...et-snmp-coders


  7. Re: net-snmp build system anomalities

    On Sat, 2008-09-20 at 09:46 +0300, Shachar Shemesh wrote:
    > I've just subscribed to this list, and not all messages in this thread
    > appear in the web archives,


    The SF mail archives are extremely slow.
    I would use http://marc.info/?l=net-snmp-coders&r=1&w=2 to get up to to
    speed on the list.

    > so forgive me if I'm answering a little out of context.


    :-)

    > Thomas Anders wrote:
    > > Magnus Fromreide wrote:
    > >
    > >> I think the compiler-choice patch is what you are asking for?
    > >> For the rest of you, do you think it is a good thing?
    > >>

    > >
    > > What problems does it promise to solve by just dropping support for
    > > --with-{cc,cflags,ld,ldflags,ar,libs}?
    > >

    > The problem with the duplicate configuration methods is that things
    > break, and it's not clear why. For example (given at the start of this
    > thread), for a simple project compiled with autoconf, merely passing
    > "--host=arm-unknown-linux-unknown" will automatically cause the
    > resulting makefile to search for "arm-unknown-linux-unknown-gcc" as the
    > compiler tool, "arm-unknown-linux-unknown-ldd" as the linker tool,
    > "arm-unknown-linux-unknown-strip" as the strip tool etc. We know that
    > this is not the case for net-snmp, and we're not sure why. The explicit
    > (read - redundant) compiler setting is a likely culprit.


    I did manage to convince my test setup to do a crossbuild from i386-...
    to i686-... and everything seemed to work for me but I did not do any
    verification beyond making sure that it built.
    With this said I think your idea of going towards automake is a much
    better solution.

    > I've started looking at what it would take to move the project over to
    > automake.


    Awesome.

    > From the preliminary work I've done it seems that automake
    > will reduce the build system source code size by about 80%(1), plus
    > solve many (if not all) of the problems mentioned in my original email.


    Even more promising.

    > ...
    >
    > 1 - I've only started to work on it, and it obviously is going to take a
    > while. So far, I haven't seen anything that cannot fit, neatly, into the
    > models suggested by automake, but I cannot claim to know the net-snmp
    > build system well enough to be sure. In any case, nuances will have to
    > be checked by someone better versed with what net-snmp needs to do than
    > me. I'll send a patch when there is something to show.


    The hard part as I see it is the autodependency handling between
    net-snmp modules using header file meta-commands.

    If you search for the string "config_require" in the configure script
    that woud take you right into the, as I understand it, most problematic
    part.

    /MF (I do like automake as well)


    -------------------------------------------------------------------------
    This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
    Build the coolest Linux based applications with Moblin SDK & win great prizes
    Grand prize is a trip for two to an Open Source event anywhere in the world
    http://moblin-contest.org/redirect.p...r_id=100&url=/
    _______________________________________________
    Net-snmp-coders mailing list
    Net-snmp-coders@lists.sourceforge.net
    https://lists.sourceforge.net/lists/...et-snmp-coders


+ Reply to Thread