Distribution security - NTP

This is a discussion on Distribution security - NTP ; Folks, Recently there have been several occasions where folks complained about one thing or another in the documentation, in particular when not using the "official" documentation via the web. The NTP maintainers and I have a very strict security policy ...

+ Reply to Thread
Results 1 to 13 of 13

Thread: Distribution security

  1. Distribution security

    Folks,

    Recently there have been several occasions where folks complained about
    one thing or another in the documentation, in particular when not using
    the "official" documentation via the web. The NTP maintainers and I have
    a very strict security policy for the NTP products maintained at U
    Delaware. In effect, there is an ironclad moat surrounding U Delaware
    and the official products can be extracted only directly or via links at
    www.ntp.org.

    In fact, all snapshots, releases, bugzilla, repositories, NTP home page
    and NTP project site physically reside at U Delaware. Only the documents
    maintained by the NTP Support Project are physically resident elsewhere
    at ISC. The intent is that, if you get something from U Delaware or ISC,
    it is authentic, official and neither modified nor infested with trojan
    horses.

    Folks should understand that, like all things in Internet social
    culture, things are always changing in one way or another. This applies
    to the sources and documentation included in each distribution. The
    documentation included in a particular distribution applies only to that
    distribution and may be different in minor ways from another distribution.

    Over the last several months there have been a raft of minor changes
    that correct minor problem leapsecond handling, secure group
    interoperability, orphan handling, pool features, rate management and
    intrusion defense. As a consequence, the doucmentation pages are being
    revised and improved on an almost continuous basis. Understand that the
    up-to-minute revision is on the web at www.ntp.org, so that may differ
    in small part from whatever distribution is in use.

    That was easy, now the hard part. The docuemntation and indeed the
    distributions themselves have been cloned in very many places, sometimes
    from rather old distributions. In quite a few cases somebody scarfs a
    current snapshot and wonders why some obscure option no longer works.
    The usual answer is to go look in the snapshot documentation, as the old
    option was probably broken and no longer needed.

    Now the ugly part. It's hard enough to deal with clones of old html
    pages, but some folks insist on man pages that are not in the official
    documentation that leaves U Delaware. The reasons for this are many,
    including the loss of web typography, diagrams and content. However,
    some folks use tools to convert html format to man format, even if that
    loses content. I have no quarrel with that as long as it is not claimed
    to be official and a disclaimer is added to that effect.

    To see how far the practice goes, I googled for "ntp-keygen man" and got
    623 hits(!). A casual check suggests that most of them are for
    reformated html pages, some really old, some incomplete and some
    modified with errors. Users of systems like Linux that may extensively
    clone and convert pages from older releases may not apply to later
    releases and especially snapshots.

    Dave

  2. Re: Distribution security

    >>> In article , "David L. Mills" writes:

    David> In fact, all snapshots, releases, bugzilla, repositories, NTP home
    David> page and NTP project site physically reside at U Delaware. Only the
    David> documents maintained by the NTP Support Project are physically
    David> resident elsewhere at ISC. The intent is that, if you get something
    David> from U Delaware or ISC, it is authentic, official and neither
    David> modified nor infested with trojan horses.

    I'll add that I generate md5 signatures at UDel when a tarball is rolled,
    and we are working on a way to pgp/gpg/crypto sign these tarballs as well.

    David> Folks should understand ...
    David> ... The documentation included in a particular distribution
    David> applies only to that distribution and may be different in minor ways
    David> from another distribution.

    David> ... Understand
    David> that the up-to-minute revision is on the web at www.ntp.org, so they
    David> may differ in small part from whatever distribution is in use.

    And one of the items on the Support Project's TODO list is to have
    web-searchable online documentation for a number of different versions of
    NTP, so folks can find the information they want more easily.

    David> That was easy, now the hard part...

    David> Now the ugly part. It's hard enough to deal with clones of old html
    David> pages, but some folks insist on man pages that are not in the
    David> official documentation that leaves U Delaware. The reasons for this
    David> are many, including the loss of web typography, diagrams and
    David> content. However, some folks use tools to convert html format to man
    David> format, even if that loses content. I have no quarrel with that as
    David> long as it is not claimed to be official and a disclaimer is added to
    David> that effect.

    Not so fast, Dave. While this *may* be true for some folks, I know of two
    exceptions.

    The first is the FreeBSD stuff, where they took your html/ pages and
    converted them in to man pages, because the documentation in FreeBSD is
    man-page based.

    The second is the official distribution. While your html/ pages are still
    your definitive pages, I have, after discussion and agreement with you,
    begun the process of converting from our old way of parsing command-line
    options and flags to AutoOpts.

    This has bought us at least 2 big overall improvements.

    First, our options processing is now much easier.

    Second, the AutoOpts tools can spit out documentation that is based (in
    part) on the actual command-line processing specification.

    On our agreed-upon agenda, this will be the new way we handle program
    documentation, because it will allow people to install the documentation in
    a variety of formats that are useful to them. This will include html, man,
    info, and some other formats.

    You have said you do not want the man pages installed at UDel, and I have
    not yet had the time to code that in to the Makefiles (I'm a busy guy and
    there have been bigger fish to fry).

    David> To see how far the practice goes, I googled for "ntp-keygen man" and
    David> got 623 hits(!). A casual check suggests that most of them are for
    David> reformated html pages, some really old, some incomplete and some
    David> modified with errors. Users of systems like Linux that may
    David> extensively clone and convert pages from older releases may not apply
    David> to later releases and especially snapshots.

    Yes, but the good news is that anybody who tries to do this should quickly
    realize that the version of the documentation they are looking at most
    likely does not match the version they are running.

    H
    --
    http://ntpforum.isc.org - be a member!

  3. Re: Distribution security

    Harlan,

    What planet are you coming from? You know I violently disagree with you
    on the options thing and it has absolutely nothing to do with the html
    documentation. The options thing is overkill, an unnecessary
    complication and not really helpful in view of the available
    documentation. I would very much rather see the low-key help hints as in
    typical Unix commands.

    Even so, as I told you, there can be serious disconnect between the
    options thing and the actual documentation, as evidence in the
    ntp-keygen program for example. I told you the coding style was
    intricate, unnecessarily complex and I would not deal with it.

    Yes, I require that man pages not be in the base distribution; however,
    there is a html2man utility in the distribution that can produce a man
    page ripoff of the html pages, although with loss of content. Then, why
    is the sntp man page still in the distribution and what is worse, why is
    that man page installed by default? You did that over my objections and
    I consider that extremely rude.

    Dave

    Harlan Stenn wrote:

    >>>>In article , "David L. Mills" writes:

    >
    >
    > David> In fact, all snapshots, releases, bugzilla, repositories, NTP home
    > David> page and NTP project site physically reside at U Delaware. Only the
    > David> documents maintained by the NTP Support Project are physically
    > David> resident elsewhere at ISC. The intent is that, if you get something
    > David> from U Delaware or ISC, it is authentic, official and neither
    > David> modified nor infested with trojan horses.
    >
    > I'll add that I generate md5 signatures at UDel when a tarball is rolled,
    > and we are working on a way to pgp/gpg/crypto sign these tarballs as well.
    >
    > David> Folks should understand ...
    > David> ... The documentation included in a particular distribution
    > David> applies only to that distribution and may be different in minor ways
    > David> from another distribution.
    >
    > David> ... Understand
    > David> that the up-to-minute revision is on the web at www.ntp.org, so they
    > David> may differ in small part from whatever distribution is in use.
    >
    > And one of the items on the Support Project's TODO list is to have
    > web-searchable online documentation for a number of different versions of
    > NTP, so folks can find the information they want more easily.
    >
    > David> That was easy, now the hard part...
    >
    > David> Now the ugly part. It's hard enough to deal with clones of old html
    > David> pages, but some folks insist on man pages that are not in the
    > David> official documentation that leaves U Delaware. The reasons for this
    > David> are many, including the loss of web typography, diagrams and
    > David> content. However, some folks use tools to convert html format to man
    > David> format, even if that loses content. I have no quarrel with that as
    > David> long as it is not claimed to be official and a disclaimer is added to
    > David> that effect.
    >
    > Not so fast, Dave. While this *may* be true for some folks, I know of two
    > exceptions.
    >
    > The first is the FreeBSD stuff, where they took your html/ pages and
    > converted them in to man pages, because the documentation in FreeBSD is
    > man-page based.
    >
    > The second is the official distribution. While your html/ pages are still
    > your definitive pages, I have, after discussion and agreement with you,
    > begun the process of converting from our old way of parsing command-line
    > options and flags to AutoOpts.
    >
    > This has bought us at least 2 big overall improvements.
    >
    > First, our options processing is now much easier.
    >
    > Second, the AutoOpts tools can spit out documentation that is based (in
    > part) on the actual command-line processing specification.
    >
    > On our agreed-upon agenda, this will be the new way we handle program
    > documentation, because it will allow people to install the documentation in
    > a variety of formats that are useful to them. This will include html, man,
    > info, and some other formats.
    >
    > You have said you do not want the man pages installed at UDel, and I have
    > not yet had the time to code that in to the Makefiles (I'm a busy guy and
    > there have been bigger fish to fry).
    >
    > David> To see how far the practice goes, I googled for "ntp-keygen man" and
    > David> got 623 hits(!). A casual check suggests that most of them are for
    > David> reformated html pages, some really old, some incomplete and some
    > David> modified with errors. Users of systems like Linux that may
    > David> extensively clone and convert pages from older releases may not apply
    > David> to later releases and especially snapshots.
    >
    > Yes, but the good news is that anybody who tries to do this should quickly
    > realize that the version of the documentation they are looking at most
    > likely does not match the version they are running.
    >
    > H


  4. Re: Distribution security

    Folks,

    I have privately followed up with Dave on his response.

    Anybody with a threaded newsreader (or access to one) should be easily able
    to find it, if they care that much.

    H
    --
    http://ntpforum.isc.org - be a member!

  5. Re: Distribution security

    Harlan,

    I don't want to pick a fight on this, but you told me offline that the
    html2man utility was imperfect and not appropriate to convert bulk NTP
    html pages to man pages. So, why is that utlity still in the
    distribution? It may yet be useful in many cases, but there needs to be
    a statement to that effect.

    Also reported offline, there is in fact an html page on the sntp progam,
    so the sntp man page is superfluous, especially if installed by default.
    I don't promote this as specific to UDel, but applicable everywhere. The
    sntp program included in the distribution itself has serious hazards and
    can be badly misused if certain program options are are enabled. The
    html page is carefully structured to avoid these hazards. In any case,
    the sntp program should be bansihed to a separate but supported
    distribution.

    On the options thing, whatever intricate facility you have installed
    creates four new files for each program. I tried to find what option
    letter coresponded to what obscure define in the ntp-keygen.c program
    and could not discover it. As result, the html documentation and the
    options list decodes are probably discordant. As I said many times, the
    official documentation should be in one place only and can be updated as
    necessary without creating little discords elsewhere. On the other hand,
    I have no objection and do encourage useful help as in the standard Unix
    command usage decodes.

    Dave

    Harlan Stenn wrote:

    > Folks,
    >
    > I have privately followed up with Dave on his response.
    >
    > Anybody with a threaded newsreader (or access to one) should be easily able
    > to find it, if they care that much.
    >
    > H


  6. Re: Distribution security

    Dave,

    >>> In article , "David L. Mills" writes:


    David> Harlan, I don't want to pick a fight on this, but you told me offline
    David> that the html2man utility was imperfect and not appropriate to
    David> convert bulk NTP html pages to man pages. So, why is that utlity
    David> still in the distribution? It may yet be useful in many cases, but
    David> there needs to be a statement to that effect.

    I think I answered this (a long time ago), but I did not repeat this answer
    in my offline reply to you.

    The html2man utility does not work on our current html pages (and has not
    worked for a long time). html2man is still there because if it is visible
    there is a chance somebody will find it and fix it. If I remove it there is
    practically no chance anybody will fix it.

    Do you care much where the "html2man is broken" statement is placed?

    David> Also reported offline, there is in fact an html page on the sntp
    David> progam, so the sntp man page is superfluous, especially if installed
    David> by default. I don't promote this as specific to UDel, but applicable
    David> everywhere. The sntp program included in the distribution itself has
    David> serious hazards and can be badly misused if certain program options
    David> are are enabled. The html page is carefully structured to avoid these
    David> hazards. In any case, the sntp program should be bansihed to a
    David> separate but supported distribution.

    As I said in my response to you:

    > The sntp man page is there because it was in the original sntp package and
    > there never was an html page for it. It is the only documentation for that
    > program that exists.


    When I wrote that I was unaware of the html page. I'm a Unix guy and I
    generally don't even consider looking for html docs - I am used to (and
    expect) man pages.

    And the original man page does document some (now obsolete) options that
    could cause trouble. However, the options have been disabled.

    See the next section for the reason I hope all this will soon be moot.

    I went on:
    > Nobody else has volunteered to convert it, and since we have recently
    > stated we want to re-implement the sntp code where is anybody's motivation
    > to produce an html page for it now?
    >
    > I'll also add that the existing sntp code has not yet been converted to
    > AutoOpts:
    >
    > http://support.ntp.org/Dev/GNUAutoGenConversion
    >
    > We had an sntp maintainer for a while; he disappeared and since nobody has
    > volunteered to take over maintenance of the old port, we now have two
    > folks who have stepped forward to implement an sntp program that meets the
    > draft spec. That front is still not making much progress yet, however.


    Please note that last paragraph - we want to get a new sntp implementation
    and the current volunteers are (slowly) working on it. We would all like
    this to go faster, as I then said:

    > If the NTP Forum is successful, there will even be a chunk of change to
    > pay people to do the sntp code/documentation:
    >
    > http://ntpforum.isc.org/Main/ForumProject00005
    >
    > But I suspect we have quite a ways to go on the NTP Forum before we have
    > the funds to pay for development projects out of excess membership
    > revenue.


    I've just started a half-time contract so I have that much less time to
    spend on NTP. We are now several months' time past the scheduled release of
    4.2.6, and I am currently spending a decent hunk of the time I still have
    available trying to get the release blockers finished.

    At this time I'm not gonna promise you I'll be able to start in on the sntp
    code myself anytime soon. I will, however, chip away at it when I can.

    David> On the options thing, whatever intricate facility you have installed
    David> creates four new files for each program. I tried to find what option
    David> letter coresponded to what obscure define in the ntp-keygen.c program
    David> and could not discover it.

    As I said in my response (modulo a typo):
    > So if we edit ntp-keygen-opts.def and skip a few lines, we start seeing:
    >
    > flag = {
    > ...
    > };
    >
    > stanzas. We note these stanzas list the flags in alphabetical order by
    > flag value (I like things like this to be in alpha order).


    David> As result, the html documentation and the
    David> options list decodes are probably discordant. As I said many times,
    David> the official documentation should be in one place only and can be
    David> updated as necessary without creating little discords elsewhere. On
    David> the other hand, I have no objection and do encourage useful help as
    David> in the standard Unix command usage decodes.

    Yes, and as I said in my response to you:

    > You have said repeatedly that there are areas of the html docs that you
    > control (most of them) and areas of the html docs I can mess with.
    >
    > I recall you saying that if I wanted to take over the documentation for
    > the programs and the ntp.conf file I could, and that until that happens
    > you will continue to maintain them. You require that should I take over
    > these pages, that they continue to be available in html format.
    >
    > This is the direction I am heading.


    When this is finished, the official documentation will be in one place, and
    the official docs will be available in html format, and that same content
    (for the programs and things like ntp.conf) will be available in man pages.
    There will also be some other formats available.

    And we will maintain this stuff by editing each master copy in one place.

    Your docs will continue to be in whatever format you like. The stuff I'm
    responsible for will be in a format that produces documents in the formats I
    need, which includes html so they can be easily integrated with your docs.

    The NTP Forum has already identified projects to get this finished:

    http://ntpforum.isc.org/Main/ForumProject00003
    http://ntpforum.isc.org/Main/ForumProject00004

    and finally,

    Harlan Stenn wrote:
    >> Folks, I have privately followed up with Dave on his response. Anybody
    >> with a threaded newsreader (or access to one) should be easily able to
    >> find it, if they care that much. H

    "it" in that last sentence was the first antecedent bit, or "Dave's
    response", not the followup I sent to Dave.
    --
    Harlan Stenn
    http://ntpforum.isc.org - be a member!

  7. Re: Distribution security


    >When I wrote that I was unaware of the html page. I'm a Unix guy and I
    >generally don't even consider looking for html docs - I am used to (and
    >expect) man pages.


    Me too.

    Would it help to ship dummy man pages that just pointed to
    the html documentation?

    --
    These are my opinions, not necessarily my employer's. I hate spam.


  8. Re: Distribution security

    >>> In article <-LOdncUxJvfOmR3anZ2dnUVZ_ovinZ2d@megapath.net>, hal-usenet@ip-64-139-1-69.sjc.megapath.net (Hal Murray) writes:

    >> When I wrote that I was unaware of the html page. I'm a Unix guy and I
    >> generally don't even consider looking for html docs - I am used to (and
    >> expect) man pages.


    Hal> Me too.

    Me too. Many other folks agree as well.

    Hal> Would it help to ship dummy man pages that just pointed to the html
    Hal> documentation?

    Please see:

    http://support.ntp.org/bin/view/Dev/...oGenConversion
    http://members.ntpforum.isc.org/bin/...umProject00003
    http://members.ntpforum.isc.org/bin/...umProject00004

    and it might be pretty easy for somebody to put the URLs in the various .def
    files to get the interim step Hal describes out there sooner.

    I'd be real happy if somebody did this, as I'm gonna be crushed for time for
    the next few months' time.
    --
    Harlan Stenn
    http://ntpforum.isc.org - be a member!

  9. Re: Distribution security

    Hal Murray wrote:
    >> When I wrote that I was unaware of the html page. I'm a Unix guy and I
    >> generally don't even consider looking for html docs - I am used to (and
    >> expect) man pages.

    >
    > Me too.
    >
    > Would it help to ship dummy man pages that just pointed to
    > the html documentation?


    Can I just get my vote in (even though my non-membership of the ntpd
    development community puts me into the whiner-not-worker category)?

    I really hate being sent from left to right in search of documentation.
    This includes stub man pages pointing me at html or info pages. Ideally
    I'd want to be able to select the documentation format on a system I'm
    responsible for myself (i.e. as a policy decision on my end).

    This issue also goes beyond the mere format of the documentation, even
    in case I am forced to accept multiple document formats. Searching the
    documentation installed on a system gets more difficult in this case.
    For example, the apropos tool won't search anything but man pages.

    Cheers, Jan

  10. Re: Distribution security

    >>> In article <4780a1e7$0$29263$ba620e4c@news.skynet.be>, Jan Ceuleers writes:

    Jan> I really hate being sent from left to right in search of
    Jan> documentation. This includes stub man pages pointing me at html or info
    Jan> pages. Ideally I'd want to be able to select the documentation format
    Jan> on a system I'm responsible for myself (i.e. as a policy decision on my
    Jan> end).

    What you describe is my goal.

    One obvious way progress toward this goal will be greatly improved will be
    for more people and institutions to join the NTP Forum.

    Please do what you can to get members to sign up for the forum - if you need
    more information to help make the decision please let me know. And if you
    decide not to join, it would also help to let me know why. Email is
    probably the best way to do these things.

    Jan> This issue also goes beyond the mere format of the documentation, even
    Jan> in case I am forced to accept multiple document formats. Searching the
    Jan> documentation installed on a system gets more difficult in this
    Jan> case. For example, the apropos tool won't search anything but man
    Jan> pages.

    One of the items on the TODO queue for support.ntp.org is to have searchable
    versions of each released version of NTP on the site.

    As for your "apropos won't search anything but man pages" complaint, the
    AutoGen documentation effort would have the same content appear in various
    formats, so at least for the "common" documentation (ie, whatever is not
    maintained by Dave), the content of the ntpd or ntp.conf documentation (for
    example) would be the same on the html, man, info, etc. pages.

    --
    Harlan Stenn
    http://ntpforum.isc.org - be a member!


  11. Re: Distribution security

    Harlan,

    Harlan Stenn wrote:
    > Jan> I really hate being sent from left to right in search of
    > Jan> documentation. This includes stub man pages pointing me at html or info
    > Jan> pages. Ideally I'd want to be able to select the documentation format
    > Jan> on a system I'm responsible for myself (i.e. as a policy decision on my
    > Jan> end).
    >
    > What you describe is my goal.


    I know and I thank you for it. My not-so-hidden agenda in writing the
    above message was to support you by showing that there is indeed demand
    for documentation on formats other than html, while ensuring consistency
    and an audit trail leading from the documentation in all formats to a
    single authoritative source.

    > One obvious way progress toward this goal will be greatly improved will be
    > for more people and institutions to join the NTP Forum.
    >
    > Please do what you can to get members to sign up for the forum - if you need
    > more information to help make the decision please let me know. And if you
    > decide not to join, it would also help to let me know why. Email is
    > probably the best way to do these things.


    I have indeed sent you email about this. Happy to help you on this issue
    as well.

    Cheers, Jan

  12. Re: Distribution security

    Jan,

    A thousand roses for saying that, in particular the point about having
    an audit trail. I've been burned many times about errors in document
    reformats that were either cached with errors or simple broken in
    translation.

    Dave

    Jan Ceuleers wrote:

    > Harlan,
    >
    > Harlan Stenn wrote:
    >
    >> Jan> I really hate being sent from left to right in search of
    >> Jan> documentation. This includes stub man pages pointing me at html
    >> or info
    >> Jan> pages. Ideally I'd want to be able to select the documentation
    >> format
    >> Jan> on a system I'm responsible for myself (i.e. as a policy decision
    >> on my
    >> Jan> end).
    >>
    >> What you describe is my goal.

    >
    >
    > I know and I thank you for it. My not-so-hidden agenda in writing the
    > above message was to support you by showing that there is indeed demand
    > for documentation on formats other than html, while ensuring consistency
    > and an audit trail leading from the documentation in all formats to a
    > single authoritative source.
    >
    >> One obvious way progress toward this goal will be greatly improved
    >> will be
    >> for more people and institutions to join the NTP Forum.
    >>
    >> Please do what you can to get members to sign up for the forum - if
    >> you need
    >> more information to help make the decision please let me know. And if
    >> you
    >> decide not to join, it would also help to let me know why. Email is
    >> probably the best way to do these things.

    >
    >
    > I have indeed sent you email about this. Happy to help you on this issue
    > as well.
    >
    > Cheers, Jan


+ Reply to Thread