drift value very large and very unstable - NTP

This is a discussion on drift value very large and very unstable - NTP ; Richard Gilbert wrote: > > > A comment in ntp.conf and/or the startup file, explaining WHY stepping > is enabled should go a long way toward solving the "dumbass" problem. > > Yes, I know, but that requires someone to ...

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2
Results 21 to 40 of 40

Thread: drift value very large and very unstable

  1. Re: drift value very large and very unstable


    Richard Gilbert wrote:
    >
    >
    > A comment in ntp.conf and/or the startup file, explaining WHY stepping
    > is enabled should go a long way toward solving the "dumbass" problem.
    >
    >


    Yes, I know, but that requires someone to actually read the comments in
    the conf files and there is always the '-x' argument that could be added
    to the command line. Adding to the fun, we use 'include' to build up
    ntp.conf so we actually have three conf files that are combined.

    At any rate, reading through other "step disable" problems in the
    mailing list, I'm getting the impression that this option isn't really
    recommended nor supported. This, of course, begs the question of why
    provide an option that is known to cause problems? How about a comment
    in the docs that states it plainly: Don't use this option. Yes, it
    does say "The kernel time discipline is disabled with this option", but
    that didn't mean anything to me when I started this work and still means
    very little. Certainly nothing I've read implies this option is
    guaranteed to cause problems. In fact, I ran for weeks with a different
    processor board and different linux kernel with stepping disabled and
    achieved excellent performance.

    Andy

  2. Re: drift value very large and very unstable

    andy.helten@dot21rts.com (Andy Helten) writes:


    >Fran Horan wrote:
    >>
    >>
    >>
    >>> So, the summary is that drift goes to 500ppm when stepping is disabled
    >>> but runs normally when stepping is enabled and both situations never
    >>> require a time step. This makes no sense to me. By the way, as
    >>> mentioned previously, we require that time does not step backward due to
    >>> a problem in some commercial software that cannot currently tolerate
    >>> time moving backwards.
    >>>
    >>> Quite frankly, I don't think it's unreasonable that a system require
    >>> time to monotonically increase.
    >>>

    >>
    >> Forgive me if this answer misses a point in the earlier details, or shows my
    >> ignorance of NTP, but a few ideas/thoughts.
    >>
    >> Oscillators and drift can go in either direction, fast or slow, its a
    >> physics-based situation. You can't write code around that and provide a
    >> software solution that is monotonic at all times. However, a single negative
    >> step just at the start may be required before going monotic after that
    >> event. (Not an expert, but that is my understanding).


    drift is the difference between the actual rate of the oscillator and 1.
    You need a drift rate of -1000000PPM for it to no longer be monotonic.
    If you have such a drift rate plug the stupid machine in!


    >>
    >> With this ref clock and a GPS-drive IRIG source, you may only see a single
    >> negative step when NTP first begins running on a new system with no drift
    >> file, or a system that has been powered off a long time with a
    >> battery-driven clock drifting over that long time. Once NTP is humming along
    >> after the initial step and some updates, you shouldn't see a step again.
    >> This makes me think that you should insert a delay in launching your
    >> sensitive application, or block the application at some point, so it does
    >> not see the (possible) first time step.
    >>
    >> Fran Horan
    >> JHU/APL
    >>
    >>

    >Hey Fran,


    >Yes, exactly, we do perform an initial time sync with stepping enabled.
    >This is done prior to initializing the commercial software and so it
    >does not cause problems if time moves backwards. And, yes, if we are


    Time should never move backward. "Steps" should be fast drifts. (10000PPM
    if necessary but not 1000000PPM.




    >below the step threshold after the initial sync (which should always be
    >the case), then we should stay below that threshold until the end of
    >time. Following this logic, we should allow time steps and be comforted
    >knowing they will never occur in a normally functioning system. I agree
    >this is reasonable and does not conflict with my own rant that "if we
    >have an offset of more than 10ms in this system, then something isn't
    >working correctly".


    >This approach is definitely worth considering and I'll bring it up with
    >the decision makers. However, there is always concern that months or
    >years from now someone will say -- "Hey, some dumbass left time stepping
    >enabled, let's disable it on all systems immediately". Surely this
    >wouldn't be done without some regression testing, but then again such a
    >mundane change shouldn't need exhaustive testing, right? Riiiiight.


    >I guess was just hoping someone will say, "Oh, right, that's a known
    >problem. You need to do 'X' to fix it."


    >Andy


  3. Re: drift value very large and very unstable

    Unruh wrote:
    > Time should never move backward. "Steps" should be fast drifts. (10000PPM
    > if necessary but not 1000000PPM.
    >
    >


    I don't know that this is unconditionally true. From
    https://lists.ntp.org/pipermail/ques...er/011548.html,
    quoting Dr. Mills:
    > 4. In the design of the nanokernel code that leaves here, the time as
    > seen by the application is not stepped backward unless the kernel clock
    > is stepped backward more than two seconds. So, if the kernel clock is
    > stepped backward more than the step threshold and less than two seconds,
    > the time as observed by an application would effectively stand still or
    > advance at a low rate for the interval required. There are other
    > features designed to avoid inconsistent reading of the kernel clock via
    > various means in and out of the kernel. These features have been
    > implemented in the stock Tru64 kernel for the Alpha, but so far as I
    > know have not been implemented in other kernels.


    Maybe I misunderstand the above statement or maybe it doesn't apply
    here, but I read it as fast drifts only happen conditionally (i.e. that
    condition being a time jump less than 2 seconds but greater than
    128ms). This makes sense, doesn't it? If "steps" were always fast
    drifts, then NTP would still require days or hours to cover a large jump
    in time. Additionally, if time *never* steps backwards, we wouldn't be
    having this conversation because I would have never needed 'tinker step
    0' or '-x' and so I would have never encountered my "problem" and so I
    would have never signed up to this mailing list and never sent in a
    question with the subject "drift value very large and very unstable".

    Andy

  4. Re: drift value very large and very unstable

    Richard B. Gilbert wrote:

    > A comment in ntp.conf and/or the startup file, explaining WHY stepping
    > is enabled should go a long way toward solving the "dumbass" problem.


    We supply neither an ntp.conf file nor a startup file so this comment
    makes no sense. This kind of thing belongs in the Support wiki which is
    constantly updated.

    Danny

  5. Re: drift value very large and very unstable

    Danny Mayer wrote:
    > Richard B. Gilbert wrote:
    >
    >
    >>A comment in ntp.conf and/or the startup file, explaining WHY stepping
    >>is enabled should go a long way toward solving the "dumbass" problem.

    >
    >
    > We supply neither an ntp.conf file nor a startup file so this comment
    > makes no sense. This kind of thing belongs in the Support wiki which is
    > constantly updated.
    >
    > Danny


    I didn't mean to suggest that you did or should!

    I intended to suggest that the OP or anyone else with the same or a
    similar problem should put the comment is HIS ntp.conf and or startup
    script.


  6. Re: drift value very large and very unstable

    Danny Mayer wrote:

    > We supply neither an ntp.conf file nor a startup file so this comment
    > makes no sense. This kind of thing belongs in the Support wiki which is
    > constantly updated.


    You need to supply both. Otherwise the packagers will do it for you and
    they will get it wrong.

  7. Re: drift value very large and very unstable

    "David Woolley" wrote in message
    news:47d7013c$0$510$5a6aecb4@news.aaisp.net.uk...
    > Danny Mayer wrote:


    >> We supply neither an ntp.conf file nor a startup file so this comment
    >> makes no sense. This kind of thing belongs in the Support wiki which
    >> is constantly updated.

    >
    > You need to supply both. Otherwise the packagers will do it for you
    > and they will get it wrong.


    ISTM there are two things that would go into a default ntp.conf to be
    supplied by the current NTP developer effort: servers, and a drift file.

    For the drift file, I suspect the Filesystem Standard (whatever it's
    called these days) defines some place where it might usefully be put.

    For the servers, we have the Pool. We still _want_ that to be used,
    right?

    Groetjes,
    Maarten Wiltink



  8. Re: drift value very large and very unstable

    David Woolley wrote:
    > Danny Mayer wrote:
    >
    >> We supply neither an ntp.conf file nor a startup file so this comment
    >> makes no sense. This kind of thing belongs in the Support wiki which
    >> is constantly updated.

    >
    >
    > You need to supply both. Otherwise the packagers will do it for you and
    > they will get it wrong.


    Anybody who runs a "packaged" configuration deserves whatever happens to
    him. Windows comes configured to use time.windows.com. AFAIK that's
    operated by Microsoft and it's not a problem. RedHat comes
    preconfigured to use servers provided by RedHat. That's not a problem
    either. Neither Microsoft nor RedHat can, or does, guarantee that the
    canned configuration will work well for you!

    Further, the NTP developers/packagers/whatever have no idea where you
    will be running your system or what your requirements will be. A
    configuration that's perfect for a system in Dover, Delaware, will be
    garbage if used in a system in Tokyo! Any canned configuration should be
    treated as an example. The ultimate user needs to pick his own servers,
    his own security, his own locations for log files, etc, etc.

    If you need a configuration you can depend on, you have a responsibility
    to think about the problem and develop a configuration that meets *your*
    needs!

    I believe that the developers have met their responsibility by providing
    documentation that tells you how to "roll your own" configuration.


  9. Re: drift value very large and very unstable

    Maarten Wiltink wrote:
    > "David Woolley" wrote in message
    > news:47d7013c$0$510$5a6aecb4@news.aaisp.net.uk...
    >
    >>Danny Mayer wrote:

    >
    >
    >>>We supply neither an ntp.conf file nor a startup file so this comment
    >>>makes no sense. This kind of thing belongs in the Support wiki which


    >
    > ISTM there are two things that would go into a default ntp.conf to be
    > supplied by the current NTP developer effort: servers, and a drift file.
    >
    > For the drift file, I suspect the Filesystem Standard (whatever it's
    > called these days) defines some place where it might usefully be put.



    File System Standard? What's that?? Unix tends to put things in more
    or less standard places but it's not guaranteed that system A's file
    system or directory naming will be compatible with system B. Once you
    get out of the Unix-Linux arena nothing is guaranteed. Windows has its
    own way of doing things. MacIntosh? It's Unix or Unix-like but I'm not
    really familiar with it; I have had maybe two hours of Mac experience in
    the last twentyfive years! OpenVMS has its own way, not compatible with
    anything else of course. Can AIX read UFS or ZFS? Can Solaris read
    whatever AIX uses?


  10. Re: drift value very large and very unstable

    "Richard B. Gilbert" writes:

    >Maarten Wiltink wrote:
    >> "David Woolley" wrote in message
    >> news:47d7013c$0$510$5a6aecb4@news.aaisp.net.uk...
    >>
    >>>Danny Mayer wrote:

    >>
    >>
    >>>>We supply neither an ntp.conf file nor a startup file so this comment
    >>>>makes no sense. This kind of thing belongs in the Support wiki which

    >
    >>
    >> ISTM there are two things that would go into a default ntp.conf to be
    >> supplied by the current NTP developer effort: servers, and a drift file.
    >>
    >> For the drift file, I suspect the Filesystem Standard (whatever it's
    >> called these days) defines some place where it might usefully be put.

    >


    >File System Standard? What's that?? Unix tends to put things in more
    >or less standard places but it's not guaranteed that system A's file
    >system or directory naming will be compatible with system B. Once you


    that is why there is a proposed file system standard.
    Log files in /var/log/ntp say.
    Drift file in /etc/ntp.drift
    config file in /etc/ntp.conf


    >get out of the Unix-Linux arena nothing is guaranteed. Windows has its
    >own way of doing things. MacIntosh? It's Unix or Unix-like but I'm not
    >really familiar with it; I have had maybe two hours of Mac experience in
    >the last twentyfive years! OpenVMS has its own way, not compatible with
    >anything else of course. Can AIX read UFS or ZFS? Can Solaris read
    >whatever AIX uses?


    Sorry-- what has the filesystem got to do with anything?


  11. Re: drift value very large and very unstable

    Unruh wrote:

    > that is why there is a proposed file system standard.
    > Log files in /var/log/ntp say.
    > Drift file in /etc/ntp.drift
    > config file in /etc/ntp.conf
    >


    I think they were referring to the Linux filesystem standard, and one of
    the things that does is to move things out of /etc. In particular,
    /etc/ntp.drift would never be allowed, because the file is updated
    dynamically.

    Incidentally, a specimen configuration needn't include real server names.

  12. Re: drift value very large and very unstable

    Richard B. Gilbert wrote:

    > Anybody who runs a "packaged" configuration deserves whatever happens to
    > him. Windows comes configured to use time.windows.com. AFAIK that's



    Most people run packaged systems these days. When they go wrong, they
    will not RTFM, or ask the packager. They come here, instead. It's part
    of the modern, instant gratification, culture.

    Note that packaging can still leave the choice of servers to the user,
    but the user will normally just substitute the things that he
    understands, like server names, and leave the rest untouched, even if it
    is a one size fits all configuration that is poor for almost all users.
    Users will not analyze how the file works.

  13. Re: drift value very large and very unstable

    David Woolley wrote:
    > Unruh wrote:
    >
    >> that is why there is a proposed file system standard. Log files in
    >> /var/log/ntp say.
    >> Drift file in /etc/ntp.drift
    >> config file in /etc/ntp.conf
    >>

    >
    > I think they were referring to the Linux filesystem standard, and one of
    > the things that does is to move things out of /etc. In particular,
    > /etc/ntp.drift would never be allowed, because the file is updated
    > dynamically.
    >
    > Incidentally, a specimen configuration needn't include real server names.


    It almost certainly SHOULD NOT contain real server names. At least a
    specimen provided by the NTP group should not. A specimen provided by
    an O/S vendor could properly include servers owned and operated by that
    vendor as RedHat does.

    If I were to write a sample ntp.conf it might look like this:

    #
    # NTP monitoring parameters
    #
    logfile /var/ntp/ntp.log
    statsdir /var/ntp/ntpstats/
    driftfile /var/ntp/ntp.drift
    statistics peerstats clockstats
    filegen peerstats file peerstats type day enable
    filegen clockstats file clockstats type day enable
    # Hardware Reference Clock
    server 127.127.3.0 prefer # PST/Traconex 1020/1030 (Type 3 Unit 0)
    # Network servers
    server 192.168.1.20 iburst # Our local GPS server
    server 10.1.1.103 # RFC-1918 address to fill out example
    server 10.2.2.104 # Another RFC-1918 address. . . .
    #
    # Authentication parameters
    #
    keys /etc/inet/ntp.keys
    trustedkey 2 3 4
    controlkey 3 # To access the ntpq utility
    requestkey 2 # To access the ntpdc utility

    I might include more comments pointing out what the user needs to
    customize and a link to the ntp documentation.


  14. Re: drift value very large and very unstable

    David Woolley wrote:
    > Richard B. Gilbert wrote:
    >
    >> Anybody who runs a "packaged" configuration deserves whatever happens
    >> to him. Windows comes configured to use time.windows.com. AFAIK that's

    >
    >
    >
    > Most people run packaged systems these days. When they go wrong, they
    > will not RTFM, or ask the packager. They come here, instead. It's part
    > of the modern, instant gratification, culture.
    >


    And some heartless person, like me, will provide a link and tell them to
    go RTFM. Or maybe I'll just tell them that "google is your friend"!

    > Note that packaging can still leave the choice of servers to the user,
    > but the user will normally just substitute the things that he
    > understands, like server names, and leave the rest untouched, even if it
    > is a one size fits all configuration that is poor for almost all users.
    > Users will not analyze how the file works.


    And that class of user may never notice, or care, that his clock is off
    by 27 seconds.

    I first started using NTP when I noticed that none of the systems I was
    responible for were closer to the correct time than five minutes.
    Indeed, one otherwise well behaved system was off by 11 hours!


  15. Re: drift value very large and very unstable

    > From: David Woolley
    > Date: Wed, 12 Mar 2008 08:11:23 +0000
    > Sender: questions-bounces+oberman=es.net@lists.ntp.org
    >
    >
    > Unruh wrote:
    >
    > > that is why there is a proposed file system standard.
    > > Log files in /var/log/ntp say.
    > > Drift file in /etc/ntp.drift
    > > config file in /etc/ntp.conf
    > >

    >
    > I think they were referring to the Linux filesystem standard, and one of
    > the things that does is to move things out of /etc. In particular,
    > /etc/ntp.drift would never be allowed, because the file is updated
    > dynamically.
    >
    > Incidentally, a specimen configuration needn't include real server names.


    Many systems mount root read-only for security reasons (and /etc is in
    the root partition. As a result, most systems all files that need
    dynamic writes (like the drift file) to an appropriate place in /var.

    In the case of FreeBSD, this is done by running ntpd with the command
    line options of "-p /var/run/ntpd.pid -f /var/db/ntpd.drift". No ntp
    configuration file is distributed with the system, but the drift file is
    moved to a much better location.
    --
    R. Kevin Oberman, Network Engineer
    Energy Sciences Network (ESnet)
    Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab)
    E-mail: oberman@es.net Phone: +1 510 486-8634
    Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751

  16. Re: drift value very large and very unstable

    David Woolley writes:

    >Richard B. Gilbert wrote:


    >> Anybody who runs a "packaged" configuration deserves whatever happens to
    >> him. Windows comes configured to use time.windows.com. AFAIK that's



    >Most people run packaged systems these days. When they go wrong, they
    >will not RTFM, or ask the packager. They come here, instead. It's part
    >of the modern, instant gratification, culture.


    Any you blame them? Most of those FMs are impossible to find anything in.
    Asking for help is an ancient human activity, far far older than manuals.


    >Note that packaging can still leave the choice of servers to the user,
    >but the user will normally just substitute the things that he
    >understands, like server names, and leave the rest untouched, even if it
    >is a one size fits all configuration that is poor for almost all users.
    > Users will not analyze how the file works.


    And they should analyse it why? You think everyone who uses the software
    has the same fascination for it that you have? You have analysed how your
    word processor stores the info? or read through its whole manual? People
    have better things to do with their lives that read manuals, unless it
    leads them into trouble.
    Most manuals are written on the "Beware of Tigers" principle (You remember
    the lines from Hitchhikers Guide where Arthur discovers the the planning
    document to destroy his house in a flooded basement down some broken stairs
    in a locked filing cabinet behind a door labeled "Beware of Tigers". It
    servers for bureaucrats to be able to point to it and say-- see it was all
    there--)


  17. Re: drift value very large and very unstable

    "Richard B. Gilbert" writes:

    >David Woolley wrote:
    >> Unruh wrote:
    >>
    >>> that is why there is a proposed file system standard. Log files in
    >>> /var/log/ntp say.
    >>> Drift file in /etc/ntp.drift
    >>> config file in /etc/ntp.conf
    >>>

    >>
    >> I think they were referring to the Linux filesystem standard, and one of
    >> the things that does is to move things out of /etc. In particular,
    >> /etc/ntp.drift would never be allowed, because the file is updated
    >> dynamically.
    >>
    >> Incidentally, a specimen configuration needn't include real server names.


    >It almost certainly SHOULD NOT contain real server names. At least a
    >specimen provided by the NTP group should not. A specimen provided by
    >an O/S vendor could properly include servers owned and operated by that
    >vendor as RedHat does.


    Well, no, that was why the pool was invented. It could well have
    a pool server included.


    >If I were to write a sample ntp.conf it might look like this:


    For whose purposes is this?
    Not for any generic user except maybe at your organization.


    >#
    ># NTP monitoring parameters
    >#
    >logfile /var/ntp/ntp.log
    >statsdir /var/ntp/ntpstats/
    >driftfile /var/ntp/ntp.drift
    >statistics peerstats clockstats
    >filegen peerstats file peerstats type day enable
    >filegen clockstats file clockstats type day enable


    I would comment those all out and state that if you want debugging info,
    uncomment them Most people will never ever look at the statistics files.

    ># Hardware Reference Clock
    >server 127.127.3.0 prefer # PST/Traconex 1020/1030 (Type 3 Unit 0)


    And that out since noone has a PST/Traconex 1020/1030 refclock.

    ># Network servers
    >server 192.168.1.20 iburst # Our local GPS server


    And you would expect anyone else to have that as well?

    >server 10.1.1.103 # RFC-1918 address to fill out example
    >server 10.2.2.104 # Another RFC-1918 address. . . .
    >#
    ># Authentication parameters
    >#
    >keys /etc/inet/ntp.keys
    >trustedkey 2 3 4
    >controlkey 3 # To access the ntpq utility
    >requestkey 2 # To access the ntpdc utility


    >I might include more comments pointing out what the user needs to
    >customize and a link to the ntp documentation.


    And where are the pool servers which the user might actually want to use?

    Things like the pool servers is something the user is highly unlikely to
    know about, and a sample config file is a great place to tell them.



  18. Re: drift value very large and very unstable

    > >File System Standard? What's that?? Unix tends to put things in
    > >more or less standard places but it's not guaranteed that system
    > >A's file system or directory naming will be compatible with system
    > >B. Once you


    > that is why there is a proposed file system standard.


    But there are so many from which to choose SYSV, probably one from
    Posix/Xopen, Linux Standard Base (?) etc etc

    rick jones
    --
    a wide gulf separates "what if" from "if only"
    these opinions are mine, all mine; HP might not want them anyway...
    feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

  19. Re: drift value very large and very unstable

    Maarten Wiltink wrote:
    >
    > ISTM there are two things that would go into a default ntp.conf to be
    > supplied by the current NTP developer effort: servers, and a drift file.
    >
    > For the drift file, I suspect the Filesystem Standard (whatever it's
    > called these days) defines some place where it might usefully be put.
    >


    There are no standard locations. Different OS's do have their own
    preferences but so do admins. And don't get me started about Windows.

    > For the servers, we have the Pool. We still _want_ that to be used,
    > right?
    >


    Yes, but where are you in the world? We don't know and it's best left to
    the installer/admin.

    Danny

  20. Re: drift value very large and very unstable

    Hello Maarten,

    On Tuesday, March 11, 2008 at 23:38:22 +0100, Maarten Wiltink wrote:

    > For the drift file, I suspect the Filesystem Standard (whatever it's
    > called these days) defines some place where it might usefully be put.


    The FHS 2.3 doesn't define explicitly a place for the ntp driftfile.
    However its rules about variable non-shareable host-specific and
    application-specific state informations lead directly to /var/lib/ntp/

    | /var/lib : Variable state information
    |
    | This hierarchy holds state information pertaining to an application or
    | the system. State information is data that programs modify while they
    | run, and that pertains to one specific host. Users must never need to
    | modify files in /var/lib to configure a package's operation.
    |
    | State information is generally used to preserve the condition of an
    | application (or a group of inter-related applications) between
    | invocations and between different instances of the same application.
    | State information should generally remain valid after a reboot, should
    | not be logging output, and should not be spooled data.
    |
    | An application (or a group of inter-related applications) must use a
    | subdirectory of /var/lib for its data.
    [...big snip...]
    | An important difference between this version of this standard and
    | previous ones is that applications are now required to use a
    | subdirectory of /var/lib.


    Serge.
    --
    Serge point Bets arobase laposte point net

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2