Speed of ntp convergence - NTP

This is a discussion on Speed of ntp convergence - NTP ; "Richard B. Gilbert" writes: >Unruh wrote: >> "Richard B. Gilbert" writes: >> >>> Unruh wrote: >>>> "David J Taylor" writes: >>>> >>>>> Hal Murray wrote: >>>>>>>> Try switching it off, changing the value int he drift file by say >>>>>>>> ...

+ Reply to Thread
Page 2 of 3 FirstFirst 1 2 3 LastLast
Results 21 to 40 of 43

Thread: Speed of ntp convergence

  1. Re: Speed of ntp convergence

    "Richard B. Gilbert" writes:

    >Unruh wrote:
    >> "Richard B. Gilbert" writes:
    >>
    >>> Unruh wrote:
    >>>> "David J Taylor" writes:
    >>>>
    >>>>> Hal Murray wrote:
    >>>>>>>> Try switching it off, changing the value int he drift file by say
    >>>>>>>> 50PPM and
    >>>>>>>> then switching it on again, and see how long it takes to recover
    >>>>>>>> from that.
    >>>>>>> Why would I do that? The drift values rarely change by more than
    >>>>>>> five, certainly not by 50. If you are seeing a change of 50, then
    >>>>>>> perhaps that it part of your problem?
    >>>>>> A big step like that makes it easy to see how the system responds.
    >>>>>> At least if it's a linear system.
    >>>>> Yes, I appreciate that, Hal, but it doesn't emulate the situation here
    >>>>> very well, which I understood to be slow convergence after a routine
    >>>>> start. It sounds as if the OP may have an incorrect drift file - it's
    >>>>> worth checking that it /is/ being updated.
    >>>>
    >>>>
    >>>> The drift file read 10. The actual drift was 250 (determined after the
    >>>> system had settled down). The drift file never changed even after a day of
    >>>> running. ntp does not seem to be rewriting the drift file. Now that is a
    >>>> problem (although with the apparent Linux bug in the timing routines where is
    >>>> miscalibrates the clock on bootup, the drift is NOT stable over reboots
    >>>> anyway, so the existence of a drift file is irrelevant. ) However, the question is
    >>>> about the bahaviour of ntp. ntp should NOT be taking 10 hours to get over a
    >>>> wrong value in the drift file.

    >>
    >>> That's easy to fix! If the drift file is not correct, remove it before
    >>> starting ntpd.

    >>
    >> Of course. However, I have no idea it is incorrect until after ntp has
    >> started up and shown me it was incorrect.
    >>
    >>> How do you tell if it's incorrect? Since ntpd is supposed to
    >>> update/rewrite the drift file every sixty minutes, a drift file more
    >>> than sixty minutes old is suspect!

    >>
    >> I think my problem was that the permissions on /etc/ntp/drift were
    >> incorrect ( owned by root rather than by ntp). But that makes no
    >> difference to how ntp behaves. ntp should do the "right thing" even if the
    >> drift file is wrong. It should take a bit longer, but not 10 hours longer.
    >> And with the current apparent bug in Linux wehre the system time is
    >> miscalibrated, it would seem that the drift file on Linux is ALWAYS wrong.
    >>


    >Do not blame ntpd for the consequences of your errors! If ntpd is
    >configured correctly and operated correctly, it behaves quite well.


    ???? I was not blaming ntp for my error. I was blaming ntp for its reaction
    to "my error" . And note a bad drift file is now the standard for
    Linux. For example between two reboots, my drift rate went from 180PPM to
    250PPM. No drift file would have fixed that.

    NTP handles drift errors badly. But that is not the question I asked.
    So far NOONE has answered the
    question-- why if my poll internal is 16 sec is the time scale for the
    error correction 1 hour?



    >And if you can write an ntpd equivalent that works better, I'm sure that
    >most of us would be interested and, perhaps, even grateful!


    already written, IF you do not want refclocks and you are running Linux. It
    is called chrony.


  2. Re: Speed of ntp convergence

    David Woolley wrote:
    > Richard B. Gilbert wrote:
    >
    >>
    >> The internet of today is similar. It may be a little better but I
    >> wouldn't count on it!
    >>

    > For one thing, the components from which it is made are 1,000 times
    > faster and have 1,000 times the memory.
    >
    > (There are other differences, like the de-skilling of sytem management.)


    And there is at least 1,000 times the traffic load that there used to be!

    I'd be more impressed if you could spell/type "system" correctly!

  3. Re: Speed of ntp convergence

    Unruh wrote:
    > "Richard B. Gilbert" writes:
    >
    >> Unruh wrote:
    >>> "Richard B. Gilbert" writes:
    >>>
    >>>> Unruh wrote:
    >>>>> "David J Taylor" writes:
    >>>>>
    >>>>>> Hal Murray wrote:
    >>>>>>>>> Try switching it off, changing the value int he drift file by say
    >>>>>>>>> 50PPM and
    >>>>>>>>> then switching it on again, and see how long it takes to recover
    >>>>>>>>> from that.
    >>>>>>>> Why would I do that? The drift values rarely change by more than
    >>>>>>>> five, certainly not by 50. If you are seeing a change of 50, then
    >>>>>>>> perhaps that it part of your problem?
    >>>>>>> A big step like that makes it easy to see how the system responds.
    >>>>>>> At least if it's a linear system.
    >>>>>> Yes, I appreciate that, Hal, but it doesn't emulate the situation here
    >>>>>> very well, which I understood to be slow convergence after a routine
    >>>>>> start. It sounds as if the OP may have an incorrect drift file - it's
    >>>>>> worth checking that it /is/ being updated.
    >>>>>
    >>>>> The drift file read 10. The actual drift was 250 (determined after the
    >>>>> system had settled down). The drift file never changed even after a day of
    >>>>> running. ntp does not seem to be rewriting the drift file. Now that is a
    >>>>> problem (although with the apparent Linux bug in the timing routines where is
    >>>>> miscalibrates the clock on bootup, the drift is NOT stable over reboots
    >>>>> anyway, so the existence of a drift file is irrelevant. ) However, the question is
    >>>>> about the bahaviour of ntp. ntp should NOT be taking 10 hours to get over a
    >>>>> wrong value in the drift file.
    >>>> That's easy to fix! If the drift file is not correct, remove it before
    >>>> starting ntpd.
    >>> Of course. However, I have no idea it is incorrect until after ntp has
    >>> started up and shown me it was incorrect.
    >>>
    >>>> How do you tell if it's incorrect? Since ntpd is supposed to
    >>>> update/rewrite the drift file every sixty minutes, a drift file more
    >>>> than sixty minutes old is suspect!
    >>> I think my problem was that the permissions on /etc/ntp/drift were
    >>> incorrect ( owned by root rather than by ntp). But that makes no
    >>> difference to how ntp behaves. ntp should do the "right thing" even if the
    >>> drift file is wrong. It should take a bit longer, but not 10 hours longer.
    >>> And with the current apparent bug in Linux wehre the system time is
    >>> miscalibrated, it would seem that the drift file on Linux is ALWAYS wrong.
    >>>

    >
    >> Do not blame ntpd for the consequences of your errors! If ntpd is
    >> configured correctly and operated correctly, it behaves quite well.

    >
    > ???? I was not blaming ntp for my error. I was blaming ntp for its reaction
    > to "my error" . And note a bad drift file is now the standard for
    > Linux. For example between two reboots, my drift rate went from 180PPM to
    > 250PPM. No drift file would have fixed that.
    >
    > NTP handles drift errors badly. But that is not the question I asked.
    > So far NOONE has answered the
    > question-- why if my poll internal is 16 sec is the time scale for the
    > error correction 1 hour?
    >


    How big is the error? Why is your poll interval 16 seconds? (If you
    are using a refclock, I withdraw the question!) Are you setting the
    correct time at startup with ntpd -g or ntpdate or sntp?

    If the drift file is known to be incorrect it's best practice to remove
    or delete it (depending on which operation your O/S supports) in which
    case nptd makes no assumptions about the drift and attempts to measure
    it. A drift file that was last modified more than 60 minutes ago should
    be assumed to be incorrect.



    One very good way to avoid startup problems is leave it running, which I do!

  4. Re: Speed of ntp convergence

    On Sat, 01 Nov 2008 03:17:15 +0000, Unruh wrote:

    > Just another data point on the behaviour of ntp. My ntp went down ( due
    > to something removing the ntp user from the password file).


    Something? FULL STOP! I'd consider the machine to be compromised and
    in need of a rebuild from scratch.

    Root Kits are known to diddle with the machine's clocks/timers in order
    to steal machine cycles undetected. In order to make the root kit work,
    the intruder most likely had to disable ntp, because it also diddles with
    the machines clocks/timers and interferes with the functioning of the
    root kit.

    [snip]

    > But never mind my concern about the markovian system feedback system ntp
    > uses. That argument I am sure everyone is tired of. What concerns me is
    > the long (1hr) time constant of the feedback loop, about 200 times
    > longer than the poll interval. Ie, it does not seem to me that ntp is
    > fulfilling its design criteria.


    If you've had a root kit installed on your machine before starting ntp,
    ntp will no longer be seeing the time in your machine, it will most likely
    be seeing whatever fake time the root kit decides to show it at any given
    instant, so it really should not be a surprise that ntp might take a bit
    longer to figure out what it is converging to. The fact that ntp
    actually does converge under these conditions only attests to the
    robustness of its design.

    If you are serious about having accurate and reliable time, you ought
    to be running on dedicated hardware with an embedded Operating System
    that can not be compromised. There is a good How-To Build A Stratum One
    Time Server published here: http://www.febo.com/pages/soekris/

    And, if you are going to go as far as to build a stratum one server
    described above, then make sure you go all the way and plug it into a
    good UPS to ensure that your time server never goes down. That way,
    ntp will just do its thing and you won't have to deal with the short
    comings, perceived or otherwise, of ntp at start up.

    >
    >
    > Here after 5.5 hours after startup is the ouput of ntpq -p
    >
    > string[root]>ntpq -p
    > remote refid st t when poll reach delay offset
    > jitter
    > ================================================== ============================
    > +tick.usask.ca .GPS. 1 u 18 64 377 44.925 1.455
    > 4.252 +ntp.ubc.ca 140.142.16.34 2 u 44 64 343 0.672
    > 0.260 0.767 *SHM(0) .PPS. 0 l 1 16 377
    > 0.000 1.136 0.023



  5. Re: Speed of ntp convergence

    "Richard B. Gilbert" writes:

    >Unruh wrote:
    >> "Richard B. Gilbert" writes:
    >>
    >>> Unruh wrote:
    >>>> "Richard B. Gilbert" writes:
    >>>>
    >>>>> Unruh wrote:
    >>>>>> "David J Taylor" writes:
    >>>>>>
    >>>>>>> Hal Murray wrote:
    >>>>>>>>>> Try switching it off, changing the value int he drift file by say
    >>>>>>>>>> 50PPM and
    >>>>>>>>>> then switching it on again, and see how long it takes to recover
    >>>>>>>>>> from that.
    >>>>>>>>> Why would I do that? The drift values rarely change by more than
    >>>>>>>>> five, certainly not by 50. If you are seeing a change of 50, then
    >>>>>>>>> perhaps that it part of your problem?
    >>>>>>>> A big step like that makes it easy to see how the system responds.
    >>>>>>>> At least if it's a linear system.
    >>>>>>> Yes, I appreciate that, Hal, but it doesn't emulate the situation here
    >>>>>>> very well, which I understood to be slow convergence after a routine
    >>>>>>> start. It sounds as if the OP may have an incorrect drift file - it's
    >>>>>>> worth checking that it /is/ being updated.
    >>>>>>
    >>>>>> The drift file read 10. The actual drift was 250 (determined after the
    >>>>>> system had settled down). The drift file never changed even after a day of
    >>>>>> running. ntp does not seem to be rewriting the drift file. Now that is a
    >>>>>> problem (although with the apparent Linux bug in the timing routines where is
    >>>>>> miscalibrates the clock on bootup, the drift is NOT stable over reboots
    >>>>>> anyway, so the existence of a drift file is irrelevant. ) However, the question is
    >>>>>> about the bahaviour of ntp. ntp should NOT be taking 10 hours to get over a
    >>>>>> wrong value in the drift file.
    >>>>> That's easy to fix! If the drift file is not correct, remove it before
    >>>>> starting ntpd.
    >>>> Of course. However, I have no idea it is incorrect until after ntp has
    >>>> started up and shown me it was incorrect.
    >>>>
    >>>>> How do you tell if it's incorrect? Since ntpd is supposed to
    >>>>> update/rewrite the drift file every sixty minutes, a drift file more
    >>>>> than sixty minutes old is suspect!
    >>>> I think my problem was that the permissions on /etc/ntp/drift were
    >>>> incorrect ( owned by root rather than by ntp). But that makes no
    >>>> difference to how ntp behaves. ntp should do the "right thing" even if the
    >>>> drift file is wrong. It should take a bit longer, but not 10 hours longer.
    >>>> And with the current apparent bug in Linux wehre the system time is
    >>>> miscalibrated, it would seem that the drift file on Linux is ALWAYS wrong.
    >>>>

    >>
    >>> Do not blame ntpd for the consequences of your errors! If ntpd is
    >>> configured correctly and operated correctly, it behaves quite well.

    >>
    >> ???? I was not blaming ntp for my error. I was blaming ntp for its reaction
    >> to "my error" . And note a bad drift file is now the standard for
    >> Linux. For example between two reboots, my drift rate went from 180PPM to
    >> 250PPM. No drift file would have fixed that.
    >>
    >> NTP handles drift errors badly. But that is not the question I asked.
    >> So far NOONE has answered the
    >> question-- why if my poll internal is 16 sec is the time scale for the
    >> error correction 1 hour?
    >>


    >How big is the error? Why is your poll interval 16 seconds? (If you


    It IS a refclock-- GPS PPS via refclock_shm.

    >are using a refclock, I withdraw the question!) Are you setting the
    >correct time at startup with ntpd -g or ntpdate or sntp?


    ntpd -g I am using the shm driver. ntp first uses other servers to set the
    time, and then starts the shm pps. The drift is way off, it seems because
    of the bug in the linux time driver ( the drift rate changes by 50PPM after
    a reboot).


    >If the drift file is known to be incorrect it's best practice to remove
    >or delete it (depending on which operation your O/S supports) in which
    >case nptd makes no assumptions about the drift and attempts to measure
    >it. A drift file that was last modified more than 60 minutes ago should
    >be assumed to be incorrect.


    I should NOT have to check up on the drift file to see if it is correct.
    That is the computer's job. ntp should NOT take 10 hours to correct a wrong
    drift file. With a 16 sec poll, ntp should NOT have a 1 hour time constant.




    >


    >One very good way to avoid startup problems is leave it running, which I do!


  6. Re: Speed of ntp convergence

    Speechless writes:

    >On Sat, 01 Nov 2008 03:17:15 +0000, Unruh wrote:


    >> Just another data point on the behaviour of ntp. My ntp went down ( due
    >> to something removing the ntp user from the password file).


    >Something? FULL STOP! I'd consider the machine to be compromised and
    >in need of a rebuild from scratch.


    Nope. It was probably the fact that on Mandriva ntp is given the uid of
    498, while my user uids start at 200. Thus when my script changes the
    users, from a master file, it wiped out the ntp user from the one machine
    on which ntp was running. It is stupid that ntp was assigned such a high
    uid.



    >Root Kits are known to diddle with the machine's clocks/timers in order
    >to steal machine cycles undetected. In order to make the root kit work,
    >the intruder most likely had to disable ntp, because it also diddles with
    >the machines clocks/timers and interferes with the functioning of the
    >root kit.


    >[snip]


    >> But never mind my concern about the markovian system feedback system ntp
    >> uses. That argument I am sure everyone is tired of. What concerns me is
    >> the long (1hr) time constant of the feedback loop, about 200 times
    >> longer than the poll interval. Ie, it does not seem to me that ntp is
    >> fulfilling its design criteria.


    >If you've had a root kit installed on your machine before starting ntp,
    >ntp will no longer be seeing the time in your machine, it will most likely
    >be seeing whatever fake time the root kit decides to show it at any given
    >instant, so it really should not be a surprise that ntp might take a bit
    >longer to figure out what it is converging to. The fact that ntp
    >actually does converge under these conditions only attests to the
    >robustness of its design.


    You are riding off into the desert on your false assumptions.

    >If you are serious about having accurate and reliable time, you ought
    >to be running on dedicated hardware with an embedded Operating System
    >that can not be compromised. There is a good How-To Build A Stratum One
    >Time Server published here: http://www.febo.com/pages/soekris/


    All systems can be comprimized. But that is irrelevant. An ordinary run of
    the mill computer with $60 worth of parts can nowadays be a good stratum 1
    server. And even your dedicated machine can go down-- disk error, hardware
    wearout, long power failure, .....


    >And, if you are going to go as far as to build a stratum one server
    >described above, then make sure you go all the way and plug it into a
    >good UPS to ensure that your time server never goes down. That way,
    >ntp will just do its thing and you won't have to deal with the short
    >comings, perceived or otherwise, of ntp at start up.


    "That's not a bug-- that is just something to avoid asking the program to
    do."

    But still no answer-- why is the time constant 1 hour when the poll
    interval is 16sec?
    Note that this ALSO affects the ability of the system to respond to temp
    changes.



    >>
    >>
    >> Here after 5.5 hours after startup is the ouput of ntpq -p
    >>
    >> string[root]>ntpq -p
    >> remote refid st t when poll reach delay offset
    >> jitter
    >> ================================================== ============================
    >> +tick.usask.ca .GPS. 1 u 18 64 377 44.925 1.455
    >> 4.252 +ntp.ubc.ca 140.142.16.34 2 u 44 64 343 0.672
    >> 0.260 0.767 *SHM(0) .PPS. 0 l 1 16 377
    >> 0.000 1.136 0.023



  7. Re: Speed of ntp convergence

    Unruh wrote:
    > "Richard B. Gilbert" writes:
    >
    >> Unruh wrote:
    >>> "Richard B. Gilbert" writes:
    >>>
    >>>> Unruh wrote:
    >>>>> "Richard B. Gilbert" writes:
    >>>>>
    >>>>>> Unruh wrote:
    >>>>>>> "David J Taylor" writes:
    >>>>>>>
    >>>>>>>> Hal Murray wrote:
    >>>>>>>>>>> Try switching it off, changing the value int he drift file by say
    >>>>>>>>>>> 50PPM and
    >>>>>>>>>>> then switching it on again, and see how long it takes to recover
    >>>>>>>>>>> from that.
    >>>>>>>>>> Why would I do that? The drift values rarely change by more than
    >>>>>>>>>> five, certainly not by 50. If you are seeing a change of 50, then
    >>>>>>>>>> perhaps that it part of your problem?
    >>>>>>>>> A big step like that makes it easy to see how the system responds.
    >>>>>>>>> At least if it's a linear system.
    >>>>>>>> Yes, I appreciate that, Hal, but it doesn't emulate the situation here
    >>>>>>>> very well, which I understood to be slow convergence after a routine
    >>>>>>>> start. It sounds as if the OP may have an incorrect drift file - it's
    >>>>>>>> worth checking that it /is/ being updated.
    >>>>>>> The drift file read 10. The actual drift was 250 (determined after the
    >>>>>>> system had settled down). The drift file never changed even after a day of
    >>>>>>> running. ntp does not seem to be rewriting the drift file. Now that is a
    >>>>>>> problem (although with the apparent Linux bug in the timing routines where is
    >>>>>>> miscalibrates the clock on bootup, the drift is NOT stable over reboots
    >>>>>>> anyway, so the existence of a drift file is irrelevant. ) However, the question is
    >>>>>>> about the bahaviour of ntp. ntp should NOT be taking 10 hours to get over a
    >>>>>>> wrong value in the drift file.
    >>>>>> That's easy to fix! If the drift file is not correct, remove it before
    >>>>>> starting ntpd.
    >>>>> Of course. However, I have no idea it is incorrect until after ntp has
    >>>>> started up and shown me it was incorrect.
    >>>>>
    >>>>>> How do you tell if it's incorrect? Since ntpd is supposed to
    >>>>>> update/rewrite the drift file every sixty minutes, a drift file more
    >>>>>> than sixty minutes old is suspect!
    >>>>> I think my problem was that the permissions on /etc/ntp/drift were
    >>>>> incorrect ( owned by root rather than by ntp). But that makes no
    >>>>> difference to how ntp behaves. ntp should do the "right thing" even if the
    >>>>> drift file is wrong. It should take a bit longer, but not 10 hours longer.
    >>>>> And with the current apparent bug in Linux wehre the system time is
    >>>>> miscalibrated, it would seem that the drift file on Linux is ALWAYS wrong.
    >>>>>
    >>>> Do not blame ntpd for the consequences of your errors! If ntpd is
    >>>> configured correctly and operated correctly, it behaves quite well.
    >>> ???? I was not blaming ntp for my error. I was blaming ntp for its reaction
    >>> to "my error" . And note a bad drift file is now the standard for
    >>> Linux. For example between two reboots, my drift rate went from 180PPM to
    >>> 250PPM. No drift file would have fixed that.
    >>>
    >>> NTP handles drift errors badly. But that is not the question I asked.
    >>> So far NOONE has answered the
    >>> question-- why if my poll internal is 16 sec is the time scale for the
    >>> error correction 1 hour?
    >>>

    >
    >> How big is the error? Why is your poll interval 16 seconds? (If you

    >
    > It IS a refclock-- GPS PPS via refclock_shm.
    >
    >> are using a refclock, I withdraw the question!) Are you setting the
    >> correct time at startup with ntpd -g or ntpdate or sntp?

    >
    > ntpd -g I am using the shm driver. ntp first uses other servers to set the
    > time, and then starts the shm pps. The drift is way off, it seems because
    > of the bug in the linux time driver ( the drift rate changes by 50PPM after
    > a reboot).
    >
    >
    >> If the drift file is known to be incorrect it's best practice to remove
    >> or delete it (depending on which operation your O/S supports) in which
    >> case nptd makes no assumptions about the drift and attempts to measure
    >> it. A drift file that was last modified more than 60 minutes ago should
    >> be assumed to be incorrect.

    >
    > I should NOT have to check up on the drift file to see if it is correct.
    > That is the computer's job. ntp should NOT take 10 hours to correct a wrong
    > drift file. With a 16 sec poll, ntp should NOT have a 1 hour time constant.


    Your opinion! The designers/developers evidently disagree.

  8. Re: Speed of ntp convergence

    On Sun, Nov 2, 2008 at 3:27 PM, David Woolley
    wrote:
    > Richard B. Gilbert wrote:
    >> The internet of today is similar. It may be a little better but I
    >> wouldn't count on it!

    > For one thing, the components from which it is made are 1,000 times
    > faster and have 1,000 times the memory.
    > (There are other differences, like the de-skilling of sytem management.)


    More significant is that the "backbone" of the Internet is no longer
    centrally controlled by an orginzation like DARPA. So one of the major
    design assumptions of NTP - symmetric routing of packets - is no
    longer valid. Asymmetric routes are now the rule rather than the
    exception (unless all traffic is local inside the same ISP).

    --
    RPM

  9. Re: Speed of ntp convergence

    Richard B. Gilbert wrote:

    >
    > Your opinion! The designers/developers evidently disagree.


    Designer, singular, as far as these issues are concerned.

    At least two new people have disagreed with the designer, recently, on
    the newsgroup, and decided that NTP is unsuitable for their application
    because of its poor startup behaviour.

  10. Re: Speed of ntp convergence

    David Woolley wrote:
    > Richard B. Gilbert wrote:
    >
    >>
    >> Your opinion! The designers/developers evidently disagree.

    >
    > Designer, singular, as far as these issues are concerned.
    >
    > At least two new people have disagreed with the designer, recently, on
    > the newsgroup, and decided that NTP is unsuitable for their application
    > because of its poor startup behaviour.


    Ntpd was not intended to be bounced up and down like a basketball and I
    doubt very much that startup speed was a design consideration. As my
    systems tend to run for months between reboots, the startup behavior of
    ntpd is not terribly important to me.

    If, for some reason, you must reboot your systems daily and you need an
    accurate clock within seconds of booting, you will just have to find a
    tool better suited to the job than ntpd.


  11. Re: Speed of ntp convergence

    "Richard B. Gilbert" writes:

    >David Woolley wrote:
    >> Richard B. Gilbert wrote:
    >>
    >>>
    >>> Your opinion! The designers/developers evidently disagree.

    >>
    >> Designer, singular, as far as these issues are concerned.
    >>
    >> At least two new people have disagreed with the designer, recently, on
    >> the newsgroup, and decided that NTP is unsuitable for their application
    >> because of its poor startup behaviour.


    >Ntpd was not intended to be bounced up and down like a basketball and I
    >doubt very much that startup speed was a design consideration. As my
    >systems tend to run for months between reboots, the startup behavior of
    >ntpd is not terribly important to me.


    >If, for some reason, you must reboot your systems daily and you need an
    >accurate clock within seconds of booting, you will just have to find a
    >tool better suited to the job than ntpd.


    Of course that is an option. However surely another option is to try to get
    ntp to start up faster-- ntp is not a force of nature, which you either
    accept or reject ( like gravity say) but is a piece of software written by
    people which can be changed, and whose design goals can be influenced. IF
    the startup behaviour of ntp were crucial to its successful operation, then
    you are right, we would simply have to accept its behaviour. But I at least
    do not think it is critical to its successful operation.
    Many features have been added to ntp over the years, and they occured
    because users requested certain features, or the designers decided they
    would be a good idea.

  12. Re: Speed of ntp convergence

    On Mon, 03 Nov 2008 06:25:15 +0000, Unruh wrote:

    > Speechless writes:
    >
    >>On Sat, 01 Nov 2008 03:17:15 +0000, Unruh wrote:

    >
    >>> Just another data point on the behaviour of ntp. My ntp went down (
    >>> due to something removing the ntp user from the password file).

    >
    >>Something? FULL STOP! I'd consider the machine to be compromised and
    >>in need of a rebuild from scratch.

    >
    > Nope. It was probably the fact that on Mandriva ntp is given the uid of
    > 498, while my user uids start at 200. Thus when my script changes the
    > users, from a master file, it wiped out the ntp user from the one
    > machine on which ntp was running. It is stupid that ntp was assigned
    > such a high uid.
    >


    Okay, that reduces but does not eliminate the possibility.

    >
    >
    >>Root Kits are known to diddle with the machine's clocks/timers in order
    >>to steal machine cycles undetected. In order to make the root kit work,
    >>the intruder most likely had to disable ntp, because it also diddles
    >>with the machines clocks/timers and interferes with the functioning of
    >>the root kit.

    >
    >>[snip]

    >
    >>> But never mind my concern about the markovian system feedback system
    >>> ntp uses. That argument I am sure everyone is tired of. What concerns
    >>> me is the long (1hr) time constant of the feedback loop, about 200
    >>> times longer than the poll interval. Ie, it does not seem to me that
    >>> ntp is fulfilling its design criteria.

    >
    >>If you've had a root kit installed on your machine before starting ntp,
    >>ntp will no longer be seeing the time in your machine, it will most
    >>likely be seeing whatever fake time the root kit decides to show it at
    >>any given instant, so it really should not be a surprise that ntp might
    >>take a bit longer to figure out what it is converging to. The fact
    >>that ntp actually does converge under these conditions only attests to
    >>the robustness of its design.

    >
    > You are riding off into the desert on your false assumptions.


    The only assumptions I've made so far are that:
    a) you possess a modicum of intelligence and
    b) you are experiencing a genuine problem not encountered by anyone else

    Thank you for setting me straight about my assumptions...you'll be added
    to my kill file in a moment...

    >
    >>If you are serious about having accurate and reliable time, you ought to
    >>be running on dedicated hardware with an embedded Operating System that
    >>can not be compromised. There is a good How-To Build A Stratum One Time
    >>Server published here: http://www.febo.com/pages/soekris/

    >
    > All systems can be comprimized. But that is irrelevant. An ordinary run
    > of the mill computer with $60 worth of parts can nowadays be a good
    > stratum 1 server. And even your dedicated machine can go down-- disk
    > error, hardware wearout, long power failure, .....
    >


    You obviously have not looked at the web site above to study in depth
    and learn what it really takes to build a stratum one time server that
    meets the requirements of a discriminating time freak. If the $60 pile
    of parts running your ntp was good enough to meet _your_ requirements,
    you would not be in this news group moaning and groaning about ntp.

    It should not be necessary to spell it out for you that if you are
    dissatisfied with the results you are getting, then you need to DO
    SOMETHING DIFFERENT to GET DIFFERENT RESULTS that might be more to your
    liking.

    For example, my assistant runs ntp on her budget priced general purpose
    machine and she is absolutely thrilled that her machine is able to keep
    time with an accuracy of "within one second" of her wrist watch. She is
    not in this news group complaining about ntp and you are. One of the
    things you might try to do DIFFERENT, is to use a better quality wrist
    watch when checking the time on your machine.

    Your other opportunity, to do something DIFFERENT, might be to put the
    pile of crap running your ntp into the thrash bin and to acquire
    timekeeping hardware with a proven track record for meeting the
    requirements of a discriminating time freak and, to run a DIFFERENT
    Operating System on said hardware that has a proven track record in
    demanding timekeeping applications. For the discriminating time freak,
    Linux is not the Operating System to use. Yes, some do use Linux,
    but not the discriminating connoisseurs of time.

    There are a number of resources on the web, in addition to the one I've
    referenced, that provide a step-by-step how-to for building a stratum
    one time server capable of tracking true UTC within a bandwidth
    of less than 150 nanoseconds, all within the budget of a typical cash
    strapped university researcher; however, building such a project entails
    allocating and expending a bit more in effort and resources than just
    $60 for a pile of parts slapped together by someone who is unable to
    demonstrate that there is some congruence between academic credentials
    and innate intelligence he might possess.

    >
    >>And, if you are going to go as far as to build a stratum one server
    >>described above, then make sure you go all the way and plug it into a
    >>good UPS to ensure that your time server never goes down. That way, ntp
    >>will just do its thing and you won't have to deal with the short
    >>comings, perceived or otherwise, of ntp at start up.

    >
    > "That's not a bug-- that is just something to avoid asking the program
    > to do."
    >
    > But still no answer-- why is the time constant 1 hour when the poll
    > interval is 16sec?


    In your case, the answer most likely lies in the realm of the ephemeral...
    perhaps better known to you as the Heisenberg uncertainty principle.


    > Note that this ALSO affects the ability of the system to respond to temp
    > changes.
    >


    Time freaks keep their time servers at a stable temperature. An
    inexpensive way to stabilize the temperature is to place the time server
    into a cavity of a large thermal mass. What you use for a thermal mass
    is left to your creative imagination. One example comes in the form of
    a 250kg cast iron truck engine block, that can be sourced from a local
    auto wrecking shop for little more than the cost of delivery. Add some
    fiber glass bat insulation at strategic places to limit air circulation
    and you are good to go.


  13. Re: Speed of ntp convergence

    Speechless wrote:

    > b) you are experiencing a genuine problem not encountered by anyone else


    Lots of people report poor startup behaviour. Two have abandoned ntpd
    on this newsgroup in the last few weeks because of this.

    >
    > It should not be necessary to spell it out for you that if you are
    > dissatisfied with the results you are getting, then you need to DO
    > SOMETHING DIFFERENT to GET DIFFERENT RESULTS that might be more to your
    > liking.


    It is perfectly valid to try and get the product improved, rather than
    simply encouraging everyone to vote with their feet.
    >
    > For example, my assistant runs ntp on her budget priced general purpose
    > machine and she is absolutely thrilled that her machine is able to keep
    > time with an accuracy of "within one second" of her wrist watch. She is


    ntpd considers accuracies of worse than about 128ms to be broken, so
    anyone who is only interested in 1 second accuracy is either getting a
    lot better than they expect, or doesn't really need ntpd.

    If you are only interested in 1 second accuracy, you don't get
    convergence issues, because ntpd will go into error recovery long before
    you reach a second. If you are out by that much at startup, you will be
    rapidly brought into that range. If you go out during operation, there
    will be an upwards of 15 minutes delay, but that will start at 128ms,
    and the time will be abruptly corrected, normally long before it reaches
    1 second. None of that really depends on the algorithms that make ntpd
    what it is.

    > not in this news group complaining about ntp and you are. One of the
    > things you might try to do DIFFERENT, is to use a better quality wrist
    > watch when checking the time on your machine.
    >


    >


  14. Re: Speed of ntp convergence

    David Woolley schrieb:
    > Speechless wrote:
    >
    >
    >> b) you are experiencing a genuine problem not encountered by anyone else
    >>

    >
    > Lots of people report poor startup behaviour. Two have abandoned ntpd
    > on this newsgroup in the last few weeks because of this.
    >
    >
    >> It should not be necessary to spell it out for you that if you are
    >> dissatisfied with the results you are getting, then you need to DO
    >> SOMETHING DIFFERENT to GET DIFFERENT RESULTS that might be more to your
    >> liking.
    >>

    >
    > It is perfectly valid to try and get the product improved, rather than
    > simply encouraging everyone to vote with their feet.
    >
    >> For example, my assistant runs ntp on her budget priced general purpose
    >> machine and she is absolutely thrilled that her machine is able to keep
    >> time with an accuracy of "within one second" of her wrist watch. She is
    >>

    >
    > ntpd considers accuracies of worse than about 128ms to be broken, so
    > anyone who is only interested in 1 second accuracy is either getting a
    > lot better than they expect, or doesn't really need ntpd.
    >
    > If you are only interested in 1 second accuracy, you don't get
    > convergence issues, because ntpd will go into error recovery long before
    > you reach a second. If you are out by that much at startup, you will be
    > rapidly brought into that range. If you go out during operation, there
    > will be an upwards of 15 minutes delay, but that will start at 128ms,
    > and the time will be abruptly corrected, normally long before it reaches
    > 1 second. None of that really depends on the algorithms that make ntpd
    > what it is.
    >
    >
    >> not in this news group complaining about ntp and you are. One of the
    >> things you might try to do DIFFERENT, is to use a better quality wrist
    >> watch when checking the time on your machine.
    >>
    >>

    Ok, I am not currently following the list on a per mail basis (WORK, a
    LOT, out of office, too..), but just caught this.. Still I want to
    reply, since it became a rather big topic, as it looks..

    I just recently tested my ntp-(pps-)machine in a large open hall and
    found it to be unable to settle down even after a day! It looks like the
    temperature changes in this open hall environment make the machine
    jitter with values between -200 and 200 ms regularly and unpredictable.
    Whenever I checked it, values were wildly varying inbetween those.. The
    same machine was able to at least settle down after an hour or so in a
    heated inside environment. I slowly come to believe that ntp simply
    lacks switches for adjusting how drastically the slewing is done in case
    the environmental temperature is varying drastically as well.

    What really bothers me is that ntp is totally capable of telling me its
    offset (and so how wrong the time /being served/ is) but not capable of
    reacting to it, due to it's programmed slowness, wich as we prove in
    some cases leads to the software being unusable.

    That's too bad and not neccesary - I wish I was a programmer...

    Best regards,
    .../nico berndt

  15. Re: Speed of ntp convergence

    Nicola Berndt wrote:

    >
    > What really bothers me is that ntp is totally capable of telling me its
    > offset (and so how wrong the time /being served/ is) but not capable of


    Careful. The theory behind ntpd is that, in normal operation, offset
    consists only of measurement error, not of real clock error. The
    problem with ntpd is that, after startup, or rapid temperature changes,
    that assumption isn't valid, and offset contains a large component of
    real error. Worse, it is obvious to a human observer, even without
    hindsight, that that is the case.

    In these situations, ntpd continues to believe that offset is noise and
    not signal.

    (One does have to beware of hindsight. In judging ntpd, one must only
    look at what has happened before its decisions.)

    > reacting to it, due to it's programmed slowness, wich as we prove in
    > some cases leads to the software being unusable.


  16. Re: Speed of ntp convergence


    I will top post this reply since there is too much garbled nonesense.

    ntp is capable of disciplining the local clock on a run of the mill PC with
    a $60 GPS receiver to a few microseconds. So lets take that as the
    standard.

    The question I have is why, given a poll interval of 16 seconds, the time
    constant of the convergence of ntp is 1 hour. This is a technical
    question, which you seem incapable of answering.

    Could I, if I designed a thermally issolated system and a better clock
    source get that long term accuracy of the system down to 1 ns? Yes. I could. It would
    be stupid for the purpose ( keeping other machines on time) since the transfer of
    time to any other computer via the net would degrade
    that accuracy to tens of microseconds, so all of the work to get the machine down to nanosecond
    accuracy would be wasted, but I could do so. But that would still be
    irrelevant to the question.

    ntp has a slow convergence, a slow reaction to errors. But as I thought I
    understood the algorithm, that timescale was of
    the order of 16 times the poll interval. Mine is 200 times
    the poll interval. That indicates one of three things.
    a) I do not properly understand the design of ntp and a time constant of
    200 times the poll interval meets the design criteria.
    b) There is some parameter in my ntp that has been misconfigured
    c) There is a bug in ntp. I would think that even you should be
    interested in bugs in ntp being fixed.

    The only reason I post here is because of the third possiblity. No-one,
    except perhaps me, is interested in the first two nor do I expect them to
    be.


    Speechless writes:

    >On Mon, 03 Nov 2008 06:25:15 +0000, Unruh wrote:


    >> Speechless writes:
    >>
    >>>On Sat, 01 Nov 2008 03:17:15 +0000, Unruh wrote:

    >>
    >>>> Just another data point on the behaviour of ntp. My ntp went down (
    >>>> due to something removing the ntp user from the password file).

    >>
    >>>Something? FULL STOP! I'd consider the machine to be compromised and
    >>>in need of a rebuild from scratch.

    >>
    >> Nope. It was probably the fact that on Mandriva ntp is given the uid of
    >> 498, while my user uids start at 200. Thus when my script changes the
    >> users, from a master file, it wiped out the ntp user from the one
    >> machine on which ntp was running. It is stupid that ntp was assigned
    >> such a high uid.
    >>


    >Okay, that reduces but does not eliminate the possibility.


    >>
    >>
    >>>Root Kits are known to diddle with the machine's clocks/timers in order
    >>>to steal machine cycles undetected. In order to make the root kit work,
    >>>the intruder most likely had to disable ntp, because it also diddles
    >>>with the machines clocks/timers and interferes with the functioning of
    >>>the root kit.

    >>
    >>>[snip]

    >>
    >>>> But never mind my concern about the markovian system feedback system
    >>>> ntp uses. That argument I am sure everyone is tired of. What concerns
    >>>> me is the long (1hr) time constant of the feedback loop, about 200
    >>>> times longer than the poll interval. Ie, it does not seem to me that
    >>>> ntp is fulfilling its design criteria.

    >>
    >>>If you've had a root kit installed on your machine before starting ntp,
    >>>ntp will no longer be seeing the time in your machine, it will most
    >>>likely be seeing whatever fake time the root kit decides to show it at
    >>>any given instant, so it really should not be a surprise that ntp might
    >>>take a bit longer to figure out what it is converging to. The fact
    >>>that ntp actually does converge under these conditions only attests to
    >>>the robustness of its design.

    >>
    >> You are riding off into the desert on your false assumptions.


    >The only assumptions I've made so far are that:
    >a) you possess a modicum of intelligence and
    >b) you are experiencing a genuine problem not encountered by anyone else


    >Thank you for setting me straight about my assumptions...you'll be added
    >to my kill file in a moment...


    >>
    >>>If you are serious about having accurate and reliable time, you ought to
    >>>be running on dedicated hardware with an embedded Operating System that
    >>>can not be compromised. There is a good How-To Build A Stratum One Time
    >>>Server published here: http://www.febo.com/pages/soekris/

    >>
    >> All systems can be comprimized. But that is irrelevant. An ordinary run
    >> of the mill computer with $60 worth of parts can nowadays be a good
    >> stratum 1 server. And even your dedicated machine can go down-- disk
    >> error, hardware wearout, long power failure, .....
    >>


    >You obviously have not looked at the web site above to study in depth
    >and learn what it really takes to build a stratum one time server that
    >meets the requirements of a discriminating time freak. If the $60 pile
    >of parts running your ntp was good enough to meet _your_ requirements,
    >you would not be in this news group moaning and groaning about ntp.


    >It should not be necessary to spell it out for you that if you are
    >dissatisfied with the results you are getting, then you need to DO
    >SOMETHING DIFFERENT to GET DIFFERENT RESULTS that might be more to your
    >liking.


    >For example, my assistant runs ntp on her budget priced general purpose
    >machine and she is absolutely thrilled that her machine is able to keep
    >time with an accuracy of "within one second" of her wrist watch. She is
    >not in this news group complaining about ntp and you are. One of the
    >things you might try to do DIFFERENT, is to use a better quality wrist
    >watch when checking the time on your machine.


    >Your other opportunity, to do something DIFFERENT, might be to put the
    >pile of crap running your ntp into the thrash bin and to acquire
    >timekeeping hardware with a proven track record for meeting the
    >requirements of a discriminating time freak and, to run a DIFFERENT
    >Operating System on said hardware that has a proven track record in
    >demanding timekeeping applications. For the discriminating time freak,
    >Linux is not the Operating System to use. Yes, some do use Linux,
    >but not the discriminating connoisseurs of time.


    >There are a number of resources on the web, in addition to the one I've
    >referenced, that provide a step-by-step how-to for building a stratum
    >one time server capable of tracking true UTC within a bandwidth
    >of less than 150 nanoseconds, all within the budget of a typical cash
    >strapped university researcher; however, building such a project entails
    >allocating and expending a bit more in effort and resources than just
    >$60 for a pile of parts slapped together by someone who is unable to
    >demonstrate that there is some congruence between academic credentials
    >and innate intelligence he might possess.


    >>
    >>>And, if you are going to go as far as to build a stratum one server
    >>>described above, then make sure you go all the way and plug it into a
    >>>good UPS to ensure that your time server never goes down. That way, ntp
    >>>will just do its thing and you won't have to deal with the short
    >>>comings, perceived or otherwise, of ntp at start up.

    >>
    >> "That's not a bug-- that is just something to avoid asking the program
    >> to do."
    >>
    >> But still no answer-- why is the time constant 1 hour when the poll
    >> interval is 16sec?


    >In your case, the answer most likely lies in the realm of the ephemeral...
    >perhaps better known to you as the Heisenberg uncertainty principle.



    >> Note that this ALSO affects the ability of the system to respond to temp
    >> changes.
    >>


    >Time freaks keep their time servers at a stable temperature. An
    >inexpensive way to stabilize the temperature is to place the time server
    >into a cavity of a large thermal mass. What you use for a thermal mass
    >is left to your creative imagination. One example comes in the form of
    >a 250kg cast iron truck engine block, that can be sourced from a local
    >auto wrecking shop for little more than the cost of delivery. Add some
    >fiber glass bat insulation at strategic places to limit air circulation
    >and you are good to go.



  17. Re: Speed of ntp convergence

    Nicola Berndt wrote:
    []
    > I just recently tested my ntp-(pps-)machine in a large open hall and
    > found it to be unable to settle down even after a day! It looks like
    > the temperature changes in this open hall environment make the machine
    > jitter with values between -200 and 200 ms regularly and
    > unpredictable. Whenever I checked it, values were wildly varying
    > inbetween those.. The same machine was able to at least settle down
    > after an hour or so in a heated inside environment. I slowly come to
    > believe that ntp simply lacks switches for adjusting how drastically
    > the slewing is done in case the environmental temperature is varying
    > drastically as well.

    []
    > Best regards,
    > ../nico berndt


    Nico,

    From what you report, I would suggest that something is very broken. I
    have a PC (FreeBSD, NTP, PPS GPS source) running in a domestic environment
    where, at the moment the central heating comes on at 0600. Here is the
    reported offset:

    http://www.satsignal.eu/mrtg/pixie_ntp.html

    Switching the room heating on causes 10usec offset, not 200msec.

    The other Windows PCs synced from that PC are generally within 10msec:

    http://www.satsignal.eu/mrtg/daily_ntp.html

    Cheers,
    David


  18. Re: Speed of ntp convergence

    Unruh wrote:
    > Evandro Menezes writes:
    > Sure. Mine is a stock Linux PC hardware, and running 4.2.4p4, not 4.2.0
    >
    > It is possible a bug has crept into the software,


    AFAIR older versions of ntpd *did* converge quite a bit faster than the
    current -stable and -dev versions. I'm afraid it's not a bug but a
    "feature" the current versions do converge so incredibly slow.

    Martin
    --
    Martin Burnicki

    Meinberg Funkuhren
    Bad Pyrmont
    Germany

  19. Re: Speed of ntp convergence

    David Woolley wrote:
    > Nicola Berndt wrote:
    >
    >>
    >> What really bothers me is that ntp is totally capable of telling me its
    >> offset (and so how wrong the time /being served/ is) but not capable of

    >
    > Careful. The theory behind ntpd is that, in normal operation, offset
    > consists only of measurement error, not of real clock error. The
    > problem with ntpd is that, after startup, or rapid temperature changes,
    > that assumption isn't valid, and offset contains a large component of
    > real error. Worse, it is obvious to a human observer, even without
    > hindsight, that that is the case.


    Some time ago I have made some tests comparing the system time disciplined
    by NTP against the time of a built-in GPS PCI card. The results showed the
    offset reported by ntpd did very well match the offset measured against the
    GPS card.

    I think the current development of NTP goes into a direction which may cover
    some special scenarions well, but matches less and less the requirements of
    "normal" users.

    Martin
    --
    Martin Burnicki

    Meinberg Funkuhren
    Bad Pyrmont
    Germany

  20. Re: Speed of ntp convergence

    David J Taylor wrote:
    > Unruh wrote:
    > []
    >> An hour later, it was still 7ms off, another hour, 2.6ms and another
    >> hour
    >> later, still 1.2 ms off. Ie, only after about 6 hours was it within a
    >> ms of
    >> the correct time. Now, usually this PPS controls the time to within
    >> about 2us (not ms, usec) but it is apparently going to take over 10
    >> hours to get
    >> there. That is of course completely rediculous.

    >
    > There sounds to be something wrong with your system.


    As mentioned in another reply, AFAIR older versions of the NTP daemon did
    converge quite a bit faster than recent versions.

    > As a comparison, I have a very old Pentium 133 system here running FreeBSD
    > with local GPS PPS and some other Internet-based stratum 2/3 servers
    > (probably NTP pool and a fixed name). I'm sure it's well within a few
    > minutes for it to reach it's full accuracy (tens of microseconds). For
    > interest, I've just (0645 UTC) switched it off and on, and we will be able
    > to watch its recovery here (30 minute updates):
    >
    > http://www.satsignal.eu/mrtg/pixie_ntp.html
    >
    > Here it is about a minute after startup:
    >
    > ntpq -p pixie
    > remote refid st t when poll reach delay offset
    > jitter
    >

    ================================================== ============================
    > +calx.pulsewidth 193.120.10.3 2 u 62 64 1 22.272 1.700
    > 0.743
    > +admin.islay.bit 192.33.96.102 2 u 62 64 1 21.131 0.921
    > 1.112
    > +dnscache-london 128.250.33.242 2 u 62 64 1 22.845 3.299
    > 0.666
    > 88-96-233-89.ds .PPS. 1 u 14 128 7 63.431 0.044
    > 2.789
    > *utserv.mcc.ac.u 193.62.22.98 2 u 64 64 1 26.494 4.312
    > 0.829
    > GPS_NMEA(1) .PPS. 0 l 12 64 3 0.000 -0.137
    > 1.654
    >
    > .. and a few minutes later ...
    >
    > ntpq -p pixie
    > remote refid st t when poll reach delay offset
    > jitter
    >

    ================================================== ============================
    > +calx.pulsewidth 193.120.10.3 2 u 54 64 37 22.348 2.946
    > 0.877
    > +admin.islay.bit 192.33.96.102 2 u 53 64 37 20.496 1.862
    > 1.057
    > +dnscache-london 128.250.33.242 2 u 57 64 37 23.090 3.809
    > 0.662
    > 88-96-233-89.ds .PPS. 1 u 134 256 17 63.431 0.044
    > 2.007
    > +utserv.mcc.ac.u 193.62.22.98 2 u 53 64 37 25.564 5.371
    > 0.868
    > *GPS_NMEA(1) .PPS. 0 l 3 64 77 0.000 -0.001
    > 0.803
    >
    > It's using the out-of-the-box NTP code, and probably a rather old version
    > of NTP.
    >
    > version="ntpd 4.2.0-a Sun May 8 06:01:21 UTC 2005 (1)"


    Now the question is how a recent version of ntpd would converge on this
    machine.

    Martin
    --
    Martin Burnicki

    Meinberg Funkuhren
    Bad Pyrmont
    Germany

+ Reply to Thread
Page 2 of 3 FirstFirst 1 2 3 LastLast