Server offset included in served time? - NTP

This is a discussion on Server offset included in served time? - NTP ; Uwe Klein writes: >David Woolley wrote: >> Martin Burnicki wrote: >>> While the system time is being slewed it may be e.g. 120 ms off, and when >>> the daemon sends the system time to its clients then it will ...

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2
Results 21 to 27 of 27

Thread: Server offset included in served time?

  1. Re: Server offset included in served time?

    Uwe Klein writes:

    >David Woolley wrote:
    >> Martin Burnicki wrote:


    >>> While the system time is being slewed it may be e.g. 120 ms off, and when
    >>> the daemon sends the system time to its clients then it will serve a time
    >>> which is 120 ms off.

    >>
    >>
    >> To some extent the fact that systems are already experiencing this
    >> suggests, to me, that one might not need to alarm the time during a
    >> temporary short loop time constant phase at startup.


    >Does ntp serve "system" time or "correct" time to clients?


    ntp has no idea of "correct time" other than server time.



  2. Re: Server offset included in served time?

    David Woolley writes:

    >Uwe Klein wrote:


    >>
    >> Does ntp serve "system" time or "correct" time to clients?


    >NTP serves system time, which its algorithms believe to be the best
    >practical estimate of correct time within statistical error. What we
    >are talking about here is making that assumption more valid during
    >startup transients.


    >Basically, as far as NTP is concerned, there is no difference between
    >system time and correct time. (Earlier versions used to serve a
    >corrected time during slew recoveries, but not during normal operation.
    > That would not make a difference for the 120ms case discussed here, as
    >that uses the normal control loop.)


    Note that this is a difference between ntp and chrony. Chrony uses past
    offset measurements ( corrected for local clock rate changes) to make an
    estimate from the measured offsets of the "correct time" and then slews the
    clock quickly to that "correct time". Obviously on startup it, and ntp,
    have no better estimates of correct time than the very few measurments of
    offset they have. Chrony rapidly gets rid of those offset estimates (if
    necessary much faster than 500PPM) by slewing the clock. If you ask it to
    it will step on startup, but in general it rapidly slews to the correct time.
    I cannot remember if it corrects for the "known offset" in the time it
    delivers to the outside.



  3. Re: Server offset included in served time?

    Martin Burnicki wrote:
    > David Woolley wrote:
    > Basically I agree. However, if the packet turnaround time is low, e.g below
    > 1 millisecond on a local net, and evaluation of the timestamps yields an
    > offset of about 120 ms (according to my example shortly after startup) then
    > the result clearly indicates the own system time is off by 120 ms.


    You need to use the full error bound calculation, not just single hop
    delay. I.E. you need to account for root dispersion (contributions from
    clock drift since the last measurement, precision, etc.) and root
    delay. You also need to account for the server being in its initial
    convergence period (I think ntpd currently converges root dispersion
    faster than it actually converges the time!

    >
    > Of course if you rely to some internet servers then there may be rather
    > large turnaround times/delays. However, given the upstream server sends
    > correct time stamps then then the error due to packet delays on the network
    > can not exceed the packet turnaround interval. So if the turnaround


    But that isn't the only source of errors.

    > interval is low (e.g. < 1 ms) and the time offset is high (e.g. ~120 ms)
    > then ntpd should indeed start compensate the time offset much quicklier
    > than it currently does.
    >
    >
    > If a complete network is powered up, inclusive of the time server, and the
    > initial time offset of the time server is 120 ms and is only slewed down
    > after hours, do you really thing it's better to slew the time that slowly
    > on the time server and providing a time to his clients which is 120 ms off?


    Dave Mills normally argues that the loop constants are carefully crafted
    so that when you chain many hops together, you still have a system that
    is stable. I was anticipating that argument by providing an escape clause.

    >
    > So the clients would start to synchronize to a reference time which is off
    > by 120 ms and then follow the slow corrections of the time server?
    >
    >


  4. Re: Server offset included in served time?

    Unruh wrote:
    > David Woolley writes:
    >
    >> But, if you use individual measurements, you will get a figure that,
    >> most of the time, is several times the true error and not necessarily in
    >> the right direction.

    >
    > No idea what that sentence means. Are you refering to the gps readings
    > which will be at worst 2 orders of magnitude better than that offsets from
    > a generic network?


    I'm assuming a time discipline algorithm that is properly matched to the
    system time noise. I tend to agree that NTP probably isn't, but in that
    case one should be changing the algorithm to make it properly matched,
    rather than trying to record how bad it is.

    With such an algorithm, one would expect the measured offsets to be more
    or less equally positive and negative and distributed fairly randomly.
    That is the mathematical assumption that I believe is the basis of the
    theoretical analysis of the behaviour of NTP. The various filters in
    NTP will low pass filter this noise and considerably reduce it in
    amplitude, resulting in the value in the system clock. As a result, the
    jitter in the system clock should be a lot less than the measurement
    jitter, and at any one instant may be result in a deviation in the same
    direction as the offset reducing the resulting offset.

    NTP also assumes that there will be a wander component, and tries to
    increase the polling interval until the wander component begins to dominate.

    Your position, and I tend to agree with it, is that there is a variation
    that there is another noise component, which results in occasional fast
    transients. But as I said, the correct approach to that is not to graph
    the resulting failure, but to improve the algorithms to give the best
    estimate of the time under those circumstances, which is, I believe,
    what you think chrony does. My guess is that chrony also produces
    offsets with a standard deviation that is significantly greater than the
    typical system clock error.

    >
    >> What you can do, is to use some hindsight, and make a slightly better
    >> estimate of the true time by combining offsets from both before and
    >> after the time the local clock was read. That gives you an advantage

    >
    > Unfortunately those offsets are not very useful, because the clock that
    > read them has had its offsets and rates changed (by ntp) since then. Ie the
    > measured offsets are not a good estimate of the offsets from "true time".


    Ideally, those adjustments are compensating for parameter changes in the
    local clock and making it keep closer to true time. With chrony, the
    correction history is also largely measuring things like temperature,
    rather than true changes in the clock rate.

    >
    >
    >> during startup and after transients, but, in the steady state, the local
    >> clock time is going to be the same as such an enhanced statistic.

    >
    >> In the steady state, all you can really deduce from the offsets is the
    >> amount of noise in the measurements. You can then expect the amount of
    >> noise in the local clock time to be several times less.

    >
    > No, the noise in the local clock may well dominate those offsets. This is
    > what happens for example to a system which is controlled by a hardware


    In that case, you have a system that is not matched to the noise
    characteristics and you need to improve the algorithms. Again, to the
    extent that you can reliably measure the error from true time, you
    should be correcting the local clock by that amount, not measuring it
    for posterity.

    My basic point in this whole thread is that if you can measure the true
    error, in real time, you can correct it in real time, resulting in an
    error that is statistically zero, based on historic measurements.

  5. Re: Server offset included in served time?

    David Woolley writes:

    >Unruh wrote:
    >> David Woolley writes:
    >>
    >>> But, if you use individual measurements, you will get a figure that,
    >>> most of the time, is several times the true error and not necessarily in
    >>> the right direction.

    >>
    >> No idea what that sentence means. Are you refering to the gps readings
    >> which will be at worst 2 orders of magnitude better than that offsets from
    >> a generic network?


    >I'm assuming a time discipline algorithm that is properly matched to the
    >system time noise. I tend to agree that NTP probably isn't, but in that
    >case one should be changing the algorithm to make it properly matched,
    >rather than trying to record how bad it is.


    >With such an algorithm, one would expect the measured offsets to be more
    >or less equally positive and negative and distributed fairly randomly.
    >That is the mathematical assumption that I believe is the basis of the
    >theoretical analysis of the behaviour of NTP. The various filters in
    >NTP will low pass filter this noise and considerably reduce it in
    >amplitude, resulting in the value in the system clock. As a result, the
    >jitter in the system clock should be a lot less than the measurement
    >jitter, and at any one instant may be result in a deviation in the same
    >direction as the offset reducing the resulting offset.


    >NTP also assumes that there will be a wander component, and tries to
    >increase the polling interval until the wander component begins to dominate.


    >Your position, and I tend to agree with it, is that there is a variation
    >that there is another noise component, which results in occasional fast
    >transients. But as I said, the correct approach to that is not to graph
    >the resulting failure, but to improve the algorithms to give the best
    >estimate of the time under those circumstances, which is, I believe,
    >what you think chrony does. My guess is that chrony also produces
    >offsets with a standard deviation that is significantly greater than the
    >typical system clock error.


    Not sure what you mean. Do you mean that chrony averages out the offsets,
    so that the actual estimated offset of the clock is less than the offsets
    measured because of statistical averaging, then yes.


    >>
    >>> What you can do, is to use some hindsight, and make a slightly better
    >>> estimate of the true time by combining offsets from both before and
    >>> after the time the local clock was read. That gives you an advantage

    >>
    >> Unfortunately those offsets are not very useful, because the clock that
    >> read them has had its offsets and rates changed (by ntp) since then. Ie the
    >> measured offsets are not a good estimate of the offsets from "true time".


    >Ideally, those adjustments are compensating for parameter changes in the
    >local clock and making it keep closer to true time. With chrony, the
    >correction history is also largely measuring things like temperature,
    >rather than true changes in the clock rate.


    Temperature causes true changes in the clock rate.



    >>
    >>
    >>> during startup and after transients, but, in the steady state, the local
    >>> clock time is going to be the same as such an enhanced statistic.

    >>
    >>> In the steady state, all you can really deduce from the offsets is the
    >>> amount of noise in the measurements. You can then expect the amount of
    >>> noise in the local clock time to be several times less.

    >>
    >> No, the noise in the local clock may well dominate those offsets. This is
    >> what happens for example to a system which is controlled by a hardware


    >In that case, you have a system that is not matched to the noise
    >characteristics and you need to improve the algorithms. Again, to the
    >extent that you can reliably measure the error from true time, you
    >should be correcting the local clock by that amount, not measuring it
    >for posterity.


    >My basic point in this whole thread is that if you can measure the true
    >error, in real time, you can correct it in real time, resulting in an
    >error that is statistically zero, based on historic measurements.


    OK. I agree.

  6. Re: Server offset included in served time?

    Unruh wrote:
    > David Woolley writes:


    >
    > Not sure what you mean. Do you mean that chrony averages out the offsets,
    > so that the actual estimated offset of the clock is less than the offsets
    > measured because of statistical averaging, then yes.
    >


    Yes. And that would be true of any algorithm that was well matched to
    the noise characteristics.
    >
    > Temperature causes true changes in the clock rate.


    Exactly. And to that extent, projecting the new clock rate back to
    adjust the apparent historic offsets is wrong.

  7. Re: Server offset included in served time?

    David Woolley writes:

    >Unruh wrote:
    >> David Woolley writes:


    >>
    >> Not sure what you mean. Do you mean that chrony averages out the offsets,
    >> so that the actual estimated offset of the clock is less than the offsets
    >> measured because of statistical averaging, then yes.
    >>


    >Yes. And that would be true of any algorithm that was well matched to
    >the noise characteristics.
    >>
    >> Temperature causes true changes in the clock rate.


    >Exactly. And to that extent, projecting the new clock rate back to
    >adjust the apparent historic offsets is wrong.


    Which is why chrony tests its fit to see whether or not a linear fit to the
    data is a good statistical fit or not. (It tests the number of times that the data
    changes sign around the fit and if it does not do so often enough, assumes
    that a straight line is not a good fit, and reduces the number of historic
    points it saves. Ie, if a linear drift is not a good fit to the data, it
    assumes that the changes in drift are more important source of error than the random
    measurement/transmission noise and adjusts the fit model. Effectively it
    is constantly trying to figure out where the minimum allan variance lies
    where random offset noise and drift noise are of equal size. )



+ Reply to Thread
Page 2 of 2 FirstFirst 1 2