Slow convergence of NTP with GPS/PPS

Hi

We are using Linux ntpd with GPS/PPS reference clock to discipline the time
on our systems.

Our application requires good time accuracy (better than 5ms) but it also
needs to get there quickly (as quickly as possible, but ideally taking no
more than about 15 minutes).
(The Linux/ntpd is running on a remote embedded device that is frequently
restarted - possibly once a day or so - so we cant wait hours for
convergence).

Currently ntpd can take hours to achieve the desired acuracy.

So, the question is simple - is there any way to significantly speedup the
convergence of ntpd (using GPS/PPS reference clock)?

We would be prepared to compromise somewhat on accuracy and jitter.
(Currently accuracy and jitter values are excellent with jitter as low as 1
microsecond and accuracy better than 10 uS but it can take a day or two to
get there).

It does not seem unreasonable to expect that the ntpd could achieve the
required accuracy within 15 minutes or so - but nothing we have tried seems
to work.
Have tried modifying some of the tinker values, but we dont really
understand what they all do - and have not really had any success.

So to summarise:

1) Is it possible to speedup ntpd convergence (using GPS/PPS reference
clock)?
2) If so, how - and what are the tradeoffs?

Any help appreciated
David

David McConnell wrote:[color=blue]
> Hi
>
> We are using Linux ntpd with GPS/PPS reference clock to discipline the time
> on our systems.
>
> Our application requires good time accuracy (better than 5ms) but it also
> needs to get there quickly (as quickly as possible, but ideally taking no
> more than about 15 minutes).
> (The Linux/ntpd is running on a remote embedded device that is frequently
> restarted - possibly once a day or so - so we cant wait hours for
> convergence).
>
> Currently ntpd can take hours to achieve the desired acuracy.
>
> So, the question is simple - is there any way to significantly speedup the
> convergence of ntpd (using GPS/PPS reference clock)?[/color]

If you are using a recent version of ntpd, start it with the "-g"
switch. That will cause it to set the clock to the correct time once
only! If you have a good drift file, you should be synchronized in
thirty seconds or so and be within ten milliseconds, or less, of the
correct time.

Try not to reboot unless absolutely necessary. I realize that some
versions of Windows need fairly frequent reboots but there are are
versions that should run for many days or weeks between reboots. My
desktop system is W/XP SP2 and has been up for at least a couple of
months and may stay up for another couple of months.

<snip>

Richard B. Gilbert wrote:
[color=blue]
>
> If you are using a recent version of ntpd, start it with the "-g"
> switch. That will cause it to set the clock to the correct time once
> only! If you have a good drift file, you should be synchronized in
> thirty seconds or so and be within ten milliseconds, or less, of the
> correct time.[/color]

My understanding was that -g turns off the 1000 second check for the
first step, but still leaves the time within +/- 128ms, which will still
take an unacceptable time to converge to +/- 5ms. Certainly the 4.2.4p4
documentation makes no claims for it beyond once only disabling the 1000
second check.
[color=blue]
>
> Try not to reboot unless absolutely necessary. I realize that some
> versions of Windows need fairly frequent reboots but there are are[/color]

I imagine this is some sort of embedded system, which he can't control,
but might be switched off outside office hours, in part because that is
the socially responsible thing to do.

[demime 1.01d removed an attachment of type multipart/signed]

Kevin Oberman wrote:[color=blue]
> [demime 1.01d removed an attachment of type multipart/signed][/color]

This has failed to reach the newsgroup.

David Woolley wrote:[color=blue]
> Richard B. Gilbert wrote:
>[color=green]
>>
>> If you are using a recent version of ntpd, start it with the "-g"
>> switch. That will cause it to set the clock to the correct time once
>> only! If you have a good drift file, you should be synchronized in
>> thirty seconds or so and be within ten milliseconds, or less, of the
>> correct time.[/color]
>
> My understanding was that -g turns off the 1000 second check for the
> first step, but still leaves the time within +/- 128ms, which will still
> take an unacceptable time to converge to +/- 5ms. Certainly the 4.2.4p4
> documentation makes no claims for it beyond once only disabling the 1000
> second check.[/color]

I don't recall that +/- 128ms is specified anywhere. If ntpd gets it's
initial time from a hardware reference clock, the time SHOULD be very
close. This will, off course, depend on the latencies in the reference
clock's response and the resolution of the time supplied.
<snip>

[email]davidm@pipstechnology.co.uk[/email] (David McConnell) writes:
[color=blue]
>Hi[/color]
[color=blue]
>We are using Linux ntpd with GPS/PPS reference clock to discipline the time
>on our systems.[/color]
[color=blue]
>Our application requires good time accuracy (better than 5ms) but it also
>needs to get there quickly (as quickly as possible, but ideally taking no
>more than about 15 minutes).
>(The Linux/ntpd is running on a remote embedded device that is frequently
>restarted - possibly once a day or so - so we cant wait hours for
>convergence).[/color]
[color=blue]
>Currently ntpd can take hours to achieve the desired acuracy.[/color]

Are you using the -g option? What is the poll interval for your gps clock?
(Use 4 whcih I think is the default for gps clocks.)

ntp is NOT designed for rapid convergence, but on a gps clock with poll
interval 4 and -g it sure should be better than hours.

[color=blue]
>So, the question is simple - is there any way to significantly speedup the
>convergence of ntpd (using GPS/PPS reference clock)?[/color]
[color=blue]
>We would be prepared to compromise somewhat on accuracy and jitter.
>(Currently accuracy and jitter values are excellent with jitter as low as 1
>microsecond and accuracy better than 10 uS but it can take a day or two to
>get there).[/color]
[color=blue]
>It does not seem unreasonable to expect that the ntpd could achieve the
>required accuracy within 15 minutes or so - but nothing we have tried seems
>to work.
>Have tried modifying some of the tinker values, but we dont really
>understand what they all do - and have not really had any success.[/color]
[color=blue]
>So to summarise:[/color]
[color=blue]
>1) Is it possible to speedup ntpd convergence (using GPS/PPS reference
>clock)?
>2) If so, how - and what are the tradeoffs?[/color]
[color=blue]
>Any help appreciated
>David[/color]

Richard B. Gilbert wrote:
[color=blue]
>
> I don't recall that +/- 128ms is specified anywhere. If ntpd gets it's
> initial time from a hardware reference clock, the time SHOULD be very
> close. This will, off course, depend on the latencies in the reference
> clock's response and the resolution of the time supplied.
> <snip>[/color]


This is where it is documented in the source code for 4.2.4p4
(ntpd/loopfilter.c):

* State < step > step Comments
* ====================================================
* NSET FREQ step, FREQ no ntp.drift
*
* FSET SYNC step, SYNC ntp.drift
*

NSET is the initial state for a cold start. FSET is the initial state
for a warm start (ntp.drift) present. For a fast start you don't want
NSET. Note that FSET goes to fully synchronised on one reading, but
only steps the time if the offset is greater than the step limit, which
normally 128ms.

Furthermore, in the branch that is conditioned thus:

if (fabs(fp_offset) > clock_max && clock_max > 0) {

there is this comment:

* In S_FSET state the initial frequency has been set
* from the frequency file. Since the time is outside
* the step threshold, the clock is stepped immediately,
* rather than after the stepout interval. Guys get
* nervous if it takes 17 minutes to set the clock for
* the first time.
*

immediately followed by:

step_systime(fp_offset);

There is no step_systime in the else branch. clock_max is the tinkered
value of the parameter that defaults to 128ms.

David Woolley <david@ex.djwhome.demon.co.uk.invalid> writes:
[color=blue]
>Richard B. Gilbert wrote:[/color]
[color=blue][color=green]
>>
>> If you are using a recent version of ntpd, start it with the "-g"
>> switch. That will cause it to set the clock to the correct time once
>> only! If you have a good drift file, you should be synchronized in
>> thirty seconds or so and be within ten milliseconds, or less, of the
>> correct time.[/color][/color]
[color=blue]
>My understanding was that -g turns off the 1000 second check for the
>first step, but still leaves the time within +/- 128ms, which will still
>take an unacceptable time to converge to +/- 5ms. Certainly the 4.2.4p4
>documentation makes no claims for it beyond once only disabling the 1000
>second check.[/color]

128ms at 500PPM is 250sec or 4 min. It will take longer than that since the
rate will not peg out, but it should not be hours.
Maybe it is best to set the clock initiallly so that it is out by by more
than 128ms (Eg advance it by 10 sec) and then use -g.


[color=blue][color=green]
>>
>> Try not to reboot unless absolutely necessary. I realize that some
>> versions of Windows need fairly frequent reboots but there are are[/color][/color]
[color=blue]
>I imagine this is some sort of embedded system, which he can't control,
>but might be switched off outside office hours, in part because that is
>the socially responsible thing to do.[/color]

"Richard B. Gilbert" <rgilbert88@comcast.net> writes:
[color=blue]
>David Woolley wrote:[color=green]
>> Richard B. Gilbert wrote:
>>[color=darkred]
>>>
>>> If you are using a recent version of ntpd, start it with the "-g"
>>> switch. That will cause it to set the clock to the correct time once
>>> only! If you have a good drift file, you should be synchronized in
>>> thirty seconds or so and be within ten milliseconds, or less, of the
>>> correct time.[/color]
>>
>> My understanding was that -g turns off the 1000 second check for the
>> first step, but still leaves the time within +/- 128ms, which will still
>> take an unacceptable time to converge to +/- 5ms. Certainly the 4.2.4p4
>> documentation makes no claims for it beyond once only disabling the 1000
>> second check.[/color][/color]
[color=blue]
>I don't recall that +/- 128ms is specified anywhere. If ntpd gets it's
>initial time from a hardware reference clock, the time SHOULD be very
>close. This will, off course, depend on the latencies in the reference
>clock's response and the resolution of the time supplied.
><snip>[/color]

ntp steps if the time is out by more than 128ms. If it is out by more than
1000 seconds it aborts. -g stops that abort once, presumably at intial
startup. Thus if at initial startup th etime is out by >128ms, it will step
the time ( to the correct time) and if the drift file is right, will then
run with that correct drift and the correct time. If it is out by <128ms it
will adjust the frequency to try to reduce that offset via the feedback
loop. Since the max rate is 500PPM if it is out by 128ms it will take min 4
min to adjust, but likely much longer. The max adjust depends on the poll
interval. Now for the refclocks I know, that is 4 (16s) so the time
constants in that feedback loop are about 4 min. if I recall correctly.
(It is such that it is longer than the max time between data which is
via the filter algorithm is one data point per 8 poll intervals.
That is the time to reduce the error by about 1/e so it will take about 5
of those intervals or 20min. (Yes, I know it is not a simple exponential
falloff).
This is a design decision. And David will defend this "slow convergence"
"to the death". chrony is much faster, but does not do refclocks at all so
that is a useless option here.

Unruh wrote:
[color=blue]
>
> 128ms at 500PPM is 250sec or 4 min. It will take longer than that since the
> rate will not peg out, but it should not be hours.[/color]

It will follow the normal transient response, which has a first zero
crossing which I believe is at about 39 minutes for phase (RFC 1305) and
rather longer for frequency). I'm not sure, but it may well overshoot
by more than 5ms, in which it could take rather longer to be acceptable
to the OP.

It should get nowhere near being slew rate limited, so the 500ppm limit
is academic.

[color=blue]
> Maybe it is best to set the clock initiallly so that it is out by by more
> than 128ms (Eg advance it by 10 sec) and then use -g.[/color]

That would be my best trick, but if you have to use -g, you probably
don't know the time well enough to be sure that adding 10 seconds won't
actually put it in the 128ms band!

David Woolley wrote:
[color=blue]
>
> It will follow the normal transient response, which has a first zero
> crossing which I believe is at about 39 minutes for phase (RFC 1305) and
> rather longer for frequency). I'm not sure, but it may well overshoot[/color]

I think those figures are based on minpoll = 6. If the reference clock
accepts a shorter minpoll, the times may be correspondingly shorter.

Unruh wrote:
[][color=blue]
> This is a design decision. And David will defend this "slow
> convergence" "to the death". chrony is much faster, but does not do
> refclocks at all so that is a useless option here.[/color]

chrony is also useless on Windows.

David

Thanks for the responses.

We have tried -g and minpoll/maxpoll are by default 4 for the GPS reference
clock.
We even recompiled ntpd with source modified to allow poll at 2sec intervals
(minpoll=1) but this did not seem to make much difference.

We have also tried fiddling some of the "tinker" settings (step and stepout)
but this just seems to lead to instability.

Also, even if we set the time pretty much perfect (within 5ms offset), ntpd
appears to first *increase* the offset to well out of our spec, then correct
through zero offset - overshooting the other way (again well out of our
spec) and then typically crawls back in after which it is stable - and
ultimately wonderfully accurate and stable.

I was hoping that some of the other tinker parameters ("allan" or
"dispersion" for e.g.) might have an effect - but what are sensible values
to use?
I realise that this will compromise ntpd's performance in other ways, but,
we could tolerate worse final accuracey and jitter in exchange for getting
to within 5ms "quickly".

The driftfile also sometimes seems to do more harm than good - especially
after a reboot.
[color=blue]
> -----Original Message-----
> From: questions-bounces+davidm=pipstechnology.co.uk@lists.ntp.org
> [mailto:questions-bounces+davidm=pipstechnology.co.uk@lists.ntp.org]On
> Behalf Of David McConnell
> Sent: 30 September 2008 14:04
> To: [email]questions@lists.ntp.org[/email]; [email]linuxpps@ml.enneenne.com[/email]
> Subject: [ntp:questions] Slow convergence of NTP with GPS/PPS
>
>
> Hi
>
> We are using Linux ntpd with GPS/PPS reference clock to
> discipline the time
> on our systems.
>
> Our application requires good time accuracy (better than 5ms)
> but it also
> needs to get there quickly (as quickly as possible, but
> ideally taking no
> more than about 15 minutes).
> (The Linux/ntpd is running on a remote embedded device that
> is frequently
> restarted - possibly once a day or so - so we cant wait hours for
> convergence).
>
> Currently ntpd can take hours to achieve the desired acuracy.
>
> So, the question is simple - is there any way to
> significantly speedup the
> convergence of ntpd (using GPS/PPS reference clock)?
>
> We would be prepared to compromise somewhat on accuracy and jitter.
> (Currently accuracy and jitter values are excellent with
> jitter as low as 1
> microsecond and accuracy better than 10 uS but it can take a
> day or two to
> get there).
>
> It does not seem unreasonable to expect that the ntpd could
> achieve the
> required accuracy within 15 minutes or so - but nothing we
> have tried seems
> to work.
> Have tried modifying some of the tinker values, but we dont really
> understand what they all do - and have not really had any success.
>
> So to summarise:
>
> 1) Is it possible to speedup ntpd convergence (using GPS/PPS reference
> clock)?
> 2) If so, how - and what are the tradeoffs?
>
> Any help appreciated
> David
>
>
> _______________________________________________
> questions mailing list
> [email]questions@lists.ntp.org[/email]
> [url]https://lists.ntp.org/mailman/listinfo/questions[/url]
>[/color]

Would the following work with a reference clock?

Step 1. Force an initial step adjustment by running ntpd in "one-shot"
mode with -gq options and "tinker step 0.001" in the config file to
get below the 128ms step threshold.

Step 2. Restart ntpd in "normal mode" (without -gq and without "tinker
step 0.001" in the config file).

Unruh wrote:[color=blue]
> David Woolley <david@ex.djwhome.demon.co.uk.invalid> writes:[color=green]
>> My understanding was that -g turns off the 1000 second check for the
>> first step, but still leaves the time within +/- 128ms, which will still
>> take an unacceptable time to converge to +/- 5ms. Certainly the 4.2.4p4
>> documentation makes no claims for it beyond once only disabling the 1000
>> second check.[/color]
>
> 128ms at 500PPM is 250sec or 4 min. It will take longer than that since the
> rate will not peg out, but it should not be hours.
> Maybe it is best to set the clock initiallly so that it is out by by more
> than 128ms (Eg advance it by 10 sec) and then use -g.[/color]

Afair the 128 ms is also a 'tinker' configuration parameter, i.e. it
should be possible to reduce it to 10 or 15 ms, at the cost of getting a
few more steps during runtime.

Terje

--
- <Terje.Mathisen@hda.hydro.com>
"almost all programming can be viewed as an exercise in caching"

Unruh wrote:[color=blue]
> [email]davidm@pipstechnology.co.uk[/email] (David McConnell) writes:
>[color=green]
>> Hi[/color]
>[color=green]
>> We are using Linux ntpd with GPS/PPS reference clock to discipline the time
>> on our systems.[/color]
>[color=green]
>> Our application requires good time accuracy (better than 5ms) but it also
>> needs to get there quickly (as quickly as possible, but ideally taking no
>> more than about 15 minutes).
>> (The Linux/ntpd is running on a remote embedded device that is frequently
>> restarted - possibly once a day or so - so we cant wait hours for
>> convergence).[/color]
>[color=green]
>> Currently ntpd can take hours to achieve the desired acuracy.[/color]
>
> Are you using the -g option? What is the poll interval for your gps clock?
> (Use 4 whcih I think is the default for gps clocks.)
>
> ntp is NOT designed for rapid convergence, but on a gps clock with poll
> interval 4 and -g it sure should be better than hours.
>
>[color=green]
>> So, the question is simple - is there any way to significantly speedup the
>> convergence of ntpd (using GPS/PPS reference clock)?[/color]
>[color=green]
>> We would be prepared to compromise somewhat on accuracy and jitter.
>> (Currently accuracy and jitter values are excellent with jitter as low as 1
>> microsecond and accuracy better than 10 uS but it can take a day or two to
>> get there).[/color]
>[color=green]
>> It does not seem unreasonable to expect that the ntpd could achieve the
>> required accuracy within 15 minutes or so - but nothing we have tried seems
>> to work.
>> Have tried modifying some of the tinker values, but we dont really
>> understand what they all do - and have not really had any success.[/color]
>[color=green]
>> So to summarise:[/color]
>[color=green]
>> 1) Is it possible to speedup ntpd convergence (using GPS/PPS reference
>> clock)?
>> 2) If so, how - and what are the tradeoffs?[/color]
>[color=green]
>> Any help appreciated
>> David[/color][/color]

On a COLD START, I would expect to have to wait for something like
thirty minutes to get really TIGHT synchronization (5 milliseconds or
less). Once you have it, you should be able to hold it until you have a
power failure or a hardware failure.

If you can't wait for synchronization, get a good UPS, a generator to
back it up and NEVER shut down or reboot!

David Woolley wrote:[color=blue]
> Richard B. Gilbert wrote:
>[color=green]
>>
>> I don't recall that +/- 128ms is specified anywhere. If ntpd gets
>> it's initial time from a hardware reference clock, the time SHOULD be
>> very close. This will, off course, depend on the latencies in the
>> reference clock's response and the resolution of the time supplied.
>> <snip>[/color]
>
>
> This is where it is documented in the source code for 4.2.4p4
> (ntpd/loopfilter.c):
>
> * State < step > step Comments
> * ====================================================
> * NSET FREQ step, FREQ no ntp.drift
> *
> * FSET SYNC step, SYNC ntp.drift
> *
>
> NSET is the initial state for a cold start. FSET is the initial state
> for a warm start (ntp.drift) present. For a fast start you don't want
> NSET. Note that FSET goes to fully synchronised on one reading, but
> only steps the time if the offset is greater than the step limit, which
> normally 128ms.
>
> Furthermore, in the branch that is conditioned thus:
>
> if (fabs(fp_offset) > clock_max && clock_max > 0) {
>
> there is this comment:
>
> * In S_FSET state the initial frequency has been set
> * from the frequency file. Since the time is outside
> * the step threshold, the clock is stepped immediately,
> * rather than after the stepout interval. Guys get
> * nervous if it takes 17 minutes to set the clock for
> * the first time.
> *
>
> immediately followed by:
>
> step_systime(fp_offset);
>
> There is no step_systime in the else branch. clock_max is the tinkered
> value of the parameter that defaults to 128ms.[/color]

The above may mean something to YOU! Sorry but I don't see it!

David Woolley <david@ex.djwhome.demon.co.uk.invalid> writes:
[color=blue]
>Unruh wrote:[/color]
[color=blue][color=green]
>>
>> 128ms at 500PPM is 250sec or 4 min. It will take longer than that since the
>> rate will not peg out, but it should not be hours.[/color][/color]
[color=blue]
>It will follow the normal transient response, which has a first zero
>crossing which I believe is at about 39 minutes for phase (RFC 1305) and
>rather longer for frequency). I'm not sure, but it may well overshoot
>by more than 5ms, in which it could take rather longer to be acceptable
>to the OP.[/color]
[color=blue]
>It should get nowhere near being slew rate limited, so the 500ppm limit
>is academic.[/color]

[color=blue][color=green]
>> Maybe it is best to set the clock initiallly so that it is out by by more
>> than 128ms (Eg advance it by 10 sec) and then use -g.[/color][/color]
[color=blue]
>That would be my best trick, but if you have to use -g, you probably
>don't know the time well enough to be sure that adding 10 seconds won't
>actually put it in the 128ms band![/color]

It could but the chances are small if he has an rtc which is not completely
broken. If 10 sec is no good, use 10 years. That will make the chance of
being within 128ms of 10 years vanishngly small.

"David J Taylor" <david-taylor@blueyonder.neither-this-part.nor-this-bit.co.uk> writes:
[color=blue]
>Unruh wrote:
>[][color=green]
>> This is a design decision. And David will defend this "slow
>> convergence" "to the death". chrony is much faster, but does not do
>> refclocks at all so that is a useless option here.[/color][/color]
[color=blue]
>chrony is also useless on Windows.[/color]
[color=blue]
>David[/color]

Agreed. It is AFAIK only useful on Linux.
It is an example however of a system which has faster response than ntp,
and has better clock control than ntp, without, afaik sacrificing
stability or anything else.