Slow convergence of NTP with GPS/PPS

Richard B. Gilbert wrote:[color=blue]
> David Woolley wrote:[color=green]
>> Richard B. Gilbert wrote:
>>[color=darkred]
>>> To turn your equipment on after months of downtime and expect it to
>>> lock on to the correct time with millisecond accuracy within seconds
>>> is asking for a hell of a lot.[/color]
>>
>> Not really. He's starting a GPS receiver at the same time and that
>> has to lock to 50ns.
>>
>> Doing it on a general purpose computer is more difficult, but not
>> particularly impossible.[/color]
>
> Even with GPS and a full four satellite fix, ten seconds to synchronize
> is extremely ambitious!! You can set the time to within whatever[/color]

As I noted elsewhere, 10 seconds isn't possible for a GPS cold start,
but most of the excess time is spent in obtaining the ephemeris. He's
probably counting GPS startup from initial power up, but NTP start up
from when it first gets run.
[color=blue]
> precision the hardware and software support but that is only half the
> problem. You also need to set the correct clock frequency. On a cold
> start, the clock frequency is a moving target as the hardware warms up.[/color]

By polling sufficiently fast you can get good time accuracy, even if
frequency takes a bit longer.
[color=blue]
>
> I would expect to wait at least thirty minutes for the system to
> stabilize with both the correct phase (time) and frequency.[/color]

[color=blue]
>It is more like 20m. And even for 2000km (Vancouver to Regina) on the
>public CanadaNet network it is only 40ms.[/color]

The time-of-flight (speed of light) delay doesn't matter (much).
It's mostly a constant. (Routing shifts on long paths introduce
shifts.)

The problem is queueing delays. They aren't stable.

--
These are my opinions, not necessarily my employer's. I hate spam.

Richard B. Gilbert schrieb:[color=blue]
> David Woolley wrote:[color=green]
>> Richard B. Gilbert wrote:
>>[color=darkred]
>>> To turn your equipment on after months of downtime and expect it to
>>> lock on to the correct time with millisecond accuracy within seconds
>>> is asking for a hell of a lot.[/color]
>> Not really. He's starting a GPS receiver at the same time and that has
>> to lock to 50ns.
>>
>> Doing it on a general purpose computer is more difficult, but not
>> particularly impossible.[/color]
>
> Even with GPS and a full four satellite fix, ten seconds to synchronize
> is extremely ambitious!! You can set the time to within whatever
> precision the hardware and software support but that is only half the
> problem. You also need to set the correct clock frequency. On a cold
> start, the clock frequency is a moving target as the hardware warms up.
>
> I would expect to wait at least thirty minutes for the system to
> stabilize with both the correct phase (time) and frequency.[/color]

To transfer the full almanac of GPS it takes roughly 12 minutes from a
cold start. Then the receiver knows everything there is for it to know.
Some receivers (like mine) you can tell it's location, wich gets you in
the 10 s range for precise time. Then again, who claimed, it has to be
10 s? I would be very happy with these 12 mins..

[color=blue]
>It isn't NTP which is the limit, but the GPS receiver acquiring lock from
>scratch after an indeterminate period of being switched off. The GPS
>could take minutes to lock and declare that it has valid time.[/color]

From the spec sheet for the Garmin GPS 18x:

Reacquisition: Less than 2 seconds
Hot: Approx. 1 second (all data known)
Warm: Approx. 38 seconds (initial position, time, and
almanac known; ephemeris unknown)
Cold: Approx. 45 seconds

I believe that's typical of modern GPS receivers.

--
These are my opinions, not necessarily my employer's. I hate spam.

Unruh schrieb:[color=blue]
> "Richard B. Gilbert" <rgilbert88@comcast.net> writes:
>[color=green]
>> David Woolley wrote:[color=darkred]
>>> Richard B. Gilbert wrote:
>>>
>>>> To turn your equipment on after months of downtime and expect it to
>>>> lock on to the correct time with millisecond accuracy within seconds
>>>> is asking for a hell of a lot.
>>> Not really. He's starting a GPS receiver at the same time and that has
>>> to lock to 50ns.
>>>
>>> Doing it on a general purpose computer is more difficult, but not
>>> particularly impossible.[/color][/color]
>[color=green]
>> Even with GPS and a full four satellite fix, ten seconds to synchronize
>> is extremely ambitious!! You can set the time to within whatever
>> precision the hardware and software support but that is only half the
>> problem. You also need to set the correct clock frequency. On a cold
>> start, the clock frequency is a moving target as the hardware warms up.[/color]
>
> With a minpoll of 4, just setting the phase correctly with zero drift
> compensation would at worst be out by 1ms by the next reading. And you can
> get a pretty good estimate of the drift, even if it is changing. The temp
> coefficient is not 10PPM/degree C. More like 1 or less. That means the
> first few measurements gives a pretty good estimate of the drift ( ie to a
> few PPM) and then the finer corrections can come while things settle down.
>
>
>
>[color=green]
>> I would expect to wait at least thirty minutes for the system to
>> stabilize with both the correct phase (time) and frequency.[/color][/color]

So i could do some more tests: I could not reproduce the strange running
off for 200 ms and more once it was gone! The thread-starter had right
the same issue and gave up on ntp after all.. Any ideas on this, anyone?

So now things look somewhat better, if I boot the machine without a
driftfile, (300 and something in there! ...) it runs away for a little
while, but not very far, some ten ms i recall. If let run then and given
the chance to write that driftfile, I can reboot and it sets time with
an offset of around 20-30 ms and from there on slowly tears it to zero.
So the good news is, the heavy drifting is in control! What looks
unneccesary is the initial offset..

Btw, I have minpoll 4 maxpoll 4 in my ntp.conf and ntpq -p says: "poll
16". Is it the poll of ntpq -p or of ntpd?

Best regards,
.../nico

Hal Murray wrote:[color=blue][color=green]
>> It isn't NTP which is the limit, but the GPS receiver acquiring lock
>> from scratch after an indeterminate period of being switched off.
>> The GPS could take minutes to lock and declare that it has valid
>> time.[/color]
>
> From the spec sheet for the Garmin GPS 18x:
>
> Reacquisition: Less than 2 seconds
> Hot: Approx. 1 second (all data known)
> Warm: Approx. 38 seconds (initial position, time, and
> almanac known; ephemeris unknown)
> Cold: Approx. 45 seconds
>
> I believe that's typical of modern GPS receivers.[/color]

Hal, thanks for those figures.

My own experience suggests that, at least with a hand-held GPS, it can be
a lot longer than 45 seconds. That figure may only apply if the GPS has a
180-degree clear view of the sky. But the GPS18 LVC does usually recover
quickly enough (mine has a less than 180-degree sky FoV).

If we accept 45s, is that shorter than the typical system boot time from
cold? Could the system boot be delayed enough so that the GPS was
guaranteed to be ready by the time it was needed?

Cheers,
David

Nicola Berndt wrote:[color=blue]
> Btw, I have minpoll 4 maxpoll 4 in my ntp.conf and ntpq -p says:
> "poll 16". Is it the poll of ntpq -p or of ntpd?
>
> Best regards,
> ../nico[/color]

Hello,
ntpq -p shows the time in seconds between two polls (i.e. 16). In the
configuration file the poll interval is noted in 2^x . This means your
entry of minpoll 4 maxpoll 4 means 2^4 seconds minpoll 2^4 seconds
maxpoll . So the display of 16 seconds as result of ntpq -p is correct.
Hope this helped,
Stefan Nottorf

David J Taylor wrote:[color=blue]
> Unruh wrote:
> [][color=green]
>> With a minpoll of 4, just setting the phase correctly with zero drift
>> compensation would at worst be out by 1ms by the next reading. And
>> you can get a pretty good estimate of the drift, even if it is
>> changing. The temp coefficient is not 10PPM/degree C. More like 1 or
>> less. That means the first few measurements gives a pretty good
>> estimate of the drift ( ie to a few PPM) and then the finer
>> corrections can come while things settle down.[/color]
>
> It isn't NTP which is the limit, but the GPS receiver acquiring lock from
> scratch after an indeterminate period of being switched off. The GPS
> could take minutes to lock and declare that it has valid time.
>
> David
>
>[/color]

And ntpd could take many minutes to bludgeon the local clock into
submission! It's easy to determine and set the correct time. It's less
easy to determine and set the correct frequency and thus KEEP the
correct time.

Nicola Berndt wrote:[color=blue]
> Unruh schrieb:[color=green]
>> "Richard B. Gilbert" <rgilbert88@comcast.net> writes:
>>[color=darkred]
>>> David Woolley wrote:
>>>> Richard B. Gilbert wrote:
>>>>
>>>>> To turn your equipment on after months of downtime and expect it to
>>>>> lock on to the correct time with millisecond accuracy within seconds
>>>>> is asking for a hell of a lot.
>>>> Not really. He's starting a GPS receiver at the same time and that has[/color][/color][/color]
<snip>[color=blue]
> Btw, I have minpoll 4 maxpoll 4 in my ntp.conf and ntpq -p says: "poll
> 16". Is it the poll of ntpq -p or of ntpd?
>[/color]

It's the poll interval of ntpd. Ntpq does not poll! The poll interval
varies between 2^MINPOLL and 2^MAXPOLL. You have set MINPOLL=MAXPOLL=4
giving a poll interval of 2^4 or 16 seconds. This is usually the
correct choice for a GPS receiver. For getting time over the internet
it would, under most circumstances, be a horrible choice!

Richard B. Gilbert wrote:
[][color=blue]
> And ntpd could take many minutes to bludgeon the local clock into
> submission! It's easy to determine and set the correct time. It's
> less easy to determine and set the correct frequency and thus KEEP the
> correct time.[/color]

Wasn't the OP looking for about 0.5s accuracy? Ah, no, that was someone
else. Here we're aiming for:

"Our application requires good time accuracy (better than 5ms) but it
also needs to get there quickly (as quickly as possible, but ideally
taking no more than about 15 minutes)."

I would have thought that a GPS and NTP would have been able to achieve
that, at least with a current drift file.

Cheers,
David

David J Taylor wrote:[color=blue]
> Richard B. Gilbert wrote:
> [][color=green]
>> And ntpd could take many minutes to bludgeon the local clock into
>> submission! It's easy to determine and set the correct time. It's
>> less easy to determine and set the correct frequency and thus KEEP the
>> correct time.[/color]
>
> Wasn't the OP looking for about 0.5s accuracy? Ah, no, that was someone
> else. Here we're aiming for:
>
> "Our application requires good time accuracy (better than 5ms) but it
> also needs to get there quickly (as quickly as possible, but ideally
> taking no more than about 15 minutes)."
>
> I would have thought that a GPS and NTP would have been able to achieve
> that, at least with a current drift file.
>
> Cheers,
> David
>
>[/color]

Try it some time! Ntpd makes a mad dash for zero offset, overshoots,
and then "rings" for a while. Eventually it gets that tight synch that
we like to see but it does take about thirty minutes.

I think I mentioned this here at the time I observed it. I believe that
I also remarked that I could have done it a lot faster by hand if I had
the "control knobs".

Richard B. Gilbert wrote:
[][color=blue]
> Try it some time! Ntpd makes a mad dash for zero offset, overshoots,
> and then "rings" for a while. Eventually it gets that tight synch
> that we like to see but it does take about thirty minutes.
>
> I think I mentioned this here at the time I observed it. I believe
> that I also remarked that I could have done it a lot faster by hand
> if I had the "control knobs".[/color]

Looking at the behaviour of client PCs synching to a GPS-stratum-1 server,
what you say surprises me somewhat. The initial transient seems to die
out very rapidly, judged on a "50ms" scale. That's with a Windows client
as well. As the stratum-1 server has a much better reference, and as the
poll is fixed at 64s, I would have thought that 5ms was no problem at all.
Just feed the PPS signal to all the clients.

Oh, well.

Thanks,
David

[email]hal-usenet@ip-64-139-1-69.sjc.megapath.net[/email] (Hal Murray) writes:

[color=blue][color=green]
>>It is more like 20m. And even for 2000km (Vancouver to Regina) on the
>>public CanadaNet network it is only 40ms.[/color][/color]
[color=blue]
>The time-of-flight (speed of light) delay doesn't matter (much).
>It's mostly a constant. (Routing shifts on long paths introduce
>shifts.)[/color]
[color=blue]
>The problem is queueing delays. They aren't stable.[/color]

Agreed. Which is also why I find it amazing that on my gps controlled
server, with a Regina level 1 backup, the regina offset is less than 1ms
even though the round trip time is 40-50 ms. Ie, assymmetric router delays
do NOT appear to be a problem.
Just one data point however..

[email]nb@komeda-berlin.de[/email] (Nicola Berndt) writes:
[color=blue]
>Richard B. Gilbert schrieb:[color=green]
>> David Woolley wrote:[color=darkred]
>>> Richard B. Gilbert wrote:
>>>
>>>> To turn your equipment on after months of downtime and expect it to
>>>> lock on to the correct time with millisecond accuracy within seconds
>>>> is asking for a hell of a lot.
>>> Not really. He's starting a GPS receiver at the same time and that has
>>> to lock to 50ns.
>>>
>>> Doing it on a general purpose computer is more difficult, but not
>>> particularly impossible.[/color]
>>
>> Even with GPS and a full four satellite fix, ten seconds to synchronize
>> is extremely ambitious!! You can set the time to within whatever
>> precision the hardware and software support but that is only half the
>> problem. You also need to set the correct clock frequency. On a cold
>> start, the clock frequency is a moving target as the hardware warms up.
>>
>> I would expect to wait at least thirty minutes for the system to
>> stabilize with both the correct phase (time) and frequency.[/color][/color]
[color=blue]
>To transfer the full almanac of GPS it takes roughly 12 minutes from a
>cold start. Then the receiver knows everything there is for it to know.
>Some receivers (like mine) you can tell it's location, wich gets you in
>the 10 s range for precise time. Then again, who claimed, it has to be
>10 s? I would be very happy with these 12 mins..[/color]

For some receivers if they know their position, they can get the time
virutally instantly from "cold start". All you need is one sattelite. If the receiver has no idea where it is, it can take much longer.
Whether or not the receiver the OP has has that
capability I do not know.

[color=blue]
>Btw, I have minpoll 4 maxpoll 4 in my ntp.conf and ntpq -p says: "poll
>16". Is it the poll of ntpq -p or of ntpd?[/color]

The poll time is e^p where p is the poll number. 2^4=16 sec.

[color=blue]
>Best regards,
>../nico[/color]

"Richard B. Gilbert" <rgilbert88@comcast.net> writes:
[color=blue]
>David J Taylor wrote:[color=green]
>> Unruh wrote:
>> [][color=darkred]
>>> With a minpoll of 4, just setting the phase correctly with zero drift
>>> compensation would at worst be out by 1ms by the next reading. And
>>> you can get a pretty good estimate of the drift, even if it is
>>> changing. The temp coefficient is not 10PPM/degree C. More like 1 or
>>> less. That means the first few measurements gives a pretty good
>>> estimate of the drift ( ie to a few PPM) and then the finer
>>> corrections can come while things settle down.[/color]
>>
>> It isn't NTP which is the limit, but the GPS receiver acquiring lock from
>> scratch after an indeterminate period of being switched off. The GPS
>> could take minutes to lock and declare that it has valid time.
>>
>> David
>>
>>[/color][/color]
[color=blue]
>And ntpd could take many minutes to bludgeon the local clock into
>submission! It's easy to determine and set the correct time. It's less
>easy to determine and set the correct frequency and thus KEEP the
>correct time.[/color]

Actually it is relatively easy to determine the frequency as well in a
short time. and then refine it. It is NOT easy if you use a Markovian strategy
like ntp uses.

"Richard B. Gilbert" <rgilbert88@comcast.net> writes:
[color=blue]
>David J Taylor wrote:[color=green]
>> Richard B. Gilbert wrote:
>> [][color=darkred]
>>> And ntpd could take many minutes to bludgeon the local clock into
>>> submission! It's easy to determine and set the correct time. It's
>>> less easy to determine and set the correct frequency and thus KEEP the
>>> correct time.[/color]
>>
>> Wasn't the OP looking for about 0.5s accuracy? Ah, no, that was someone
>> else. Here we're aiming for:
>>
>> "Our application requires good time accuracy (better than 5ms) but it
>> also needs to get there quickly (as quickly as possible, but ideally
>> taking no more than about 15 minutes)."
>>
>> I would have thought that a GPS and NTP would have been able to achieve
>> that, at least with a current drift file.
>>
>> Cheers,
>> David
>>
>>[/color][/color]
[color=blue]
>Try it some time! Ntpd makes a mad dash for zero offset, overshoots,
>and then "rings" for a while. Eventually it gets that tight synch that
>we like to see but it does take about thirty minutes.[/color]
[color=blue]
>I think I mentioned this here at the time I observed it. I believe that
>I also remarked that I could have done it a lot faster by hand if I had
>the "control knobs".[/color]

a) The clock filter means that 7/8 of the data points get tossed. That
means that it is 128 sec between data points. Ntp must have a damping time
longer than that. and is about twice that. Ie, the error is reduced by
approximately 1/e in that damping time. Ie, if the original error was 30
ms, it will be about 12mx after 4 min, 5 after 8 min,.... but that is for
offset errors. For frequency errors it is worse. First it has to wait for
the frequency actually to create an offset error, and then start to fix
it so frequency offsets take even longer to damp out.

b) Yes, you might have been able to do it faster, but it is unclear.
Remember what ntpo does is correct errors only by changing the frequency of
the clock. Ie, the only knob you are allowed to twidle is the frequency
knob. That is tougher. (It is like driving-- If you find yourself driving
on the wrong side of the road, all you can change is the direction the car
is driving not its position. It is really really easy to overshoot and find
yourself in the ditch. ntp is designed NOT to land in the ditch, ever. That
means it is conservative in its stearing.

[color=blue]
>It's the poll interval of ntpd. Ntpq does not poll! The poll interval
>varies between 2^MINPOLL and 2^MAXPOLL. You have set MINPOLL=MAXPOLL=4
>giving a poll interval of 2^4 or 16 seconds. This is usually the
>correct choice for a GPS receiver.[/color]

Why do you say that? Or let me ask it another way, how would you
decide what the right polling interval is?

--
These are my opinions, not necessarily my employer's. I hate spam.

[color=blue]
>My own experience suggests that, at least with a hand-held GPS, it can be
>a lot longer than 45 seconds. That figure may only apply if the GPS has a
>180-degree clear view of the sky. But the GPS18 LVC does usually recover
>quickly enough (mine has a less than 180-degree sky FoV).[/color]

The 45 second number if probably marketing. I'm not sure
how to translate that into reality.

I wouldn't be surprised if older units took a lot longer.

[color=blue]
>If we accept 45s, is that shorter than the typical system boot time from
>cold? Could the system boot be delayed enough so that the GPS was
>guaranteed to be ready by the time it was needed?[/color]

With a bit of work, you can boot in much less that 45 seconds.

--
These are my opinions, not necessarily my employer's. I hate spam.

[color=blue]
>Agreed. Which is also why I find it amazing that on my gps controlled
>server, with a Regina level 1 backup, the regina offset is less than 1ms
>even though the round trip time is 40-50 ms. Ie, assymmetric router delays
>do NOT appear to be a problem.
>Just one data point however..[/color]

Look at your statistics while you download a huge file, for example
a CD.

My DSL line has 100 ms of queueing delays.

--
These are my opinions, not necessarily my employer's. I hate spam.