Slow convergence of NTP with GPS/PPS

Hal Murray schrieb:[color=blue]
>[color=green]
>> A termistor on the crystal on the other hand might be useful to compensate
>> the temperature ( there is an alteration of ntp which also calculates the
>> temp compensation of the crystal and uses that to calculate the required
>> drift rate.-- unfortunately I do not remember its name of location on the
>> web)[/color]
>
> NTP temperature compensation
> Mark Martinec, 2001-01-08
> [url]http://www.ijs.si/time/temp-compensation/[/url]
>
> A wonderful read.
>[/color]
Really nice one, thx for the link!

Also thx to everyone thinking about a solution.

A good idea might be to find someone who would program GPS/PPS support
for chrony. Many people would benefit from it. We also have a good
programmer working with us and he is already looking into things..

I also have to further investigate the sudden change of behaviour I
experienced and if I found the reason I will have to try to start the
machine in a very cold room, on the heating and so on.. If it would
settle down within 15 Minutes, we could live with it...

Regards,
.../nico

David Woolley wrote:[color=blue]
> Richard B. Gilbert wrote:
>[color=green]
>> The clock in a PC is basically the guts of a cheap "Quartz" watch. It
>> wouldn't surprise me if the manufacturers bought the crystals rejected
>> by the watch makers. I suspect that the clock exists MOSTLY so the
>> machine will have the correct date for things like letters and checks.[/color]
>
> That describes the RTC, and may not even be valid for HPET systems. The
> clock that ntpd disciplines is not based on a 32kHz watch crystal, but
> on a much higher frequency crystal. Historically, the primary purpose
> of the latter crystal is to provide a logic clock for the processor and
> memory, not for time keeping.[/color]

And it probably varies in frequency with temperature and age. And
probably no one cares if the frequency is off by a percent or two,
especially if it's off on the high side. Who is going to complain if
his 2.4 GHz processor is actually operating at 2.45 GHZ??

[color=blue]
>And it probably varies in frequency with temperature and age. And
>probably no one cares if the frequency is off by a percent or two,
>especially if it's off on the high side. Who is going to complain if
>his 2.4 GHz processor is actually operating at 2.45 GHZ??[/color]

Actually, it's probably low.

If it was fast, you would be overclocking, maybe not by much, but
there is no longer any guarantee that your CPU will work.
[If it would work reliabily at x% faster, all the manufacturers
would bump things up a bit.]

Another reason it's probably slow is that almost everybody uses
spread sprectum clocking to reduce EMI, or rather to get past
the FCC regulation. It doesn't reduce it, just spreads it around.
That's typically in the 1/2 % range, and it's slower to make sure
they don't break things by going faster.

--
These are my opinions, not necessarily my employer's. I hate spam.

Hal Murray <hal-usenet@ip-64-139-1-69.sjc.megapath.net> wrote:[color=blue]
> Actually, it's probably low.[/color]
[color=blue]
> If it was fast, you would be overclocking, maybe not by much, but
> there is no longer any guarantee that your CPU will work. [If it
> would work reliabily at x% faster, all the manufacturers would bump
> things up a bit.][/color]

Maybe... if they could also bump-up the price a bit. :) And then there
is binning...

rick jones
--
denial, anger, bargaining, depression, acceptance, rebirth...
where do you want to be today?
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

[email]nb@komeda-berlin.de[/email] (Nicola Berndt) writes:
[color=blue]
>Hal Murray schrieb:[color=green]
>>[color=darkred]
>>> A termistor on the crystal on the other hand might be useful to compensate
>>> the temperature ( there is an alteration of ntp which also calculates the
>>> temp compensation of the crystal and uses that to calculate the required
>>> drift rate.-- unfortunately I do not remember its name of location on the
>>> web)[/color]
>>
>> NTP temperature compensation
>> Mark Martinec, 2001-01-08
>> [url]http://www.ijs.si/time/temp-compensation/[/url]
>>
>> A wonderful read.
>>[/color]
>Really nice one, thx for the link![/color]
[color=blue]
>Also thx to everyone thinking about a solution.[/color]
[color=blue]
>A good idea might be to find someone who would program GPS/PPS support
>for chrony. Many people would benefit from it. We also have a good
>programmer working with us and he is already looking into things..[/color]

I keep thinking about it, but I am not a good programmer, and first one has
to understand chrony well enough to start hacking it. What would be really
nice is to install a glue layer so one could simply take the ntp refclock
files and compile them into chrony. Unfortunately the ntp people did not
isolate the refclock behaviour terribly well, but it should be relatively
easy for a good programmer to write such glue ( chrony also has a
reasonably well issolated glue to the server querying stuff)


[color=blue]
>I also have to further investigate the sudden change of behaviour I
>experienced and if I found the reason I will have to try to start the
>machine in a very cold room, on the heating and so on.. If it would
>settle down within 15 Minutes, we could live with it...[/color]
[color=blue]
>Regards,
>../nico[/color]

"Richard B. Gilbert" <rgilbert88@comcast.net> writes:
[color=blue]
>David Woolley wrote:[color=green]
>> Richard B. Gilbert wrote:
>>[color=darkred]
>>> The clock in a PC is basically the guts of a cheap "Quartz" watch. It
>>> wouldn't surprise me if the manufacturers bought the crystals rejected
>>> by the watch makers. I suspect that the clock exists MOSTLY so the
>>> machine will have the correct date for things like letters and checks.[/color]
>>
>> That describes the RTC, and may not even be valid for HPET systems. The
>> clock that ntpd disciplines is not based on a 32kHz watch crystal, but
>> on a much higher frequency crystal. Historically, the primary purpose
>> of the latter crystal is to provide a logic clock for the processor and
>> memory, not for time keeping.[/color][/color]
[color=blue]
>And it probably varies in frequency with temperature and age. And
>probably no one cares if the frequency is off by a percent or two,
>especially if it's off on the high side. Who is going to complain if
>his 2.4 GHz processor is actually operating at 2.45 GHZ??[/color]

And from experiment it is actually off by less than .01%.(100PPM) Most
commercial computers are that good. In fact if they are off by .05% ntp is
useless and will refuse to even try to discipline it.

Hal Murray <hal-usenet@ip-64-139-1-69.sjc.megapath.net> wrote:
[color=blue]
> NTP temperature compensation
> Mark Martinec, 2001-01-08
> [url]http://www.ijs.si/time/temp-compensation/[/url][/color]
[color=blue]
> A wonderful read.[/color]

I wonder how closely Mark's results could be replicated using some
(set of) temperature readings from components already in the system
rather than adding another temp probe? Stuff like CPU temps and other
intra-system components. I'm not sure they have nearly the same
accuracy and resolution though :(

rick jones
--
No need to believe in either side, or any side. There is no cause.
There's only yourself. The belief is in your own precision. - Jobert
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

Unruh <unruh-spam@physics.ubc.ca> wrote:[color=blue]
> ( assuming that the network noise is at the 100us type level).[/color]

That feels like a rather large assumption given the target environment
does not seem to allow the system to be synced to be up long-term.

rick jones
--
Wisdom Teeth are impacted, people are affected by the effects of events.
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

Rick Jones <rick.jones2@hp.com> writes:
[color=blue]
>Hal Murray <hal-usenet@ip-64-139-1-69.sjc.megapath.net> wrote:[/color]
[color=blue][color=green]
>> NTP temperature compensation
>> Mark Martinec, 2001-01-08
>> [url]http://www.ijs.si/time/temp-compensation/[/url][/color][/color]
[color=blue][color=green]
>> A wonderful read.[/color][/color]
[color=blue]
>I wonder how closely Mark's results could be replicated using some
>(set of) temperature readings from components already in the system
>rather than adding another temp probe? Stuff like CPU temps and other
>intra-system components. I'm not sure they have nearly the same
>accuracy and resolution though :([/color]


The problem is not accuracy and resolution, the problem is time delay. If
the cpu heats up, it will take a while for the clock crystal to heat up and
will not heat up as much. Ie, the cpu could jump to 70C briefly while some
calculation was being done, while the clock crystal heated up almost
nothing. It is probably better than nothing but if the computer has a very
variable useage, so the temp fluctuates a lot, this will be much noisier
than directly measuring the temp of the crystal. If the computer is pretty
stable, it should be fine to use other proxies.

[color=blue]
>I wonder how closely Mark's results could be replicated using some
>(set of) temperature readings from components already in the system
>rather than adding another temp probe? Stuff like CPU temps and other
>intra-system components. I'm not sure they have nearly the same
>accuracy and resolution though :([/color]

One system I have reads temperature in quanta of 1C.
(There might be ways to get better. I haven't pushed all that hard.)
That's not very good on this scale.

I played around in this area a while ago. I didn't get good results
until I glued the temperature probe to the xtal. That one reads
to 0.1F,

--
These are my opinions, not necessarily my employer's. I hate spam.

Rick Jones <rick.jones2@hp.com> writes:
[color=blue]
>Unruh <unruh-spam@physics.ubc.ca> wrote:[color=green]
>> ( assuming that the network noise is at the 100us type level).[/color][/color]
[color=blue]
>That feels like a rather large assumption given the target environment
>does not seem to allow the system to be synced to be up long-term.[/color]

Of course it might be. However he also talks about atomic clocks and gps
and wanting quick ms accuracy, which would only be possible on a fast
network connection, etc. Since we have absolutely no idea what his setup
is, I make my assumptions explicit. It may be that he has Mills's "slow
network to Malasia" where any network packet takes an hour to traverse the
net, but in that case even he might recognize that what he wants is
impossible.

[color=blue][color=green]
>>A good idea might be to find someone who would program GPS/PPS support
>>for chrony. Many people would benefit from it. We also have a good
>>programmer working with us and he is already looking into things..[/color]
>
>I keep thinking about it, but I am not a good programmer, and first one has
>to understand chrony well enough to start hacking it. What would be really
>nice is to install a glue layer so one could simply take the ntp refclock
>files and compile them into chrony. Unfortunately the ntp people did not
>isolate the refclock behaviour terribly well, but it should be relatively
>easy for a good programmer to write such glue ( chrony also has a
>reasonably well issolated glue to the server querying stuff)[/color]

Look at the shared memory stuff.

gpsd supports many GPS devices.

--
These are my opinions, not necessarily my employer's. I hate spam.

Hal Murray <hal-usenet@ip-64-139-1-69.sjc.megapath.net> wrote:
[color=blue][color=green]
> >I wonder how closely Mark's results could be replicated using some
> >(set of) temperature readings from components already in the system
> >rather than adding another temp probe? Stuff like CPU temps and other
> >intra-system components. I'm not sure they have nearly the same
> >accuracy and resolution though :([/color][/color]
[color=blue]
> One system I have reads temperature in quanta of 1C.
> (There might be ways to get better. I haven't pushed all that hard.)
> That's not very good on this scale.[/color]
[color=blue]
> I played around in this area a while ago. I didn't get good results
> until I glued the temperature probe to the xtal. That one reads
> to 0.1F,[/color]

Sigh. I was hoping there might be a middle ground using stuff already
present in the system.

rick jones
--
The computing industry isn't as much a game of "Follow The Leader" as
it is one of "Ring Around the Rosy" or perhaps "Duck Duck Goose."
- Rick Jones
these opinions are mine, all mine; HP might not want them anyway... :)
feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

Richard B. Gilbert wrote:
[color=blue]
> To turn your equipment on after months of downtime and expect it to lock
> on to the correct time with millisecond accuracy within seconds is
> asking for a hell of a lot.[/color]

Not really. He's starting a GPS receiver at the same time and that has
to lock to 50ns.

Doing it on a general purpose computer is more difficult, but not
particularly impossible.

Unfortunately, the GPS receiver I have only has one channel, so I'm not
sure how fast a parallel receiver can lock when it no longer has valid
ephemeris data.

Unruh wrote:
[color=blue]
>
> Assuming the network noise is the 100us level, (which it is for example
> between me and Regina 3000km away) you should get the accuracy easily to
> 1ms in 1 sec. if all you want is the phase error. One packet exchange will
> give it to you.[/color]

You have an unused network. For about 20km between work and ISP it is
more like 100ms peak to peak. (2Mb/s 1:1)

[color=blue]
> receiver would be better, especially if it know its location. Then within
> seconds it would know the time to nanoseconds. If it has to diiscover its[/color]

It's going to take at least 30 seconds to do that, as it will need the
detailed emphemeris data and that takes 30 seconds to transmit (it might
well pick the strongest signal and try to read the coarse ephemeris data
first; I'm not sure if that fits in the 30 seconds.

Unruh wrote:
[color=blue]
> Of course it might be. However he also talks about atomic clocks and gps[/color]

The marketing hype is such in this area that most people who use the
term mean a radio clock (including GPS), rather than a true, physically
local, atomic clock. It's particularly used for VLF clocks, using the 1
baud slow time code (e.g. MSF, without the fine time signal). Such VLF
clocks are not normally even corrected for light travel time.

NTP users are more likely to mean GPS based devices.

David Woolley <david@ex.djwhome.demon.co.uk.invalid> writes:
[color=blue]
>Unruh wrote:[/color]
[color=blue][color=green]
>>
>> Assuming the network noise is the 100us level, (which it is for example
>> between me and Regina 3000km away) you should get the accuracy easily to
>> 1ms in 1 sec. if all you want is the phase error. One packet exchange will
>> give it to you.[/color][/color]
[color=blue]
>You have an unused network. For about 20km between work and ISP it is
>more like 100ms peak to peak. (2Mb/s 1:1)[/color]

It is more like 20m. And even for 2000km (Vancouver to Regina) on the
public CanadaNet network it is only 40ms.


[color=blue][color=green]
>> receiver would be better, especially if it know its location. Then within
>> seconds it would know the time to nanoseconds. If it has to diiscover its[/color][/color]
[color=blue]
>It's going to take at least 30 seconds to do that, as it will need the
>detailed emphemeris data and that takes 30 seconds to transmit (it might
>well pick the strongest signal and try to read the coarse ephemeris data
>first; I'm not sure if that fits in the 30 seconds.[/color]

David Woolley wrote:[color=blue]
> Richard B. Gilbert wrote:
>[color=green]
>> To turn your equipment on after months of downtime and expect it to
>> lock on to the correct time with millisecond accuracy within seconds
>> is asking for a hell of a lot.[/color]
>
> Not really. He's starting a GPS receiver at the same time and that has
> to lock to 50ns.
>
> Doing it on a general purpose computer is more difficult, but not
> particularly impossible.[/color]

Even with GPS and a full four satellite fix, ten seconds to synchronize
is extremely ambitious!! You can set the time to within whatever
precision the hardware and software support but that is only half the
problem. You also need to set the correct clock frequency. On a cold
start, the clock frequency is a moving target as the hardware warms up.

I would expect to wait at least thirty minutes for the system to
stabilize with both the correct phase (time) and frequency.

"Richard B. Gilbert" <rgilbert88@comcast.net> writes:
[color=blue]
>David Woolley wrote:[color=green]
>> Richard B. Gilbert wrote:
>>[color=darkred]
>>> To turn your equipment on after months of downtime and expect it to
>>> lock on to the correct time with millisecond accuracy within seconds
>>> is asking for a hell of a lot.[/color]
>>
>> Not really. He's starting a GPS receiver at the same time and that has
>> to lock to 50ns.
>>
>> Doing it on a general purpose computer is more difficult, but not
>> particularly impossible.[/color][/color]
[color=blue]
>Even with GPS and a full four satellite fix, ten seconds to synchronize
>is extremely ambitious!! You can set the time to within whatever
>precision the hardware and software support but that is only half the
>problem. You also need to set the correct clock frequency. On a cold
>start, the clock frequency is a moving target as the hardware warms up.[/color]

With a minpoll of 4, just setting the phase correctly with zero drift
compensation would at worst be out by 1ms by the next reading. And you can
get a pretty good estimate of the drift, even if it is changing. The temp
coefficient is not 10PPM/degree C. More like 1 or less. That means the
first few measurements gives a pretty good estimate of the drift ( ie to a
few PPM) and then the finer corrections can come while things settle down.



[color=blue]
>I would expect to wait at least thirty minutes for the system to
>stabilize with both the correct phase (time) and frequency.[/color]

Unruh wrote:
[][color=blue]
> With a minpoll of 4, just setting the phase correctly with zero drift
> compensation would at worst be out by 1ms by the next reading. And
> you can get a pretty good estimate of the drift, even if it is
> changing. The temp coefficient is not 10PPM/degree C. More like 1 or
> less. That means the first few measurements gives a pretty good
> estimate of the drift ( ie to a few PPM) and then the finer
> corrections can come while things settle down.[/color]

It isn't NTP which is the limit, but the GPS receiver acquiring lock from
scratch after an indeterminate period of being switched off. The GPS
could take minutes to lock and declare that it has valid time.

David