In article ,
shy author writes:
>Has anyone thought about removing both the linear terms and quadratic
>terms in the drift, by utilizing the temperature sensor readings
>available on many of the latest motherboards?

Linear works pretty well. I think it would be hard to get quadratic.

NTP temperature compensation
Mark Martinec, 2001-01-08
http://www.ijs.si/time/temp-compensation/


>Just a thought, but it seems a shame that we're not taking advantage of
>the thermal data available to us, when correcting for clock drift and
>simply using a linear correction.

The temperature sensor is probably in the CPU chip. You probably
want it on the crystal. The sensors on CPUs tend to be coarse.

--
These are my opinions, not necessarily my employer's. I hate spam.

shy author writes:

>Has anyone thought about removing both the linear terms and quadratic
>terms in the drift, by utilizing the temperature sensor readings
>available on many of the latest motherboards?

>Crystal oscillators tend to have both a linear bias and a quadratic bias,
>determinate upon temperature, leaving a stochastic component, which I
>believe we're trying to compensate for all along by the operation of ntp.

>Integration of the temperature readings and knowledge of the current
>drift should suggest the appropriate coefficients for the corrections.

>Just a thought, but it seems a shame that we're not taking advantage of
>the thermal data available to us, when correcting for clock drift and
>simply using a linear correction.

A Quadratic correction tends to run away pretty badly if you suddenly have
a period of no ntp readings. But I am not at all sure that is what you
mean. Rather you seem to mean that one use the temp to estimate what the
linear drift is ( from the past readings of temp and drift.)

> A Quadratic correction tends to run away pretty badly if you suddenly
> have a period of no ntp readings. But I am not at all sure that is what
> you mean. Rather you seem to mean that one use the temp to estimate what
> the linear drift is ( from the past readings of temp and drift.)

Correct, in that the linear correction will vary with temperature (and
could be matched closely by a quadratic)

I am thinking of a least-squares fit to the drift data, versus
temperature. Another poster mentioned that we really need temperature of
the crystal, I agree, but there should be strong correlation between that
(unknown) temperature and the motherboard or processor chip temperatures.

shy author writes:

>> A Quadratic correction tends to run away pretty badly if you suddenly
>> have a period of no ntp readings. But I am not at all sure that is what
>> you mean. Rather you seem to mean that one use the temp to estimate what
>> the linear drift is ( from the past readings of temp and drift.)

>Correct, in that the linear correction will vary with temperature (and
>could be matched closely by a quadratic)

That is NOT a quadratic correction. It is a linear correction where the
linear coefficient depends on temperature.

>I am thinking of a least-squares fit to the drift data, versus
>temperature. Another poster mentioned that we really need temperature of
>the crystal, I agree, but there should be strong correlation between that
>(unknown) temperature and the motherboard or processor chip temperatures.

And a lag.

>Correct, in that the linear correction will vary with temperature (and
>could be matched closely by a quadratic)

I don't think you can measure the temperature and/or drift accurately
enough to make quadratic corrections interesting.


>I am thinking of a least-squares fit to the drift data, versus
>temperature. Another poster mentioned that we really need temperature of
>the crystal, I agree, but there should be strong correlation between that
>(unknown) temperature and the motherboard or processor chip temperatures.

Years ago, I tried to measure things.

Most PCs have 2 xtals. One at 14.xxx MHz (cheap, 4X color burst)
that drives the CPU and most motherboard logic through a magic clock
generator (PLL) chip, and another that is a 32 KHz watch crystal for
keeping time when the CPU is off. The latter also makes interrupts
for the scheduler.

I had the temperature probe on the 14 MHz xtal. It didn't work very well.

I was assuming that the system used something like a cycle counter
(TSC) for timekeeping. That has troubles in multi-CPU systems.
The code I was actually running used the timer interrupts for
timekeeping and the TSC to interpolate between ticks.

Things got a lot better when I moved the probe to the 32 KHz xtal which
was in a corner of the board far from the CPU.

Here are a couple of graphs:
http://www.megapathdsl.net/~hmurray/ntp/drift-ex.gif
http://www.megapathdsl.net/~hmurray/ntp/drift.gif
http://www.megapathdsl.net/~hmurray/ntp/slope.gif
http://www.megapathdsl.net/~hmurray/ntp/slope2.gif

Again, read Mark Martinec's web page. He's got a lot of good
data.
http://www.ijs.si/time/temp-compensation/


--
These are my opinions, not necessarily my employer's. I hate spam.

shy author wrote:
>>A Quadratic correction tends to run away pretty badly if you suddenly
>>have a period of no ntp readings. But I am not at all sure that is what
>>you mean. Rather you seem to mean that one use the temp to estimate what
>>the linear drift is ( from the past readings of temp and drift.)
>
>
> Correct, in that the linear correction will vary with temperature (and
> could be matched closely by a quadratic)
>
> I am thinking of a least-squares fit to the drift data, versus
> temperature. Another poster mentioned that we really need temperature of
> the crystal, I agree, but there should be strong correlation between that
> (unknown) temperature and the motherboard or processor chip temperatures.
>
One could stick a LM75 Temp Sensor ( or similar with better resolution )
to the Xtal and connect it to the MotherBoard SM-Bus.
Should not be all that difficult to integrate.

uwe

Hal Murray wrote:
>
> Most PCs have 2 xtals. One at 14.xxx MHz (cheap, 4X color burst)
> that drives the CPU and most motherboard logic through a magic clock
> generator (PLL) chip, and another that is a 32 KHz watch crystal for
> keeping time when the CPU is off. The latter also makes interrupts
> for the scheduler.

Historically interrupts from the 32kHz clock have not been used, except,
possibly, in powered down states to initiate a restart from suspend or
hibernate. It is possible that has changed very recently, but they
certainly weren't used historically.

>
> I had the temperature probe on the 14 MHz xtal. It didn't work very well.
>
> I was assuming that the system used something like a cycle counter
> (TSC) for timekeeping. That has troubles in multi-CPU systems.

I think the most common way of doing timekeeping at the 1ms and higher
level is to use the counter timer, which is driven from a signal divided
down to approximately 1MHz. TSC is mainly used to interpolate between
ticks or to detect missed ticks.

> The code I was actually running used the timer interrupts for
> timekeeping and the TSC to interpolate between ticks.

but note that the timer interrupts are not based on the 32kHz
oscillator, in the typical system.

>Historically interrupts from the 32kHz clock have not been used, except,
>possibly, in powered down states to initiate a restart from suspend or
>hibernate. It is possible that has changed very recently, but they
>certainly weren't used historically.

Well, at least one of us is confused. Or maybe my history starts
long before yours.

I'm pretty sure that some of the systems I've worked on used
the interrupt from the 32 KHz clock chip to drive the scheduler.

Some/many systems have long had troubles keeping time if interrups
get lost. That wouldn't make sense if something like the TSC was
used for timekeeping.

Here is the message from Dave Mills that got me thinking in the
right direction:
Message-ID: <3CE6FB0C.C1A206@udel.edu>
http://groups.google.com/group/comp....d26948e56531d4

After that, I moved the temperature probe over to the 32 KHz crystal
and my temperature data looked much cleaner. The 32 KHz crystal is
off in a corner of the board. THe main CPU crystal is reasonably close
to the center of the board where all the heat is generated.

In the last year or two, the Linux timekeeping stuff has changed
a lot, partly in order to support laptops and such that go into
serious power down mode and don't want to waste a lot of battery
on each tick when there is nothing to do.

--
These are my opinions, not necessarily my employer's. I hate spam.

Hal Murray wrote:

>
> I'm pretty sure that some of the systems I've worked on used
> the interrupt from the 32 KHz clock chip to drive the scheduler.

From the beginning, "IBM" PCs did not use the 32kHz clock. It is
possible that some other hardware used a 32kHz derived clock for powered
up timing, but I'm not aware of anything before Linux, recently, started
having options for every possible source. Historically, what tended to
precede the counter-timer clocks running at about a MHz, was mains
frequency clocks, running at 50 or 60 Hz.






>
> Some/many systems have long had troubles keeping time if interrups
> get lost. That wouldn't make sense if something like the TSC was
> used for timekeeping.

TSC is a relatively very recent feature. The interrupts come from a
counter timer, which was originally part of a counter-timer chip (Intel
8254-2), but is now just part of the overall ASIC. It's clocked at
1.190 MHz (that's what the PC Technical Reference says, but I suspect,
if I traced the circuit, I would find it is 1.19318667. The original
MS-DOS clock rate is a result of counting this for the full 16 bits of
the counter. (Other channels, on the 8254, clocked at the same rate,
where used to generate memory refresh timing and speaker beeps.)
>
>

David Woolley wrote:

> 1.190 MHz (that's what the PC Technical Reference says, but I suspect,

That's actually the AT, not the basic PC. It looks like the AT used a
completely separate clock for for the processor clock, whereas I think
the PC used the 14.1... MHz one as the basic source for that. Modern
PCs tend to have only one crystal besides the 32kHz one. The former
being used for everything except the CMOS clock.

> Again, read Mark Martinec's web page. He's got a lot of good data.
> http://www.ijs.si/time/temp-compensation/

This is almost exactly what I was thinking. Mark's 7 years ahead of me
(laugh)

Thanks for the post, I appreciate the information.

David and Hal,

David Woolley wrote:
> Hal Murray wrote:
>>
>> Most PCs have 2 xtals. One at 14.xxx MHz (cheap, 4X color burst)
>> that drives the CPU and most motherboard logic through a magic clock
>> generator (PLL) chip, and another that is a 32 KHz watch crystal for
>> keeping time when the CPU is off. The latter also makes interrupts
>> for the scheduler.
>
> Historically interrupts from the 32kHz clock have not been used, except,
> possibly, in powered down states to initiate a restart from suspend or
> hibernate. It is possible that has changed very recently, but they
> certainly weren't used historically.

About which operating system(s) are you talking?

The PC's standard RTC chip can certainly generate cyclic interrupts.
However, if a cyclic interrupt from the RTC or from another timer chip is
used to drive the scheduler depends on the type and eventually on the
version of an operating system, isn't it?

So one system may be using the RTC's interrupts and another one may not.


Martin
--
Martin Burnicki

Meinberg Funkuhren
Bad Pyrmont
Germany

Martin Burnicki writes:

>David and Hal,

>David Woolley wrote:
>> Hal Murray wrote:
>>>
>>> Most PCs have 2 xtals. One at 14.xxx MHz (cheap, 4X color burst)
>>> that drives the CPU and most motherboard logic through a magic clock
>>> generator (PLL) chip, and another that is a 32 KHz watch crystal for
>>> keeping time when the CPU is off. The latter also makes interrupts
>>> for the scheduler.
>>
>> Historically interrupts from the 32kHz clock have not been used, except,
>> possibly, in powered down states to initiate a restart from suspend or
>> hibernate. It is possible that has changed very recently, but they
>> certainly weren't used historically.

>About which operating system(s) are you talking?

>The PC's standard RTC chip can certainly generate cyclic interrupts.
>However, if a cyclic interrupt from the RTC or from another timer chip is
>used to drive the scheduler depends on the type and eventually on the
>version of an operating system, isn't it?

>So one system may be using the RTC's interrupts and another one may not.

So the question is, do you know of any operating systems which use the RTC
to drive the scheduler?

The rtc has been going through a bunch of changes recently-- from Motorola
MC146818 to HPET to more recent rtc chipsets.

Under Linux support is a mess. For example if you turn on an interrupt, the
system returns immediately even though the conditions of the interrupt have
not been met. Some bug in the code.

Martin Burnicki wrote:
>
> About which operating system(s) are you talking?

For powered up timing, MS-DOS, its predecessors if they implemented a
software clock at all, the MS-DOS Windows (3.0, 3.1, 95, 98, ME, 98SE),
most, if not all of the NT Windows (NT 3.5, NT 4.0, Windows 2000,
Windows XP, probably Windows 2003), Linux from start to 2.4, and mostly
for 2.6, SCO OpenServer.......

(The tick rate for MS-DOS family systems is a good clue to which timer
they use.)
>
> The PC's standard RTC chip can certainly generate cyclic interrupts.
> However, if a cyclic interrupt from the RTC or from another timer chip is

But generally isn't used for that. ISTR that some early PCs didn't have
an RTC and had to be set when booted.

> used to drive the scheduler depends on the type and eventually on the
> version of an operating system, isn't it?

Divergence into alternative periodic sources on IBM PC type machines is
very recent.

David Woolley wrote:
> Martin Burnicki wrote:
>>
>> About which operating system(s) are you talking?
>
> For powered up timing, MS-DOS, its predecessors if they implemented a
> software clock at all, the MS-DOS Windows (3.0, 3.1, 95, 98, ME, 98SE),
> most, if not all of the NT Windows (NT 3.5, NT 4.0, Windows 2000,
> Windows XP, probably Windows 2003), Linux from start to 2.4, and mostly
> for 2.6, SCO OpenServer.......
>
> (The tick rate for MS-DOS family systems is a good clue to which timer
> they use.)
>>
>> The PC's standard RTC chip can certainly generate cyclic interrupts.
>> However, if a cyclic interrupt from the RTC or from another timer chip is
>
> But generally isn't used for that. ISTR that some early PCs didn't have
> an RTC and had to be set when booted.

The original IBM PC and the PC/XT did not have hardware clocks. If you
wanted a clock you purchased a "multifunction card" and plugged it into
the bus. I believe that the PC/AT was the first IBM PC with a native
clock. I have forgotten what else the multifunction card did but it
supplied a few goodies that were not native.

Clone manufacturers may have beaten IBM and installed clock as standard
first.

Unruh wrote:
>
> Under Linux support is a mess. For example if you turn on an interrupt, the
> system returns immediately even though the conditions of the interrupt have
> not been met. Some bug in the code.
>
could you be more specific?

uwe

David Woolley writes:

>Martin Burnicki wrote:
>>
>> About which operating system(s) are you talking?

>For powered up timing, MS-DOS, its predecessors if they implemented a
>software clock at all, the MS-DOS Windows (3.0, 3.1, 95, 98, ME, 98SE),
>most, if not all of the NT Windows (NT 3.5, NT 4.0, Windows 2000,
>Windows XP, probably Windows 2003), Linux from start to 2.4, and mostly
>for 2.6, SCO OpenServer.......

>(The tick rate for MS-DOS family systems is a good clue to which timer
>they use.)
>>
>> The PC's standard RTC chip can certainly generate cyclic interrupts.
>> However, if a cyclic interrupt from the RTC or from another timer chip is

>But generally isn't used for that. ISTR that some early PCs didn't have
>an RTC and had to be set when booted.

>> used to drive the scheduler depends on the type and eventually on the
>> version of an operating system, isn't it?

>Divergence into alternative periodic sources on IBM PC type machines is
>very recent.

There is the rtc, and then there is the timer chip on the PC board. I
thought that they were very different. The RTC is the on board real time
clock powered by a battery on the cmos. The timer interrups are a divisor
of the main bus clock that drives the computer bus I believe. I thought we
were talking about the rtc, not the bus clock. ( int 8 not int 0)

Unruh wrote:

>
> There is the rtc, and then there is the timer chip on the PC board. I
> thought that they were very different. The RTC is the on board real time
> clock powered by a battery on the cmos. The timer interrups are a divisor
> of the main bus clock that drives the computer bus I believe. I thought we
> were talking about the rtc, not the bus clock. ( int 8 not int 0)
>

That's what I was saying. Martin was suggesting that the RTC was being
used, and that alternative sources were more common than in reality.
The RTC runs off the 32kHz crystal.

Note that on the PC/AT, for which I have detailed circuit diagrams, the
main bus clock and the timer clock are distinct. I think that is
because the original clock was right for the PC but too slow for the AT.
They have merged again, because the crystal frequency is now
multiplied to get the main bus clock.

HPET timers are recent, and generally use the non-32kHz crystal, i.e.
they have the same temperature dependence as the counter timer interrupt
and TSC, because they are derived from the same oscillator.

Uwe Klein writes:

>Unruh wrote:
>>
>> Under Linux support is a mess. For example if you turn on an interrupt, the
>> system returns immediately even though the conditions of the interrupt have
>> not been met. Some bug in the code.
>>
>could you be more specific?

Sure
bugzilla.kernel.org bug 11112
On more recent systems (eg systems with HPET) the kernel code was not
cleaning up old interrupt information when the intrrupt UIE ( the update
interrupt which tell the rtc to issue an interrupt once per second just
when the seconds marker was turning over in the rtc) was switched on. This
meant that the interrupt was retured immediately. The usual way of reading
the rtc for accurate work was to switch on the UIE, and then do a select() or
a read() on the /dev/rtc fd, and wait for the update. However, it returned
immediately meaning that the first select/read after the UIE was turned on
returned immediately and not when the rtc updated. Ie, the time would be
out by up to a second. Now if you did not care whether or not the rtc was
accurate, that is ok. However if you wanted to set or read the rtc
accurately, that is terrible. Here you discipline your computer to within 1
microsecond, and you cannot discipline your rtc to better than a second.

I discovered this about 2 years ago while trying to understand chrony, but
at that time thought it is was just some weirdness with the rtc I did not
understand. Last week, when a new computer could not run chrony at all, I
looked into it further, and discovered the above bug in the rtc driver (the
hpet and also the new rtc-cmos drivers). (There was also a further bug in
the hpet glue and the Mandriva kernel setup which was why chrony was not
working, but that is another story).

Note that there are now kernel fixes for the bug which will probably make
it into the 2.6.28 kernel. In the meantime, make sure that if you want to
accurately read the rtc, do a select or read twice-- the second one will be
at the update boundary (well on an hpet system, that is not clear either.
It seems that the rtc situation is a bit of a mess these days caused by
very bad hpet/bios behaviour.)


>uwe