>>> In article <20080309231536.67e0fc80.miallen@ioplex.com>, Michael B Allen writes:

Michael> For example, I just installed the "ntp" package on a Ubuntu
Michael> server. So I waited an hour and absolutely nothing happened. The
Michael> time has been out of sync by 3 minutes since ntpd was installed.

Michael> I did a capture and I can see requests are being answered. There
Michael> are no firewalls in the way.

Michael> The default config is inlined below. The only thing I changed was
Michael> 'server 192.168.2.15' which is the time server (which I know
Michael> works).

I think you need a "restrict" line for 192.168.2.15 .

See http://support.ntp.org/Support/AccessRestrictions .

--
Harlan Stenn
http://ntpforum.isc.org - be a member!

On Mon, 10 Mar 2008 03:52:17 +0000
Harlan Stenn wrote:

> >>> In article <20080309231536.67e0fc80.miallen@ioplex.com>, Michael B Allen writes:
>
> Michael> For example, I just installed the "ntp" package on a Ubuntu
> Michael> server. So I waited an hour and absolutely nothing happened. The
> Michael> time has been out of sync by 3 minutes since ntpd was installed.
>
> Michael> I did a capture and I can see requests are being answered. There
> Michael> are no firewalls in the way.
>
> Michael> The default config is inlined below. The only thing I changed was
> Michael> 'server 192.168.2.15' which is the time server (which I know
> Michael> works).
>
> I think you need a "restrict" line for 192.168.2.15 .
>
> See http://support.ntp.org/Support/AccessRestrictions .

Hi Harlan,

Adding 'restrict 192.168.2.15' had no effect after 5 minutes and counting.

Below is a request and response if that helps in the diagnosis.

Mike

No. Time Source Destination Protocol Info
107 62.564960 192.168.2.23 192.168.2.15 NTP NTP client

Frame 107 (90 bytes on wire, 90 bytes captured)
Ethernet II, Src: Vmware_82:21:c2 (00:0c:29:82:21:c2), Dst: 3com_cf:5f:18 (00:01:02:cf:5f:18)
Internet Protocol, Src: 192.168.2.23 (192.168.2.23), Dst: 192.168.2.15 (192.168.2.15)
User Datagram Protocol, Src Port: ntp (123), Dst Port: ntp (123)
Network Time Protocol
Flags: 0xe3
11.. .... = Leap Indicator: alarm condition (clock not synchronized) (3)
..10 0... = Version number: NTP Version 4 (4)
.... .011 = Mode: client (3)
Peer Clock Stratum: unspecified or unavailable (0)
Peer Polling Interval: 6 (64 sec)
Peer Clock Precision: 0.000001 sec
Root Delay: 0.0000 sec
Root Dispersion: 0.0068 sec
Reference Clock ID: (Initialization)
Reference Clock Update Time: NULL
Originate Time Stamp: Mar 10, 2008 06:43:58.9069 UTC
Receive Time Stamp: Mar 10, 2008 06:48:13.7477 UTC
Transmit Time Stamp: Mar 10, 2008 06:49:18.7473 UTC

No. Time Source Destination Protocol Info
108 62.565348 192.168.2.15 192.168.2.23 NTP NTP server

Frame 108 (90 bytes on wire, 90 bytes captured)
Ethernet II, Src: 3com_cf:5f:18 (00:01:02:cf:5f:18), Dst: Vmware_82:21:c2 (00:0c:29:82:21:c2)
Internet Protocol, Src: 192.168.2.15 (192.168.2.15), Dst: 192.168.2.23 (192.168.2.23)
User Datagram Protocol, Src Port: ntp (123), Dst Port: ntp (123)
Network Time Protocol
Flags: 0x24
00.. .... = Leap Indicator: no warning (0)
..10 0... = Version number: NTP Version 4 (4)
.... .100 = Mode: server (4)
Peer Clock Stratum: secondary reference (3)
Peer Polling Interval: 6 (64 sec)
Peer Clock Precision: 0.000004 sec
Root Delay: 0.0916 sec
Root Dispersion: 0.0664 sec
Reference Clock ID: 69.31.13.210
Reference Clock Update Time: Mar 10, 2008 06:39:27.6109 UTC
Originate Time Stamp: Mar 10, 2008 06:49:18.7473 UTC
Receive Time Stamp: Mar 10, 2008 06:45:03.5936 UTC
Transmit Time Stamp: Mar 10, 2008 06:45:03.5936 UTC

Michael B Allen schreef:

> When installing an NTPD package on Linux would you expect it to work
> by default or must one always be expected to modify the config?

Some hints:

- Have you checked ntpd's logging? What interfaces is it using?
- If you stop and start ntpd, does it work then?
- Do you have the network-manager package installed?

I had the same symptoms as you describe. It turned out that, if you use
network-manager, Ubuntu does not have any network interfaces configured
at the time it starts ntpd, which is a major SNAFU.

N

Michael B Allen wrote:
>
> When installing an NTPD package on Linux would you expect it to work
> by default or must one always be expected to modify the config?

I would not expect the config to have appropriate servers, so one would
always have to modify it. Including a valid server is likely to result
in most people using it, which is good for neither them nor the server.

> The default config is inlined below. The only thing I changed was
> 'server 192.168.2.15' which is the time server (which I know works).

This is Ubuntu's default, not THE default. You need to take this up
with Ubuntu.

Note, whilst I thought it was the restricts, at first, however, I think
they are actually OK.

PS, The standard diagnostics are the result of running the peers and rv
subcommands of ntpq, preferably with rv's for the servers, for which you
may need assoc, to get the reference number.

On 2008-03-10, Harlan Stenn wrote:

> Michael B Allen writes:
>
>># By default, exchange time with everybody, but don't allow
>># configuration.
>># See /usr/share/doc/ntp-doc/html/accopt.html for details.
>>restrict -4 default kod notrap nomodify nopeer noquery
>>restrict -6 default kod notrap nomodify nopeer noquery

These default restrictions ALLOW global access to/from time service.

>># Local users may interrogate the ntp server more closely.
>>restrict 127.0.0.1
>>restrict ::1
>
> I think you need a "restrict" line for 192.168.2.15 .

No, he does not.

--
Steve Kostecke
NTP Public Services Project - http://support.ntp.org/

On 2008-03-10, Michael B Allen wrote:
> Harlan Stenn wrote:
>
>> Michael B Allen writes:
>>
>> The default config is inlined below. The only thing I changed was
>> server 192.168.2.15' which is the time server (which I know
>> works).
>>
>> I think you need a "restrict" line for 192.168.2.15 .
>>
>> See http://support.ntp.org/Support/AccessRestrictions .
>
> Adding 'restrict 192.168.2.15' had no effect after 5 minutes and counting.

That's because there was nothing in your restrict lines blocking time
service. If you want to know more about how the restrict lines work
please take the time to read Support.AccessRestrictions.

> Below is a request and response if that helps in the diagnosis.

The output of 'ntpq -pcrv' is what we need to see.

BTW: You should append 'iburst' to your server lines to speed up initial sync
from a bit over 5 minutes to a bit less than 20 seconds (assuming no other
issues exist).

--
Steve Kostecke
NTP Public Services Project - http://support.ntp.org/

On 10 Mar 2008 12:08:53 GMT
Steve Kostecke wrote:

> On 2008-03-10, Michael B Allen wrote:
> > Harlan Stenn wrote:
> >
> >> Michael B Allen writes:
> >>
> >> The default config is inlined below. The only thing I changed was
> >> server 192.168.2.15' which is the time server (which I know
> >> works).
> >>
> >> I think you need a "restrict" line for 192.168.2.15 .
> >>
> >> See http://support.ntp.org/Support/AccessRestrictions .
> >
> > Adding 'restrict 192.168.2.15' had no effect after 5 minutes and counting.
>
> That's because there was nothing in your restrict lines blocking time
> service. If you want to know more about how the restrict lines work
> please take the time to read Support.AccessRestrictions.
>
> > Below is a request and response if that helps in the diagnosis.
>
> The output of 'ntpq -pcrv' is what we need to see.
>
> BTW: You should append 'iburst' to your server lines to speed up initial sync
> from a bit over 5 minutes to a bit less than 20 seconds (assuming no other
> issues exist).

Hi Steve,

Output of ntpq:

# ntpq -pcrv
assID=0 status=c011 sync_alarm, sync_unspec, 1 event, event_restart,
version="ntpd 4.2.4p0@1.1472-o Thu Oct 4 20:58:45 UTC 2007 (1)",
processor="i686", system="Linux/2.6.22-14-server", leap=11, stratum=16,
precision=-20, rootdelay=0.000, rootdispersion=0.210, peer=0,
refid=INIT, reftime=00000000.00000000 Thu, Feb 7 2036 1:28:16.000,
poll=6, clock=cb7fdd9b.a61cc7fd Mon, Mar 10 2008 12:17:31.648, state=1,
offset=0.000, frequency=0.000, jitter=0.001, noise=0.001,
stability=0.000, tai=0
remote refid st t when poll reach delay offset jitter
================================================== ============================
nano.foo.net 69.31.13.210 3 u 13 64 1 0.364 -420416 0.001

And here's syslog on startup:

Mar 10 12:17:17 ls3 ntpd[5923]: ntpd 4.2.4p0@1.1472-o Thu Oct 4 20:58:45 UTC 2007 (1)
Mar 10 12:17:17 ls3 ntpd[5924]: precision = 1.000 usec
Mar 10 12:17:17 ls3 ntpd[5924]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled
Mar 10 12:17:17 ls3 ntpd[5924]: Listening on interface #1 wildcard, ::#123 Disabled
Mar 10 12:17:17 ls3 ntpd[5924]: Listening on interface #2 lo, ::1#123 Enabled
Mar 10 12:17:17 ls3 ntpd[5924]: Listening on interface #3 eth0, fe80::20c:29ff:fe82:21c2#123 Enabled
Mar 10 12:17:17 ls3 ntpd[5924]: Listening on interface #4 lo, 127.0.0.1#123 Enabled
Mar 10 12:17:17 ls3 ntpd[5924]: Listening on interface #5 eth0, 192.168.2.23#123 Enabled
Mar 10 12:17:17 ls3 ntpd[5924]: kernel time sync status 0040
Mar 10 12:17:17 ls3 ntpd[5924]: frequency initialized 0.000 PPM from /var/lib/ntp/ntp.drift

Note: I removed the insignificant restrict line and restarted ntpd.

Mike

On 2008-03-10, Michael B Allen wrote:

> Steve Kostecke wrote:
>
>> The output of 'ntpq -pcrv' is what we need to see.
>>
>> BTW: You should append 'iburst' to your server lines to speed up
>> initial sync from a bit over 5 minutes to a bit less than 20 seconds
>> (assuming no other issues exist).
>
> Output of ntpq:

[snip]

> refid=INIT, reftime=00000000.00000000 Thu, Feb 7 2036 1:28:16.000,
> poll=6, clock=cb7fdd9b.a61cc7fd Mon, Mar 10 2008 12:17:31.648, state=1,

Your ntpd is stuck in its early start-up stages.

> remote refid st t when poll reach delay offset jitter
>================================================== =================
>nano.foo.net 69.31.13.210 3 u 13 64 1 0.364 -420416 0.001

Although this was taken a bit prematurely (i.e. right after ntpd was
started) ...

We can see that the remote time server is answering your polls. So we
can probably rule out a network problem.

The offset is excessive. It may be because ntpd has not run long enough
to poll the remote time server and determine that a step is required.

You should either (a) start ntpd with -g to allow a large initial
time step, or (b) preset the clock with 'ntpq -gq' (or the deprecated
ntpdate) before starting the daemon instance of ntpd

> Mar 10 12:17:17 ls3 ntpd[5924]: frequency initialized 0.000 PPM \
> from /var/lib/ntp/ntp.drift

A driftfile containing 0.00 is not a good sign.

Please add the '-g' command-line argument to your ntp init script (if it
is not already there) then (a) stop ntpd, (b) delete the driftfile, and
(c) start ntpd.

Make sure ntpd is allowed to run undisturbed until a non-zero value is
written to the driftfile (this may take an hour). Your ntpd will spend
about 20 minutes testing the clock to determine the needed frequency
correction. This is the value which is written to the driftfile. Once
the test is complete ntpd will be able to synchronize its time sources.

ntpd will synchronize quickly to its time sources on subsequent
start-ups when your drift file contains a "good" frequency correction
value.

BTW: for interactive support please visit #ntp on irc.freenode.net

--
Steve Kostecke
NTP Public Services Project - http://support.ntp.org/

Steve Kostecke wrote:
>
> The offset is excessive. It may be because ntpd has not run long enough
> to poll the remote time server and determine that a step is required.
>
> You should either (a) start ntpd with -g to allow a large initial
> time step, or (b) preset the clock with 'ntpq -gq' (or the deprecated
> ntpdate) before starting the daemon instance of ntpd
>

The offset is less than 1000 seconds, so -g is not essential.
>

On 10 Mar 2008 19:43:34 GMT
Steve Kostecke wrote:

> On 2008-03-10, Michael B Allen wrote:
>
> > Steve Kostecke wrote:
> >
> >> The output of 'ntpq -pcrv' is what we need to see.
> >>
> >> BTW: You should append 'iburst' to your server lines to speed up
> >> initial sync from a bit over 5 minutes to a bit less than 20 seconds
> >> (assuming no other issues exist).
> >
> > Output of ntpq:
>
> [snip]
>
> > refid=INIT, reftime=00000000.00000000 Thu, Feb 7 2036 1:28:16.000,
> > poll=6, clock=cb7fdd9b.a61cc7fd Mon, Mar 10 2008 12:17:31.648, state=1,
>
> Your ntpd is stuck in its early start-up stages.
>
> > remote refid st t when poll reach delay offset jitter
> >================================================== =================
> >nano.foo.net 69.31.13.210 3 u 13 64 1 0.364 -420416 0.001
>
> Although this was taken a bit prematurely (i.e. right after ntpd was
> started) ...
>
> We can see that the remote time server is answering your polls. So we
> can probably rule out a network problem.
>
> The offset is excessive. It may be because ntpd has not run long enough
> to poll the remote time server and determine that a step is required.
>
> You should either (a) start ntpd with -g to allow a large initial
> time step, or (b) preset the clock with 'ntpq -gq' (or the deprecated
> ntpdate) before starting the daemon instance of ntpd
>
> > Mar 10 12:17:17 ls3 ntpd[5924]: frequency initialized 0.000 PPM \
> > from /var/lib/ntp/ntp.drift
>
> A driftfile containing 0.00 is not a good sign.
>
> Please add the '-g' command-line argument to your ntp init script (if it
> is not already there) then (a) stop ntpd, (b) delete the driftfile, and
> (c) start ntpd.
>
> Make sure ntpd is allowed to run undisturbed until a non-zero value is
> written to the driftfile (this may take an hour). Your ntpd will spend
> about 20 minutes testing the clock to determine the needed frequency
> correction. This is the value which is written to the driftfile. Once
> the test is complete ntpd will be able to synchronize its time sources.
>
> ntpd will synchronize quickly to its time sources on subsequent
> start-ups when your drift file contains a "good" frequency correction
> value.

I stopped ntpd, deleted the drift file, ran 'ntpdate 192.168.2.15',
verified that the time was sync'd and started ntpd.

Ninty minutes later the time is ahead of the time server by 36 seconds.

# ntpq -pcrv
assID=0 status=c011 sync_alarm, sync_unspec, 1 event, event_restart,
version="ntpd 4.2.4p0@1.1472-o Thu Oct 4 20:58:45 UTC 2007 (1)",
processor="i686", system="Linux/2.6.22-14-server", leap=11, stratum=16,
precision=-20, rootdelay=0.000, rootdispersion=98.190, peer=0,
refid=INIT, reftime=00000000.00000000 Thu, Feb 7 2036 1:28:16.000,
poll=6, clock=cb8034d0.c2c6254f Mon, Mar 10 2008 18:29:36.760, state=0,
offset=0.000, frequency=0.000, jitter=0.001, noise=0.001,
stability=0.000, tai=0
remote refid st t when poll reach delay offset jitter
================================================== ============================
nano.foo.net 69.31.13.210 3 u 37 64 377 0.637 -30672. 1355.63

Restarting ntpd shows:

Mar 10 18:33:10 ls3 ntpd[6545]: ntpd 4.2.4p0@1.1472-o Thu Oct 4 20:58:45 UTC 2007 (1)
Mar 10 18:33:10 ls3 ntpd[6546]: precision = 1.000 usec
Mar 10 18:33:10 ls3 ntpd[6546]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled
Mar 10 18:33:10 ls3 ntpd[6546]: Listening on interface #1 wildcard, ::#123 Disabled
Mar 10 18:33:10 ls3 ntpd[6546]: Listening on interface #2 lo, ::1#123 Enabled
Mar 10 18:33:10 ls3 ntpd[6546]: Listening on interface #3 eth0, fe80::20c:29ff:fe82:21c2#123 Enabled
Mar 10 18:33:10 ls3 ntpd[6546]: Listening on interface #4 lo, 127.0.0.1#123 Enabled
Mar 10 18:33:10 ls3 ntpd[6546]: Listening on interface #5 eth0, 192.168.2.23#123 Enabled
Mar 10 18:33:10 ls3 ntpd[6546]: kernel time sync status 0040
Mar 10 18:33:10 ls3 ntpd[6546]: frequency initialized 0.000 PPM from /var/lib/ntp/ntp.drift

After hearing your description of "testing the clock" I have to wonder
if the fact that the server is running in VMWare matters. The clock
drifts quite a bit (e.g. 36 seconds in an 90 minutes). Could the fact that
Ubuntu is running in VMWare Server be a problem?

Mike

The following config works:

driftfile /var/lib/ntp/ntp.drift
server 192.168.2.15 iburst

The clock was sync'd in a matter of seconds.

I kinda figured it would work since I have other servers that use it
and I've asked about this on this list before and was told that the
above was the minimum client config. I was just intrigued that Ubuntu
could ship with a default config that didn't actually work and wanted to
see if someone could pinpoint why. I have a feeling there are a lot of
Ubuntu users out there who think their clocks are syncing when in fact
they're not.

Thanks for the entertainment,
Mike

Michael B Allen wrote:

> Ninty minutes later the time is ahead of the time server by 36 seconds.

That drift rate is uncorrectable by ntpd. (It may step repeatedly,)
Normally this would indicate a seriously broken motherboard, although,
in this case, it may be software related, particularly if any other
virtual machine is trying to control the clock.

Michael B Allen wrote:
> The following config works:
>
> driftfile /var/lib/ntp/ntp.drift
> server 192.168.2.15 iburst
>
> The clock was sync'd in a matter of seconds.
>
> I kinda figured it would work since I have other servers that use it
> and I've asked about this on this list before and was told that the
> above was the minimum client config. I was just intrigued that Ubuntu

It's not minimum; it includes some, but not all, of the optional things
that are considered best current practice (it doesn't contain 4 servers,
which is the other common BCP requirement).

Given that your drift rate is 6666 ppm and ntpd needs it to be somewhat
less than 500ppm, I suspect you are getting a continuous sequence of
steps and you just happened to look after a step (and iburst is allowing
it to actually get a time measurement fast enough that it does exceed
128ms before the time is measured.

I don't know how VMWare handles the TSC, but unless it handles it in a
away intended for real time measurement, rather than CPU usage
measurement the calibration may be run at a time that doesn't match
average conditions. If that theory is correct, you need to disable TSC
as a time source.

> could ship with a default config that didn't actually work and wanted to
> see if someone could pinpoint why. I have a feeling there are a lot of
> Ubuntu users out there who think their clocks are syncing when in fact
> they're not.
>
> Thanks for the entertainment,
> Mike

"Michael B Allen" wrote in message
news:20080310183745.2979d85d.miallen@ioplex.com...

> [...] Could the fact that Ubuntu is running in VMWare Server be a problem?

Yes. Very much so. Install the VMWare tools and let the virtual machine
host control the passage of time on the clients. Only run NTP on the
host machine.

There's a whitepaper somewhere. It boils down to 'Virtual machines are
not suited to real-time software. Let the host control time, it will
do a better job.'

Time synchronisation in the clients is off by default, and they _will_
drift. I have two old Linux machines now virtualised, both without VMWare
tools installed. With adjtimex I managed to get one below 1PPM and it
now steps half a second each week; for the other one (kernel 2.0) I
didn't find or compile an adjtimex yet and it steps 2.5 seconds each
day.

Groetjes,
Maarten Wiltink

Maarten Wiltink wrote:
> "Michael B Allen" wrote in message
> news:20080310183745.2979d85d.miallen@ioplex.com...
>
>
>>[...] Could the fact that Ubuntu is running in VMWare Server be a problem?
>
>
> Yes. Very much so. Install the VMWare tools and let the virtual machine
> host control the passage of time on the clients. Only run NTP on the
> host machine.
>
> There's a whitepaper somewhere. It boils down to 'Virtual machines are
> not suited to real-time software. Let the host control time, it will
> do a better job.'
>
> Time synchronisation in the clients is off by default, and they _will_
> drift. I have two old Linux machines now virtualised, both without VMWare
> tools installed. With adjtimex I managed to get one below 1PPM and it
> now steps half a second each week; for the other one (kernel 2.0) I
> didn't find or compile an adjtimex yet and it steps 2.5 seconds each
> day.
>
> Groetjes,
> Maarten Wiltink
>
>

My experience with VMWare is limited to VMWare ESX. With ESX all you
need to do is to log in as root, edit ntp.conf to include your favorite
servers, start ntpd and enjoy.

"Richard B. Gilbert" wrote in message
news:47D680B2.1050806@comcast.net...
[...]
> My experience with VMWare is limited to VMWare ESX. With ESX all you
> need to do is to log in as root, edit ntp.conf to include your favorite
> servers, start ntpd and enjoy.

Is that the free one? (The free one is what I have at home. It's free.)

With the free version, you have to install the VMware tools in every
client, and enable 'Time synchronization between the virtual machine
and the host operating system' on the options tab. (This is on a Windows
client.)

Groetjes,
Maarten Wiltink

On Tue, 11 Mar 2008 07:30:20 +0000
David Woolley wrote:

> I don't know how VMWare handles the TSC, but unless it handles it in a
> away intended for real time measurement, rather than CPU usage
> measurement the calibration may be run at a time that doesn't match
> average conditions. If that theory is correct, you need to disable TSC
> as a time source.

Well I have no idea what you just said but I suppose it means that the
problem isn't really "SOLVED" after all. And I just rebooted and the
time is off again so I wonder if iburst is really sufficient for a
robust solution.

For posterity I wanted to see if installing VMWare tools made a difference
(I bet it does) but after downloading gcc and 45MB of kernel source
it wanted me to build the kernel and that's where I throw in the
towel (the Linux kernel guys really need to figure out how to make symbol
versioning more forgiving).

And of course all of this says nothing about the default ntp.conf on
Ubuntu. I suspect it does in fact work and that the problem is just
VMWare Server (the "free" one, not ESX).

Mike

Maarten Wiltink wrote:
> "Richard B. Gilbert" wrote in message
> news:47D680B2.1050806@comcast.net...
> [...]
>
>>My experience with VMWare is limited to VMWare ESX. With ESX all you
>>need to do is to log in as root, edit ntp.conf to include your favorite
>>servers, start ntpd and enjoy.
>
>
> Is that the free one? (The free one is what I have at home. It's free.)
>
> With the free version, you have to install the VMware tools in every
> client, and enable 'Time synchronization between the virtual machine
> and the host operating system' on the options tab. (This is on a Windows
> client.)
>
> Groetjes,
> Maarten Wiltink
>
>

I don't think the ESX version is free! My employer at that time bought
a copy. We installed it on a server and ran two or three virtual
machines on it. I configured NTP on it. This was ca. 2003.