Martin Burnicki writes:

>Dave,

>David L. Mills wrote:
>> Serge,
>>
>> The behavior after a step is deliberate. The iburst volley after a step
>> is delayed a random fraction of the poll interval to avoid implosion
>> at a busy server. An additional delay may be enforced to avoid violating
>> the headway restrictions. This is not to protect your applications; it
>> is to protect the server.

>Is it really necessary to insert a random delay after a step? There has
>already been a random delay immediately after startup, before the client
>has decided that a step was required.

>So even if a bunch of clients started up at the same time and had to step,
>they wouln't step at the same time, and thus wouldn't do the next iburst
>volley at the same time anyway.

Why not? The power comes on on your computer farm of 2000 machines, all the clients are the same type so the
bootup sequence is identical. They all start ntp at the same time, to
within a second or so. And suddenly the poor server is flooded.

>Martin
>--
>Martin Burnicki

>Meinberg Funkuhren
>Bad Pyrmont
>Germany

Martin Burnicki wrote:

> Wouldn't it make sense to adjust the time constant depending on the time
> after startup, and/or the quality of the responses from the upstream
> servers?
>
It does get adjusted. We are talking about the minimum value!

Unruh wrote:
> David Woolley writes:
>
>> Harlan Stenn wrote:
>>> For the general use case (LAN and/or WAN and/or jerky path) ntpd behaves
>>> well.
>
>> We are talking typical rather than general cases. In the typical case,
>> 1ms after 1 second is a reasonable expectation on a WAN, especially when
>> a site is restarting, e.g. after a power failure, or a home system
>> switching on, and, therefore, the network load is low.
>
> I think you go t your units mixed up. computer A goes down for three days
> due to an avalanch cutting the power. It takes a lot longer than one second
> to resync that computer. A few hours is more like it.

I was talking about what people could expect from software that behaved
well; I think you are describing what ntpd actually does here. My point
was that ntpd's ability to tolerate really rotten links is irrelevant
for most users, who are only about 20ms away from their ISP's time
server, and can expect to read it to about 1ms accuracy.

> If you mean, I shut down ntp and restart it immediately , then 1ms in 1
> minute is reasonable ( you cannot have made enough measurements in 1 sec to
> even know if it is accurate.)
>

Bill,

>>> In article , Unruh writes:

Unruh> Why not? The power comes on on your computer farm of 2000 machines,
Unruh> all the clients are the same type so the bootup sequence is
Unruh> identical. They all start ntp at the same time, to within a second or
Unruh> so. And suddenly the poor server is flooded.

2000 packets hitting ntpd all at once should not be a problem for an ntp
server in that environment.
--
Harlan Stenn
http://ntpforum.isc.org - be a member!

Hello David,

On Tuesday, February 12, 2008 at 15:04:45 +0000, David L. Mills wrote:

> Serge Bets wrote:
>> ntpd -q can make use of the driftfile to set the kernel frequency
> That was removed as a significant security hazard.

Why exactly?


> If you want to rxplicitly set the frequency, use ntptime -f.

Sure: I can preset the frequency by hand. But not setting the frequency
is not a sensible option: it's required for good ntpq -q operations,
otherwise slews don't end on the zero.


> Ths scheme is designed so you can run ntpd until the kernel frequency
> has stabilized, then kill ntpd and run SNTP client at regular
> intervals.

There is no obstacle to that. When ntpd quits, the kernel runs on the
last computed frequency. Without driftfile, ntpd -q runs above this
frequency. With a driftfile, ntpd -q could even run above this frequency
after a reboot.

The obstacle if one existed would be a frequency reset to zero at
startup, like done by loop_config(LOOP_DRIFTINIT). Fortunately this
doesn't happen in mode_ntpdate (the -q flag).


Serge.
--
Serge point Bets arobase laposte point net

Maarten,

No. However, there is a small dither of a few percent at all poll
intervals to resist self-synchronization.

Dave

Maarten Wiltink wrote:
> "David L. Mills" wrote in message
> news:fosb7r$i9s$1@scrotar.nss.udel.edu...
>
>
>>No, there is no random delay at startup. Each association starts one
>>second after the previous one. The random backoff occurs only after a
>>step.
>
>
> Is there also a random backoff after an increase of the polling interval?
>
> Groetjes,
> Maarten Wiltink
>
>

Unruh,

Depends who the clients are. An ntpd client will not come up in the
first second, although successive associations will come up at 2-s
intervals. I would not expect 2000 clients to come up at the same exact
time anyway due ordinary latency variations in the boot process. I would
be more worried about a broacast server coming up with 2000 corporate
broadcast clients, but in that case the initial client response is
randomized over the poll interval.

Dave

Unruh wrote:

> Martin Burnicki writes:
>
>
>>Dave,
>
>
>>David L. Mills wrote:
>>
>>>Serge,
>>>
>>>The behavior after a step is deliberate. The iburst volley after a step
>>> is delayed a random fraction of the poll interval to avoid implosion
>>>at a busy server. An additional delay may be enforced to avoid violating
>>>the headway restrictions. This is not to protect your applications; it
>>>is to protect the server.
>
>
>>Is it really necessary to insert a random delay after a step? There has
>>already been a random delay immediately after startup, before the client
>>has decided that a step was required.
>
>
>>So even if a bunch of clients started up at the same time and had to step,
>>they wouln't step at the same time, and thus wouldn't do the next iburst
>>volley at the same time anyway.
>
>
> Why not? The power comes on on your computer farm of 2000 machines, all the clients are the same type so the
> bootup sequence is identical. They all start ntp at the same time, to
> within a second or so. And suddenly the poor server is flooded.
>
>
>>Martin
>>--
>>Martin Burnicki
>
>
>>Meinberg Funkuhren
>>Bad Pyrmont
>>Germany

>I was talking about what people could expect from software that behaved
>well; I think you are describing what ntpd actually does here. My point
>was that ntpd's ability to tolerate really rotten links is irrelevant
>for most users, who are only about 20ms away from their ISP's time
>server, and can expect to read it to about 1ms accuracy.

20 ms sounds like a typical DSL link. That 1ms accuracy goes out
the window if you are doing a big download. (At least on my DSL
link.)

--
These are my opinions, not necessarily my employer's. I hate spam.

Hal Murray wrote:

> 20 ms sounds like a typical DSL link. That 1ms accuracy goes out
> the window if you are doing a big download. (At least on my DSL
> link.)
>

People don't generally do big downloads during the boot of a machine!
On a big network, the most likely reason for rebooting a timeserver in
prime time is a power failure. In which case the whole network is
likely to be down.

At worst, using ntpdate -b, you only get something like the current ntpd
behavour. Typically you end up within a millisecond.

"David L. Mills" wrote in message
news:fots64$2g7$1@scrotar.nss.udel.edu...

>>> No, there is no random delay at startup. Each association starts one
>>> second after the previous one. The random backoff occurs only after
>>> a step.
>>
>> Is there also a random backoff after an increase of the polling
>> interval?

> No. However, there is a small dither of a few percent at all poll
> intervals to resist self-synchronization.

Wouldn't that be a nice feature to add? If it's currently polling a
server on, say second 100 (reckoned externally) of 256, to go to
either 100 _or 356_ of 512.

I understand that there are already some random waits in the client
code and Internet servers are well protected by random noise. But
for large numbers of clients in a uniform environment that were all
started at about the same time, is there any way they tend to
naturally disperse across the final 1024s polling interval?

Groetjes,
Maarten Wiltink

Maarten,

The natural behavior of a bunch of oscillators near the same frequency
is to become one giant phase-locked oscillator. Adding a bit of random
fuzz at each poll turns each oscillator into a mini random-walk which
breaks up that tendency. The fuzz is not a lot, like 10 percent.

Dave

Maarten Wiltink wrote:
> "David L. Mills" wrote in message
> news:fots64$2g7$1@scrotar.nss.udel.edu...
>
>
>>>>No, there is no random delay at startup. Each association starts one
>>>>second after the previous one. The random backoff occurs only after
>>>>a step.
>>>
>>>Is there also a random backoff after an increase of the polling
>>>interval?
>
>
>>No. However, there is a small dither of a few percent at all poll
>>intervals to resist self-synchronization.
>
>
> Wouldn't that be a nice feature to add? If it's currently polling a
> server on, say second 100 (reckoned externally) of 256, to go to
> either 100 _or 356_ of 512.
>
> I understand that there are already some random waits in the client
> code and Internet servers are well protected by random noise. But
> for large numbers of clients in a uniform environment that were all
> started at about the same time, is there any way they tend to
> naturally disperse across the final 1024s polling interval?
>
> Groetjes,
> Maarten Wiltink
>
>

"David L. Mills" wrote in message
news:fp0fpa$p5c$1@scrotar.nss.udel.edu...

>>>> Is there also a random backoff after an increase of the polling
>>>> interval?

>>> No. However, there is a small dither of a few percent at all poll
>>> intervals to resist self-synchronization.

> The natural behavior of a bunch of oscillators near the same frequency
> is to become one giant phase-locked oscillator. Adding a bit of random
> fuzz at each poll turns each oscillator into a mini random-walk which
> breaks up that tendency. The fuzz is not a lot, like 10 percent.

Do you mean the dither alluded to above is cumulative?

I was never much good with statistics and remember only that the
expectation of the offset after N steps in a random walk is sqrt(N)
times the average step size. Not a clue what the distribution might
be. Intuitively, I would be aiming for uniform, and randomly adding
half a polling interval delay when doubling it seemed to me like it
would do that.

Groetjes,
Maarten Wiltink

Maarten,

You are correct about the frequency, but the statistic of interest here
is the phase or offset from one server to another. Accumulated over many
poll intervals, the phase does a random walk. In your terms, the
integral of a Gaussian distribution is a random walk.

Dave

Maarten Wiltink wrote:
> "David L. Mills" wrote in message
> news:fp0fpa$p5c$1@scrotar.nss.udel.edu...
>
>
>>>>>Is there also a random backoff after an increase of the polling
>>>>>interval?
>
>
>>>>No. However, there is a small dither of a few percent at all poll
>>>>intervals to resist self-synchronization.
>
>
>>The natural behavior of a bunch of oscillators near the same frequency
>>is to become one giant phase-locked oscillator. Adding a bit of random
>>fuzz at each poll turns each oscillator into a mini random-walk which
>>breaks up that tendency. The fuzz is not a lot, like 10 percent.
>
>
> Do you mean the dither alluded to above is cumulative?
>
> I was never much good with statistics and remember only that the
> expectation of the offset after N steps in a random walk is sqrt(N)
> times the average step size. Not a clue what the distribution might
> be. Intuitively, I would be aiming for uniform, and randomly adding
> half a polling interval delay when doubling it seemed to me like it
> would do that.
>
> Groetjes,
> Maarten Wiltink
>
>

Unruh writes:

> In ntpdate.c around line 542 (4.2.4p4)is the sequence
> if (!authistrusted(sys_authkey)) {
> char buf[10];
>
> (void) sprintf(buf, "%lu", (unsigned long)sys_authkey);
> msyslog(LOG_ERR, "authentication key %s unknown", buf);

Is that too simple?
msyslog(LOG_ERR, "authentication key %lu unknown",
(unsigned long)sys_authkey);


> exit(1);
> }
>
> Since unsigned long does not have a definite length on all machines, and with the trailing
> zero certainly is potentially longer than 10 bytes, that buf is ripe for
> buffer overflow.
> It should be something like
> char buf[(sizeof(unsigned long)*12/5+2)];
> And/or the sprintf should be an snprintf.

Ulrich Windl wrote:
> Unruh writes:
>
>> In ntpdate.c around line 542 (4.2.4p4)is the sequence
>> if (!authistrusted(sys_authkey)) {
>> char buf[10];
>>
>> (void) sprintf(buf, "%lu", (unsigned long)sys_authkey);
>> msyslog(LOG_ERR, "authentication key %s unknown", buf);
>
> Is that too simple?
> msyslog(LOG_ERR, "authentication key %lu unknown",
> (unsigned long)sys_authkey);
>

In this case it's the right solution. There's no need for an
intermediate buffer here.

Danny

Hello Ulrich,

Ulrich Windl wrote:
> Is that too simple?
> msyslog(LOG_ERR, "authentication key %lu unknown",
> (unsigned long)sys_authkey);

Oooh, of course that't the best fix.

I have already prepared a patch but I must have been blind that I didn't see
the obvious solution. Since the patch has not yet been committed I'll
update it once more.

Thanks,

Martin
--
Martin Burnicki

Meinberg Funkuhren
Bad Pyrmont
Germany

Hello David,

On Monday, February 11, 2008 at 19:03:36 +0000, David L. Mills wrote:

> While not admitted in public, the latest snapshot can set the poll
> interval to 3 (8 s), so the risetime is 250 s. This works just fine on
> a LAN, but I would never do this on an outside circuit.

Setting ntp-dev 4.2.5p113 to minpoll 3 doesn't seem to work well under
Linux. At startup, the offset doesn't converge to zero, the frequency
remains constant, and in loopstats the stated offset is 0.000000000 and
the wander is 0.000001.

Normal behaviour and true loopstats values come back soon after the poll
interval ramps up to 16 seconds, around 7 minutes after startup.


Serge.
--
Serge point Bets arobase laposte point net

Serge,

The ntp-dev version says 5p113, but I can't confirm this is the same as
the current snapshot. In any case, you need to set the minimum average
headway/system minimum poll interval to 3 in both the server and client;
otherwise, the server will declare a rate violation and toss you a KoD.
Use the discard average 3 command in both the server and client. More in
the web documentation on the Rate Management and Kiss-o'-Death page.
Also, you can find an informal report on recent code upgrades at
www.eecis.udel.edu/~mills/upgrade.txt.

Dave

Serge Bets wrote:
> Hello David,
>
> On Monday, February 11, 2008 at 19:03:36 +0000, David L. Mills wrote:
>
>
>>While not admitted in public, the latest snapshot can set the poll
>>interval to 3 (8 s), so the risetime is 250 s. This works just fine on
>>a LAN, but I would never do this on an outside circuit.
>
>
> Setting ntp-dev 4.2.5p113 to minpoll 3 doesn't seem to work well under
> Linux. At startup, the offset doesn't converge to zero, the frequency
> remains constant, and in loopstats the stated offset is 0.000000000 and
> the wander is 0.000001.
>
> Normal behaviour and true loopstats values come back soon after the poll
> interval ramps up to 16 seconds, around 7 minutes after startup.
>
>
> Serge.

Hello David,

On Tuesday, March 18, 2008 at 2:44:12 +0000, David L. Mills wrote:

> you need to set the minimum average headway/system minimum poll
> interval to 3 in both the server and client; otherwise, the server
> will declare a rate violation and toss you a KoD. Use the discard
> average 3 command in both the server and client.

Thank you: I had no discard command. So now I tried on both the server
and the client:

| discard average 3
| server minpoll 3

Still same symptoms: The machines don't discipline their clocks. Until
the poll interval ramps to 16 seconds on the client. Forever on the
server, whose refclock sticks to minpoll. And ntptime shows time
constant 0 for minpoll 3 as well as for minpoll 4.


Serge.
--
Serge point Bets arobase laposte point net