Ruh, roh. Leap Second? - NTP

This is a discussion on Ruh, roh. Leap Second? - NTP ; 1 Time(s): Clock: inserting leap second 23:59:60 UTC RHEL 5.2 system running ntpq 4.2.2p1@1.1570-o Thu Jan 17 18:14:14 UTC 2008 (1) which is the default for that distribution. Grepping around in the logs it appears that most or all of ...

+ Reply to Thread
Results 1 to 16 of 16

Thread: Ruh, roh. Leap Second?

  1. Ruh, roh. Leap Second?

    1 Time(s): Clock: inserting leap second 23:59:60 UTC

    RHEL 5.2 system running ntpq 4.2.2p1@1.1570-o Thu Jan 17 18:14:14 UTC 2008
    (1) which is the default for that distribution. Grepping around in the
    logs it appears that most or all of my RHEL systems did it.

    I got this, too, on at least one system:

    Jun 30 19:34:53 ozz-1300 ntpd[2659]: time reset +1.000360 s


    Thing is, IERS, who I understand is in charge of these things, says: "There
    will NOT be a leap second introduced in UTC on 30 June 2008."
    (http://maia.usno.navy.mil/)

    So was that bogus? A bug in RH's NTP code? A problem with NTP itself?

    Operator error? :-)


    --
    Peter Laws / N5UWY
    National Weather Center / Network Operations Center
    University of Oklahoma Information Technology
    plaws@ou.edu
    -----------------------------------------------------------------------
    Feedback? Contact my director, Craig Cochell, craigc@ou.edu. Thank you!

  2. Re: Ruh, roh. Leap Second?

    Once upon a time, Peter Laws said:
    >1 Time(s): Clock: inserting leap second 23:59:60 UTC
    >
    >RHEL 5.2 system running ntpq 4.2.2p1@1.1570-o Thu Jan 17 18:14:14 UTC 2008
    >(1) which is the default for that distribution. Grepping around in the
    >logs it appears that most or all of my RHEL systems did it.
    >
    >I got this, too, on at least one system:
    >
    >Jun 30 19:34:53 ozz-1300 ntpd[2659]: time reset +1.000360 s


    You should be more specific with versions on RHEL (or any RPM based
    distribution); do "rpm -q ntp" to get the full NVR. For example, RHEL
    5.2 systems should have ntp-4.2.2p1-8.el5.

    In any case, I just checked RHEL 5.2, 5.1, 4.6, and 3.9 systems, and
    none show a leap second or a time reset log entry.

    --
    Chris Adams
    Systems and Network Administrator - HiWAAY Internet Services
    I don't speak for anybody but myself - that's enough trouble.

  3. Re: Ruh, roh. Leap Second?

    Peter,

    Peter Laws wrote:
    > 1 Time(s): Clock: inserting leap second 23:59:60 UTC
    >
    > RHEL 5.2 system running ntpq 4.2.2p1@1.1570-o Thu Jan 17 18:14:14 UTC 2008
    > (1) which is the default for that distribution. Grepping around in the
    > logs it appears that most or all of my RHEL systems did it.
    >
    > I got this, too, on at least one system:
    >
    > Jun 30 19:34:53 ozz-1300 ntpd[2659]: time reset +1.000360 s
    >
    >
    > Thing is, IERS, who I understand is in charge of these things, says:
    > "There will NOT be a leap second introduced in UTC on 30 June 2008."
    > (http://maia.usno.navy.mil/)
    >
    > So was that bogus? A bug in RH's NTP code? A problem with NTP itself?
    >
    > Operator error? :-)


    Unless there is a bug in that specific version of ntpd you are running, ntpd
    only propagates leap second announcements if it has received such an
    announcement from either an upstream NTP server or reference clock, or from
    a leap seconds file.

    Which upstream servers are you using?

    Are you using any local reference clock, e.g. a GPS receiver?

    Your ntpd may also have received a faulty leap second announcement from an
    upstream server which in turn has received the announcement from its time
    source.

    For a short summary how leap seconds could/should be handled, see:
    http://www.meinberg.de/english/info/leap-second.htm

    Martin
    --
    Martin Burnicki

    Meinberg Funkuhren
    Bad Pyrmont
    Germany

  4. Re: Ruh, roh. Leap Second?

    Martin Burnicki wrote:

    > Unless there is a bug in that specific version of ntpd you are running, ntpd
    > only propagates leap second announcements if it has received such an
    > announcement from either an upstream NTP server or reference clock, or from
    > a leap seconds file.
    >
    > Which upstream servers are you using?
    >
    > Are you using any local reference clock, e.g. a GPS receiver?


    Of course. I have two TymServe 2100s and another GPS clock which I admit
    to never having seen. :-) The servers in question also get time from two
    offsite Stratum 2 servers (louie.udel.edu and ntp1.kansas.net) and are
    themselves peered together



    > For a short summary how leap seconds could/should be handled, see:
    > http://www.meinberg.de/english/info/leap-second.htm


    Thanks, Martin.

    --
    Peter Laws / N5UWY
    National Weather Center / Network Operations Center
    University of Oklahoma Information Technology
    plaws@ou.edu
    -----------------------------------------------------------------------
    Feedback? Contact my director, Craig Cochell, craigc@ou.edu. Thank you!

  5. Re: Ruh, roh. Leap Second?

    Peter,

    I head the same comment over the jungle telegraph. However, in the
    distributions that leave here there is no such comment, so it would have
    to come from somewhere else or from a modified distribution. To help
    spot feckless fingers, recent versions have a protostats file in the
    statistics collection and "official" reports go there, as well as the
    system log if configured. These reports are described in the decode.html
    page in the docuentation.

    Dave

    Peter Laws wrote:
    > 1 Time(s): Clock: inserting leap second 23:59:60 UTC
    >
    > RHEL 5.2 system running ntpq 4.2.2p1@1.1570-o Thu Jan 17 18:14:14 UTC 2008
    > (1) which is the default for that distribution. Grepping around in the
    > logs it appears that most or all of my RHEL systems did it.
    >
    > I got this, too, on at least one system:
    >
    > Jun 30 19:34:53 ozz-1300 ntpd[2659]: time reset +1.000360 s
    >
    >
    > Thing is, IERS, who I understand is in charge of these things, says: "There
    > will NOT be a leap second introduced in UTC on 30 June 2008."
    > (http://maia.usno.navy.mil/)
    >
    > So was that bogus? A bug in RH's NTP code? A problem with NTP itself?
    >
    > Operator error? :-)
    >
    >


  6. Re: Ruh, roh. Leap Second?

    Dave,

    David L. Mills wrote:
    > Peter,
    >
    > I head the same comment over the jungle telegraph. However, in the
    > distributions that leave here there is no such comment, so it would have
    > to come from somewhere else or from a modified distribution.


    The message:

    Clock: inserting leap second 23:59:60 UTC

    can be found in the Linux kernel sources. So this indicates ntpd has indeed
    received a leap second announcement from an upstream time source and has
    passed that announcement to the kernel which has then inserted the leap
    second. AFAIK that's the way leap seconds should be handled on systems
    which provide a kernel PLL.

    The following "time reset" happened after that to undo the faulty leap
    second and step back to the correct time.

    So the basic question is from which upstream time source the faulty leap
    second announcement has been received.

    > To help
    > spot feckless fingers, recent versions have a protostats file in the
    > statistics collection and "official" reports go there, as well as the
    > system log if configured. These reports are described in the decode.html
    > page in the docuentation.


    I'm not familiar with the protostats file, but I assume it is only generated
    if explicitely configured in ntp.conf, similar to the other stats files.

    Since such faulty leap second announcements have been reported here several
    times I'd really appreciate if there would be a log entry generated by
    default if a leap second announcement is received. That log entry should
    also indicate from which time source that announcement has been received,
    so a faulty time source could easily be identified and fixed or discarded
    in order to prevent those faults the next time.

    Martin
    --
    Martin Burnicki

    Meinberg Funkuhren
    Bad Pyrmont
    Germany

  7. Re: Ruh, roh. Leap Second?

    Martin,

    There can be a disconnect of serveral months between some gimmick in the
    development branch and when it appears on supermarket shelves; however,
    the current version does explicitly announce impending leaps in the
    protostats file and does take a vote among the sources on whether to
    mark the calendar for a leap. All this is overridden if the NIST
    leapseconds file is present.

    Dave

    Martin Burnicki wrote:
    > Dave,
    >
    > David L. Mills wrote:
    >
    >>Peter,
    >>
    >>I head the same comment over the jungle telegraph. However, in the
    >>distributions that leave here there is no such comment, so it would have
    >>to come from somewhere else or from a modified distribution.

    >
    >
    > The message:
    >
    > Clock: inserting leap second 23:59:60 UTC
    >
    > can be found in the Linux kernel sources. So this indicates ntpd has indeed
    > received a leap second announcement from an upstream time source and has
    > passed that announcement to the kernel which has then inserted the leap
    > second. AFAIK that's the way leap seconds should be handled on systems
    > which provide a kernel PLL.
    >
    > The following "time reset" happened after that to undo the faulty leap
    > second and step back to the correct time.
    >
    > So the basic question is from which upstream time source the faulty leap
    > second announcement has been received.
    >
    >
    >>To help
    >>spot feckless fingers, recent versions have a protostats file in the
    >>statistics collection and "official" reports go there, as well as the
    >>system log if configured. These reports are described in the decode.html
    >>page in the docuentation.

    >
    >
    > I'm not familiar with the protostats file, but I assume it is only generated
    > if explicitely configured in ntp.conf, similar to the other stats files.
    >
    > Since such faulty leap second announcements have been reported here several
    > times I'd really appreciate if there would be a log entry generated by
    > default if a leap second announcement is received. That log entry should
    > also indicate from which time source that announcement has been received,
    > so a faulty time source could easily be identified and fixed or discarded
    > in order to prevent those faults the next time.
    >
    > Martin


  8. Re: Ruh, roh. Leap Second?

    In article <486A32E6.4050201@ou.edu>,
    plaws@ou.edu (Peter Laws) writes:
    >1 Time(s): Clock: inserting leap second 23:59:60 UTC
    >
    >RHEL 5.2 system running ntpq 4.2.2p1@1.1570-o Thu Jan 17 18:14:14 UTC 2008
    >(1) which is the default for that distribution. Grepping around in the
    >logs it appears that most or all of my RHEL systems did it.
    >
    >I got this, too, on at least one system:
    >
    >Jun 30 19:34:53 ozz-1300 ntpd[2659]: time reset +1.000360 s



    Humm. I got one too, back in 2005, 6 months ahead of the last leap second.

    I wonder what was going on back then,


    Jun 21 01:09:33 shuksan ntpd[1794]: ntpd 4.2.0a@1.1190-r Thu Apr 14 07:47:25 EDT 2005 (1)

    Jun 30 16:59:59 shuksan kernel: Clock: inserting leap second 23:59:60 UTC
    Jun 30 17:57:14 shuksan ntpd[1794]: time reset +1.014226 s

    --
    These are my opinions, not necessarily my employer's. I hate spam.


  9. Re: Ruh, roh. Leap Second?

    Dave,

    David L. Mills wrote:
    > Martin,
    >
    > There can be a disconnect of serveral months between some gimmick in the
    > development branch and when it appears on supermarket shelves;


    Of course.

    > however,
    > the current version does explicitly announce impending leaps in the
    > protostats file and does take a vote among the sources on whether to
    > mark the calendar for a leap. All this is overridden if the NIST
    > leapseconds file is present.


    Understood. However, does it also write to the log file from *which*
    source(s) a leap second announcement has been received?

    Martin
    --
    Martin Burnicki

    Meinberg Funkuhren
    Bad Pyrmont
    Germany

  10. Re: Ruh, roh. Leap Second?

    Hal,

    Hal Murray wrote:
    > In article <486A32E6.4050201@ou.edu>,
    > plaws@ou.edu (Peter Laws) writes:
    >>1 Time(s): Clock: inserting leap second 23:59:60 UTC
    >>
    >>RHEL 5.2 system running ntpq 4.2.2p1@1.1570-o Thu Jan 17 18:14:14 UTC 2008
    >>(1) which is the default for that distribution. Grepping around in the
    >>logs it appears that most or all of my RHEL systems did it.
    >>
    >>I got this, too, on at least one system:
    >>
    >>Jun 30 19:34:53 ozz-1300 ntpd[2659]: time reset +1.000360 s

    >
    >
    > Humm. I got one too, back in 2005, 6 months ahead of the last leap
    > second.
    >
    > I wonder what was going on back then,
    >
    >
    > Jun 21 01:09:33 shuksan ntpd[1794]: ntpd 4.2.0a@1.1190-r Thu Apr 14
    > 07:47:25 EDT 2005 (1)
    >
    > Jun 30 16:59:59 shuksan kernel: Clock: inserting leap second 23:59:60 UTC
    > Jun 30 17:57:14 shuksan ntpd[1794]: time reset +1.014226 s


    If ntpd had logged the source from which it had received the leap second
    announcement then we would have a clue ...

    Martin
    --
    Martin Burnicki

    Meinberg Funkuhren
    Bad Pyrmont
    Germany

  11. Re: Ruh, roh. Leap Second?

    Martin Burnicki wrote:

    > If ntpd had logged the source from which it had received the leap second
    > announcement then we would have a clue ...


    Which would be helpful.

    Looks like I had at least 14 systems report a leap second insertion.

    All of them get time from either of two sets of peered systems I maintain.
    Both of those sets get their time from the aforementioned GPS clocks (2
    TymServe 2100 GPS clocks and another unknown GPS clock). One set also uses
    louie.udel.edu and ntp1.kansas.net, while the other does NOT.

    Both sets of peers got leap second insertions.

    All are running RHEL 4 or 5 and are reasonably current on patches. None
    use anything but the NTP version distributed with RHEL.

    Since one set of peers only uses the three GPS clocks as references, is it
    reasonable to assume that GPS is the source of the leap second?

    Or might it be something related to the GPS clock units themselves? One of
    the TymServes has the last firmware available, while the other doesn't.
    The other is, as mentioned, unknown.





    --
    Peter Laws / N5UWY
    National Weather Center / Network Operations Center
    University of Oklahoma Information Technology
    plaws@ou.edu
    -----------------------------------------------------------------------
    Feedback? Contact my director, Craig Cochell, craigc@ou.edu. Thank you!

  12. Re: Ruh, roh. Leap Second?

    Peter Laws wrote:


    > All are running RHEL 4 or 5 and are reasonably current on patches. None
    > use anything but the NTP version distributed with RHEL.


    Looks like my brand new, not-yet-in-service DNS/DHCP appliances recorded
    the leap second as well, so that effectively eliminates RHEL, since none of
    the appliances talk to anyone but themselves and those same three clocks.

    Since none of y'all got leaped GPS is probably not the culprit ... So, I
    guess I need at least update the other TymServ 2100 to the last version of
    it's code and then to determine what that unknown GPS clock is and upgrade
    it if I can.

    Nobody else got leaped the other day? *No* one??


    --
    Peter Laws / N5UWY
    National Weather Center / Network Operations Center
    University of Oklahoma Information Technology
    plaws@ou.edu
    -----------------------------------------------------------------------
    Feedback? Contact my director, Craig Cochell, craigc@ou.edu. Thank you!

  13. Re: Ruh, roh. Leap Second?

    Peter,

    Peter Laws wrote:
    > Peter Laws wrote:
    >
    >
    >> All are running RHEL 4 or 5 and are reasonably current on patches. None
    >> use anything but the NTP version distributed with RHEL.

    >
    > Looks like my brand new, not-yet-in-service DNS/DHCP appliances recorded
    > the leap second as well, so that effectively eliminates RHEL, since none
    > of the appliances talk to anyone but themselves and those same three
    > clocks.


    That's also my conclusion.

    > Since none of y'all got leaped GPS is probably not the culprit ...


    The GPS satellites have not announced a leap second for the end of June, so
    either (at least) one of the GPS receivers seems to have a firmware bug, or
    at least one of your timeservers can be configured to insert a leap second
    at a predefined date, and has been configured in a wrong way.

    > So, I
    > guess I need at least update the other TymServ 2100 to the last version of
    > it's code and then to determine what that unknown GPS clock is and upgrade
    > it if I can.
    >
    > Nobody else got leaped the other day? *No* one??


    At least I haven't heard about any other wrong leap second event.

    Martin
    --
    Martin Burnicki

    Meinberg Funkuhren
    Bad Pyrmont
    Germany

  14. Re: Ruh, roh. Leap Second?

    David L. Mills schreef:

    > the current version [...] does take a vote among the sources on
    > whether to mark the calendar for a leap. All this is overridden if
    > the NIST leapseconds file is present.


    How interesting. I had a question about this in another thread. Can I
    take this to mean that *if* you have configured a leapseconds file, it
    will be the *only* source of information used? If yes, perhaps this
    could be mentioned explicitly in the docs. I believe that is not the
    case right now.

    N

  15. Re: Ruh, roh. Leap Second?

    Nero,

    I think it pretty clear what "overridden" means. Additional information
    on leap seconds is in the ntpd documentation on the web.

    Dave

    Nero Imhard wrote:
    > David L. Mills schreef:
    >
    >> the current version [...] does take a vote among the sources on
    >> whether to mark the calendar for a leap. All this is overridden if
    >> the NIST leapseconds file is present.

    >
    >
    > How interesting. I had a question about this in another thread. Can I
    > take this to mean that *if* you have configured a leapseconds file, it
    > will be the *only* source of information used? If yes, perhaps this
    > could be mentioned explicitly in the docs. I believe that is not the
    > case right now.
    >
    > N


  16. Re: Ruh, roh. Leap Second?

    Martin,

    Yes.

    Dave

    Martin Burnicki wrote:
    > Dave,
    >
    > David L. Mills wrote:
    >
    >>Martin,
    >>
    >>There can be a disconnect of serveral months between some gimmick in the
    >>development branch and when it appears on supermarket shelves;

    >
    >
    > Of course.
    >
    >
    >>however,
    >>the current version does explicitly announce impending leaps in the
    >>protostats file and does take a vote among the sources on whether to
    >>mark the calendar for a leap. All this is overridden if the NIST
    >>leapseconds file is present.

    >
    >
    > Understood. However, does it also write to the log file from *which*
    > source(s) a leap second announcement has been received?
    >
    > Martin


+ Reply to Thread