Bad clock on a linux server? - Hardware

This is a discussion on Bad clock on a linux server? - Hardware ; This probably is not a linux question. I have a server (one of 16 linux prod. servers) and its clocks drifts A LOT. Over exactly 10 minutes, it drifted by 9 seconds!!! (appx. one second per minute) The drift rate ...

+ Reply to Thread
Results 1 to 8 of 8

Thread: Bad clock on a linux server?

  1. Bad clock on a linux server?

    This probably is not a linux question.

    I have a server (one of 16 linux prod. servers) and its clocks drifts
    A LOT. Over exactly 10 minutes, it drifted by 9 seconds!!! (appx. one
    second per minute)

    The drift rate is truly alarming!

    The server is not particularly busy, it is an 8 CPU machine, 0.84 load
    average.

    We have several of these machines and the rest seem to perform fine.

    I disabled NTP service for right now to see how it goes without NTP
    interfering. And that's how I get this drift rate, without NTP
    involved. (normally all my servers run NTP)

    Any ideas?
    --
    Due to extreme spam originating from Google Groups, and their inattention
    to spammers, I and many others block all articles originating
    from Google Groups. If you want your postings to be seen by
    more readers you will need to find a different means of
    posting on Usenet.
    http://improve-usenet.org/

  2. Re: Bad clock on a linux server?

    Ignoramus32753 wrote:
    > This probably is not a linux question.
    >
    > I have a server (one of 16 linux prod. servers) and its clocks drifts
    > A LOT. Over exactly 10 minutes, it drifted by 9 seconds!!! (appx. one
    > second per minute)
    >
    > The drift rate is truly alarming!
    >
    > The server is not particularly busy, it is an 8 CPU machine, 0.84 load
    > average.
    >
    > We have several of these machines and the rest seem to perform fine.
    >
    > I disabled NTP service for right now to see how it goes without NTP
    > interfering. And that's how I get this drift rate, without NTP
    > involved. (normally all my servers run NTP)
    >
    > Any ideas?


    crystal is simple edge of spec.

    Run NTP, or simply get another motherboard.

    Is this the RTC or the 'when I am booted' clock?

    The latter is not highly specced anyway.

  3. Re: Bad clock on a linux server?

    Ignoramus32753 wrote:
    > This probably is not a linux question.
    >
    > I have a server (one of 16 linux prod. servers) and its clocks drifts
    > A LOT. Over exactly 10 minutes, it drifted by 9 seconds!!! (appx. one
    > second per minute)
    >
    > The drift rate is truly alarming!
    >
    > The server is not particularly busy, it is an 8 CPU machine, 0.84 load
    > average.
    >
    > We have several of these machines and the rest seem to perform fine.
    >
    > I disabled NTP service for right now to see how it goes without NTP
    > interfering. And that's how I get this drift rate, without NTP
    > involved. (normally all my servers run NTP)
    >
    > Any ideas?


    Yes. You might try deleting the file /etc/adjtime (it will come back). If it
    is a little off, it will slowly get to be right, but if it is way off, all
    kinds of perplexing things happen. There is also one called drift (that can
    be anywhere) maintained by ntpd that you might wish to zap.

    --
    .~. Jean-David Beyer Registered Linux User 85642.
    /V\ PGP-Key: 9A2FC99A Registered Machine 241939.
    /( )\ Shrewsbury, New Jersey http://counter.li.org
    ^^-^^ 11:30:01 up 1 day, 17:36, 4 users, load average: 4.13, 4.48, 4.42

  4. Re: Bad clock on a linux server?

    On 2008-08-08, mblume wrote:
    > Am Fri, 08 Aug 2008 09:36:15 -0500 schrieb Ignoramus32753:
    >> This probably is not a linux question.
    >>
    >> I have a server (one of 16 linux prod. servers) and its clocks drifts A
    >> LOT. Over exactly 10 minutes, it drifted by 9 seconds!!! (appx. one
    >> second per minute)
    >>
    >> The drift rate is truly alarming!
    >>

    > May be a hardware problem.
    > AFAIK this may also happen if your /etc/adjtime specfies bad values.
    > Delete /etc/adjtime, set the time manually (if your processes on this machine
    > allow time jumps) and see whether it gets better.
    >


    Here's my adjtime from that machine:

    badmachine:root:/etc ==] cat adjtime
    0.170262 1218171731 0.000000
    1218171731
    UTC

    Here's one from a good machine

    goodmachine:root:~ ==] cat /etc/adjtime
    37.896042 1218210127 0.000000
    1218210127
    UTC

    Can I somehow effect changes WITHOUT rebooting badmachine? It runs
    something important and cannot be rebooted for a few hours.

    --
    Due to extreme spam originating from Google Groups, and their inattention
    to spammers, I and many others block all articles originating
    from Google Groups. If you want your postings to be seen by
    more readers you will need to find a different means of
    posting on Usenet.
    http://improve-usenet.org/

  5. Re: Bad clock on a linux server?

    Ignoramus32753 writes:

    >This probably is not a linux question.


    >I have a server (one of 16 linux prod. servers) and its clocks drifts
    >A LOT. Over exactly 10 minutes, it drifted by 9 seconds!!! (appx. one
    >second per minute)


    >The drift rate is truly alarming!


    What clock is the system using? Try some of the others. See the
    /usr/src/linux/Documentation/kenel-source.txt and look for clock-source
    parameter.
    Or maybe sombody instituted a different clock source for that server and
    forgot to tell anyone.



    >The server is not particularly busy, it is an 8 CPU machine, 0.84 load
    >average.


    >We have several of these machines and the rest seem to perform fine.


    >I disabled NTP service for right now to see how it goes without NTP
    >interfering. And that's how I get this drift rate, without NTP
    >involved. (normally all my servers run NTP)


    ntp cannot correct a drift rate of 15000 PPM. Its max ability is 500PPM.

    >Any ideas?
    >--
    > Due to extreme spam originating from Google Groups, and their inattention
    > to spammers, I and many others block all articles originating
    > from Google Groups. If you want your postings to be seen by
    > more readers you will need to find a different means of
    > posting on Usenet.
    > http://improve-usenet.org/


  6. Re: Bad clock on a linux server?

    The Natural Philosopher writes:

    >Ignoramus32753 wrote:
    >> This probably is not a linux question.
    >>
    >> I have a server (one of 16 linux prod. servers) and its clocks drifts
    >> A LOT. Over exactly 10 minutes, it drifted by 9 seconds!!! (appx. one
    >> second per minute)
    >>
    >> The drift rate is truly alarming!
    >>
    >> The server is not particularly busy, it is an 8 CPU machine, 0.84 load
    >> average.
    >>
    >> We have several of these machines and the rest seem to perform fine.
    >>
    >> I disabled NTP service for right now to see how it goes without NTP
    >> interfering. And that's how I get this drift rate, without NTP
    >> involved. (normally all my servers run NTP)
    >>
    >> Any ideas?


    >crystal is simple edge of spec.


    Sorry, no, that crystal is way way out of spec. .015 error is nowhere near
    spec.

    >Run NTP, or simply get another motherboard.


    NTP cannot correct a clock that far out. It is limited by 500PPM.

    >Is this the RTC or the 'when I am booted' clock?


    >The latter is not highly specced anyway.


    It is definitely speced better than that.


  7. Re: Bad clock on a linux server?

    Ignoramus32753 writes:

    >On 2008-08-08, mblume wrote:
    >> Am Fri, 08 Aug 2008 09:36:15 -0500 schrieb Ignoramus32753:
    >>> This probably is not a linux question.
    >>>
    >>> I have a server (one of 16 linux prod. servers) and its clocks drifts A
    >>> LOT. Over exactly 10 minutes, it drifted by 9 seconds!!! (appx. one
    >>> second per minute)
    >>>
    >>> The drift rate is truly alarming!
    >>>

    >> May be a hardware problem.
    >> AFAIK this may also happen if your /etc/adjtime specfies bad values.
    >> Delete /etc/adjtime, set the time manually (if your processes on this machine
    >> allow time jumps) and see whether it gets better.
    >>


    >Here's my adjtime from that machine:


    >badmachine:root:/etc ==] cat adjtime
    >0.170262 1218171731 0.000000
    >1218171731
    >UTC


    >Here's one from a good machine


    >goodmachine:root:~ ==] cat /etc/adjtime
    >37.896042 1218210127 0.000000
    >1218210127
    >UTC


    >Can I somehow effect changes WITHOUT rebooting badmachine? It runs
    >something important and cannot be rebooted for a few hours.


    No. adjtime is for translation of the rtc (real time clock) not the system
    clock. For the system clock, you need something like ntp or chrony (for
    yuor situation chrony might work, since it does not have the same limits on
    the PPM it will adjust, but I suspect 15000 PPM is still not handleable.)

    It could also be lost ticks-- for some reason your system is loosing timer
    tick interrupts. If those machines run at 100 Hz, only one or two a second
    would give your results. This could occur if the machines were overloading
    their irq, causing something to hog the irqs for over 1/100of a sec. Since
    we do not know what that crucial thing that is running is doing, maybe it
    is responsible. You could try using the hpet as the timer rathr than the
    jiffies, that is if your system is new enough to have an hpet.
    But that requires a reboot
    ..




    >--
    > Due to extreme spam originating from Google Groups, and their inattention
    > to spammers, I and many others block all articles originating
    > from Google Groups. If you want your postings to be seen by
    > more readers you will need to find a different means of
    > posting on Usenet.
    > http://improve-usenet.org/


  8. Re: Bad clock on a linux server?

    On 2008-08-08, Unruh wrote:
    > Ignoramus32753 writes:
    >
    >>This probably is not a linux question.

    >
    >>I have a server (one of 16 linux prod. servers) and its clocks drifts
    >>A LOT. Over exactly 10 minutes, it drifted by 9 seconds!!! (appx. one
    >>second per minute)

    >
    >>The drift rate is truly alarming!

    >
    > What clock is the system using? Try some of the others. See the
    > /usr/src/linux/Documentation/kenel-source.txt and look for clock-source
    > parameter.


    Which file, again?

    > Or maybe sombody instituted a different clock source for that server and
    > forgot to tell anyone.


    They are all set up the same way by my script.


    >
    >
    >>The server is not particularly busy, it is an 8 CPU machine, 0.84 load
    >>average.

    >
    >>We have several of these machines and the rest seem to perform fine.

    >
    >>I disabled NTP service for right now to see how it goes without NTP
    >>interfering. And that's how I get this drift rate, without NTP
    >>involved. (normally all my servers run NTP)

    >
    > ntp cannot correct a drift rate of 15000 PPM. Its max ability is 500PPM.
    >
    >>Any ideas?


    --
    Due to extreme spam originating from Google Groups, and their inattention
    to spammers, I and many others block all articles originating
    from Google Groups. If you want your postings to be seen by
    more readers you will need to find a different means of
    posting on Usenet.
    http://improve-usenet.org/

+ Reply to Thread