On Monday, 17 of March 2008, Linus Torvalds wrote:
>
> On Mon, 17 Mar 2008, Rafael J. Wysocki wrote:
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=9983
> > Subject : PROBLEM: 2.6.25-rc1-git2 freezes when accessing external USB hard disk (ehci-hcd)
> > Submitter : Linas ®virblis <0x0007@gmail.com>
> > Date : 2008-02-13 22:38 (33 days old)
> > References : http://lkml.org/lkml/2008/2/13/566
>
> This is most likely already fixed by commit
> e82cc1288fa57857c6af8c57f3d07096d4bcd9d9.
>
> Unless Linas can reproduce it with a newer kernel (I'm cutting an -rc6
> right now, but any -git snapshot in the last few days should work) this
> one should be closed. We can't keep things open just because the tester
> hasn't tested.

Sure, I'll close it if there's no response in a couple of days.

Thanks,
Rafael
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Mon, 17 Mar 2008, Rafael J. Wysocki wrote:
>
> Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=9983
> Subject : PROBLEM: 2.6.25-rc1-git2 freezes when accessing external USB hard disk (ehci-hcd)
> Submitter : Linas ®virblis <0x0007@gmail.com>
> Date : 2008-02-13 22:38 (33 days old)
> References : http://lkml.org/lkml/2008/2/13/566

This is most likely already fixed by commit
e82cc1288fa57857c6af8c57f3d07096d4bcd9d9.

Unless Linas can reproduce it with a newer kernel (I'm cutting an -rc6
right now, but any -git snapshot in the last few days should work) this
one should be closed. We can't keep things open just because the tester
hasn't tested.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Rafael J. Wysocki wrote:

Hi,

> This message contains a list of some regressions from 2.6.24 reported since
> 2.6.25-rc1 was released, for which there are no fixes in the mainline I know
> of. If any of them have been fixed already, please let me know.

> Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10152
> Subject : Clocksource tsc is always unstable with 2.6.25-* kernels and CONFIG_NO_HZ=y on my box
> Submitter : Gabriel C
> Date : 2008-02-24 01:31 (22 days old)
> References : http://lkml.org/lkml/2008/2/23/380
> http://lkml.org/lkml/2008/2/24/281
> Handled-By : Thomas Gleixner
>

Thomas do you want me to bisect ?

Or do you have any patches I could try ( really does not matter how experimental they are ) ?

Rafael the bug report is saying x86-64 Component while my box is 32bit Could you please correct this ?

Best Regards

Gabriel



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Mon, 17 Mar 2008, Gabriel C wrote:
> > Subject : Clocksource tsc is always unstable with 2.6.25-* kernels and CONFIG_NO_HZ=y on my box
> > Submitter : Gabriel C
> > Date : 2008-02-24 01:31 (22 days old)
> > References : http://lkml.org/lkml/2008/2/23/380
> > http://lkml.org/lkml/2008/2/24/281
> > Handled-By : Thomas Gleixner
> >
>
> Thomas do you want me to bisect ?

That'd be great.

> Or do you have any patches I could try ( really does not matter how experimental they are ) ?

No, I have not the lightest clue whats going on.

Thanks,

tglx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Thomas Gleixner wrote:
> On Mon, 17 Mar 2008, Gabriel C wrote:
>>> Subject : Clocksource tsc is always unstable with 2.6.25-* kernels and CONFIG_NO_HZ=y on my box
>>> Submitter : Gabriel C
>>> Date : 2008-02-24 01:31 (22 days old)
>>> References : http://lkml.org/lkml/2008/2/23/380
>>> http://lkml.org/lkml/2008/2/24/281
>>> Handled-By : Thomas Gleixner
>>>
>> Thomas do you want me to bisect ?
>
> That'd be great.

Ok I'll start doing that later on today.

>
>> Or do you have any patches I could try ( really does not matter how experimental they are ) ?
>
> No, I have not the lightest clue whats going on.
>
> Thanks,
>
> tglx
>

Gabriel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Monday, 17 of March 2008, Jason Wu wrote:
> 2008/3/17, Rafael J. Wysocki :
> > This message contains a list of some regressions from 2.6.24 reported since
> > 2.6.25-rc1 was released, for which there are no fixes in the mainline I know
> > of. If any of them have been fixed already, please let me know.
> >
> > If you know of any other unresolved regressions from 2.6.24, please let me know
> > either and I'll add them to the list. Also, please let me know if any of the
> > entries below are invalid.
> >
> >
> > Listed regressions statistics:
> >
> > Date Total Pending Unresolved
> > ----------------------------------------
> > 2008-03-17 148 38 30
> > 2008-03-16 146 42 35
> > 2008-03-14 145 45 39
> > 2008-03-12 143 51 41
> > 2008-03-11 141 58 43
> > 2008-03-10 138 66 47
> > 2008-03-03 115 65 49
> > 2008-02-25 90 51 39
> > 2008-02-17 61 45 37
> >
> >
> > Unresolved regressions
> > ----------------------
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=9962
> > Subject : mount: could not find filesystem
> > Submitter : Kamalesh Babulal
> > Date : 2008-02-12 14:34 (34 days old)
> > References : http://lkml.org/lkml/2008/2/12/91
> > Handled-By : Bartlomiej Zolnierkiewicz
> > Yinghai Lu
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=9976
> > Subject : BUG: 2.6.25-rc1: iptables postrouting setup causes oops
> > Submitter : Ben Nizette
> > Date : 2008-02-12 12:46 (34 days old)
> > References : http://lkml.org/lkml/2008/2/12/148
> > Handled-By : Haavard Skinnemoen
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=9978
> > Subject : 2.6.25-rc1: volanoMark regression
> > Submitter : Zhang, Yanmin
> > Date : 2008-02-13 10:30 (33 days old)
> > References : http://lkml.org/lkml/2008/2/13/128
> > http://lkml.org/lkml/2008/3/12/52
> > Handled-By : Srivatsa Vaddagiri
> > Balbir Singh
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=9980
> > Subject : 2.6.25-rc1 on Sun Ultra 40- HPET clocksource which causes it to hang
> > Submitter : Jasper Bryant-Greene
> > Date : 2008-02-13 12:25 (33 days old)
> > References : http://lkml.org/lkml/2008/2/13/181
> > Handled-By : Yinghai Lu
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=9983
> > Subject : PROBLEM: 2.6.25-rc1-git2 freezes when accessing external USB hard disk (ehci-hcd)
> > Submitter : Linas ®virblis <0x0007@gmail.com>
> > Date : 2008-02-13 22:38 (33 days old)
> > References : http://lkml.org/lkml/2008/2/13/566
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=9995
> > Subject : 2.6.25-rc1 regression - backlight controlls do not work - ThinkPad T61
> > Submitter : Lukas Hejtmanek
> > Date : 2008-02-15 04:51 (31 days old)
> > Handled-By : Zhang Rui
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10011
> > Subject : The computer is blocked when X is started - unless max_cstate=2 - Acer Travelmate 4001 Lmi
> > Submitter : François Valenduc
> > Date : 2008-02-17 06:28 (29 days old)
> > Handled-By : Thomas Gleixner
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10027
> > Subject : 2.6.25-rc[12] Video4Linux Bttv Regression
> > Submitter : Bongani Hlope
> > Date : 2008-02-17 09:36 (29 days old)
> > References : http://lkml.org/lkml/2008/2/17/55
> > Handled-By : Mauro Carvalho Chehab
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10041
> > Subject : 2.6.25-rc1/2 regression: first-time login into gnome fails
> > Submitter : Romano Giannetti
> > Date : 2008-02-18 11:56 (28 days old)
> > References : http://lkml.org/lkml/2008/2/18/145
> > Handled-By : Ray Lee
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10051
> > Subject : Spurious messages at boot, eventually hangs the usb subsustem
> > Submitter : Jean-Luc Coulon
> > Date : 2008-02-20 09:10 (26 days old)
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10065
> > Subject : 2.6.25-rc2 regression - hang on suspend
> > Submitter : Soeren Sonnenburg
> > Date : 2008-02-19 12:59 (27 days old)
> > References : http://lkml.org/lkml/2008/2/19/165
> > http://lkml.org/lkml/2008/2/17/381
> > Handled-By : Rafael J. Wysocki
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10067
> > Subject : TUNER_TDA8290=y, VIDEO_DEV=n build error
> > Submitter : Toralf Förster
> > Date : 2008-02-22 10:36 (24 days old)
> > References : http://lkml.org/lkml/2008/2/19/262
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10082
> > Subject : 2.6.25-rc2-git4 - Kernel oops while running kernbench and tbench on powerpc
> > Submitter : Kamalesh Babulal
> > Date : 2008-02-20 16:01 (26 days old)
> > References : http://lkml.org/lkml/2008/2/20/218
> > http://lkml.org/lkml/2008/1/18/71
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10086
> > Subject : 2.6.25-rc2 + smartd = hang
> > Submitter : Anders Eriksson
> > Date : 2008-02-22 17:51 (24 days old)
> > References : http://lkml.org/lkml/2008/2/22/239
> > Handled-By : Bartlomiej Zolnierkiewicz
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10093
> > Subject : 2.6.25-current-git hangs on boot unless CONFIG_CPU_IDLE=n - Apple
> > Submitter : Soeren Sonnenburg
> > Date : 2008-02-23 18:55 (23 days old)
> > References : http://lkml.org/lkml/2008/2/23/263
> > http://marc.info/?l=linux-acpi&m=120387537018467&w=4
> > Handled-By : Pallipadi, Venkatesh
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10117
> > Subject : 2.6.25-current-git hangs on boot (pci=nommconf helps)
> > Submitter : Soeren Sonnenburg
> > Date : 2008-02-23 18:55 (23 days old)
> > References : http://lkml.org/lkml/2008/2/23/263
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10133
> > Subject : INFO: possible circular locking in the resume
> > Submitter : Zdenek Kabelac
> > Date : 2008-02-27 (19 days old)
> > References : http://lkml.org/lkml/2008/2/26/479
> > Handled-By : Gautham R Shenoy
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10146
> > Subject : 2.6.25-rc: complete lockup on boot/start of X (bisected)
> > Submitter : Marcin Slusarz
> > Date : 2008-03-02 20:00 (15 days old)
> > References : http://lkml.org/lkml/2008/3/2/91
> > Handled-By : Peter Zijlstra
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10152
> > Subject : Clocksource tsc is always unstable with 2.6.25-* kernels and CONFIG_NO_HZ=y on my box
> > Submitter : Gabriel C
> > Date : 2008-02-24 01:31 (22 days old)
> > References : http://lkml.org/lkml/2008/2/23/380
> > http://lkml.org/lkml/2008/2/24/281
> > Handled-By : Thomas Gleixner
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10156
> > Subject : KVM & Qemu crashed with infinite recursive kernel loop in the guest
> > Submitter : Zdenek Kabelac
> > Date : 2008-02-28 11:25 (18 days old)
> > References : http://lkml.org/lkml/2008/2/28/106
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10172
> > Subject : kvm: INFO: inconsistent lock state
> > Submitter : Zdenek Kabelac
> > Date : 2008-03-05 03:26 (12 days old)
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10190
> > Subject : [BUG] Linux-2.6.25-rc4 (and also in rc3) Compile Error
> > Submitter : Tarkan Erimer
> > Date : 2008-03-05 05:01 (12 days old)
> > References : http://www.ussg.iu.edu/hypermail/lin...03.0/1867.html
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10203
> > Subject : 2.6.25 IOMMU breaks DMA for b43 on x86_64
> > Submitter : Christian Casteyde
> > Date : 2008-03-09 00:55 (8 days old)
> > Handled-By : Michael Buesch
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10207
> > Subject : INFO: task mount:11202 blocked for more than 120 seconds
> > Submitter : Christian Kujau
> > Date : 2008-03-07 21:32 (10 days old)
> > References : http://lkml.org/lkml/2008/3/7/308
> > http://lkml.org/lkml/2008/3/9/186
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=10211
> > Subject : drivers/media/video/cx2341x.c: undefined references
> > Submitter : Toralf Förster
> > Date : 2008-03-07 13:48 (10 days old)
> > References : http://lkml.org/lkml/2008/3/7/168
> >
> I think patch of Mauro Carvalho Chehab can fix this bug.
> http://linuxtv.org/hg/v4l-dvb/rev/ba1a6a7bd53b

Thanks, I updated the entry.

Rafael
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Gabriel C wrote:
> Thomas Gleixner wrote:
>> On Mon, 17 Mar 2008, Gabriel C wrote:
>>>> Subject : Clocksource tsc is always unstable with 2.6.25-* kernels and CONFIG_NO_HZ=y on my box
>>>> Submitter : Gabriel C
>>>> Date : 2008-02-24 01:31 (22 days old)
>>>> References : http://lkml.org/lkml/2008/2/23/380
>>>> http://lkml.org/lkml/2008/2/24/281
>>>> Handled-By : Thomas Gleixner
>>>>
>>> Thomas do you want me to bisect ?
>> That'd be great.
>
> Ok I'll start doing that later on today.
>

I managed to bisect 'one of the bugs' down , I got some problems and used skip once because a revision didn't compiled ,
but it seems bisect got the right commit still. Sadly it seems there are 2 different bugs.

Also before I've started the bisect I've tested linux-next to be sure the bug(s) still exists and while rc1 got that already
I've started to bisect 2.6.24 -> 2.6.25-rc1.

cat .git/refs/bisect/bad
1ada5cba6a0318f90e45b38557e7b5206a9cba38

git show 1ada5cba6a0318f90e45b38557e7b5206a9cba38
commit 1ada5cba6a0318f90e45b38557e7b5206a9cba38
Author: Andi Kleen
Date: Wed Jan 30 13:30:02 2008 +0100

clocksource: make clocksource watchdog cycle through online CPUs

This way it checks if the clocks are synchronized between CPUs too.
This might be able to detect slowly drifting TSCs which only
go wrong over longer time.

Signed-off-by: Andi Kleen
Signed-off-by: Ingo Molnar
Signed-off-by: Thomas Gleixner

diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
index cabfa19..edd5ef8 100644
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -142,8 +142,13 @@ static void clocksource_watchdog(unsigned long data)
}

if (!list_empty(&watchdog_list)) {
- __mod_timer(&watchdog_timer,
- watchdog_timer.expires + WATCHDOG_INTERVAL);
+ /* Cycle through CPUs to check if the CPUs stay synchronized to
+ * each other. */
+ int next_cpu = next_cpu(raw_smp_processor_id(), cpu_online_map);
+ if (next_cpu >= NR_CPUS)
+ next_cpu = first_cpu(cpu_online_map);
+ watchdog_timer.expires += WATCHDOG_INTERVAL;
+ add_timer_on(&watchdog_timer, next_cpu);
}
spin_unlock(&watchdog_lock);
}
@@ -165,7 +170,7 @@ static void clocksource_check_watchdog(struct clocksource *cs)
if (!started && watchdog) {
watchdog_last = watchdog->read();
watchdog_timer.expires = jiffies + WATCHDOG_INTERVAL;
- add_timer(&watchdog_timer);
+ add_timer_on(&watchdog_timer, first_cpu(cpu_online_map));
}
} else {
if (cs->flags & CLOCK_SOURCE_IS_CONTINUOUS)
@@ -186,7 +191,8 @@ static void clocksource_check_watchdog(struct clocksource *cs)
watchdog_last = watchdog->read();
watchdog_timer.expires =
jiffies + WATCHDOG_INTERVAL;
- add_timer(&watchdog_timer);
+ add_timer_on(&watchdog_timer,
+ first_cpu(cpu_online_map));
}
}
}


git bisect log
git-bisect start
# bad: [19af35546de68c872dcb687613e0902a602cb20e] Linux 2.6.25-rc1
git-bisect bad 19af35546de68c872dcb687613e0902a602cb20e
# good: [49914084e797530d9baaf51df9eda77babc98fa8] Linux 2.6.24
git-bisect good 49914084e797530d9baaf51df9eda77babc98fa8
# bad: [d2e626f45cc450c00f5f98a89b8b4c4ac3c9bf5f] x86: add PAGE_KERNEL_EXEC_NOCACHE
git-bisect bad d2e626f45cc450c00f5f98a89b8b4c4ac3c9bf5f
# good: [fb46990dba94866462e90623e183d02ec591cf8f] [NETFILTER]: nf_queue: remove unnecessary hook existance check
git-bisect good fb46990dba94866462e90623e183d02ec591cf8f
# good: [936722922f6d2366378de606a40c14f96915474d] [IPV4] fib_trie: compute size when needed
git-bisect good 936722922f6d2366378de606a40c14f96915474d
# bad: [ff14c6164bd532a6dc9025c07d3b562f839f00a9] x86: x86-64 ia32 ptrace pt_regs cleanup
git-bisect bad ff14c6164bd532a6dc9025c07d3b562f839f00a9
# good: [c087567d3ffb2c7c61e091982e6ca45478394f1a] SUNRPC: Remove the obsolete RPC_WAITQ macro
git-bisect good c087567d3ffb2c7c61e091982e6ca45478394f1a
# bad: [af7a78e9258ffcca681e080cbc857f854869144f] x86: move mce related declarations
git-bisect bad af7a78e9258ffcca681e080cbc857f854869144f
# good: [34f5b4662bf4b54f22b32ce76ce70eccd7ebc68a] SUNRPC: Don't bother changing the sigmask for asynchronous RPC calls
git-bisect good 34f5b4662bf4b54f22b32ce76ce70eccd7ebc68a
# bad: [83bd01024b1fdfc41d9b758e5669e80fca72df66] x86: protect against sigaltstack wraparound
git-bisect bad 83bd01024b1fdfc41d9b758e5669e80fca72df66
# good: [efd9ac8630e89b9ee7ce64008bd7783952374f37] time: fold __get_realtime_clock_ts() into getnstimeofday()
git-bisect good efd9ac8630e89b9ee7ce64008bd7783952374f37
# bad: [37a47db8d7f0f38dac5acf5a13abbc8f401707fa] x86: assign IRQs to HPET timers, fix
git-bisect bad 37a47db8d7f0f38dac5acf5a13abbc8f401707fa
# skip: [316da3b3fc8efa9a5d2c99e0d449f01ff38c6aba] x86: restrict PIT clocksource usage
git-bisect skip 316da3b3fc8efa9a5d2c99e0d449f01ff38c6aba
# bad: [4713e22ce81eb8b3353e16435362eb3d0ec95640] clocksource: add unregister function to disable unusable clocksources
git-bisect bad 4713e22ce81eb8b3353e16435362eb3d0ec95640
# bad: [1ada5cba6a0318f90e45b38557e7b5206a9cba38] clocksource: make clocksource watchdog cycle through online CPUs
git-bisect bad 1ada5cba6a0318f90e45b38557e7b5206a9cba38
# good: [1077f5a917b7c630231037826b344b2f7f5b903f] clocksource.c: use init_timer_deferrable for clocksource_watchdog
git-bisect good 1077f5a917b7c630231037826b344b2f7f5b903f


Also the broken revision died with that :

arch/x86/kernel/i8253.c: In function 'init_pit_clocksource':
arch/x86/kernel/i8253.c:207: error: implicit declaration of function 'is_hpet_enabled'
make[1]: *** [arch/x86/kernel/i8253.o] Error 1
make: *** [arch/x86/kernel] Error 2

If you tell me on how to fix that I'll restart the bisect from there , just in case ..


Also reverting the commit from 2.6.25-rc1 fixes the 'Tsc being unstable thing' but it does not fix the hang
when I boot with clocksource=acpi_pm so that seems to be introduced in a different commit.

I will try to bisect this hang also , most probably on weekend.


Also I reverted that commit from git head and an kernel compiles right now, I'll let you know in a bit if that worked out.

Please let me know if you need more informations.


Best Regards,

Gabriel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Gabriel C wrote:
> Gabriel C wrote:
>> Thomas Gleixner wrote:
>>> On Mon, 17 Mar 2008, Gabriel C wrote:
>>>>> Subject : Clocksource tsc is always unstable with 2.6.25-* kernels and CONFIG_NO_HZ=y on my box
>>>>> Submitter : Gabriel C
>>>>> Date : 2008-02-24 01:31 (22 days old)
>>>>> References : http://lkml.org/lkml/2008/2/23/380
>>>>> http://lkml.org/lkml/2008/2/24/281
>>>>> Handled-By : Thomas Gleixner
>>>>>
>>>> Thomas do you want me to bisect ?
>>> That'd be great.
>> Ok I'll start doing that later on today.
>>
>
> I managed to bisect 'one of the bugs' down , I got some problems and used skip once because a revision didn't compiled ,
> but it seems bisect got the right commit still. Sadly it seems there are 2 different bugs.
>
> Also before I've started the bisect I've tested linux-next to be sure the bug(s) still exists and while rc1 got that already
> I've started to bisect 2.6.24 -> 2.6.25-rc1.
>
> cat .git/refs/bisect/bad
> 1ada5cba6a0318f90e45b38557e7b5206a9cba38
>
> git show 1ada5cba6a0318f90e45b38557e7b5206a9cba38
> commit 1ada5cba6a0318f90e45b38557e7b5206a9cba38
> Author: Andi Kleen
> Date: Wed Jan 30 13:30:02 2008 +0100
>
> clocksource: make clocksource watchdog cycle through online CPUs
>
> This way it checks if the clocks are synchronized between CPUs too.
> This might be able to detect slowly drifting TSCs which only
> go wrong over longer time.
>
> Signed-off-by: Andi Kleen
> Signed-off-by: Ingo Molnar
> Signed-off-by: Thomas Gleixner
>
> diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c
> index cabfa19..edd5ef8 100644
> --- a/kernel/time/clocksource.c
> +++ b/kernel/time/clocksource.c
> @@ -142,8 +142,13 @@ static void clocksource_watchdog(unsigned long data)
> }
>
> if (!list_empty(&watchdog_list)) {
> - __mod_timer(&watchdog_timer,
> - watchdog_timer.expires + WATCHDOG_INTERVAL);
> + /* Cycle through CPUs to check if the CPUs stay synchronized to
> + * each other. */
> + int next_cpu = next_cpu(raw_smp_processor_id(), cpu_online_map);
> + if (next_cpu >= NR_CPUS)
> + next_cpu = first_cpu(cpu_online_map);
> + watchdog_timer.expires += WATCHDOG_INTERVAL;
> + add_timer_on(&watchdog_timer, next_cpu);
> }
> spin_unlock(&watchdog_lock);
> }
> @@ -165,7 +170,7 @@ static void clocksource_check_watchdog(struct clocksource *cs)
> if (!started && watchdog) {
> watchdog_last = watchdog->read();
> watchdog_timer.expires = jiffies + WATCHDOG_INTERVAL;
> - add_timer(&watchdog_timer);
> + add_timer_on(&watchdog_timer, first_cpu(cpu_online_map));
> }
> } else {
> if (cs->flags & CLOCK_SOURCE_IS_CONTINUOUS)
> @@ -186,7 +191,8 @@ static void clocksource_check_watchdog(struct clocksource *cs)
> watchdog_last = watchdog->read();
> watchdog_timer.expires =
> jiffies + WATCHDOG_INTERVAL;
> - add_timer(&watchdog_timer);
> + add_timer_on(&watchdog_timer,
> + first_cpu(cpu_online_map));
> }
> }
> }
>
>
> git bisect log
> git-bisect start
> # bad: [19af35546de68c872dcb687613e0902a602cb20e] Linux 2.6.25-rc1
> git-bisect bad 19af35546de68c872dcb687613e0902a602cb20e
> # good: [49914084e797530d9baaf51df9eda77babc98fa8] Linux 2.6.24
> git-bisect good 49914084e797530d9baaf51df9eda77babc98fa8
> # bad: [d2e626f45cc450c00f5f98a89b8b4c4ac3c9bf5f] x86: add PAGE_KERNEL_EXEC_NOCACHE
> git-bisect bad d2e626f45cc450c00f5f98a89b8b4c4ac3c9bf5f
> # good: [fb46990dba94866462e90623e183d02ec591cf8f] [NETFILTER]: nf_queue: remove unnecessary hook existance check
> git-bisect good fb46990dba94866462e90623e183d02ec591cf8f
> # good: [936722922f6d2366378de606a40c14f96915474d] [IPV4] fib_trie: compute size when needed
> git-bisect good 936722922f6d2366378de606a40c14f96915474d
> # bad: [ff14c6164bd532a6dc9025c07d3b562f839f00a9] x86: x86-64 ia32 ptrace pt_regs cleanup
> git-bisect bad ff14c6164bd532a6dc9025c07d3b562f839f00a9
> # good: [c087567d3ffb2c7c61e091982e6ca45478394f1a] SUNRPC: Remove the obsolete RPC_WAITQ macro
> git-bisect good c087567d3ffb2c7c61e091982e6ca45478394f1a
> # bad: [af7a78e9258ffcca681e080cbc857f854869144f] x86: move mce related declarations
> git-bisect bad af7a78e9258ffcca681e080cbc857f854869144f
> # good: [34f5b4662bf4b54f22b32ce76ce70eccd7ebc68a] SUNRPC: Don't bother changing the sigmask for asynchronous RPC calls
> git-bisect good 34f5b4662bf4b54f22b32ce76ce70eccd7ebc68a
> # bad: [83bd01024b1fdfc41d9b758e5669e80fca72df66] x86: protect against sigaltstack wraparound
> git-bisect bad 83bd01024b1fdfc41d9b758e5669e80fca72df66
> # good: [efd9ac8630e89b9ee7ce64008bd7783952374f37] time: fold __get_realtime_clock_ts() into getnstimeofday()
> git-bisect good efd9ac8630e89b9ee7ce64008bd7783952374f37
> # bad: [37a47db8d7f0f38dac5acf5a13abbc8f401707fa] x86: assign IRQs to HPET timers, fix
> git-bisect bad 37a47db8d7f0f38dac5acf5a13abbc8f401707fa
> # skip: [316da3b3fc8efa9a5d2c99e0d449f01ff38c6aba] x86: restrict PIT clocksource usage
> git-bisect skip 316da3b3fc8efa9a5d2c99e0d449f01ff38c6aba
> # bad: [4713e22ce81eb8b3353e16435362eb3d0ec95640] clocksource: add unregister function to disable unusable clocksources
> git-bisect bad 4713e22ce81eb8b3353e16435362eb3d0ec95640
> # bad: [1ada5cba6a0318f90e45b38557e7b5206a9cba38] clocksource: make clocksource watchdog cycle through online CPUs
> git-bisect bad 1ada5cba6a0318f90e45b38557e7b5206a9cba38
> # good: [1077f5a917b7c630231037826b344b2f7f5b903f] clocksource.c: use init_timer_deferrable for clocksource_watchdog
> git-bisect good 1077f5a917b7c630231037826b344b2f7f5b903f
>
>
> Also the broken revision died with that :
>
> arch/x86/kernel/i8253.c: In function 'init_pit_clocksource':
> arch/x86/kernel/i8253.c:207: error: implicit declaration of function 'is_hpet_enabled'
> make[1]: *** [arch/x86/kernel/i8253.o] Error 1
> make: *** [arch/x86/kernel] Error 2
>
> If you tell me on how to fix that I'll restart the bisect from there , just in case ..
>
>
> Also reverting the commit from 2.6.25-rc1 fixes the 'Tsc being unstable thing' but it does not fix the hang
> when I boot with clocksource=acpi_pm so that seems to be introduced in a different commit.
>
> I will try to bisect this hang also , most probably on weekend.
>
>
> Also I reverted that commit from git head and an kernel compiles right now, I'll let you know in a bit if that worked out.

Worked out

git head - 1ada5cba6a0318f90e45b38557e7b5206a9cba38 works here.

dmesg|grep clocksource
[ 0.563915] Time: tsc clocksource has been installed.

uname -a
Linux lara 2.6.25-rc6-00014-gbde4f8f-dirty #2 SMP PREEMPT Tue Mar 18 04:48:53 CET 2008 i686 GNU/Linux


Gabriel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Gabriel C wrote:
> Gabriel C wrote:
>> Thomas Gleixner wrote:
>>> On Mon, 17 Mar 2008, Gabriel C wrote:
>>>>> Subject : Clocksource tsc is always unstable with 2.6.25-* kernels and CONFIG_NO_HZ=y on my box
>>>>> Submitter : Gabriel C
>>>>> Date : 2008-02-24 01:31 (22 days old)
>>>>> References : http://lkml.org/lkml/2008/2/23/380
>>>>> http://lkml.org/lkml/2008/2/24/281
>>>>> Handled-By : Thomas Gleixner
>>>>>
>>>> Thomas do you want me to bisect ?
>>> That'd be great.
>> Ok I'll start doing that later on today.
>>
>
[ snip ]

> still hangs when I boot with clocksource=acpi_pm so that seems to be introduced in a different commit.
>
> I will try to bisect this hang also , most probably on weekend.
>

Correction on this one.

Current git head boots just fine with clocksource=acpi_pm here , I just don't know which commit fixed it.

Gabriel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Fri, 21 Mar 2008, Gabriel C wrote:

> > still hangs when I boot with clocksource=acpi_pm so that seems to
> > be introduced in a different commit.
> >
> > I will try to bisect this hang also , most probably on weekend.
> >
>
> Correction on this one.
>
> Current git head boots just fine with clocksource=acpi_pm here , I
> just don't know which commit fixed it.

Hmm. Very dubious. I'm a bit afraid of self healing problems. It would
be interesting to find the commit which fixed the acpi_pm timer
problem unvoluntary.

Also, can you please reapply the reverted clocksource patch ? I have
the feeling that the acpi_pm one was the real problem which was
triggered the modfied watchdog.

Thanks,

tglx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Thomas Gleixner wrote:
> On Fri, 21 Mar 2008, Gabriel C wrote:
>
>>> still hangs when I boot with clocksource=acpi_pm so that seems to
>>> be introduced in a different commit.
>>>
>>> I will try to bisect this hang also , most probably on weekend.
>>>
>> Correction on this one.
>>
>> Current git head boots just fine with clocksource=acpi_pm here , I
>> just don't know which commit fixed it.
>
> Hmm. Very dubious. I'm a bit afraid of self healing problems. It would
> be interesting to find the commit which fixed the acpi_pm timer
> problem unvoluntary.

I can try to find it.

>
> Also, can you please reapply the reverted clocksource patch ? I have
> the feeling that the acpi_pm one was the real problem which was
> triggered the modfied watchdog.

Sure I can , will do so in some minutes and let you know.

>
> Thanks,
>
> tglx


Gabriel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Gabriel C wrote:
> Thomas Gleixner wrote:
>> On Fri, 21 Mar 2008, Gabriel C wrote:
>>
>>>> still hangs when I boot with clocksource=acpi_pm so that seems to
>>>> be introduced in a different commit.
>>>>
>>>> I will try to bisect this hang also , most probably on weekend.
>>>>
>>> Correction on this one.
>>>
>>> Current git head boots just fine with clocksource=acpi_pm here , I
>>> just don't know which commit fixed it.
>> Hmm. Very dubious. I'm a bit afraid of self healing problems. It would
>> be interesting to find the commit which fixed the acpi_pm timer
>> problem unvoluntary.
>
> I can try to find it.
>
>> Also, can you please reapply the reverted clocksource patch ? I have
>> the feeling that the acpi_pm one was the real problem which was
>> triggered the modfied watchdog.
>
> Sure I can , will do so in some minutes and let you know.

It took a bit longer sorry but I have more infos now.

The acpi_pm was not related to that I still get the problem.

Of course I still can try to find the commit which magically fixed acpi_pm if you really want.

It seems like it breaks only when you enable HT and only on 2 socket motherboards.
( at least the ones I own , I know is old hardware but worked fine for me )

Also disabling the second CPU and enabling HT works , enabling both CPUs and disabling HT works ,
booting with enabled HT and both CPUs but maxcpus=2 also works , booting with 2 CPUs and HT on breaks ,
booting with both CPUs HT on but maxcpus=3 breaks also.

Also I have another dual motherboard here 604 socket with 2 2,4 GHz Xeon's.
The motherboard has the storage controller somewhat broken but for a quick test it is still good and I see the
same thing.

Does that make any sense ?

Also all that tested on 2.6.25-rc6-00224-gae51801-dirty ( dirty while I reverted the revert =) )

Gabriel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Fri, 21 Mar 2008, Gabriel C wrote:
> >> Also, can you please reapply the reverted clocksource patch ? I have
> >> the feeling that the acpi_pm one was the real problem which was
> >> triggered the modfied watchdog.
> >
> > Sure I can , will do so in some minutes and let you know.
>
> It took a bit longer sorry but I have more infos now.
>
> The acpi_pm was not related to that I still get the problem.
>
> Of course I still can try to find the commit which magically fixed acpi_pm if you really want.

Just if you are really bored. I would have asked if it had fixed
the TSC issue.

> It seems like it breaks only when you enable HT and only on 2 socket motherboards.
> ( at least the ones I own , I know is old hardware but worked fine for me )

Hmm. I wonder why a dual socket board survives the initial sync test.

> Also disabling the second CPU and enabling HT works , enabling both
> CPUs and disabling HT works , booting with enabled HT and both CPUs
> but maxcpus=2 also works , booting with 2 CPUs and HT on breaks ,
> booting with both CPUs HT on but maxcpus=3 breaks also.
>
> Also I have another dual motherboard here 604 socket with 2 2,4 GHz
> Xeon's. The motherboard has the storage controller somewhat broken
> but for a quick test it is still good and I see the same thing.
>
> Does that make any sense ?

Not really. Can you please revert the reverted revert again and run

http://people.redhat.com/mingo/time-...me-warp-test.c

on your machine with all CPUs and HT enabled ?

Thanks,
tglx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Thomas Gleixner wrote:
> On Fri, 21 Mar 2008, Gabriel C wrote:
>>>> Also, can you please reapply the reverted clocksource patch ? I have
>>>> the feeling that the acpi_pm one was the real problem which was
>>>> triggered the modfied watchdog.
>>> Sure I can , will do so in some minutes and let you know.
>> It took a bit longer sorry but I have more infos now.
>>
>> The acpi_pm was not related to that I still get the problem.
>>
>> Of course I still can try to find the commit which magically fixed acpi_pm if you really want.
>
> Just if you are really bored. I would have asked if it had fixed
> the TSC issue.
>
>> It seems like it breaks only when you enable HT and only on 2 socket motherboards.
>> ( at least the ones I own , I know is old hardware but worked fine for me )
>
> Hmm. I wonder why a dual socket board survives the initial sync test.
>
>> Also disabling the second CPU and enabling HT works , enabling both
>> CPUs and disabling HT works , booting with enabled HT and both CPUs
>> but maxcpus=2 also works , booting with 2 CPUs and HT on breaks ,
>> booting with both CPUs HT on but maxcpus=3 breaks also.
>>
>> Also I have another dual motherboard here 604 socket with 2 2,4 GHz
>> Xeon's. The motherboard has the storage controller somewhat broken
>> but for a quick test it is still good and I see the same thing.
>>
>> Does that make any sense ?
>
> Not really. Can you please revert the reverted revert again and run
>
> http://people.redhat.com/mingo/time-...me-warp-test.c
>
> on your machine with all CPUs and HT enabled ?

Sure , doing so now.

>
> Thanks,
> tglx


Gabriel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Gabriel C wrote:
> Thomas Gleixner wrote:
>> On Fri, 21 Mar 2008, Gabriel C wrote:
>>>>> Also, can you please reapply the reverted clocksource patch ? I have
>>>>> the feeling that the acpi_pm one was the real problem which was
>>>>> triggered the modfied watchdog.
>>>> Sure I can , will do so in some minutes and let you know.
>>> It took a bit longer sorry but I have more infos now.
>>>
>>> The acpi_pm was not related to that I still get the problem.
>>>
>>> Of course I still can try to find the commit which magically fixed acpi_pm if you really want.
>> Just if you are really bored. I would have asked if it had fixed
>> the TSC issue.
>>
>>> It seems like it breaks only when you enable HT and only on 2 socket motherboards.
>>> ( at least the ones I own , I know is old hardware but worked fine for me )
>> Hmm. I wonder why a dual socket board survives the initial sync test.
>>
>>> Also disabling the second CPU and enabling HT works , enabling both
>>> CPUs and disabling HT works , booting with enabled HT and both CPUs
>>> but maxcpus=2 also works , booting with 2 CPUs and HT on breaks ,
>>> booting with both CPUs HT on but maxcpus=3 breaks also.
>>>
>>> Also I have another dual motherboard here 604 socket with 2 2,4 GHz
>>> Xeon's. The motherboard has the storage controller somewhat broken
>>> but for a quick test it is still good and I see the same thing.
>>>
>>> Does that make any sense ?
>> Not really. Can you please revert the reverted revert again and run
>>
>> http://people.redhat.com/mingo/time-...me-warp-test.c
>>
>> on your machine with all CPUs and HT enabled ?
>
> Sure , doing so now.
>

Here the result on 2.6.25-rc6-00243-g028011e ( it was running 30++ minutes the time I was away for food =) )

....

4 CPUs, running 4 parallel test-tasks.
checking for time-warps via:
- read time stamp counter (RDTSC) instruction (cycle resolution)
- gettimeofday (TOD) syscall (usec resolution)
- clock_gettime(CLOCK_MONOTONIC) syscall (nsec resolution)

| 1.46 us, TSC-warps:0 | 16.01 us, TOD-warps:0 | 16.10 us, CLOCK-warps:0

....

Gabriel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Fri, 21 Mar 2008, Gabriel C wrote:
> >>> Does that make any sense ?
> >> Not really. Can you please revert the reverted revert again and run
> >>
> >> http://people.redhat.com/mingo/time-...me-warp-test.c
> >>
> >> on your machine with all CPUs and HT enabled ?
> >
> > Sure , doing so now.
> >
>
> Here the result on 2.6.25-rc6-00243-g028011e ( it was running 30++
> minutes the time I was away for food =) )
> ...
>
> 4 CPUs, running 4 parallel test-tasks.
> checking for time-warps via:
> - read time stamp counter (RDTSC) instruction (cycle resolution)
> - gettimeofday (TOD) syscall (usec resolution)
> - clock_gettime(CLOCK_MONOTONIC) syscall (nsec resolution)
>
> | 1.46 us, TSC-warps:0 | 16.01 us, TOD-warps:0 | 16.10 us, CLOCK-warps:0

Amazing. I never found a multi socket box where the TSC's were in sync.

So the rotating watchdog triggers for a yet to figure out reason.

Oh, now that the pm timer seems to work again, can you try the following:

apply the reverted patch again and let the box boot. At some point the
TSC is marked unstable and is replaced by acpi_pm clocksource.

What result does timewarp.c show in that situation ?

Thanks,
tglx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Thomas Gleixner wrote:
> On Fri, 21 Mar 2008, Gabriel C wrote:
>>>>> Does that make any sense ?
>>>> Not really. Can you please revert the reverted revert again and run
>>>>
>>>> http://people.redhat.com/mingo/time-...me-warp-test.c
>>>>
>>>> on your machine with all CPUs and HT enabled ?
>>> Sure , doing so now.
>>>
>> Here the result on 2.6.25-rc6-00243-g028011e ( it was running 30++
>> minutes the time I was away for food =) )
>> ...
>>
>> 4 CPUs, running 4 parallel test-tasks.
>> checking for time-warps via:
>> - read time stamp counter (RDTSC) instruction (cycle resolution)
>> - gettimeofday (TOD) syscall (usec resolution)
>> - clock_gettime(CLOCK_MONOTONIC) syscall (nsec resolution)
>>
>> | 1.46 us, TSC-warps:0 | 16.01 us, TOD-warps:0 | 16.10 us, CLOCK-warps:0
>
> Amazing. I never found a multi socket box where the TSC's were in sync.
>
> So the rotating watchdog triggers for a yet to figure out reason.
>
> Oh, now that the pm timer seems to work again, can you try the following:
>
> apply the reverted patch again and let the box boot. At some point the
> TSC is marked unstable and is replaced by acpi_pm clocksource.
>
> What result does timewarp.c show in that situation ?

Here it is , same kernel + Andi's patch :

../time-warp-test
4 CPUs, running 4 parallel test-tasks.
checking for time-warps via:
- read time stamp counter (RDTSC) instruction (cycle resolution)
- gettimeofday (TOD) syscall (usec resolution)
- clock_gettime(CLOCK_MONOTONIC) syscall (nsec resolution)

| 1.78 us, TSC-warps:0 | 19.27 us, TOD-warps:0 | 19.37 us, CLOCK-warps:0

>
> Thanks,
> tglx


Gabriel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Fri, 21 Mar 2008, Gabriel C wrote:
> > So the rotating watchdog triggers for a yet to figure out reason.
> >
> > Oh, now that the pm timer seems to work again, can you try the following:
> >
> > apply the reverted patch again and let the box boot. At some point the
> > TSC is marked unstable and is replaced by acpi_pm clocksource.
> >
> > What result does timewarp.c show in that situation ?
>
> Here it is , same kernel + Andi's patch :
>
> ./time-warp-test
> 4 CPUs, running 4 parallel test-tasks.
> checking for time-warps via:
> - read time stamp counter (RDTSC) instruction (cycle resolution)
> - gettimeofday (TOD) syscall (usec resolution)
> - clock_gettime(CLOCK_MONOTONIC) syscall (nsec resolution)
>
> | 1.78 us, TSC-warps:0 | 19.27 us, TOD-warps:0 | 19.37 us, CLOCK-warps:0

Ok. So the watchdog trigger is a false positive.

Thinking more about it, it looks like Andi's change triggers some
hidden bug in the combination of NO_HZ and add_timer_on(), where the
CPU on which the timer is added is likely in a long idle sleep. I look
into this tomorrow.

Thanks for testing

tglx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

On Fri, 21 Mar 2008, Thomas Gleixner wrote:
> >
> > | 1.78 us, TSC-warps:0 | 19.27 us, TOD-warps:0 | 19.37 us, CLOCK-warps:0
>
> Ok. So the watchdog trigger is a false positive.
>
> Thinking more about it, it looks like Andi's change triggers some
> hidden bug in the combination of NO_HZ and add_timer_on(), where the
> CPU on which the timer is added is likely in a long idle sleep. I look
> into this tomorrow.

Ok. Here is what's happening:

CPU0 runs the watchdog timer and schedules it on CPU1.

With NO_HZ enabled CPU1 is in a long idle sleep. At this point of the
boot process there is probably no timer pending on CPU1, which means
the idle sleep is infinite.

Now some time later CPU1 gets woken by an interrupt/IPI and runs the
timer wheel. At this point the pm_timer which is the reference clock
has already wrapped around, so the watchdog thinks that there is a
huge time difference and marks the TSC unstable.

Aside of that watchdog issue this also affects the other users of
add_timer_on(): e.g. queue_delayed_work_on().

Can you please apply the patch below and verify it with Andi's
watchdog patch applied ?

Thanks,

tglx

---
include/linux/tick.h | 4 ++++
kernel/time/tick-sched.c | 30 ++++++++++++++++++++++++++++++
kernel/timer.c | 14 +++++++++++++-
3 files changed, 47 insertions(+), 1 deletion(-)

Index: linux-2.6/include/linux/tick.h
================================================== =================
--- linux-2.6.orig/include/linux/tick.h
+++ linux-2.6/include/linux/tick.h
@@ -111,6 +111,8 @@ extern void tick_nohz_update_jiffies(voi
extern ktime_t tick_nohz_get_sleep_length(void);
extern void tick_nohz_stop_idle(int cpu);
extern u64 get_cpu_idle_time_us(int cpu, u64 *last_update_time);
+extern int tick_nohz_cpu_needs_wakeup(int cpu);
+extern void tick_nohz_rescan_timers_on(int cpu);
# else
static inline void tick_nohz_stop_sched_tick(void) { }
static inline void tick_nohz_restart_sched_tick(void) { }
@@ -123,6 +125,8 @@ static inline ktime_t tick_nohz_get_slee
}
static inline void tick_nohz_stop_idle(int cpu) { }
static inline u64 get_cpu_idle_time_us(int cpu, u64 *unused) { return 0; }
+static inline int tick_nohz_cpu_needs_wakeup(int cpu) { return 0; }
+static inline void tick_nohz_rescan_timers_on(int cpu) { }
# endif /* !NO_HZ */

#endif
Index: linux-2.6/kernel/time/tick-sched.c
================================================== =================
--- linux-2.6.orig/kernel/time/tick-sched.c
+++ linux-2.6/kernel/time/tick-sched.c
@@ -183,6 +183,36 @@ u64 get_cpu_idle_time_us(int cpu, u64 *l
}

/**
+ * tick_nohz_cpu_needs_wakeup - check possible wakeup of cpu in add_timer_on()
+ *
+ * when add_timer_on() happens on a CPU which is in a long idle sleep,
+ * then we need to wake it up so the timer wheel gets reevaluated.
+ *
+ * Note: we use idle_cpu() which checks the idle state lockless, but
+ * we are ordered against the other cpu which might be on the way to
+ * idle by the timer base lock, which we hold.
+ */
+int tick_nohz_cpu_needs_wakeup(int cpu)
+{
+ return tick_nohz_enabled && idle_cpu(cpu) &&
+ (cpu != smp_processor_id());
+}
+
+/**
+ * tick_nohz_rescan_timers_on - reevaluate the idle sleep time of a CPU
+ *
+ * When a CPU is idle and a timer got added to this CPU timer wheel
+ * via add_timer_on() then we need to make sure that the CPU
+ * reevaluates the timer wheel. Otherwise the timer might be delayed
+ * for a real long time.
+ */
+void tick_nohz_rescan_timers_on(int cpu)
+{
+ if (tick_nohz_enabled && idle_cpu(cpu))
+ smp_send_reschedule(cpu);
+}
+
+/**
* tick_nohz_stop_sched_tick - stop the idle tick from the idle task
*
* When the next event is more than a tick into the future, stop the idle tick
Index: linux-2.6/kernel/timer.c
================================================== =================
--- linux-2.6.orig/kernel/timer.c
+++ linux-2.6/kernel/timer.c
@@ -445,15 +445,27 @@ void add_timer_on(struct timer_list *tim
{
struct tvec_base *base = per_cpu(tvec_bases, cpu);
unsigned long flags;
+ int wakeidle;

timer_stats_timer_set_start_info(timer);
BUG_ON(timer_pending(timer) || !timer->function);
spin_lock_irqsave(&base->lock, flags);
timer_set_base(timer, base);
internal_add_timer(base, timer);
+ /*
+ * Check whether the other CPU is idle and needs to be
+ * triggered to reevaluate the timer wheel when nohz is
+ * active. We are protected against the other CPU fiddling
+ * with the timer by holding the timer base lock. This also
+ * makes sure that a CPU on the way to idle can not evaluate
+ * the timer wheel.
+ */
+ wakeidle = tick_nohz_cpu_needs_wakeup(cpu);
spin_unlock_irqrestore(&base->lock, flags);
-}

+ if (wakeidle)
+ tick_nohz_rescan_timers_on(cpu);
+}

/**
* mod_timer - modify a timer's timeout
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/