[PATCH] x86-64: Disable local APIC timer use on AMD systems with C1E

commit 3556ddfa9284a86a59a9b78fe5894430f6ab4eef titled

[PATCH] x86-64: Disable local APIC timer use on AMD systems with C1E

solves a problem with AMD dual core laptops e.g. HP nx6325 (Turion 64
X2) with C1E enabled:

When both cores go into idle at the same time, then the system switches
into C1E state, which is basically the same as C3. This stops the local
apic timer.

This was debugged right after the dyntick merge on i386 and despite the
patch title it fixes only the 32 bit path.

x86_64 is still missing this fix. It seems that mainline is not really
affected by this issue, as the PIT is running and keeps jiffies
incrementing, but that's just waiting for trouble.

-mm suffers from this problem due to the x86_64 high resolution timer
patches.

This is a quick and dirty port of the i386 code to x86_64.

I spent quite a time with Rafael to debug the -mm / hrt wreckage until
someone pointed us to this. I really had forgotten that we debugged this
half a year ago already.

Sigh, is it just me or is there something yelling arch/x86 into my ear?

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

diff --git a/arch/x86_64/kernel/setup.c b/arch/x86_64/kernel/setup.c
index af838f6..32054bf 100644
--- a/arch/x86_64/kernel/setup.c
+++ b/arch/x86_64/kernel/setup.c
@@ -546,6 +546,37 @@ static void __init amd_detect_cmp(struct cpuinfo_x86 *c)
#endif
}

+#define ENABLE_C1E_MASK 0x18000000
+#define CPUID_PROCESSOR_SIGNATURE 1
+#define CPUID_XFAM 0x0ff00000
+#define CPUID_XFAM_K8 0x00000000
+#define CPUID_XFAM_10H 0x00100000
+#define CPUID_XFAM_11H 0x00200000
+#define CPUID_XMOD 0x000f0000
+#define CPUID_XMOD_REV_F 0x00040000
+
+/* AMD systems with C1E don't have a working lAPIC timer. Check for that. */
+static __cpuinit int amd_apic_timer_broken(void)
+{
+ u32 lo, hi;
+ u32 eax = cpuid_eax(CPUID_PROCESSOR_SIGNATURE);
+ switch (eax & CPUID_XFAM) {
+ case CPUID_XFAM_K8:
+ if ((eax & CPUID_XMOD) < CPUID_XMOD_REV_F)
+ break;
+ case CPUID_XFAM_10H:
+ case CPUID_XFAM_11H:
+ rdmsr(MSR_K8_ENABLE_C1E, lo, hi);
+ if (lo & ENABLE_C1E_MASK)
+ return 1;
+ break;
+ default:
+ /* err on the side of caution */
+ return 1;
+ }
+ return 0;
+}
+
static void __cpuinit init_amd(struct cpuinfo_x86 *c)
{
unsigned level;
@@ -617,6 +648,9 @@ static void __cpuinit init_amd(struct cpuinfo_x86 *c)
/* Family 10 doesn't support C states in MWAIT so don't use it */
if (c->x86 == 0x10 && !force_mwait)
clear_bit(X86_FEATURE_MWAIT, &c->x86_capability);
+
+ if (amd_apic_timer_broken())
+ disable_apic_timer = 1;
}

static void __cpuinit detect_ht(struct cpuinfo_x86 *c)
diff --git a/include/asm-x86_64/apic.h b/include/asm-x86_64/apic.h
index 85125ef..e458020 100644
--- a/include/asm-x86_64/apic.h
+++ b/include/asm-x86_64/apic.h
@@ -20,6 +20,7 @@ extern int apic_verbosity;
extern int apic_runs_main_timer;
extern int ioapic_force;
extern int apic_mapped;
+extern int disable_apic_timer;

/*
* Define the default level of output to be very little


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email]majordomo@vger.kernel.org[/email]
More majordomo info at [url]http://vger.kernel.org/majordomo-info.html[/url]
Please read the FAQ at [url]http://www.tux.org/lkml/[/url]

On Tuesday, 25 September 2007 21:37, Thomas Gleixner wrote:[color=blue]
> commit 3556ddfa9284a86a59a9b78fe5894430f6ab4eef titled
>
> [PATCH] x86-64: Disable local APIC timer use on AMD systems with C1E
>
> solves a problem with AMD dual core laptops e.g. HP nx6325 (Turion 64
> X2) with C1E enabled:
>
> When both cores go into idle at the same time, then the system switches
> into C1E state, which is basically the same as C3. This stops the local
> apic timer.
>
> This was debugged right after the dyntick merge on i386 and despite the
> patch title it fixes only the 32 bit path.
>
> x86_64 is still missing this fix. It seems that mainline is not really
> affected by this issue, as the PIT is running and keeps jiffies
> incrementing, but that's just waiting for trouble.
>
> -mm suffers from this problem due to the x86_64 high resolution timer
> patches.
>
> This is a quick and dirty port of the i386 code to x86_64.
>
> I spent quite a time with Rafael to debug the -mm / hrt wreckage until
> someone pointed us to this. I really had forgotten that we debugged this
> half a year ago already.
>
> Sigh, is it just me or is there something yelling arch/x86 into my ear?
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>[/color]

I have reworked the patch a bit so that it applies on top of 2.6.23-rc8-mm1
and compiles (my version is attached).

With this patch applied, the kernel boots correctly on the nx6325.

Greetings,
Rafael

On Tue, 2007-09-25 at 22:55 +0200, Rafael J. Wysocki wrote:[color=blue]
> I have reworked the patch a bit so that it applies on top of 2.6.23-rc8-mm1
> and compiles (my version is attached).
>
> With this patch applied, the kernel boots correctly on the nx6325.[/color]

I know. It's basically enforced "noapictimer".

But this still does not explain why your nasty box booted current
mainline with "apicmaintimer" on the kernel command line.

tglx


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email]majordomo@vger.kernel.org[/email]
More majordomo info at [url]http://vger.kernel.org/majordomo-info.html[/url]
Please read the FAQ at [url]http://www.tux.org/lkml/[/url]

On Tuesday, 25 September 2007 22:55, Thomas Gleixner wrote:[color=blue]
> On Tue, 2007-09-25 at 22:55 +0200, Rafael J. Wysocki wrote:[color=green]
> > I have reworked the patch a bit so that it applies on top of 2.6.23-rc8-mm1
> > and compiles (my version is attached).
> >
> > With this patch applied, the kernel boots correctly on the nx6325.[/color]
>
> I know. It's basically enforced "noapictimer".
>
> But this still does not explain why your nasty box booted current
> mainline with "apicmaintimer" on the kernel command line.[/color]

Now it doesn't do that any more.

As I wrote in the other thread, I think that _sometimes_ the CPU doesn't enter
the C1E state (or how it's called) and then the box boots and works.

Greetings,
Rafael
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email]majordomo@vger.kernel.org[/email]
More majordomo info at [url]http://vger.kernel.org/majordomo-info.html[/url]
Please read the FAQ at [url]http://www.tux.org/lkml/[/url]

* Rafael J. Wysocki <rjw@sisk.pl> wrote:
[color=blue]
> On Tuesday, 25 September 2007 22:55, Thomas Gleixner wrote:[color=green]
> > On Tue, 2007-09-25 at 22:55 +0200, Rafael J. Wysocki wrote:[color=darkred]
> > > I have reworked the patch a bit so that it applies on top of 2.6.23-rc8-mm1
> > > and compiles (my version is attached).
> > >
> > > With this patch applied, the kernel boots correctly on the nx6325.[/color]
> >
> > I know. It's basically enforced "noapictimer".
> >
> > But this still does not explain why your nasty box booted current
> > mainline with "apicmaintimer" on the kernel command line.[/color]
>
> Now it doesn't do that any more.
>
> As I wrote in the other thread, I think that _sometimes_ the CPU
> doesn't enter the C1E state (or how it's called) and then the box
> boots and works.[/color]

Can you see any correlation between being on AC or on battery and this
bug occuring? Sometimes the BIOS might even do C states depending on
whether the box was booted up on AC or on battery. (and sometimes the
kernel calibrates things differently when booting up on battery [lower
speed] - so this can have an effect too.)

Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email]majordomo@vger.kernel.org[/email]
More majordomo info at [url]http://vger.kernel.org/majordomo-info.html[/url]
Please read the FAQ at [url]http://www.tux.org/lkml/[/url]

On Wednesday, 26 September 2007 10:31, Ingo Molnar wrote:[color=blue]
>
> * Rafael J. Wysocki <rjw@sisk.pl> wrote:
>[color=green]
> > On Tuesday, 25 September 2007 22:55, Thomas Gleixner wrote:[color=darkred]
> > > On Tue, 2007-09-25 at 22:55 +0200, Rafael J. Wysocki wrote:
> > > > I have reworked the patch a bit so that it applies on top of 2.6.23-rc8-mm1
> > > > and compiles (my version is attached).
> > > >
> > > > With this patch applied, the kernel boots correctly on the nx6325.
> > >
> > > I know. It's basically enforced "noapictimer".
> > >
> > > But this still does not explain why your nasty box booted current
> > > mainline with "apicmaintimer" on the kernel command line.[/color]
> >
> > Now it doesn't do that any more.
> >
> > As I wrote in the other thread, I think that _sometimes_ the CPU
> > doesn't enter the C1E state (or how it's called) and then the box
> > boots and works.[/color]
>
> Can you see any correlation between being on AC or on battery and this
> bug occuring?[/color]

No. All tests were done with the AC power supply plugged in.
[color=blue]
> Sometimes the BIOS might even do C states depending on
> whether the box was booted up on AC or on battery. (and sometimes the
> kernel calibrates things differently when booting up on battery [lower
> speed] - so this can have an effect too.)[/color]

On my box there are hidden history effects related to other things too (eg.
sometimes after a fresh boot the processor generates lots of ACPI events
such that it takes 25% of CPU power to handle them alone).

Greetings,
Rafael
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email]majordomo@vger.kernel.org[/email]
More majordomo info at [url]http://vger.kernel.org/majordomo-info.html[/url]
Please read the FAQ at [url]http://www.tux.org/lkml/[/url]