2.6.28-rc4 mem_cgroup_charge_common panic - Kernel

This is a discussion on 2.6.28-rc4 mem_cgroup_charge_common panic - Kernel ; Hi KAME, Thank you for the fix for online/offline page_cgroup panic. While running memory offline/online tests ran into another mem_cgroup panic. Thanks, Badari Unable to handle kernel paging request for data at address 0x00000020 Faulting instruction address: 0xc0000000001055e4 Oops: Kernel ...

+ Reply to Thread
Results 1 to 3 of 3

Thread: 2.6.28-rc4 mem_cgroup_charge_common panic

  1. 2.6.28-rc4 mem_cgroup_charge_common panic

    Hi KAME,

    Thank you for the fix for online/offline page_cgroup panic.

    While running memory offline/online tests ran into another
    mem_cgroup panic.

    Thanks,
    Badari

    Unable to handle kernel paging request for data at address 0x00000020
    Faulting instruction address: 0xc0000000001055e4
    Oops: Kernel access of bad area, sig: 11 [#2]
    SMP NR_CPUS=32 NUMA pSeries
    Modules linked in:
    NIP: c0000000001055e4 LR: c00000000010557c CTR: c0000000000bfb74
    REGS: c0000000f6c7f1b0 TRAP: 0300 Tainted: G D (2.6.28-rc4)
    MSR: 8000000000009032 CR: 44044422 XER: 20000018
    DAR: 0000000000000020, DSISR: 0000000042000000
    TASK = c0000000f6c56cc0[4610] 'crash' THREAD: c0000000f6c7c000 CPU: 0
    GPR00: c0000000e910b560 c0000000f6c7f430 c000000000b36fc0 0000000000000001
    GPR04: c000000005355278 0000000000000001 0000000000000000 0000000000000000
    GPR08: c000000005355290 0000000000000018 c0000000e910b558 c0000000e910b548
    GPR12: 0000000000000000 c000000000b58300 00000400001ca30a 0000000000000000
    GPR16: 0000000000000000 0000000000000006 c0000000d43cb5c0 c0000000e66d0b88
    GPR20: 0000000000000004 0000000000000000 c0000000e64c6180 0000000000000000
    GPR24: 00000000000000d0 0000000000000005 c000000000bac418 0000000000000001
    GPR28: c0000000e910b538 c000000005355278 c000000000aacad8 c0000000f6c7f430
    NIP [c0000000001055e4] .mem_cgroup_charge_common+0x26c/0x330
    LR [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    Call Trace:
    [c0000000f6c7f430] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330 (unreliable)
    [c0000000f6c7f4f0] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    [c0000000f6c7f590] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    [c0000000f6c7f640] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    [c0000000f6c7f6e0] [c000000000144348] .mpage_readpages+0xc8/0x170
    [c0000000f6c7f810] [c000000000182e68] .reiserfs_readpages+0x50/0x78
    [c0000000f6c7f8b0] [c0000000000cee80] .__do_page_cache_readahead+0x174/0x280
    [c0000000f6c7f980] [c0000000000cf6e0] .do_page_cache_readahead+0xa4/0xd0
    [c0000000f6c7fa20] [c0000000000c5274] .filemap_fault+0x198/0x420
    [c0000000f6c7fb00] [c0000000000d9660] .__do_fault+0xb8/0x664
    [c0000000f6c7fc10] [c0000000000dbcc4] .handle_mm_fault+0x1ec/0xaf4
    [c0000000f6c7fd00] [c0000000005a8b10] .do_page_fault+0x384/0x570
    [c0000000f6c7fe30] [c00000000000517c] handle_page_fault+0x20/0x5c
    Instruction dump:
    794a26e4 391d0018 38a00001 7d6be214 7d5c5214 7fa4eb78 e92b0048 380a0008
    39290001 f92b0048 60000000 e92a0008 f93d0018 f8080008 f90a0008
    ---[ end trace aaa19ed35042c148 ]---
    BUG: soft lockup - CPU#1 stuck for 61s! [udevd:1249]
    Modules linked in:
    NIP: c0000000005a69fc LR: c0000000005a69f4 CTR: c0000000000bfb74
    REGS: c0000000e7f9b040 TRAP: 0901 Tainted: G D (2.6.28-rc4)
    MSR: 8000000000009032 CR: 80004424 XER: 20000018
    TASK = c0000000e9b5ccc0[1249] 'udevd' THREAD: c0000000e7f98000 CPU: 1
    GPR00: 00000000c0000000 c0000000e7f9b2c0 c000000000b36fc0 0000000000000001
    GPR04: c00000000010557c c0000000000bfb74 0000000000000000 0000000000000000
    GPR08: c000000000bd7700 00000000c0000000 00000000004d3000 c0000000007296c0
    GPR12: 000000000000d032 c000000000b58500
    NIP [c0000000005a69fc] ._spin_lock_irqsave+0x84/0xd4
    LR [c0000000005a69f4] ._spin_lock_irqsave+0x7c/0xd4
    Call Trace:
    [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    Instruction dump:
    40a2fff0 4c00012c 2fa90000 41be0050 8b8d01da 2fbd0000 38600000 419e0008
    7fa3eb78 4ba65179 60000000 7c210b78 <801b0000> 2fa00000 40befff4 7c421378
    RCU detected CPU 1 stall (t=4299517593/1725750 jiffies)
    Call Trace:
    [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    LR = ._spin_lock_irqsave+0x7c/0xd4
    [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    RCU detected CPU 1 stall (t=4299525093/1733250 jiffies)
    Call Trace:
    [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    LR = ._spin_lock_irqsave+0x7c/0xd4
    [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    RCU detected CPU 1 stall (t=4299532593/1740750 jiffies)
    Call Trace:
    [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    LR = ._spin_lock_irqsave+0x7c/0xd4
    [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    Unable to handle kernel paging request for data at address 0x00000008
    Faulting instruction address: 0xc0000000001055e4
    Oops: Kernel access of bad area, sig: 11 [#3]
    SMP NR_CPUS=32 NUMA pSeries
    Modules linked in:
    NIP: c0000000001055e4 LR: c00000000010557c CTR: c0000000000bfb74
    REGS: c0000000f6c87720 TRAP: 0300 Tainted: G D (2.6.28-rc4)
    MSR: 8000000000009032 CR: 28044482 XER: 20000010
    DAR: 0000000000000008, DSISR: 0000000042000000
    TASK = c0000000f6bdecc0[4614] 'sshd' THREAD: c0000000f6c84000 CPU: 3
    GPR00: c0000000e9009150 c0000000f6c879a0 c000000000b36fc0 0000000000000001
    GPR04: c000000005355688 0000000000000001 0000000000000001 0000000000000000
    GPR08: c0000000053556a0 0000000000000000 c0000000e9009148 c0000000e9009140
    GPR12: 0000000000000000 c000000000b58900 00000400000382d0 0000000000000006
    GPR16: 0000000000000000 0000000000000001 0000000000000001 c0000000e612e818
    GPR20: 00000fffffdba4e0 0000040000744d98 c0000000e66d2138 0000000000000001
    GPR24: 00000000000000d0 0000000000000005 c000000000bac418 0000000000000001
    GPR28: c0000000e9009138 c000000005355688 c000000000aacad8 c0000000f6c879a0
    NIP [c0000000001055e4] .mem_cgroup_charge_common+0x26c/0x330
    LR [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    Call Trace:
    [c0000000f6c879a0] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330 (unreliable)
    [c0000000f6c87a60] [c0000000001057e4] .mem_cgroup_charge+0x9c/0xc8
    [c0000000f6c87b00] [c0000000000d96fc] .__do_fault+0x154/0x664
    [c0000000f6c87c10] [c0000000000dbcc4] .handle_mm_fault+0x1ec/0xaf4
    [c0000000f6c87d00] [c0000000005a8b10] .do_page_fault+0x384/0x570
    [c0000000f6c87e30] [c00000000000517c] handle_page_fault+0x20/0x5c
    Instruction dump:
    794a26e4 391d0018 38a00001 7d6be214 7d5c5214 7fa4eb78 e92b0048 380a0008
    39290001 f92b0048 60000000 e92a0008 f93d0018 f8080008 f90a0008
    ---[ end trace aaa19ed35042c148 ]---
    RCU detected CPU 1 stall (t=4299540093/1748250 jiffies)
    Call Trace:
    [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    LR = ._spin_lock_irqsave+0x7c/0xd4
    [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    BUG: soft lockup - CPU#0 stuck for 61s! [sshd:3665]
    Modules linked in:
    NIP: c0000000005a69fc LR: c0000000005a69f4 CTR: c0000000000bfb74
    REGS: c0000000e667f6c0 TRAP: 0901 Tainted: G D (2.6.28-rc4)
    MSR: 8000000000009032 CR: 88004484 XER: 20000010
    TASK = c0000000e9905980[3665] 'sshd' THREAD: c0000000e667c000 CPU: 0
    GPR00: 0000000080000000 c0000000e667f940 c000000000b36fc0 0000000000000001
    GPR04: c00000000010557c c0000000000bfb74 0000000000000001 0000000000000000
    GPR08: c000000000bd7700 0000000080000000 00000000004cc000 c0000000007296c0
    GPR12: 0000000000000000 c000000000b58300
    NIP [c0000000005a69fc] ._spin_lock_irqsave+0x84/0xd4
    LR [c0000000005a69f4] ._spin_lock_irqsave+0x7c/0xd4
    Call Trace:
    [c0000000e667f940] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    [c0000000e667f9e0] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    [c0000000e667faa0] [c0000000001057e4] .mem_cgroup_charge+0x9c/0xc8
    [c0000000e667fb40] [c0000000000da170] .do_wp_page+0x564/0x8ec
    [c0000000e667fc10] [c0000000000dc500] .handle_mm_fault+0xa28/0xaf4
    [c0000000e667fd00] [c0000000005a8b10] .do_page_fault+0x384/0x570
    [c0000000e667fe30] [c00000000000517c] handle_page_fault+0x20/0x5c
    Instruction dump:









    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  2. Re: 2.6.28-rc4 mem_cgroup_charge_common panic

    On Mon, 10 Nov 2008 13:43:28 -0800
    Badari Pulavarty wrote:

    > Hi KAME,
    >
    > Thank you for the fix for online/offline page_cgroup panic.
    >
    > While running memory offline/online tests ran into another
    > mem_cgroup panic.
    >


    Hm, should I avoid freeing mem_cgroup at memory Offline ?
    (memmap is also not free AFAIK.)

    Anyway, I'll dig this. thanks.

    -Kame

    > Thanks,
    > Badari
    >
    > Unable to handle kernel paging request for data at address 0x00000020
    > Faulting instruction address: 0xc0000000001055e4
    > Oops: Kernel access of bad area, sig: 11 [#2]
    > SMP NR_CPUS=32 NUMA pSeries
    > Modules linked in:
    > NIP: c0000000001055e4 LR: c00000000010557c CTR: c0000000000bfb74
    > REGS: c0000000f6c7f1b0 TRAP: 0300 Tainted: G D (2.6.28-rc4)
    > MSR: 8000000000009032 CR: 44044422 XER: 20000018
    > DAR: 0000000000000020, DSISR: 0000000042000000
    > TASK = c0000000f6c56cc0[4610] 'crash' THREAD: c0000000f6c7c000 CPU: 0
    > GPR00: c0000000e910b560 c0000000f6c7f430 c000000000b36fc0 0000000000000001
    > GPR04: c000000005355278 0000000000000001 0000000000000000 0000000000000000
    > GPR08: c000000005355290 0000000000000018 c0000000e910b558 c0000000e910b548
    > GPR12: 0000000000000000 c000000000b58300 00000400001ca30a 0000000000000000
    > GPR16: 0000000000000000 0000000000000006 c0000000d43cb5c0 c0000000e66d0b88
    > GPR20: 0000000000000004 0000000000000000 c0000000e64c6180 0000000000000000
    > GPR24: 00000000000000d0 0000000000000005 c000000000bac418 0000000000000001
    > GPR28: c0000000e910b538 c000000005355278 c000000000aacad8 c0000000f6c7f430
    > NIP [c0000000001055e4] .mem_cgroup_charge_common+0x26c/0x330
    > LR [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > Call Trace:
    > [c0000000f6c7f430] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330 (unreliable)
    > [c0000000f6c7f4f0] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > [c0000000f6c7f590] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > [c0000000f6c7f640] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > [c0000000f6c7f6e0] [c000000000144348] .mpage_readpages+0xc8/0x170
    > [c0000000f6c7f810] [c000000000182e68] .reiserfs_readpages+0x50/0x78
    > [c0000000f6c7f8b0] [c0000000000cee80] .__do_page_cache_readahead+0x174/0x280
    > [c0000000f6c7f980] [c0000000000cf6e0] .do_page_cache_readahead+0xa4/0xd0
    > [c0000000f6c7fa20] [c0000000000c5274] .filemap_fault+0x198/0x420
    > [c0000000f6c7fb00] [c0000000000d9660] .__do_fault+0xb8/0x664
    > [c0000000f6c7fc10] [c0000000000dbcc4] .handle_mm_fault+0x1ec/0xaf4
    > [c0000000f6c7fd00] [c0000000005a8b10] .do_page_fault+0x384/0x570
    > [c0000000f6c7fe30] [c00000000000517c] handle_page_fault+0x20/0x5c
    > Instruction dump:
    > 794a26e4 391d0018 38a00001 7d6be214 7d5c5214 7fa4eb78 e92b0048 380a0008
    > 39290001 f92b0048 60000000 e92a0008 f93d0018 f8080008 f90a0008
    > ---[ end trace aaa19ed35042c148 ]---
    > BUG: soft lockup - CPU#1 stuck for 61s! [udevd:1249]
    > Modules linked in:
    > NIP: c0000000005a69fc LR: c0000000005a69f4 CTR: c0000000000bfb74
    > REGS: c0000000e7f9b040 TRAP: 0901 Tainted: G D (2.6.28-rc4)
    > MSR: 8000000000009032 CR: 80004424 XER: 20000018
    > TASK = c0000000e9b5ccc0[1249] 'udevd' THREAD: c0000000e7f98000 CPU: 1
    > GPR00: 00000000c0000000 c0000000e7f9b2c0 c000000000b36fc0 0000000000000001
    > GPR04: c00000000010557c c0000000000bfb74 0000000000000000 0000000000000000
    > GPR08: c000000000bd7700 00000000c0000000 00000000004d3000 c0000000007296c0
    > GPR12: 000000000000d032 c000000000b58500
    > NIP [c0000000005a69fc] ._spin_lock_irqsave+0x84/0xd4
    > LR [c0000000005a69f4] ._spin_lock_irqsave+0x7c/0xd4
    > Call Trace:
    > [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    > [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    > [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    > [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    > [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    > [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    > [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    > [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    > [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    > [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    > Instruction dump:
    > 40a2fff0 4c00012c 2fa90000 41be0050 8b8d01da 2fbd0000 38600000 419e0008
    > 7fa3eb78 4ba65179 60000000 7c210b78 <801b0000> 2fa00000 40befff4 7c421378
    > RCU detected CPU 1 stall (t=4299517593/1725750 jiffies)
    > Call Trace:
    > [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    > [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    > [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    > [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    > [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    > [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    > [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    > [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    > [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    > [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    > --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    > LR = ._spin_lock_irqsave+0x7c/0xd4
    > [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    > [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    > [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    > [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    > [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    > [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    > [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    > [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    > [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    > [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    > RCU detected CPU 1 stall (t=4299525093/1733250 jiffies)
    > Call Trace:
    > [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    > [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    > [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    > [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    > [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    > [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    > [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    > [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    > [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    > [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    > --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    > LR = ._spin_lock_irqsave+0x7c/0xd4
    > [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    > [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    > [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    > [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    > [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    > [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    > [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    > [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    > [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    > [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    > RCU detected CPU 1 stall (t=4299532593/1740750 jiffies)
    > Call Trace:
    > [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    > [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    > [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    > [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    > [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    > [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    > [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    > [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    > [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    > [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    > --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    > LR = ._spin_lock_irqsave+0x7c/0xd4
    > [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    > [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    > [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    > [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    > [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    > [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    > [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    > [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    > [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    > [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    > Unable to handle kernel paging request for data at address 0x00000008
    > Faulting instruction address: 0xc0000000001055e4
    > Oops: Kernel access of bad area, sig: 11 [#3]
    > SMP NR_CPUS=32 NUMA pSeries
    > Modules linked in:
    > NIP: c0000000001055e4 LR: c00000000010557c CTR: c0000000000bfb74
    > REGS: c0000000f6c87720 TRAP: 0300 Tainted: G D (2.6.28-rc4)
    > MSR: 8000000000009032 CR: 28044482 XER: 20000010
    > DAR: 0000000000000008, DSISR: 0000000042000000
    > TASK = c0000000f6bdecc0[4614] 'sshd' THREAD: c0000000f6c84000 CPU: 3
    > GPR00: c0000000e9009150 c0000000f6c879a0 c000000000b36fc0 0000000000000001
    > GPR04: c000000005355688 0000000000000001 0000000000000001 0000000000000000
    > GPR08: c0000000053556a0 0000000000000000 c0000000e9009148 c0000000e9009140
    > GPR12: 0000000000000000 c000000000b58900 00000400000382d0 0000000000000006
    > GPR16: 0000000000000000 0000000000000001 0000000000000001 c0000000e612e818
    > GPR20: 00000fffffdba4e0 0000040000744d98 c0000000e66d2138 0000000000000001
    > GPR24: 00000000000000d0 0000000000000005 c000000000bac418 0000000000000001
    > GPR28: c0000000e9009138 c000000005355688 c000000000aacad8 c0000000f6c879a0
    > NIP [c0000000001055e4] .mem_cgroup_charge_common+0x26c/0x330
    > LR [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > Call Trace:
    > [c0000000f6c879a0] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330 (unreliable)
    > [c0000000f6c87a60] [c0000000001057e4] .mem_cgroup_charge+0x9c/0xc8
    > [c0000000f6c87b00] [c0000000000d96fc] .__do_fault+0x154/0x664
    > [c0000000f6c87c10] [c0000000000dbcc4] .handle_mm_fault+0x1ec/0xaf4
    > [c0000000f6c87d00] [c0000000005a8b10] .do_page_fault+0x384/0x570
    > [c0000000f6c87e30] [c00000000000517c] handle_page_fault+0x20/0x5c
    > Instruction dump:
    > 794a26e4 391d0018 38a00001 7d6be214 7d5c5214 7fa4eb78 e92b0048 380a0008
    > 39290001 f92b0048 60000000 e92a0008 f93d0018 f8080008 f90a0008
    > ---[ end trace aaa19ed35042c148 ]---
    > RCU detected CPU 1 stall (t=4299540093/1748250 jiffies)
    > Call Trace:
    > [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    > [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    > [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    > [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    > [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    > [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    > [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    > [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    > [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    > [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    > --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    > LR = ._spin_lock_irqsave+0x7c/0xd4
    > [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    > [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    > [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    > [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    > [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    > [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    > [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    > [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    > [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    > [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    > BUG: soft lockup - CPU#0 stuck for 61s! [sshd:3665]
    > Modules linked in:
    > NIP: c0000000005a69fc LR: c0000000005a69f4 CTR: c0000000000bfb74
    > REGS: c0000000e667f6c0 TRAP: 0901 Tainted: G D (2.6.28-rc4)
    > MSR: 8000000000009032 CR: 88004484 XER: 20000010
    > TASK = c0000000e9905980[3665] 'sshd' THREAD: c0000000e667c000 CPU: 0
    > GPR00: 0000000080000000 c0000000e667f940 c000000000b36fc0 0000000000000001
    > GPR04: c00000000010557c c0000000000bfb74 0000000000000001 0000000000000000
    > GPR08: c000000000bd7700 0000000080000000 00000000004cc000 c0000000007296c0
    > GPR12: 0000000000000000 c000000000b58300
    > NIP [c0000000005a69fc] ._spin_lock_irqsave+0x84/0xd4
    > LR [c0000000005a69f4] ._spin_lock_irqsave+0x7c/0xd4
    > Call Trace:
    > [c0000000e667f940] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > [c0000000e667f9e0] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > [c0000000e667faa0] [c0000000001057e4] .mem_cgroup_charge+0x9c/0xc8
    > [c0000000e667fb40] [c0000000000da170] .do_wp_page+0x564/0x8ec
    > [c0000000e667fc10] [c0000000000dc500] .handle_mm_fault+0xa28/0xaf4
    > [c0000000e667fd00] [c0000000005a8b10] .do_page_fault+0x384/0x570
    > [c0000000e667fe30] [c00000000000517c] handle_page_fault+0x20/0x5c
    > Instruction dump:
    >
    >
    >
    >
    >
    >
    >
    >
    >
    >


    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  3. Re: 2.6.28-rc4 mem_cgroup_charge_common panic

    On Tue, 11 Nov 2008 10:14:40 +0900
    KAMEZAWA Hiroyuki wrote:

    > On Mon, 10 Nov 2008 13:43:28 -0800
    > Badari Pulavarty wrote:
    >
    > > Hi KAME,
    > >
    > > Thank you for the fix for online/offline page_cgroup panic.
    > >
    > > While running memory offline/online tests ran into another
    > > mem_cgroup panic.
    > >

    >
    > Hm, should I avoid freeing mem_cgroup at memory Offline ?
    > (memmap is also not free AFAIK.)
    >
    > Anyway, I'll dig this. thanks.
    >

    it seems not the same kind of bug..

    Could you give me disassemble of mem_cgroup_charge_common() ?
    (I'm not sure I can read ppc asm but I want to know what is "0x20"
    of fault address....)

    As first impression, it comes from page migration..
    rc4's page migration handler of memcg handles *usual* path but not so good.

    new migration code of memcg in mmotm is much better, I think.
    Could you try mmotm if you have time ?

    Thanks,
    -Kame


    > -Kame
    >
    > > Thanks,
    > > Badari
    > >
    > > Unable to handle kernel paging request for data at address 0x00000020
    > > Faulting instruction address: 0xc0000000001055e4
    > > Oops: Kernel access of bad area, sig: 11 [#2]
    > > SMP NR_CPUS=32 NUMA pSeries
    > > Modules linked in:
    > > NIP: c0000000001055e4 LR: c00000000010557c CTR: c0000000000bfb74
    > > REGS: c0000000f6c7f1b0 TRAP: 0300 Tainted: G D (2.6.28-rc4)
    > > MSR: 8000000000009032 CR: 44044422 XER: 20000018
    > > DAR: 0000000000000020, DSISR: 0000000042000000
    > > TASK = c0000000f6c56cc0[4610] 'crash' THREAD: c0000000f6c7c000 CPU: 0
    > > GPR00: c0000000e910b560 c0000000f6c7f430 c000000000b36fc0 0000000000000001
    > > GPR04: c000000005355278 0000000000000001 0000000000000000 0000000000000000
    > > GPR08: c000000005355290 0000000000000018 c0000000e910b558 c0000000e910b548
    > > GPR12: 0000000000000000 c000000000b58300 00000400001ca30a 0000000000000000
    > > GPR16: 0000000000000000 0000000000000006 c0000000d43cb5c0 c0000000e66d0b88
    > > GPR20: 0000000000000004 0000000000000000 c0000000e64c6180 0000000000000000
    > > GPR24: 00000000000000d0 0000000000000005 c000000000bac418 0000000000000001
    > > GPR28: c0000000e910b538 c000000005355278 c000000000aacad8 c0000000f6c7f430
    > > NIP [c0000000001055e4] .mem_cgroup_charge_common+0x26c/0x330
    > > LR [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > > Call Trace:
    > > [c0000000f6c7f430] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330 (unreliable)
    > > [c0000000f6c7f4f0] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > > [c0000000f6c7f590] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > > [c0000000f6c7f640] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > > [c0000000f6c7f6e0] [c000000000144348] .mpage_readpages+0xc8/0x170
    > > [c0000000f6c7f810] [c000000000182e68] .reiserfs_readpages+0x50/0x78
    > > [c0000000f6c7f8b0] [c0000000000cee80] .__do_page_cache_readahead+0x174/0x280
    > > [c0000000f6c7f980] [c0000000000cf6e0] .do_page_cache_readahead+0xa4/0xd0
    > > [c0000000f6c7fa20] [c0000000000c5274] .filemap_fault+0x198/0x420
    > > [c0000000f6c7fb00] [c0000000000d9660] .__do_fault+0xb8/0x664
    > > [c0000000f6c7fc10] [c0000000000dbcc4] .handle_mm_fault+0x1ec/0xaf4
    > > [c0000000f6c7fd00] [c0000000005a8b10] .do_page_fault+0x384/0x570
    > > [c0000000f6c7fe30] [c00000000000517c] handle_page_fault+0x20/0x5c
    > > Instruction dump:
    > > 794a26e4 391d0018 38a00001 7d6be214 7d5c5214 7fa4eb78 e92b0048 380a0008
    > > 39290001 f92b0048 60000000 e92a0008 f93d0018 f8080008 f90a0008
    > > ---[ end trace aaa19ed35042c148 ]---
    > > BUG: soft lockup - CPU#1 stuck for 61s! [udevd:1249]
    > > Modules linked in:
    > > NIP: c0000000005a69fc LR: c0000000005a69f4 CTR: c0000000000bfb74
    > > REGS: c0000000e7f9b040 TRAP: 0901 Tainted: G D (2.6.28-rc4)
    > > MSR: 8000000000009032 CR: 80004424 XER: 20000018
    > > TASK = c0000000e9b5ccc0[1249] 'udevd' THREAD: c0000000e7f98000 CPU: 1
    > > GPR00: 00000000c0000000 c0000000e7f9b2c0 c000000000b36fc0 0000000000000001
    > > GPR04: c00000000010557c c0000000000bfb74 0000000000000000 0000000000000000
    > > GPR08: c000000000bd7700 00000000c0000000 00000000004d3000 c0000000007296c0
    > > GPR12: 000000000000d032 c000000000b58500
    > > NIP [c0000000005a69fc] ._spin_lock_irqsave+0x84/0xd4
    > > LR [c0000000005a69f4] ._spin_lock_irqsave+0x7c/0xd4
    > > Call Trace:
    > > [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > > [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > > [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > > [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > > [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > > [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    > > [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    > > [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    > > [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    > > [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    > > [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    > > [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    > > [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    > > [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    > > [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    > > Instruction dump:
    > > 40a2fff0 4c00012c 2fa90000 41be0050 8b8d01da 2fbd0000 38600000 419e0008
    > > 7fa3eb78 4ba65179 60000000 7c210b78 <801b0000> 2fa00000 40befff4 7c421378
    > > RCU detected CPU 1 stall (t=4299517593/1725750 jiffies)
    > > Call Trace:
    > > [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    > > [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    > > [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    > > [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    > > [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    > > [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    > > [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    > > [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    > > [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    > > [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    > > --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    > > LR = ._spin_lock_irqsave+0x7c/0xd4
    > > [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > > [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > > [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > > [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > > [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > > [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    > > [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    > > [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    > > [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    > > [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    > > [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    > > [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    > > [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    > > [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    > > [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    > > RCU detected CPU 1 stall (t=4299525093/1733250 jiffies)
    > > Call Trace:
    > > [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    > > [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    > > [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    > > [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    > > [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    > > [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    > > [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    > > [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    > > [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    > > [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    > > --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    > > LR = ._spin_lock_irqsave+0x7c/0xd4
    > > [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > > [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > > [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > > [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > > [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > > [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    > > [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    > > [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    > > [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    > > [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    > > [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    > > [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    > > [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    > > [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    > > [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    > > RCU detected CPU 1 stall (t=4299532593/1740750 jiffies)
    > > Call Trace:
    > > [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    > > [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    > > [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    > > [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    > > [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    > > [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    > > [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    > > [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    > > [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    > > [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    > > --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    > > LR = ._spin_lock_irqsave+0x7c/0xd4
    > > [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > > [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > > [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > > [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > > [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > > [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    > > [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    > > [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    > > [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    > > [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    > > [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    > > [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    > > [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    > > [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    > > [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    > > Unable to handle kernel paging request for data at address 0x00000008
    > > Faulting instruction address: 0xc0000000001055e4
    > > Oops: Kernel access of bad area, sig: 11 [#3]
    > > SMP NR_CPUS=32 NUMA pSeries
    > > Modules linked in:
    > > NIP: c0000000001055e4 LR: c00000000010557c CTR: c0000000000bfb74
    > > REGS: c0000000f6c87720 TRAP: 0300 Tainted: G D (2.6.28-rc4)
    > > MSR: 8000000000009032 CR: 28044482 XER: 20000010
    > > DAR: 0000000000000008, DSISR: 0000000042000000
    > > TASK = c0000000f6bdecc0[4614] 'sshd' THREAD: c0000000f6c84000 CPU: 3
    > > GPR00: c0000000e9009150 c0000000f6c879a0 c000000000b36fc0 0000000000000001
    > > GPR04: c000000005355688 0000000000000001 0000000000000001 0000000000000000
    > > GPR08: c0000000053556a0 0000000000000000 c0000000e9009148 c0000000e9009140
    > > GPR12: 0000000000000000 c000000000b58900 00000400000382d0 0000000000000006
    > > GPR16: 0000000000000000 0000000000000001 0000000000000001 c0000000e612e818
    > > GPR20: 00000fffffdba4e0 0000040000744d98 c0000000e66d2138 0000000000000001
    > > GPR24: 00000000000000d0 0000000000000005 c000000000bac418 0000000000000001
    > > GPR28: c0000000e9009138 c000000005355688 c000000000aacad8 c0000000f6c879a0
    > > NIP [c0000000001055e4] .mem_cgroup_charge_common+0x26c/0x330
    > > LR [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > > Call Trace:
    > > [c0000000f6c879a0] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330 (unreliable)
    > > [c0000000f6c87a60] [c0000000001057e4] .mem_cgroup_charge+0x9c/0xc8
    > > [c0000000f6c87b00] [c0000000000d96fc] .__do_fault+0x154/0x664
    > > [c0000000f6c87c10] [c0000000000dbcc4] .handle_mm_fault+0x1ec/0xaf4
    > > [c0000000f6c87d00] [c0000000005a8b10] .do_page_fault+0x384/0x570
    > > [c0000000f6c87e30] [c00000000000517c] handle_page_fault+0x20/0x5c
    > > Instruction dump:
    > > 794a26e4 391d0018 38a00001 7d6be214 7d5c5214 7fa4eb78 e92b0048 380a0008
    > > 39290001 f92b0048 60000000 e92a0008 f93d0018 f8080008 f90a0008
    > > ---[ end trace aaa19ed35042c148 ]---
    > > RCU detected CPU 1 stall (t=4299540093/1748250 jiffies)
    > > Call Trace:
    > > [c0000000e7f9aa00] [c0000000000102a4] .show_stack+0x94/0x198 (unreliable)
    > > [c0000000e7f9aab0] [c0000000000103d0] .dump_stack+0x28/0x3c
    > > [c0000000e7f9ab30] [c0000000000b1020] .__rcu_pending+0xa8/0x2c4
    > > [c0000000e7f9abd0] [c0000000000b1288] .rcu_pending+0x4c/0xa0
    > > [c0000000e7f9ac60] [c000000000076a8c] .update_process_times+0x50/0xa8
    > > [c0000000e7f9ad00] [c000000000095e88] .tick_sched_timer+0xb0/0x100
    > > [c0000000e7f9adb0] [c00000000008ae98] .__run_hrtimer+0xa4/0x13c
    > > [c0000000e7f9ae50] [c00000000008c0b8] .hrtimer_interrupt+0x128/0x200
    > > [c0000000e7f9af30] [c00000000002858c] .timer_interrupt+0xc0/0x11c
    > > [c0000000e7f9afd0] [c000000000003710] decrementer_common+0x110/0x180
    > > --- Exception: 901 at ._spin_lock_irqsave+0x84/0xd4
    > > LR = ._spin_lock_irqsave+0x7c/0xd4
    > > [c0000000e7f9b2c0] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > > [c0000000e7f9b360] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > > [c0000000e7f9b420] [c000000000105c70] .mem_cgroup_cache_charge+0x130/0x154
    > > [c0000000e7f9b4c0] [c0000000000c29bc] .add_to_page_cache_locked+0x64/0x18c
    > > [c0000000e7f9b570] [c0000000000c2b64] .add_to_page_cache_lru+0x80/0xe4
    > > [c0000000e7f9b610] [c0000000000c2c34] .__grab_cache_page+0x6c/0xb4
    > > [c0000000e7f9b6b0] [c000000000187628] .reiserfs_write_begin+0xb0/0x2bc
    > > [c0000000e7f9b790] [c0000000000c38a8] .generic_file_buffered_write+0x150/0x354
    > > [c0000000e7f9b8d0] [c0000000000c40a8] .__generic_file_aio_write_nolock+0x384/0x3fc
    > > [c0000000e7f9b9d0] [c0000000000c41b0] .generic_file_aio_write+0x90/0x128
    > > [c0000000e7f9ba90] [c0000000001093a4] .do_sync_write+0xe0/0x148
    > > [c0000000e7f9bc30] [c000000000188868] .reiserfs_file_write+0x8c/0xd4
    > > [c0000000e7f9bcd0] [c000000000109d00] .vfs_write+0xf0/0x1c4
    > > [c0000000e7f9bd80] [c00000000010a69c] .sys_write+0x6c/0xb8
    > > [c0000000e7f9be30] [c00000000000852c] syscall_exit+0x0/0x40
    > > BUG: soft lockup - CPU#0 stuck for 61s! [sshd:3665]
    > > Modules linked in:
    > > NIP: c0000000005a69fc LR: c0000000005a69f4 CTR: c0000000000bfb74
    > > REGS: c0000000e667f6c0 TRAP: 0901 Tainted: G D (2.6.28-rc4)
    > > MSR: 8000000000009032 CR: 88004484 XER: 20000010
    > > TASK = c0000000e9905980[3665] 'sshd' THREAD: c0000000e667c000 CPU: 0
    > > GPR00: 0000000080000000 c0000000e667f940 c000000000b36fc0 0000000000000001
    > > GPR04: c00000000010557c c0000000000bfb74 0000000000000001 0000000000000000
    > > GPR08: c000000000bd7700 0000000080000000 00000000004cc000 c0000000007296c0
    > > GPR12: 0000000000000000 c000000000b58300
    > > NIP [c0000000005a69fc] ._spin_lock_irqsave+0x84/0xd4
    > > LR [c0000000005a69f4] ._spin_lock_irqsave+0x7c/0xd4
    > > Call Trace:
    > > [c0000000e667f940] [c0000000005a69a0] ._spin_lock_irqsave+0x28/0xd4 (unreliable)
    > > [c0000000e667f9e0] [c00000000010557c] .mem_cgroup_charge_common+0x204/0x330
    > > [c0000000e667faa0] [c0000000001057e4] .mem_cgroup_charge+0x9c/0xc8
    > > [c0000000e667fb40] [c0000000000da170] .do_wp_page+0x564/0x8ec
    > > [c0000000e667fc10] [c0000000000dc500] .handle_mm_fault+0xa28/0xaf4
    > > [c0000000e667fd00] [c0000000005a8b10] .do_page_fault+0x384/0x570
    > > [c0000000e667fe30] [c00000000000517c] handle_page_fault+0x20/0x5c
    > > Instruction dump:
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >

    >
    > --
    > To unsubscribe, send a message with 'unsubscribe linux-mm' in
    > the body to majordomo@kvack.org. For more info on Linux MM,
    > see: http://www.linux-mm.org/ .
    > Don't email: email@kvack.org
    >


    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

+ Reply to Thread