proc_stack for x86_64 (spinning process) - Kernel

This is a discussion on proc_stack for x86_64 (spinning process) - Kernel ; Hi, I have a spinning process that it's locking my server (eating 100% CPU). I can't kill it even with kill -9 I'm trying to use proc_stack to check what's the problem, compiling the kernel with http://linuxhacker.ru/~nikita/patche...oc-stack.patch But this patch ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: proc_stack for x86_64 (spinning process)

  1. proc_stack for x86_64 (spinning process)

    Hi,

    I have a spinning process that it's locking my server (eating 100% CPU). I
    can't kill it even with kill -9

    I'm trying to use proc_stack to check what's the problem, compiling the kernel
    with

    http://linuxhacker.ru/~nikita/patche...oc-stack.patch

    But this patch is only for x86 as my arch is x86_64. Does anyone know if
    there's a patch for x86_64?

    Thanks
    Nuno Fernandes
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  2. Re: proc_stack for x86_64 (spinning process)

    Nuno Pais Fernandes writes:

    > Hi,
    >
    > I have a spinning process that it's locking my server (eating 100% CPU). I
    > can't kill it even with kill -9
    >
    > I'm trying to use proc_stack to check what's the problem, compiling the kernel
    > with
    >
    > http://linuxhacker.ru/~nikita/patche...oc-stack.patch
    >
    > But this patch is only for x86 as my arch is x86_64. Does anyone know if
    > there's a patch for x86_64?


    The standard way to handle that is to just do

    echo 1 > /proc/sys/kernel/sysrq
    echo t > /proc/sysrq-trigger

    and then look for the process backtrace in the kernel log.

    Another alternative is to use crash to take a live dump and then analyse that
    using crash.

    -Andi
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  3. Re: proc_stack for x86_64 (spinning process)

    On Tuesday 08 April 2008 12:21:53 Andi Kleen wrote:
    > Nuno Pais Fernandes writes:
    > > Hi,
    > >
    > > I have a spinning process that it's locking my server (eating 100% CPU).
    > > I can't kill it even with kill -9
    > >
    > > I'm trying to use proc_stack to check what's the problem, compiling the
    > > kernel with
    > >
    > > http://linuxhacker.ru/~nikita/patche.../vm_07-proc-st
    > >ack.patch
    > >
    > > But this patch is only for x86 as my arch is x86_64. Does anyone know if
    > > there's a patch for x86_64?

    >
    > The standard way to handle that is to just do
    >
    > echo 1 > /proc/sys/kernel/sysrq
    > echo t > /proc/sysrq-trigger
    >
    > and then look for the process backtrace in the kernel log.

    I've already done that but the problem is that the "R" script does not show
    any backtrack. Apparently it's spinning inside kernel and all other processes
    become in D state (uninterrutable sleep) because of that one.

    Apr 7 15:22:03 app14 kernel: comentarios R running task 0 7615
    8051 (NOTLB)
    Apr 7 15:22:03 app14 kernel: comentarios D ffffff80e2570030 0 8150
    7328 (NOTLB)
    Apr 7 15:22:03 app14 kernel: ffffff80cc8bfcf8 0000000000000282
    00000000f69539c0 ffffff80eea385b0
    Apr 7 15:22:03 app14 kernel: ffffff80e2570030 000000000011d7be
    001769ecf4f60e59 ffffff8000e26030
    Apr 7 15:22:03 app14 kernel: ffffff80e25702c8 0000000000000000
    Apr 7 15:22:03 app14 kernel: Call
    Trace:{cfs2cfs2_dentry_revalidate+369}
    Apr 7 15:22:03 app14 kernel: {do_lookup+419}
    {__down+147}
    Apr 7 15:22:03 app14 kernel:
    {default_wake_function+0}
    {lease_alloc+96}
    Apr 7 15:22:03 app14 kernel: {__down_failed+53}
    {dummy_inode_permission+0}
    Apr 7 15:22:03 app14 kernel:
    {dummy_inode_permission+0}
    {.text.lock.open+5}
    Apr 7 15:22:03 app14 kernel: {__d_lookup+287}
    {get_write_access+62}
    Apr 7 15:22:03 app14 kernel: {may_open+481}
    {open_namei+788}
    Apr 7 15:22:03 app14 kernel: {filp_open+80}
    {dput+56}
    Apr 7 15:22:03 app14 kernel: {strncpy_from_user+74}
    {sys32_open+54}
    Apr 7 15:22:03 app14 kernel: {ia32_sysret+0}

    Thanks,
    Nuno Fernandes

    > Another alternative is to use crash to take a live dump and then analyse
    > that using crash.
    >
    > -Andi



    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  4. Re: proc_stack for x86_64 (spinning process)

    On Tue, Apr 08, 2008 at 12:41:45PM +0100, Nuno Fernandes wrote:
    > On Tuesday 08 April 2008 12:21:53 Andi Kleen wrote:
    > > Nuno Pais Fernandes writes:
    > > > Hi,
    > > >
    > > > I have a spinning process that it's locking my server (eating 100% CPU).
    > > > I can't kill it even with kill -9
    > > >
    > > > I'm trying to use proc_stack to check what's the problem, compiling the
    > > > kernel with
    > > >
    > > > http://linuxhacker.ru/~nikita/patche.../vm_07-proc-st
    > > >ack.patch
    > > >
    > > > But this patch is only for x86 as my arch is x86_64. Does anyone know if
    > > > there's a patch for x86_64?

    > >
    > > The standard way to handle that is to just do
    > >
    > > echo 1 > /proc/sys/kernel/sysrq
    > > echo t > /proc/sysrq-trigger
    > >
    > > and then look for the process backtrace in the kernel log.

    > I've already done that but the problem is that the "R" script does not show
    > any backtrack. Apparently it's spinning inside kernel and all other processes
    > become in D state (uninterrutable sleep) because of that one.


    Then you can either try to catch it with sysrq-p (but only works rarely)
    or better use the crash method. But if the process is running live dump
    might not work, you might need to force a non live dump using sysrq-c

    Another way to identify spinning code is to use a profiler like oprofile
    or readprofile. Something that spins tends to be on top.

    -Andi

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

+ Reply to Thread