Interrupt storm - BSD

This is a discussion on Interrupt storm - BSD ; I just upgraded to 6.3-RELEASE, and after the obligatory reboot I have the following message: interrupt storm detected on "irq12": throttling interrupt source On my system, irq12 is rl0. After booting, this message just scrolls up the console, printing every ...

+ Reply to Thread
Results 1 to 8 of 8

Thread: Interrupt storm

  1. Interrupt storm

    I just upgraded to 6.3-RELEASE, and after the obligatory reboot I have
    the following message:

    interrupt storm detected on "irq12": throttling interrupt source

    On my system, irq12 is rl0. After booting, this message just scrolls up
    the console, printing every couple of seconds.

    I have tried the suggestions in
    http://docs.freebsd.org/cgi/getmsg.c...reebsd-current,
    but when I do

    set debug.acpi.quirks=4

    the SCSI device scan no longer completes and I cannot boot.

    Does this mean that the Realtek card has suddenly gone bad? It does not
    seem to have affected speed too much - the commands below were run via
    SSH, over that same rl0 interface. Unfortunately I just gave my spare
    NIC away yesterday, so I can't test (yes, I know, I jinxed myself).


    Output from vmstat -i and dmesg follows:

    interrupt total rate
    irq0: clk 315202 997
    irq1: atkbd0 116 0
    irq3: sio1 1 0
    irq4: sio0 1 0
    irq5: xl0 815 2
    irq6: fdc0 8 0
    irq8: rtc 40344 127
    irq10: adv0 117 0
    irq11: ath0 ohci0 596 1
    irq12: rl0 315617 998
    irq14: ata0 2004 6
    Total 674821 2135


    Copyright (c) 1992-2008 The FreeBSD Project.
    Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    The Regents of the University of California. All rights reserved.
    FreeBSD is a registered trademark of The FreeBSD Foundation.
    FreeBSD 6.3-RELEASE #0: Wed Jan 16 04:18:52 UTC 2008
    root@dessler.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC
    Timecounter "i8254" frequency 1193182 Hz quality 0
    CPU: AMD Athlon(tm) Processor (598.84-MHz 686-class CPU)
    Origin = "AuthenticAMD" Id = 0x621 Stepping = 1

    Features=0x183f9ff
    AMD Features=0xc0400800
    real memory = 402587648 (383 MB)
    avail memory = 380260352 (362 MB)
    wlan: mac acl policy registered
    kbd1 at kbdmux0
    ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
    hptrr: HPT RocketRAID controller driver v1.1 (Jan 16 2008 04:16:19)
    ACPI disabled by blacklist. Contact your BIOS vendor.
    cpu0 on motherboard
    pcib0: pcibus 0 on motherboard
    pci0: on pcib0
    agp0: port 0xd000-0xd003 mem
    0xd8000000-0xdfffffff,0xea010000-0xea010fff at device 0.0 on pci0
    pcib1: at device 1.0 on pci0
    pci1: on pcib1
    pci1: at device 5.0 (no driver attached)
    isab0: at device 7.0 on pci0
    isa0: on isab0
    atapci0: port
    0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 7.1 on pci0
    ata0: on atapci0
    ata1: on atapci0
    pci0: at device 7.3 (no driver attached)
    ohci0: mem 0xea013000-0xea013fff irq 11 at
    device 7.4 on pci0
    ohci0: [GIANT-LOCKED]
    usb0: OHCI version 1.0, legacy support
    usb0: on ohci0
    usb0: USB revision 1.0
    uhub0: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
    uhub0: 4 ports with 4 removable, self powered
    xl0: <3Com 3c905C-TX Fast Etherlink XL> port 0xd400-0xd47f mem
    0xea014000-0xea01407f irq 5 at device 8.0 on pci0
    miibus0: on xl0
    xlphy0: <3c905C 10/100 internal PHY> on miibus0
    xlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    xl0: Ethernet address: 00:50:da:2d:f4:61
    rl0: port 0xd800-0xd8ff mem
    0xea011000-0xea0110ff irq 12 at device 9.0 on pci0
    miibus1: on rl0
    rlphy0: on miibus1
    rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
    rl0: Ethernet address: 00:50:fc:e8:a1:7d
    adv0: port 0xdc00-0xdcff mem
    0xea012000-0xea0120ff irq 10 at device 10.0 on pci0
    adv0: AdvanSys Ultra SCSI Host Adapter, SCSI ID 7, queue depth 240
    adv0: [GIANT-LOCKED]
    ath0: mem 0xea000000-0xea00ffff irq 11 at device 11.0 on pci0
    ath0: Ethernet address: 00:0f:b5:89:cf:44
    ath0: mac 7.9 phy 4.5 radio 5.6
    pci0: at device 12.0 (no driver
    attached)
    pmtimer0 on isa0
    orm0: at iomem
    0xc0000-0xc7fff,0xc8000-0xc87ff,0xc9000-0xccfff on isa0
    atkbdc0: at port 0x60,0x64 on isa0
    atkbd0: irq 1 on atkbdc0
    kbd0 at atkbd0
    atkbd0: [GIANT-LOCKED]
    fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2
    on isa0
    fdc0: [FAST]
    fd0: <1440-KB 3.5" drive> on fdc0 drive 0
    ppc0: at port 0x378-0x37f irq 7 on isa0
    ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
    ppbus0: on ppc0
    ppbus0: IEEE1284 device found /NIBBLE
    Probing for PnP devices on ppbus0:
    ppbus0: PRINTER BJ,LQ,BJL,BJRaster,BSCC
    plip0: on ppbus0
    lpt0: on ppbus0
    lpt0: Interrupt-driven port
    ppi0: on ppbus0
    sc0: at flags 0x100 on isa0
    sc0: VGA <16 virtual consoles, flags=0x300>
    sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
    sio0: type 16550A
    sio1 at port 0x2f8-0x2ff irq 3 on isa0
    sio1: type 16550A
    vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
    unknown: can't assign resources (port)
    unknown: can't assign resources (port)
    unknown: can't assign resources (memory)
    unknown: can't assign resources (port)
    unknown: can't assign resources (port)
    unknown: can't assign resources (port)
    unknown: can't assign resources (port)
    Timecounter "TSC" frequency 598841459 Hz quality 800
    Timecounters tick every 1.000 msec
    hptrr: no controller detected.
    interrupt storm detected on "irq12:"; throttling interrupt source
    ad0: 8063MB at ata0-master UDMA66
    ad1: 39205MB at ata0-slave UDMA66
    Waiting 5 seconds for SCSI devices to settle
    interrupt storm detected on "irq12:"; throttling interrupt source
    interrupt storm detected on "irq12:"; throttling interrupt source
    interrupt storm detected on "irq12:"; throttling interrupt source
    interrupt storm detected on "irq12:"; throttling interrupt source
    interrupt storm detected on "irq12:"; throttling interrupt source
    interrupt storm detected on "irq12:"; throttling interrupt source
    sa0 at adv0 bus 0 target 0 lun 0
    sa0: Removable Sequential Access SCSI-2 device
    sa0: 10.000MB/s transfers (10.000MHz, offset 15)
    cd0 at adv0 bus 0 target 3 lun 0
    cd0: Removable CD-ROM SCSI-2 device
    cd0: 10.000MB/s transfers (10.000MHz, offset 15)
    cd0: Attempt to query device size failed: NOT READY, Medium not present
    da0 at adv0 bus 0 target 4 lun 0
    da0: Fixed Direct Access SCSI-3 device
    da0: 20.000MB/s transfers (20.000MHz, offset 15)
    da0: 17501MB (35843670 512 byte sectors: 255H 63S/T 2231C)
    Trying to mount root from ufs:/dev/ad0s1a
    interrupt storm detected on "irq12:"; throttling interrupt source

    The interrupt storm message repeates from here.

    Thanks for any suggestions!


  2. Re: Interrupt storm

    On Sun, 20 Jan 2008 18:16:07 +0100, riotnrrd wrote:
    : I just upgraded to 6.3-RELEASE, and after the obligatory reboot I have
    : the following message:
    :
    : interrupt storm detected on "irq12": throttling interrupt source
    :
    : On my system, irq12 is rl0. After booting, this message just scrolls up
    : the console, printing every couple of seconds.

    Does vmstat -i show anyone else on irq12? Does the rl device support polling?


  3. Re: Interrupt storm

    On Sun, 20 Jan 2008 17:37:49 GMT, Howard Goldstein wrote:
    : On Sun, 20 Jan 2008 18:16:07 +0100, riotnrrd wrote:
    : : I just upgraded to 6.3-RELEASE, and after the obligatory reboot I have
    : : the following message:
    : :
    : : interrupt storm detected on "irq12": throttling interrupt source
    : :
    : : On my system, irq12 is rl0. After booting, this message just scrolls up
    : : the console, printing every couple of seconds.
    :
    : Does vmstat -i show anyone else on irq12? Does the rl device support polling?

    Duh I snipped your vmstat and see it now. Might try rl in polling.
    Do you have a ps2 mouse attached?

  4. Re: Interrupt storm

    Howard Goldstein wrote:
    > On Sun, 20 Jan 2008 18:16:07 +0100, riotnrrd wrote:
    > : I just upgraded to 6.3-RELEASE, and after the obligatory reboot I have
    > : the following message:
    > :
    > : interrupt storm detected on "irq12": throttling interrupt source
    > :
    > : On my system, irq12 is rl0. After booting, this message just scrolls up
    > : the console, printing every couple of seconds.
    >
    > Does vmstat -i show anyone else on irq12? Does the rl device support polling?



    No, there is only one irq12, but with a huge number of interrupts - and
    the system has not been up 40 minutes yet:

    $ vmstat -i
    interrupt total rate
    irq0: clk 2253766 999
    irq1: atkbd0 481 0
    irq3: sio1 1 0
    irq4: sio0 1 0
    irq5: xl0 21441 9
    irq6: fdc0 8 0
    irq8: rtc 288463 127
    irq10: adv0 117 0
    irq11: ath0 ohci0 596 0
    irq12: rl0 2252038 998
    irq14: ata0 53862 23
    Total 4870774 2159

    As for the polling, I understand that requires a custom kernel, right? I
    tried that when I was playing with a more-than-VGA console, but it was a
    pain getting that to work with freebsd-update.

    If you have a suggestion, I would be very interested, as I never did get
    round to sorting out the console.

  5. Re: Interrupt storm

    Howard Goldstein wrote:
    > On Sun, 20 Jan 2008 17:37:49 GMT, Howard Goldstein wrote:
    > : On Sun, 20 Jan 2008 18:16:07 +0100, riotnrrd wrote:
    > : : I just upgraded to 6.3-RELEASE, and after the obligatory reboot I have
    > : : the following message:
    > : :
    > : : interrupt storm detected on "irq12": throttling interrupt source
    > : :
    > : : On my system, irq12 is rl0. After booting, this message just scrolls up
    > : : the console, printing every couple of seconds.
    > :
    > : Does vmstat -i show anyone else on irq12? Does the rl device support polling?
    >
    > Duh I snipped your vmstat and see it now. Might try rl in polling.
    > Do you have a ps2 mouse attached?


    Heh, no worries.

    No, no mouse, but the keyboard is ps2.

  6. Re: Interrupt storm

    On Sun, 20 Jan 2008 18:44:44 +0100, riotnrrd wrote:
    : Howard Goldstein wrote:
    : > On Sun, 20 Jan 2008 18:16:07 +0100, riotnrrd wrote:
    : > : I just upgraded to 6.3-RELEASE, and after the obligatory reboot I have
    : > : the following message:
    : > :
    : > : interrupt storm detected on "irq12": throttling interrupt source
    : > :
    : > : On my system, irq12 is rl0. After booting, this message just scrolls up
    : > : the console, printing every couple of seconds.
    : >
    : > Does vmstat -i show anyone else on irq12? Does the rl device support polling?
    :
    :
    : No, there is only one irq12, but with a huge number of interrupts - and
    : the system has not been up 40 minutes yet:
    :
    : $ vmstat -i
    : interrupt total rate
    : irq0: clk 2253766 999
    : irq1: atkbd0 481 0
    : irq3: sio1 1 0
    : irq4: sio0 1 0
    : irq5: xl0 21441 9
    : irq6: fdc0 8 0
    : irq8: rtc 288463 127
    : irq10: adv0 117 0
    : irq11: ath0 ohci0 596 0
    : irq12: rl0 2252038 998
    : irq14: ata0 53862 23
    : Total 4870774 2159
    :
    : As for the polling, I understand that requires a custom kernel, right? I
    : tried that when I was playing with a more-than-VGA console, but it was a
    : pain getting that to work with freebsd-update.

    Yes, and a switch on ifconfig to get the driver in polling mode

    :
    : If you have a suggestion, I would be very interested, as I never did get
    : round to sorting out the console.

    Maybe try other PCI slots for the rl interface? I've never seen a
    system that came up with acpi disabled and blacklisted as your boot up
    message does. Do you see anything on the acpi man page regarding the
    blacklist that you could try?


  7. Re: Interrupt storm

    Howard Goldstein wrote:
    > On Sun, 20 Jan 2008 18:44:44 +0100, riotnrrd wrote:
    > : Howard Goldstein wrote:
    > : > On Sun, 20 Jan 2008 18:16:07 +0100, riotnrrd wrote:
    > : > : I just upgraded to 6.3-RELEASE, and after the obligatory reboot I have
    > : > : the following message:
    > : > :
    > : > : interrupt storm detected on "irq12": throttling interrupt source
    > : > :
    > : > : On my system, irq12 is rl0. After booting, this message just scrolls up
    > : > : the console, printing every couple of seconds.
    > : >
    > : > Does vmstat -i show anyone else on irq12? Does the rl device support polling?
    > :
    > :
    > : No, there is only one irq12, but with a huge number of interrupts - and
    > : the system has not been up 40 minutes yet:
    > :
    > : $ vmstat -i
    > : interrupt total rate
    > : irq0: clk 2253766 999
    > : irq1: atkbd0 481 0
    > : irq3: sio1 1 0
    > : irq4: sio0 1 0
    > : irq5: xl0 21441 9
    > : irq6: fdc0 8 0
    > : irq8: rtc 288463 127
    > : irq10: adv0 117 0
    > : irq11: ath0 ohci0 596 0
    > : irq12: rl0 2252038 998
    > : irq14: ata0 53862 23
    > : Total 4870774 2159
    > :
    > : As for the polling, I understand that requires a custom kernel, right? I
    > : tried that when I was playing with a more-than-VGA console, but it was a
    > : pain getting that to work with freebsd-update.
    >
    > Yes, and a switch on ifconfig to get the driver in polling mode
    >
    > :
    > : If you have a suggestion, I would be very interested, as I never did get
    > : round to sorting out the console.
    >
    > Maybe try other PCI slots for the rl interface? I've never seen a
    > system that came up with acpi disabled and blacklisted as your boot up
    > message does. Do you see anything on the acpi man page regarding the
    > blacklist that you could try?
    >


    Do not meddle in the ways of IRQ, for it is utterly incomprehensible and
    quick to induce anger.

    I moved the rl0 to another PCI slot, with no change in the message. What
    I did not realize was that irq12 had been reassigned behind my back, and
    it was now adv0 on irq12, while rl0 had moved in with xl0 and was cozily
    tucked up in irq11.

    I only realized this once I had obtained and plugged in a different
    Realtek NIC, noticed that the message was still there, and re-ran vmstat
    -i just to check.

    Removing irq12 from the ESCD pool seems to have fixed the problem - for
    now at least.

    Ah, the joys of franken-servers...

    --
    RiotNrrd
    Checking whether build environment is sane ...
    build environment is grinning and holding a spatula. Guess not.

  8. Re: Interrupt storm

    On Fri, 25 Jan 2008 11:08:48 +0100, riotnrrd wrote:
    : I only realized this once I had obtained and plugged in a different
    : Realtek NIC, noticed that the message was still there, and re-ran vmstat
    : -i just to check.
    :
    : Removing irq12 from the ESCD pool seems to have fixed the problem - for
    : now at least.

    Wow talk about bizarre. WTF is going on with vmstat -i then? Weird.
    Good on you tracking it down.

+ Reply to Thread