Hello everyone:

Please see complete dmesg output for two LOR's

I'm tracking: *default release=cvs tag=RELENG_7

For now, this is a spare box that can be subjected to any type of
testing - you only need to ask. I'm an avid advocate of FreeBSD but make
no claims in understanding internals from anything but a 7956 foot level
- and that varies ;-)

FreeBSD zega.mikej.com 7.0-PRERELEASE FreeBSD 7.0-PRERELEASE #0: Sun Feb
24 12:44:54 UTC 2008 mikej@zega.mikej.com:/usr/obj/usr/src/sys/ZEGA
i386

GENERIC kernel other than:

options KVA_PAGES=512
options KDB # Enable kernel debugger support.
options DDB # Support DDB.
options GDB # Support remote GDB.
options INVARIANTS # Enable calls of extra sanity
checking
options INVARIANT_SUPPORT # Extra sanity checks of internal
structures, required by INVARIANTS
options WITNESS # Enable checks to detect
deadlocks and cycles
options WITNESS_SKIPSPIN # Don't run witness on spinlocks
for speed.

AND: /boot/loader.conf

kern.sync_on_panic=0
console="comconsole vidconsole"
vfs.zfs.prefetch_disable="1"
zfs_load="YES"
vfs.zfs.arc_max="320M"
vm.kmem_size_max="1500M"
vm.kmem_size="1500M"


Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights
reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.0-PRERELEASE #0: Sun Feb 24 12:44:54 UTC 2008
mikej@zega.mikej.com:/usr/obj/usr/src/sys/ZEGA
WARNING: WITNESS option enabled, expect reduced performance.
WARNING: DIAGNOSTIC option enabled, expect reduced performance.
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Pentium(R) 4 CPU 2.66GHz (2660.02-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf29 Stepping = 9

Features=0xbfebfbff ,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE, SSE2,SS,HTT,TM,PBE>
Features2=0x4400
real memory = 2146697216 (2047 MB)
avail memory = 2087800832 (1991 MB)
ACPI APIC Table:
WITNESS: spin lock intrcnt not in order list
ioapic0 irqs 0-23 on motherboard
kbd1 at kbdmux0
ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413,
RF5413)
hptrr: HPT RocketRAID controller driver v1.1 (Feb 24 2008 12:41:19)
acpi0:
on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, 7ff00000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
cpu0: on acpi0
p4tcc0: on cpu0
pcib0: port 0xcf8-0xcff on acpi0
pci0: on pcib0
agp0: on hostb0
pcib1: at device 1.0 on pci0
pci1: on pcib1
vgapci0: port 0xac00-0xac7f mem
0xe0000000-0xefffffff,0xfd9c0000-0xfd9fffff irq 16 at device 0.0 on pci1
uhci0: port 0xe000-0xe01f
irq 16 at device 29.0 on pci0
uhci0: [GIANT-LOCKED]
uhci0: [ITHREAD]
usb0: on uhci0
usb0: USB revision 1.0
uhub0: on usb0
uhub0: 2 ports with 2 removable, self powered
uhci1: port 0xe400-0xe41f
irq 19 at device 29.1 on pci0
uhci1: [GIANT-LOCKED]
uhci1: [ITHREAD]
usb1: on uhci1
usb1: USB revision 1.0
uhub1: on usb1
uhub1: 2 ports with 2 removable, self powered
uhci2: port 0xe800-0xe81f
irq 18 at device 29.2 on pci0
uhci2: [GIANT-LOCKED]
uhci2: [ITHREAD]
usb2: on uhci2
usb2: USB revision 1.0
uhub2: on usb2
uhub2: 2 ports with 2 removable, self powered
uhci3: port 0xec00-0xec1f
irq 16 at device 29.3 on pci0
uhci3: [GIANT-LOCKED]
uhci3: [ITHREAD]
usb3: on uhci3
usb3: USB revision 1.0
uhub3: on usb3
uhub3: 2 ports with 2 removable, self powered
ehci0: mem
0xfebffc00-0xfebfffff irq 23 at device 29.7 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb4: EHCI version 1.0
usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
usb4: on ehci0
usb4: USB revision 2.0
uhub4: on usb4
uhub4: 8 ports with 8 removable, self powered
pcib2: at device 30.0 on pci0
pci2: on pcib2
em0: port
0xbc00-0xbc3f mem 0xfeae0000-0xfeafffff,0xfeac0000-0xfeadffff irq 18 at
device 0.0 on pci2
em0: Ethernet address: 00:1b:21:10:1d:55
em0: [FILTER]
twe0: <3ware Storage Controller. Driver version 1.50.01.002> port
0xb800-0xb80f mem 0xfea9fc00-0xfea9fc0f,0xfe000000-0xfe7fffff irq 19 at
device 1.0 on pci2
twe0: [GIANT-LOCKED]
twe0: [ITHREAD]
twe0: 4 ports, Firmware FE7X 1.05.00.068, BIOS BE7X 1.08.00.048
rl0: port 0xb400-0xb4ff mem
0xfea9f800-0xfea9f8ff irq 22 at device 5.0 on pci2
miibus0: on rl0
rlphy0: PHY 0 on miibus0
rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
rl0: Ethernet address: 00:50:2c:a4:f7:b3
rl0: [ITHREAD]
isab0: at device 31.0 on pci0
isa0: on isab0
atapci0: port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0
ata0: on atapci0
ata0: [ITHREAD]
ata1: on atapci0
ata1: [ITHREAD]
pci0: at device 31.5 (no driver attached)
acpi_button0: on acpi0
acpi_button1: on acpi0
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on
acpi0
sio0: type 16550A, console
sio0: [FILTER]
fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2
on acpi0
fdc0: [FILTER]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
atkbdc0: port 0x60,0x64 irq 1 on acpi0
atkbd0: irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
pmtimer0 on isa0
orm0: at iomem
0xc0000-0xcffff,0xd0000-0xd0fff,0xd1000-0xd1fff,0xd2800-0xd37ff pnpid
ORM0000 on isa0
ppc0: at port 0x378-0x37f irq 7 on isa0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/9 bytes threshold
ppbus0: on ppc0
ppbus0: [ITHREAD]
plip0: on ppbus0
lpt0: on ppbus0
lpt0: Interrupt-driven port
ppi0: on ppbus0
ppc0: [GIANT-LOCKED]
ppc0: [ITHREAD]
sc0: at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on
isa0
WARNING: ZFS is considered to be an experimental feature in FreeBSD.
Timecounter "TSC" frequency 2660018636 Hz quality 800
Timecounters tick every 1.000 msec
hptrr: no controller detected.
ZFS filesystem version 6
ZFS storage pool version 6
ad0: 194479MB at ata0-master UDMA100
ad1: 194481MB at ata0-slave UDMA100
ad2: 194481MB at ata1-master UDMA100
ad3: 194481MB at ata1-slave UDMA100
twed0: on twe0
twed0: 715422MB (1465185024 sectors)
WARNING: WITNESS option enabled, expect reduced performance.
WARNING: DIAGNOSTIC option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/twed0s1a
em0: link state changed to UP
lock order reversal:
1st 0x80ba3c20 sched lock (sched lock) @
/usr/src/sys/kern/subr_trap.c:86
2nd 0x80c081c8 sio (sio) @ /usr/src/sys/dev/sio/sio.c:2556
KDB: stack backtrace:
db_trace_self_wrapper(80a964f8,fae2aae0,8076f525,8 0a978e6,80c081c8,...)
at db_trace_self_wrapper+0x26
kdb_backtrace(80a978e6,80c081c8,80b649c0,80b649c0, 80ab5543,...) at
kdb_backtrace+0x29
witness_checkorder(80c081c8,9,80ab5543,9fc,fae2ab1 0,...) at
witness_checkorder+0x5e5
_mtx_lock_spin_flags(80c081c8,0,80ab5543,9fc,fae2a b2c,...) at
_mtx_lock_spin_flags+0x32
sio_cnputc(80b649e0,66,fae2acc0,5,66,...) at sio_cnputc+0x86
cnputc(66,fae2acc0,fae2ab80,80765a61,3c6,...) at cnputc+0x5f
putcons(3c6,80a94f2d,15c3804,885c8000,80a971d1,... ) at putcons+0x17
putchar(66,fae2acc0,80ba3cc0,893fdc34,0,...) at putchar+0x61
kvprintf(80a971d0,80765a00,fae2acc0,a,fae2acec,... ) at kvprintf+0x75
printf(80a971d0,0,80a971b2,56,885c3894,...) at printf+0x4e
userret(885c8000,fae2ad38,80a971b2,fc,1020804) at userret+0xab
ast(fae2ad38) at ast+0x34b
doreti_ast() at doreti_ast+0x17
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()
failed to set signal flags properly for ast()

lock order reversal:
1st 0x85e07ac8 process slock (process slock) @
/usr/src/sys/kern/kern_resource.c:1066
2nd 0x80c081c8 sio (sio) @ /usr/src/sys/dev/sio/sio.c:2556
KDB: stack backtrace:
db_trace_self_wrapper(80a964f8,fad3a93c,8076f525,8 0a978e6,80c081c8,...)
at db_trace_self_wrapper+0x26
kdb_backtrace(80a978e6,80c081c8,80b649c0,80b649c0, 80ab5543,...) at
kdb_backtrace+0x29
witness_checkorder(80c081c8,9,80ab5543,9fc,85e1263 0,...) at
witness_checkorder+0x5e5
_mtx_lock_spin_flags(80c081c8,0,80ab5543,9fc,80ba8 738,...) at
_mtx_lock_spin_flags+0x32
sio_cnputc(80b649e0,63,fad3ab1c,5,63,...) at sio_cnputc+0x86
cnputc(63,fad3ab1c,fad3a9dc,80765a61,0,...) at cnputc+0x5f
putcons(0,80beeba8,1e126c8,c36,80a94179,...) at putcons+0x17
putchar(63,fad3ab1c,200246,0,fad3aa10,...) at putchar+0x61
kvprintf(80a94178,80765a00,fad3ab1c,a,fad3ab48,... ) at kvprintf+0x75
printf(80a94178,4291db58,1f,6cf6ce72,1d,...) at printf+0x4e
calcru1(fad3ac38,8073b690,fad3ac30,85e126ec,8a1503 2a,15aa0) at
calcru1+0x25e
calcru(85e07ab0,fad3ac30,fad3ac38,42a,85e07ab0,... ) at calcru+0xdd
rufetchcalc(85e07ab0,fad3ac30,fad3ac30,fad3ac38,fa d3acfc,...) at
rufetchcalc+0x53
kern_getrusage(85e12630,0,fad3ac30,0,0,...) at kern_getrusage+0x61
getrusage(85e12630,fad3acfc,8,80a97500,80b39598,.. .) at getrusage+0x27
syscall(fad3ad38) at syscall+0x2d3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (117, FreeBSD ELF32, getrusage), eip = 0x809ed83, esp =
0x7fbfe88c, ebp = 0x8148000 ---
calcru: runtime went backwards from 134260841304 usec to 126382165618
usec for pid 827 (dnetc)
calcru: runtime went backwards from 4534 usec to 4267 usec for pid 919
(getty)
calcru: runtime went backwards from 4652 usec to 4378 usec for pid 904
(getty)
calcru: runtime went backwards from 4572 usec to 4303 usec for pid 903
(getty)
calcru: runtime went backwards from 4610 usec to 4339 usec for pid 902
(getty)
calcru: runtime went backwards from 4495 usec to 4230 usec for pid 901
(getty)
calcru: runtime went backwards from 4648 usec to 4375 usec for pid 900
(getty)
calcru: runtime went backwards from 4420 usec to 4160 usec for pid 899
(getty)
calcru: runtime went backwards from 4481 usec to 4218 usec for pid 898
(getty)
calcru: runtime went backwards from 53281990 usec to 51509780 usec for
pid 855 (cron)
calcru: runtime went backwards from 7674352130 usec to 7224054531 usec
for pid 839 (sshd)
calcru: runtime went backwards from 43225 usec to 40688 usec for pid 745
(nfsd)
calcru: runtime went backwards from 7988 usec to 7519 usec for pid 743
(mountd)
calcru: runtime went backwards from 3613 usec to 3401 usec for pid 592
(devd)
calcru: runtime went backwards from 28022 usec to 26242 usec for pid 592
(devd)
calcru: runtime went backwards from 83141 usec to 79411 usec for pid 128
(spa_zio_intr_1)
calcru: runtime went backwards from 18948528 usec to 18427925 usec for
pid 50 (softdepflush)
calcru: runtime went backwards from 1551354 usec to 1528787 usec for pid
47 (bufdaemon)
calcru: runtime went backwards from 8955 usec to 8430 usec for pid 40
(irq1: atkbd0)
calcru: runtime went backwards from 8022203 usec to 7770714 usec for pid
27 (irq19: uhci1 twe0)
calcru: runtime went backwards from 22096 usec to 20692 usec for pid 20
(swi5: +)
calcru: runtime went backwards from 277929 usec to 260275 usec for pid
17 (swi6: task queue)
calcru: runtime went backwards from 2438 usec to 2295 usec for pid 12
(swi1: net)
calcru: runtime went backwards from 89469281 usec to 84551522 usec for
pid 11 (idle: cpu0)
calcru: runtime went backwards from 1088731 usec to 1024840 usec for pid
1 (init)
calcru: runtime went backwards from 49829588 usec to 46905404 usec for
pid 1 (init)
calcru: runtime went backwards from 146817 usec to 137491 usec for pid 0
(swapper)


#ps -ax

PID TT STAT TIME COMMAND
0 ?? WLs 0:00.14 [swapper]
1 ?? ILs 0:01.04 /sbin/init --
2 ?? DL 0:10.63 [g_event]
3 ?? DL 0:21.83 [g_up]
4 ?? DL 0:16.37 [g_down]
5 ?? DL 0:00.00 [system_taskq]
6 ?? DL 0:00.00 [kqueue taskq]
7 ?? DL 0:00.00 [acpi_task_0]
8 ?? DL 0:00.00 [acpi_task_1]
9 ?? DL 0:00.00 [acpi_task_2]
10 ?? DL 0:00.00 [audit]
11 ?? RL 1:43.84 [idle: cpu0]
12 ?? WL 0:00.00 [swi1: net]
13 ?? WL 6:41.69 [swi4: clock sio]
14 ?? WL 0:00.00 [swi3: vm]
15 ?? DL 0:07.92 [yarrow]
16 ?? WL 0:00.30 [swi6: Giant taskq]
17 ?? WL 0:00.26 [swi6: task queue]
18 ?? DL 0:00.00 [xpt_thrd]
19 ?? WL 0:00.00 [swi2: cambio]
20 ?? WL 0:00.02 [swi5: +]
21 ?? DL 0:00.00 [thread taskq]
22 ?? WL 0:00.00 [irq9: acpi0]
23 ?? WL 0:00.00 [irq16: uhci0 uhci3]
24 ?? DL 0:00.01 [usb0]
25 ?? DL 0:00.00 [usbtask-hc]
26 ?? DL 0:00.00 [usbtask-dr]
27 ?? WL 0:09.38 [irq19: uhci1 twe0]
28 ?? DL 0:00.01 [usb1]
29 ?? WL 0:02.81 [irq18: em0 uhci2]
30 ?? DL 0:00.01 [usb2]
31 ?? DL 0:00.01 [usb3]
32 ?? WL 0:00.00 [irq23: ehci0]
33 ?? DL 0:00.02 [usb4]
34 ?? DL 0:10.49 [em0 taskq]
35 ?? WL 0:00.00 [irq22: rl0]
36 ?? WL 0:00.22 [irq14: ata0]
37 ?? WL 0:00.23 [irq15: ata1]
38 ?? WL 0:00.00 [swi0: sio]
39 ?? DL 0:00.79 [fdc0]
40 ?? WL 0:00.01 [irq1: atkbd0]
41 ?? WL 0:00.00 [irq7: ppbus0 ppc0]
42 ?? DL 0:00.00 [sctp_iterator]
43 ?? DL 0:00.99 [arc_reclaim_thread]
44 ?? DL 0:00.22 [pagedaemon]
45 ?? DL 0:00.00 [vmdaemon]
46 ?? DL 0:00.00 [pagezero]
47 ?? DL 0:01.74 [bufdaemon]
48 ?? DL 9:47.21 [syncer]
49 ?? DL 0:01.94 [vnlru]
50 ?? DL 0:21.50 [softdepflush]
51 ?? DL 0:22.08 [schedcpu]
125 ?? DL 0:00.00 [spa_zio_issue_0]
126 ?? DL 0:00.00 [spa_zio_intr_0]
127 ?? DL 0:00.00 [spa_zio_issue_1]
128 ?? DL 0:00.08 [spa_zio_intr_1]
129 ?? DL 0:00.03 [spa_zio_issue_2]
130 ?? DL 0:02.80 [spa_zio_intr_2]
131 ?? DL 0:00.00 [spa_zio_issue_3]
132 ?? DL 0:00.00 [spa_zio_intr_3] 133 ?? DL 0:00.00
[spa_zio_issue_4]
134 ?? DL 0:00.00 [spa_zio_intr_4]
135 ?? DL 0:00.00 [spa_zio_issue_5]
136 ?? DL 0:00.00 [spa_zio_intr_5]
137 ?? DL 0:00.03 [vdev:worker ad0]
138 ?? DL 0:00.03 [vdev:worker ad1]
139 ?? DL 0:00.03 [vdev:worker ad2]
140 ?? DL 0:00.03 [vdev:worker ad3]
141 ?? DL 0:00.40 [txg_thread_enter]
142 ?? DL 0:03.00 [txg_thread_enter]
143 ?? DL 0:00.40 [txg_thread_enter]
144 ?? DL 0:00.00 [zil_clean]
145 ?? DL 0:00.01 [zil_clean]
592 ?? Is 0:00.00 /sbin/devd
646 ?? Is 0:00.45 /usr/sbin/syslogd -s
722 ?? Ss 0:00.25 /usr/sbin/rpcbind
743 ?? Is 0:00.01 /usr/sbin/mountd -r
745 ?? Is 0:00.04 nfsd: master (nfsd)
747 ?? I 0:19.35 nfsd: server (nfsd)
748 ?? I 0:00.24 nfsd: server (nfsd)
749 ?? I 0:00.02 nfsd: server (nfsd)
750 ?? I 0:00.01 nfsd: server (nfsd)
801 ?? Ss 0:07.91 /usr/sbin/ntpd -c /etc/ntp.conf -p
/var/run/ntpd.pid -f /var/db/ntpd.drift
827 ?? RNs 2585:41.13 /usr/local/distributed.net/dnetc -quiet
839 ?? Is 0:00.01 /usr/sbin/sshd
845 ?? Ss 0:03.38 sendmail: accepting connections (sendmail)
849 ?? Is 0:00.08 sendmail: Queue runner@00:30:00 for
/var/spool/clientmqueue (sendmail)
855 ?? Is 0:00.60 /usr/sbin/cron -s
45427 ?? Is 0:00.08 sshd: mikej [priv] (sshd)
45431 ?? S 0:04.18 sshd: mikej@ttyp0 (sshd)
919 v0 Is+ 0:00.00 /usr/libexec/getty Pc ttyv0
898 v1 Is+ 0:00.00 /usr/libexec/getty Pc ttyv1
899 v2 Is+ 0:00.00 /usr/libexec/getty Pc ttyv2
900 v3 Is+ 0:00.00 /usr/libexec/getty Pc ttyv3
901 v4 Is+ 0:00.00 /usr/libexec/getty Pc ttyv4
902 v5 Is+ 0:00.00 /usr/libexec/getty Pc ttyv5
903 v6 Is+ 0:00.00 /usr/libexec/getty Pc ttyv6
904 v7 Is+ 0:00.00 /usr/libexec/getty Pc ttyv7
10793 p0 R+ 0:00.00 ps -ax
10794 p0 R+ 0:00.00 su (bash)
45433 p0 Is 0:00.03 -bash (bash)
45453 p0 I 0:00.01 su
45454 p0 S 0:00.20 su (bash)


DUMB? QUESTION: Why did the LOR not trigger a kernel TRAP? - At 46 I'm
still trying to learn.

These errors did not occur at boot, but sometime I'm estimating 1+days
after the system was up and running

CONFIDENTIALITY NOTE: This message is intended only for the use
of the individual or entity to whom it is addressed and may contain
information that is privileged, confidential, and exempt from
disclosure under applicable law. If the reader of this message is
not the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this transmission
in error, please notify us by telephone at (502) 212-4001 or
notify us at PAI , Dept. 99, 11857 Commonwealth Drive,
Louisville, KY 40299. Thank you.
_______________________________________________
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/lis...reebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"