ZFS panic in zone_dataset_visible - FreeBSD

This is a discussion on ZFS panic in zone_dataset_visible - FreeBSD ; Hello, I am running several servers using Pawel's July 27 ZFS patchset, applied against 8-current source from the same day. I have seen a similar panic on two different servers: Server #1: Fatal trap 12: page fault while in kernel ...

+ Reply to Thread
Results 1 to 2 of 2

Thread: ZFS panic in zone_dataset_visible

  1. ZFS panic in zone_dataset_visible

    Hello,

    I am running several servers using Pawel's July 27 ZFS patchset, applied
    against 8-current source from the same day. I have seen a similar panic
    on two different servers:

    Server #1:

    Fatal trap 12: page fault while in kernel mode
    cpuid = 0; apic id = 00
    fault virtual address = 0x570
    fault code = supervisor write data, page not present
    instruction pointer = 0x8:0xffffffff802d60a5
    stack pointer = 0x10:0xfffffffec89f3280
    frame pointer = 0x10:0xfffffffec89f3290
    code segment = base 0x0, limit 0xfffff, type 0x1b
    = DPL 0, pres 1, long 1, def32 0, gran 1
    processor eflags = interrupt enabled, resume, IOPL = 0
    current process = 95276 (ftpd)
    [thread pid 95276 tid 100432 ]
    Stopped at _mtx_lock_flags+0x15: lock cmpxchgq %rsi,0x18(%rdi)
    db> bt
    Tracing pid 95276 tid 100432 td 0xffffff010b3cc000
    _mtx_lock_flags() at _mtx_lock_flags+0x15
    zone_dataset_visible() at zone_dataset_visible+0x94
    zfs_mount() at zfs_mount+0x3e5
    domount() at domount+0x216
    zfsctl_snapdir_lookup() at zfsctl_snapdir_lookup+0x3ba
    VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x40
    lookup() at lookup+0x518
    namei() at namei+0x515
    kern_statat() at kern_statat+0x92
    lstat() at lstat+0x2a
    syscall() at syscall+0x264
    Xfast_syscall() at Xfast_syscall+0xab
    --- syscall (190, FreeBSD ELF64, lstat), rip = 0x800d9153c, rsp =
    0x7fffffffa5c8, rbp = 0x6581d0 ---
    db>

    Server #1 (again):

    Fatal trap 12: page fault while in kernel mode
    cpuid = 1; apic id = 01
    fault virtual address = 0x570
    fault code = supervisor write data, page not present
    instruction pointer = 0x8:0xffffffff802d60a5
    stack pointer = 0x10:0xfffffffec8b3d280
    frame pointer = 0x10:0xfffffffec8b3d290
    code segment = base 0x0, limit 0xfffff, type 0x1b
    = DPL 0, pres 1, long 1, def32 0, gran 1
    processor eflags = interrupt enabled, resume, IOPL = 0
    current process = 87967 (sftp-server)
    [thread pid 87967 tid 100498 ]
    Stopped at _mtx_lock_flags+0x15: lock cmpxchgq %rsi,0x18(%rdi)
    db> bt
    Tracing pid 87967 tid 100498 td 0xffffff00a4c81700
    _mtx_lock_flags() at _mtx_lock_flags+0x15
    zone_dataset_visible() at zone_dataset_visible+0x94
    zfs_mount() at zfs_mount+0x3e5
    domount() at domount+0x216
    zfsctl_snapdir_lookup() at zfsctl_snapdir_lookup+0x3ba
    VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x40
    lookup() at lookup+0x518
    namei() at namei+0x515
    kern_statat() at kern_statat+0x92
    lstat() at lstat+0x2a
    syscall() at syscall+0x264
    Xfast_syscall() at Xfast_syscall+0xab
    --- syscall (190, FreeBSD ELF64, lstat), rip = 0x800d1c53c, rsp =
    0x7ffffffea5b8, rbp = 0x630020 ---
    db>

    Server #2:

    Fatal trap 12: page fault while in kernel mode
    cpuid = 0; apic id = 00
    fault virtual address = 0x570
    fault code = supervisor write data, page not present
    instruction pointer = 0x8:0xffffffff802d60a5
    stack pointer = 0x10:0xfffffffec8813280
    frame pointer = 0x10:0xfffffffec8813290
    code segment = base 0x0, limit 0xfffff, type 0x1b
    = DPL 0, pres 1, long 1, def32 0, gran 1
    processor eflags = interrupt enabled, resume, IOPL = 0
    current process = 2851 (sh)
    [thread pid 2851 tid 100336 ]
    Stopped at _mtx_lock_flags+0x15: lock cmpxchgq %rsi,0x18(%rdi)
    db> bt
    Tracing pid 2851 tid 100336 td 0xffffff004f0cf000
    _mtx_lock_flags() at _mtx_lock_flags+0x15
    zone_dataset_visible() at zone_dataset_visible+0x94
    zfs_mount() at zfs_mount+0x3e5
    domount() at domount+0x216
    zfsctl_snapdir_lookup() at zfsctl_snapdir_lookup+0x3ba
    VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x40
    lookup() at lookup+0x518
    namei() at namei+0x515
    kern_statat() at kern_statat+0x92
    stat() at stat+0x2a
    syscall() at syscall+0x264
    Xfast_syscall() at Xfast_syscall+0xab
    --- syscall (188, FreeBSD ELF64, stat), rip = 0x80099855c, rsp =
    0x7fffffffe868, rbp = 0x1 ---
    db>

    Has anyone else encountered this panic?

    --
    Scott Burns
    System Administrator
    BQ Internet Corporation
    _______________________________________________
    freebsd-current@freebsd.org mailing list
    http://lists.freebsd.org/mailman/lis...reebsd-current
    To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"


  2. Re: ZFS panic in zone_dataset_visible

    On Mon, Sep 22, 2008 at 01:12:50PM -0400, Scott Burns wrote:
    > Scott Burns wrote:
    > >Hello,
    > >
    > >I am running several servers using Pawel's July 27 ZFS patchset, applied
    > >against 8-current source from the same day. I have seen a similar panic
    > >on two different servers:

    > ...
    > >Stopped at _mtx_lock_flags+0x15: lock cmpxchgq %rsi,0x18(%rdi)
    > >db> bt
    > >Tracing pid 95276 tid 100432 td 0xffffff010b3cc000
    > >_mtx_lock_flags() at _mtx_lock_flags+0x15
    > >zone_dataset_visible() at zone_dataset_visible+0x94
    > >zfs_mount() at zfs_mount+0x3e5

    > ...
    >
    > With a bit of testing, I found that this panic is easily reproducible.
    > Simply try to list the contents of a snapshot from within a jail, as
    > long as the snapshot isn't already mounted, and the system panics. If I
    > mount the snapshot from outside of the jail first, and then list it
    > inside the jail, it does not panic.
    >
    > I spent a bit of time debugging this weekend. Trying to list an
    > unmounted snapshot triggers a zfs_mount() for the snapshot, which calls
    > zone_dataset_visible() to determine if the snapshot should be visible in
    > the current zone. When it is run outside of a jail, it returns true
    > early on because INGLOBALZONE(curproc) is true, otherwise it takes
    > another code path.
    >
    > The panic is happening after that check, at mtx_lock(&pr->cr_mtx),
    > because (pr = curthread->td_ucred->cr_prison) is NULL. Interestingly,
    > it's not NULL if zone_dataset_visible() is triggered by a "zfs list"
    > command, but it is NULL if zone_dataset_visible() is called from
    > zfs_mount().
    >
    > As a temporary workaround, I modified my copy of
    > cddl/compat/opensolaris/kern/opensolaris_zone.c to have
    > zone_dataset_visible() return true if it is being called for a snapshot.
    > I modified it as below:
    >
    > -if (INGLOBALZONE(curproc))
    > +if (INGLOBALZONE(curproc) || strchr(dataset, '@'))
    >
    > This is obviously not ideal, since it allows the manipulation of the
    > snapshot from another jail if the caller knows that it exists. Since I
    > am the only one with root access to any of the jails, I am not concerned
    > with that. "zfs list" continues to behave normally.
    >
    > I will continue looking at this, but since my main goal of working
    > around the panic has been taken care of, I am not sure how long my
    > attention span will last. If the cause of
    > curthread->td_ucred->cr_prison being NULL under these conditions is
    > obvious to anyone, please let me know.


    Thanks for the report. The problem is that we have an ugly hack to allow
    regular users to mount snapshots automatically. It works by changing
    td_ucred to kcred for VFS_MOUNT() call. This makes p_ucred to point at
    jailed cred and td_ucred to point at unjailed thread in
    zone_dataset_visible(), which is confusing of course. I fixed it by
    reimplementing INGLOBALZONE() macro to take thread, not process.

    --
    Pawel Jakub Dawidek http://www.wheel.pl
    pjd@FreeBSD.org http://www.FreeBSD.org
    FreeBSD committer Am I Evil? Yes, I Am!

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.4 (FreeBSD)

    iD8DBQFJFbwrForvXbEpPzQRAvq3AJwMDKLqrE46CDqUC4iBgO LqWeX7BACgg3jF
    FknEhzFJE3W2BOO6vubg560=
    =xAWM
    -----END PGP SIGNATURE-----


+ Reply to Thread