unmount and fuser funnies (Vol Mgr, possibly FS?)

On our development system, we are running 4 instances of
Oracle, each with uniquely named sets of directories
of the form /ora/x/name... where "x" is the system-id, lets say
one of a,b,c,d. (there are 9 "names").

We wanted to restore all of instance a from backup, and shut it down
ready for this.

As I have done before, I unmounted /ora/a/name..., etc, in order to
make sure they were not in use, prior to the restore.

To my surprise, I could not unmount one of them (Oracle binaries), and
when I ran "fuser" on the /ora/a/name.. file-system I was trying to
unmount, I found that apparently, processes from the other instances were
trying to use it. Logically, there is no reason why they should, and my
DBA colleague checked everything very carefully, and could not find where
they might be doing this.

We are now suspecting that somehow, with Veritas involved,
we are actually getting some kind of lock on the physical
disk, rather than the logical one or filesystem.
That is, there are file-systems from more than one instance
on each physical disk, so if the disk is busy, could that
be preventing me from unmounting the file-systems of the
shut-down instance?

I am wondering if "fuser" tells lies when Veritas is
involved, and also perhaps umount has limitations here.

Another thought I had was that if the files are being
mapped into memory, Oracle is somehow picking up
the "wrong" copies, and then relating them back
to the disks they came from.

Are there any (Veritas or otherwise) tools I could
use to find out what's going on? ("lsof" comes to
mind, and I intend to try that).

Thanks for any pointers.


(Solaris 5.8 VM 3.2 FS 3.4)

--
[email]mike@ellwoods.org.uk[/email]

I've had a similar experience, but didn't tie the processes down to the oracle
binaries specifically. The processes I saw were using the oracle home directory
and it was caused by our dba's su'ing to oracle and then changing directories.
For some reason this caused the current directory to be in use for the oracle
home and it was not released even though the dba had changed out of the oracle
home. After checking out was the process was I just killed those processes
and umounted. (for i in `fuser -ck /whatever`; do kill -9 $i; done) What
is the name of the process that was using the directory? (for i in `fuser
-ck /dir`; do ps -lp $i; done)

-glenn


Mike Ellwood <mike@ellwoods.org.uk> wrote:[color=blue]
>
>On our development system, we are running 4 instances of
>Oracle, each with uniquely named sets of directories
>of the form /ora/x/name... where "x" is the system-id, lets say
>one of a,b,c,d. (there are 9 "names").
>
>We wanted to restore all of instance a from backup, and shut it down
>ready for this.
>
>As I have done before, I unmounted /ora/a/name..., etc, in order to
>make sure they were not in use, prior to the restore.
>
>To my surprise, I could not unmount one of them (Oracle binaries), and
>when I ran "fuser" on the /ora/a/name.. file-system I was trying to
>unmount, I found that apparently, processes from the other instances were
>trying to use it. Logically, there is no reason why they should, and my
>DBA colleague checked everything very carefully, and could not find where
>they might be doing this.
>
>We are now suspecting that somehow, with Veritas involved,
>we are actually getting some kind of lock on the physical
>disk, rather than the logical one or filesystem.
>That is, there are file-systems from more than one instance
>on each physical disk, so if the disk is busy, could that
>be preventing me from unmounting the file-systems of the
>shut-down instance?
>
>I am wondering if "fuser" tells lies when Veritas is
>involved, and also perhaps umount has limitations here.
>
>Another thought I had was that if the files are being
>mapped into memory, Oracle is somehow picking up
>the "wrong" copies, and then relating them back
>to the disks they came from.
>
>Are there any (Veritas or otherwise) tools I could
>use to find out what's going on? ("lsof" comes to
>mind, and I intend to try that).
>
>Thanks for any pointers.
>
>
>(Solaris 5.8 VM 3.2 FS 3.4)
>
>--
>mike@ellwoods.org.uk[/color]