Please critique my backup practices - VMS
This is a discussion on Please critique my backup practices - VMS ; "Michael Moroney" wrote in message
news:fs9n15$vfi$1@pcls4.std.com...
> "John Wallace" writes:
>
>
> >"Michael Moroney" wrote in message
> >news:fs918n$qf8$1@pcls4.std.com...
> >> We have heard many time: "Thou shalt not use BACKUP/IGNORE=INTERLOCK".
> >> But I have an application that ...
-
Re: Please critique my backup practices
"Michael Moroney" wrote in message
news:fs9n15$vfi$1@pcls4.std.com...
> "John Wallace" writes:
>
>
> >"Michael Moroney" wrote in message
> >news:fs918n$qf8$1@pcls4.std.com...
> >> We have heard many time: "Thou shalt not use BACKUP/IGNORE=INTERLOCK".
> >> But I have an application that runs continuously and doesn't close its
> >> files, so the alternative is no backup of them. I had the idea of
doing
> >> two incremental backups, the first being a
BACKUP/MOD/SINCE=BACKUP/RECORD
> >> of the disk, and the second being
BACKUP/MOD/SINCE=BACKUP/IGNORE=INTERLOCK
> >> (note: no /RECORD) of the disk. The first is a "clean" incremental
backup
> >> of the disk, and the second, since it is run immediately after the
first,
> >> will contain only open files plus perhaps a few modified very recently.
> >> It would only be used in emergencies. The lack of /RECORD is so the
files
> >> backed up on the second backup are not considered properly backed up,
and
> >> any file closed before the next pass of incremental backups will be
> >> properly backed up during the next pass. Comments?
> >>
>
> >What do you know about the relationship between the continuously-running
> >application's in-memory copy of the file (which the application will
see),
> >and the on-disk copy of the file (which BACKUP will see)? Without that
kind
> >of knowledge, comments on the efficacy of /IGNORE=INTERLOCK are likely
> >nothing more than speculation (unless the comment is "it's pointless",
which
> >isn't news to you).
>
> The application maps data files via $CRMPSC/$MGBLSC and generally forces a
> writeback ($UPDSEC) when something important changes. The application is
> homegrown and plays *very* fast and loose with the data, but due to the
> nature of this beast, that's not as bad as it sounds.
If the backup happens to contain valid data from these section files,
you're presumably better off than if it didn't? But how do you *know* it
will contain valid restorable data?
In addition to the usual open file concerns, I'd also want to know about the
relationship between the files - as the apps are still active, backups of
files may be individually valid but may relate to different points in time,
which, depending on the app, may or may not actually be a valid consistent
restorable dataset?
Even if you test the backup/restore and it appears to work when you test it,
as far as I can see you've no guarantee that the standard backup will not,
one day, contain inconsistent data e.g. because of unfortunate timing.
Murphy says that sooner or later, you will actually need to restore the data
but it won't be there in a usable form. What would the impact be if (when)
that happened?
If this data really is important to you, and you really really really cannot
quiesce the app somehow, maybe you might want to think about doing something
application-specific (but probably outside the main application) to save a
copy of the important live data; maybe something which "knows" about the
application's behaviour and knows when a good time to take a self-consistent
snapshot might be???? I'm sure there are folks around who could provide
better advice than I'm offering. They might want $$$, which would
presumably be worthwhile if the data in these files is valuable to you.
Regards
John
-
Re: Please critique my backup practices
On Mar 25, 6:53 pm, David J Dachtera
wrote:
> AEF wrote:
>
> > On Mar 21, 11:41 am, David J Dachtera
> > wrote:
> > > Dale Dellutri wrote:
>
> > > > On Thu, 20 Mar 2008 12:36:19 -0700 (PDT), tadamsmar wrote:
> > > > > On Mar 20, 2:36?pm, Dale Dellutri wrote:
> > > > > > On Thu, 20 Mar 2008 07:02:14 -0700 (PDT), tadamsmar wrote:
> > > > > > > To do a backup, I pop a drive out of a shadowset, back it up and put
> > > > > > > it back.
>
> > > > > > I assume you're talking about VMS Volume Shadowing. ?As far as I know,
> > > > > > taking a drive out of a shadowset causes the drive to look the same as
> > > > > > if there'd been a power failure. ?In other words, it does not properly
> > > > > > close open files.
>
> > > > > > I assume that you take image backups. ?If so, there's no benefit to
> > > > > > taking the drive out of the shadowset to take the image backup.
> > > > > > According to service techs that I talked to when I first set up
> > > > > > volume shadowing, there's no advantage over taking an image backup
> > > > > > of the volume set (the DSA device).
>
> > > > > > I assume that this is still true.
>
> > > > > > > The problem is, I don't record the backup dates.
>
> > > > > > You could record backup dates if you simply took an image backup
> > > > > > of the volume set.
>
> > > > > Don't I have to shutdown my system to do that?
>
> > > > I finally found the relevant item in "HP Volume Shadowing for
> > > > OpenVMS", v7.3-2. Chapter 7, Section titled "Data Consistency
> > > > Requirements": "Removal of a shadow set member results in what
> > > > is called a crash-consistent copy. That is, the copy of the
> > > > data on the removed member is of the same level of consistency
> > > > as what would result if the system had failed at that instant."
> > > > (pg 124 in my copy).
>
> > > > Actually, reading the entire section titled "Guidelines for
> > > > for Using a Shadow Member for Backup" would be very useful.
>
> > > The usual technique is to quiesce the application (cause it to close its
> > > files), THEN split the shadow-sets. We do something similar with BCVs on
> > > an EMC DMX array. This minimizes your "backup window".
>
> > Why not...
> > shut the app
> > check that no files are open (except for INDEXF.SYS by the system)
> > dismount the shadows set entirely
>
> Doing so disables mini-copy. See HELP DISMOUNT /POLICY
>
> David J Dachtera
> (formerly dba) DJE Systems
Then why does the fine manual say to do it that way?
I guess it's a case of safety vs. speed.
AEF
-
Re: Please critique my backup practices
AEF wrote:
> On Mar 25, 6:53 pm, David J Dachtera
> wrote:
>
>>AEF wrote:
>>
>>
>>>On Mar 21, 11:41 am, David J Dachtera
>>>wrote:
>>>
>>>>Dale Dellutri wrote:
>>
>>>>>On Thu, 20 Mar 2008 12:36:19 -0700 (PDT), tadamsmar wrote:
>>>>>
>>>>>>On Mar 20, 2:36?pm, Dale Dellutri wrote:
>>>>>>
>>>>>>>On Thu, 20 Mar 2008 07:02:14 -0700 (PDT), tadamsmar wrote:
>>>>>>>
>>>>>>>>To do a backup, I pop a drive out of a shadowset, back it up and put
>>>>>>>>it back.
>>
>>>>>>>I assume you're talking about VMS Volume Shadowing. ?As far as I know,
>>>>>>>taking a drive out of a shadowset causes the drive to look the same as
>>>>>>>if there'd been a power failure. ?In other words, it does not properly
>>>>>>>close open files.
>>
>>>>>>>I assume that you take image backups. ?If so, there's no benefit to
>>>>>>>taking the drive out of the shadowset to take the image backup.
>>>>>>>According to service techs that I talked to when I first set up
>>>>>>>volume shadowing, there's no advantage over taking an image backup
>>>>>>>of the volume set (the DSA device).
>>
>>>>>>>I assume that this is still true.
>>
>>>>>>>>The problem is, I don't record the backup dates.
>>
>>>>>>>You could record backup dates if you simply took an image backup
>>>>>>>of the volume set.
>>
>>>>>>Don't I have to shutdown my system to do that?
>>
>>>>>I finally found the relevant item in "HP Volume Shadowing for
>>>>>OpenVMS", v7.3-2. Chapter 7, Section titled "Data Consistency
>>>>>Requirements": "Removal of a shadow set member results in what
>>>>>is called a crash-consistent copy. That is, the copy of the
>>>>>data on the removed member is of the same level of consistency
>>>>>as what would result if the system had failed at that instant."
>>>>>(pg 124 in my copy).
>>
>>>>>Actually, reading the entire section titled "Guidelines for
>>>>>for Using a Shadow Member for Backup" would be very useful.
>>
>>>>The usual technique is to quiesce the application (cause it to close its
>>>>files), THEN split the shadow-sets. We do something similar with BCVs on
>>>>an EMC DMX array. This minimizes your "backup window".
>>
>>>Why not...
>>>shut the app
>>>check that no files are open (except for INDEXF.SYS by the system)
>>>dismount the shadows set entirely
>>
>>Doing so disables mini-copy. See HELP DISMOUNT /POLICY
>>
>>David J Dachtera
>>(formerly dba) DJE Systems
>
>
> Then why does the fine manual say to do it that way?
>
> I guess it's a case of safety vs. speed.
>
> AEF
What version of the manual? Maybe it predates minicopy, or maybe
it wasn't updated when minicopy was added, or maybe it is referring
to the safe way to do a backup when you can't follow the 237 caveats
that have been posted in this thread about the dangers of doing an
online backup of an active application.
--
John Santos
Evans Griffiths & Hart, Inc.
781-861-0670 ext 539
-
Re: Please critique my backup practices
On Mar 26, 7:35 pm, John Santos wrote:
> AEF wrote:
> > On Mar 25, 6:53 pm, David J Dachtera
> > wrote:
>
> >>AEF wrote:
>
> >>>On Mar 21, 11:41 am, David J Dachtera
> >>>wrote:
>
> >>>>Dale Dellutri wrote:
>
> >>>>>On Thu, 20 Mar 2008 12:36:19 -0700 (PDT), tadamsmar wrote:
>
> >>>>>>On Mar 20, 2:36?pm, Dale Dellutri wrote:
>
> >>>>>>>On Thu, 20 Mar 2008 07:02:14 -0700 (PDT), tadamsmar wrote:
>
> >>>>>>>>To do a backup, I pop a drive out of a shadowset, back it up and put
> >>>>>>>>it back.
>
> >>>>>>>I assume you're talking about VMS Volume Shadowing. ?As far as I know,
> >>>>>>>taking a drive out of a shadowset causes the drive to look the same as
> >>>>>>>if there'd been a power failure. ?In other words, it does not properly
> >>>>>>>close open files.
>
> >>>>>>>I assume that you take image backups. ?If so, there's no benefit to
> >>>>>>>taking the drive out of the shadowset to take the image backup.
> >>>>>>>According to service techs that I talked to when I first set up
> >>>>>>>volume shadowing, there's no advantage over taking an image backup
> >>>>>>>of the volume set (the DSA device).
>
> >>>>>>>I assume that this is still true.
>
> >>>>>>>>The problem is, I don't record the backup dates.
>
> >>>>>>>You could record backup dates if you simply took an image backup
> >>>>>>>of the volume set.
>
> >>>>>>Don't I have to shutdown my system to do that?
>
> >>>>>I finally found the relevant item in "HP Volume Shadowing for
> >>>>>OpenVMS", v7.3-2. Chapter 7, Section titled "Data Consistency
> >>>>>Requirements": "Removal of a shadow set member results in what
> >>>>>is called a crash-consistent copy. That is, the copy of the
> >>>>>data on the removed member is of the same level of consistency
> >>>>>as what would result if the system had failed at that instant."
> >>>>>(pg 124 in my copy).
>
> >>>>>Actually, reading the entire section titled "Guidelines for
> >>>>>for Using a Shadow Member for Backup" would be very useful.
>
> >>>>The usual technique is to quiesce the application (cause it to close its
> >>>>files), THEN split the shadow-sets. We do something similar with BCVs on
> >>>>an EMC DMX array. This minimizes your "backup window".
>
> >>>Why not...
> >>>shut the app
> >>>check that no files are open (except for INDEXF.SYS by the system)
> >>>dismount the shadows set entirely
>
> >>Doing so disables mini-copy. See HELP DISMOUNT /POLICY
>
> >>David J Dachtera
> >>(formerly dba) DJE Systems
>
> > Then why does the fine manual say to do it that way?
>
> > I guess it's a case of safety vs. speed.
>
> > AEF
>
> What version of the manual? Maybe it predates minicopy, or maybe
> it wasn't updated when minicopy was added, or maybe it is referring
> to the safe way to do a backup when you can't follow the 237 caveats
> that have been posted in this thread about the dangers of doing an
> online backup of an active application.
>
> --
> John Santos
> Evans Griffiths & Hart, Inc.
> 781-861-0670 ext 539
The most current version on www.hp.com: The one under the V8.3 tab,
but the manual itself is OpenVMS Alpha 7.3-2. I assume that's the most
current. Has anything been added since? If so, why is it not
documented at www.hp.com ?
AEF
-
Re: Please critique my backup practices
"John Wallace" writes:
>"Michael Moroney" wrote in message
>news:fs9n15$vfi$1@pcls4.std.com...
>>
>> The application maps data files via $CRMPSC/$MGBLSC and generally forces a
>> writeback ($UPDSEC) when something important changes. The application is
>> homegrown and plays *very* fast and loose with the data, but due to the
>> nature of this beast, that's not as bad as it sounds.
> If the backup happens to contain valid data from these section files,
>you're presumably better off than if it didn't? But how do you *know* it
>will contain valid restorable data?
These guys have been running wild for years, and have had problems with
bad data, but they brush it off. I figure a bad backup might be better
than no backup.
>In addition to the usual open file concerns, I'd also want to know about the
>relationship between the files - as the apps are still active, backups of
>files may be individually valid but may relate to different points in time,
>which, depending on the app, may or may not actually be a valid consistent
>restorable dataset?
>Even if you test the backup/restore and it appears to work when you test it,
>as far as I can see you've no guarantee that the standard backup will not,
>one day, contain inconsistent data e.g. because of unfortunate timing.
>Murphy says that sooner or later, you will actually need to restore the data
>but it won't be there in a usable form. What would the impact be if (when)
>that happened?
It has happened. The system is mostly for tracking and they reenter
things manually when there are problems.
>If this data really is important to you, and you really really really cannot
>quiesce the app somehow, maybe you might want to think about doing something
>application-specific (but probably outside the main application) to save a
>copy of the important live data; maybe something which "knows" about the
>application's behaviour and knows when a good time to take a self-consistent
>snapshot might be???? I'm sure there are folks around who could provide
>better advice than I'm offering. They might want $$$, which would
>presumably be worthwhile if the data in these files is valuable to you.
We're the ones trying to make some sanity out of the customer's system.
If we get the contract we will do a rewrite, including MUCH better data
protection, including such things as quiescing the app for proper backups,
and so much more. You don't want to know.