Backup tape reuse strategies? - Linux

This is a discussion on Backup tape reuse strategies? - Linux ; Hi all, We have a centralized backup system of several dozen machines. My boss is cracking down on tape costs, so we are trying to come up with a strategy to recycle old backup tapes. The problem is that it's ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: Backup tape reuse strategies?

  1. Backup tape reuse strategies?

    Hi all,

    We have a centralized backup system of several dozen machines. My boss
    is cracking down on tape costs, so we are trying to come up with a
    strategy to recycle old backup tapes. The problem is that it's not
    easy to do it because the backup software (Time Navigator) writes
    files from different machines and from different dates onto the same
    tape. Thus, if I want to keep a full backup only of a certain subset
    of machines from a year ago, it's very time-consuming to track down
    the tapes involved.

    Obviously, the software allows to backup each host to a different set
    of tapes, but managing all this becomes hard very quickly, and it
    would waste tapes anyway. I can also set cut-off dates where I close
    all tapes and start anew, say, each month, but this also adds to the
    management time and tape wastage.

    Has anyone run into this problem (ease-of-use vs precise
    accountability) with backup tapes, especially regarding tape reuse?
    This would probably arise with any kind of centralized backup
    solution...

    Thanks,
    Simon Ilyushchenko

  2. Re: Backup tape reuse strategies?

    Raven and Peter Håkanson,

    Thank you for your responses.

    > Can I ask if he has a valid point or is it just a case of penny pinching?
    > How much value do you place on the tapes if you actually need to use them
    > for a restore?


    I never thought I'd regret I am not in corporate world. Here is a
    bigger picture: we are a research institution. We *are* on a tight
    budget, and doing monthly fulls for the available disk space without
    recycling tapes would run into six figures (USD). Backups are
    important mostly for disaster recovery and short-term data loss - I
    have had virtually no requests for files over half a year old.
    However, we have had cases in the past when years-old data had to be
    dug up regarding some patent issues. On top of all, our attempts to
    ask the backup policy on a higher level have not had any reply yet - I
    guess nobody wants to be responsible for formulating it. (Looks like
    this is the first thing to pursue - there are still even higher people
    to consult.)

    > Could it be time to reconsider your whole strategy including the choice of
    > backup software?


    I am not averse to the idea - one of the reasons for the post is to
    find out what other people are doing in similar situations.

    > How many tape units and locations? By the sounds of it you need to run
    > separate jobs for each machine so as to keep groups of tapes for each.


    Two locations, two tape units in each, for Windows and Unix servers
    separately.
    Close to a hundred servers in total with about 10 Tb of disks storage
    (and growing), though, thankfully, it's only about 50% utilized so
    far.

    The problem with separate jobs is increased maintenance time. After
    all, we are not only minimizing the tape cost, but the man-hours
    spent, which can be more productively used elsewhere.

    > Agreed and agreed. Perhaps it's worth re-evaluating whether the cost of
    > tapes under the current system is really that excessive.


    From what I understand, the budgets are *really* tight.

    > How many tapes are we talking about in total? Surely the value of the tape
    > is irrelevant if the backups are so critical to the operation?


    Judging by the frequency of the requests, long-term backups are not
    very critical, but as I mentioned, in the absence of the policy you
    never know.

    > As far as accountability, if they really are tracking the tape costs
    > themselves simply serial number each tape and do a semi-regular audit. Tapes
    > that go bad get officially written off, tapes that go missing get
    > questioned.


    I did not mean just keeping track of tapes themselves, but rather of
    what exactly is on each tape. And with centralized backup software,
    it's non-trivial.

    Simon Ilyushchenko

  3. Re: Backup tape reuse strategies?

    BTW, I forgot to ask: is anyone checking whether a tape is good before
    using it? I mean the whole tape, not just a small part in the
    beginning. My backup software does not do it.

    Thanks,
    Simon

  4. Re: Backup tape reuse strategies?

    In comp.unix.admin Simon wrote:
    > Raven and Peter Håkanson,


    > Thank you for your responses.


    Is the tapes fully utilized ? If not ( and the contents are not compressed
    on non-compressing drives) lots of space might be wasted.

    You might also gain significant amount if you can separate things that
    can be recovered from distributions ( OS and tools ) from the actual
    data. This might include forcing data to "fileservers-only" where
    regular backups are done. Creating schedules with full backups
    intermixed with "levels" ( aka hanoi-tower) might spare a number
    of tapes. ( I have had great success using freely avaliable amanda
    in a network of machines). The perfect case is where you can say :
    "We don't backup workstations, they are recreated via fresh install".

    Now a good policy is essential to define the "acceptable service", without
    that you are always on the loosing side whenever data is lost. If
    management avoids this you could create one yourself and state "We will
    use this policy until something else is decided".

    This policy might stipulate that "backup is for disaster purposes and
    limited to (xx ) months. Any archiving has to be done via other purposes.
    ( remember that backup and archiving is two totally different things,
    using different tools and with differing goals)

    --
    Peter Håkanson
    IPSec Sverige ( At Gothenburg Riverside )
    Sorry about my e-mail address, but i'm trying to keep spam out,
    remove "icke-reklam" if you feel for mailing me. Thanx.

  5. Re: Backup tape reuse strategies?

    On 29 Jul 2003 09:59:11 -0700, Simon wrote:
    >
    > I never thought I'd regret I am not in corporate world. Here is a
    > bigger picture: we are a research institution. We *are* on a tight
    > budget, and doing monthly fulls for the available disk space without
    > recycling tapes would run into six figures (USD). Backups are
    > important mostly for disaster recovery and short-term data loss - I
    > have had virtually no requests for files over half a year old.


    Yes, IRS auditors were looking into a million $ plus project
    which we wrote off as development research several years ago.
    I think their computer brought it up because time was about to expire
    and they would not be able to look into it.

    Our lawyer calls me up and asks for project plans, requirement/design
    docs, memos, source. Glad my operations group have yearly backups
    going way back.

    What is even better, when the media changed from round reel to
    cartridge, they copied all the data onto the new media.

    We would had to have paid a bit of a penalty without the items.
    Penalty would have wiped all those *savings*.

    Penny wise can be pound foolish.

    Hey site the risk, the example. Ask them to check with the lawyer
    if Management is libel for the policy. I find when I put management
    in harm's way, we can find the better solution.


  6. Re: Backup tape reuse strategies?

    In comp.unix.admin Bit Twister wrote:

    > Our lawyer calls me up and asks for project plans, requirement/design
    > docs, memos, source. Glad my operations group have yearly backups
    > going way back.
    > What is even better, when the media changed from round reel to
    > cartridge, they copied all the data onto the new media.


    Similar situation: I used to work for $VERY_BIG_COMPANY who was being
    sued for a (bogus) patent infringement claim. By going back to backup
    tapes, again, 2 generations older than current technology, I was able to
    retrieve the weekly "report to my boss what I was working on this week"
    file from the key developer who had been on the project. Using these files,
    we were able to show what, when, and by whom, the work had been done.

    If we had reused those tapes, it would have been a finger-pointing contest,
    at which point a jury could have very well decided to give the advantage
    to the "little guy" fighting the very very big company. We kept all of our
    full backups (done weekly) offsite, and pay an obscene amount of money to
    store them (and for the tapes that are effectively "lost"), but that one
    lawsuit could have more than wiped out any savings that we would have seen
    by reusing those tapes.

    If nothing else, decide on a policy...full backups have *x* months of
    retention ('forever' might be the right answer), but incrementals have
    just a few months before they're reused. Maybe you only want to keep one
    set of full backups per month, but keep those forever; it depends on
    how much risk you're willing to take, compared to the cost of tapes.

    One other thing - if your software can be configured so that full and
    incrementals get written to different tapes, do it. That way, only full
    backups will be on the tapes you need to keep around.

    > Penny wise can be pound foolish.


    Absolutely correct. Tape is cheaper than the work needed to generate the
    data on it, almost always.

    Dave Hinz


  7. Re: Backup tape reuse strategies?

    On Mon, 28 Jul 2003 19:21:51 -0500, "Kermit T Tensmeyer"
    wrote:

    [ Snip ]

    > Most tape systems have a hard time figuring out that a tape is full, so it
    >will use tape errors to identify that it can't write any more to the tape.
    > The downside to this is that the system has a hard
    > time detecting that the tape is worn out.


    As a general rule, in the tape drive business we call tape systems
    that behave as you describe "broken".

    Reaching End-of-Tape is a specific non-error exception; unfortunately,
    a lot of *software* fails to distinguish the case where a write
    successfully wrote only part of the requested data because of an EOT
    was reached (partial completion, no error), and the case where the
    same thing happened because of an unrecoverable write error (partial
    completion, write error).

    [ Snip ]

    Malc.

  8. Re: Backup tape reuse strategies?

    In comp.unix.admin Nicholas Bachmann wrote:
    > davehinz@spamcop.net wrote:
    >>
    >> Similar situation: I used to work for $VERY_BIG_COMPANY who was being
    >> sued for a (bogus) patent infringement claim. By going back to backup


    > Let me guess: some company nobody's ever heard of whose sole business is
    > owning patents and filing lawsuits against big companies (like GE, IBM,
    > or Microsoft) who will most likely settle quietly and finance the
    > purchase of more stupid patents and more bogus lawsuits. Am I right?


    I think you know I worked for GE, right? So, you're kind of right...
    it was a case of "David vs. Goliath". To be honest, I'm not sure how
    (or if) it finally was settled.

    It was an MRI scanning technology that, yes, there was a patent that
    vaguely described something like what GE was also doing, and the little
    company claimed "We thought of it first, see, here." It was something
    like "Instead of taking scans along just the X, Y, or Z axis, let's take
    them at an angle" which, to my non-legal mind, falls into the category
    of "blisteringly ****ing obvious application of the technology" - kind of
    like patenting using the side of a crayon for coloring instead of just
    the tip.

    It did have one relevant (to this thread) effect, though...my backup
    tape budget went _way_ up, and I was able to switch from 8mm
    (boo, hiss) to DLT-7000 drives. About the same cost per gig for tape,
    but much cheaper to pay for storage (they charge by the tub, not by
    the byte).

    It all depends on the priorities. If the OP's needs are more of a backup/
    restore than an archiving, then that changes the answers somewhat. If the
    decision comes down to "Do we pay for archive tapes, or continue research",
    well, that seems like a pretty straightforward question to answer...making
    the organization become history just to preserve the history would be silly.

    Dave Hinz



  9. Re: Backup tape reuse strategies?

    First of all Thank God I don't have that kind of problem.

    The problem you have is a little strange in that way that the cost of
    tape compared to the cost of the time you spend on minimizing the tape
    cost, it would be more easy, secure and cost less just to buy the
    tapes you need.

    In our small shop we use NetBackup Datacenter as backupsoftware (good
    software, ok support, you can trust it and it has the features most
    people need.
    On the hardware side we use a StorageTek L-series library and
    StorageTek 9940B tapedrives.

    Basicly our tapemanagement is based on the standard features in
    NetBackup (no fancy stuff at all)
    In the different policies we configure how long to keep a given
    backup, and when there is no longer a valid backup on the tape
    NetBackup uses it again.
    The only real planning on minimizing the numbers of tape we use are on
    the number of days we keep the backups. I found out that we used less
    tapes on a daily basis when we used the same period for all backups.
    In the beginning it seemed a little strange to me, but basicly it's
    because we didn't fill all the tapes.

    We run around 250 backups a day (could be more since we don't count
    them) and the total amount written to tape is in the area of 1 TB a
    day.

    Compared to what the backupsoftware, backuphardware and the people to
    run the backups cost, tapecost is a minor issue (I think we pay less
    then EUR 100 a tape)

    If I wanted to keep some of my backups for a longer period I would
    duplicate them to a seperate volume pool (standard feature in NBU but
    I don't know about you software) or use vaulting and let NetBackup do
    it automaticly)

    It seems to me that you want to do some realy fancy backup stuff with
    a low cost software and hardware installation. And the only way to get
    that to work is to use a lot of manpower for it.

    /johnny

    On 28 Jul 2003 14:43:18 -0700, simonf@simonf.com (Simon) wrote:

    >Hi all,
    >
    >We have a centralized backup system of several dozen machines. My boss
    >is cracking down on tape costs, so we are trying to come up with a
    >strategy to recycle old backup tapes. The problem is that it's not
    >easy to do it because the backup software (Time Navigator) writes
    >files from different machines and from different dates onto the same
    >tape. Thus, if I want to keep a full backup only of a certain subset
    >of machines from a year ago, it's very time-consuming to track down
    >the tapes involved.
    >
    >Obviously, the software allows to backup each host to a different set
    >of tapes, but managing all this becomes hard very quickly, and it
    >would waste tapes anyway. I can also set cut-off dates where I close
    >all tapes and start anew, say, each month, but this also adds to the
    >management time and tape wastage.
    >
    >Has anyone run into this problem (ease-of-use vs precise
    >accountability) with backup tapes, especially regarding tape reuse?
    >This would probably arise with any kind of centralized backup
    >solution...
    >
    >Thanks,
    >Simon Ilyushchenko



+ Reply to Thread