Incremental backups? - Questions

This is a discussion on Incremental backups? - Questions ; I am trying to set up Red Hat Linux to use as a file server. I have used Windows in the past with a program called Dantz Retrospect for incremental backups to another hard drive in the server. Everything I ...

+ Reply to Thread
Results 1 to 13 of 13

Thread: Incremental backups?

  1. Incremental backups?

    I am trying to set up Red Hat Linux to use as a file server. I have
    used Windows in the past with a program called Dantz Retrospect for
    incremental backups to another hard drive in the server. Everything I
    have read so far deals with booting up and making a full back up of
    the server. What software (less than $75.00) is availible to backup my
    Linux File Server to a Hard drive on the same Server.

    Thanks for your help
    Ryan

  2. Re: Incremental backups?

    In comp.os.linux.help Ryan Ashline wrote:
    > the server. What software (less than $75.00) is availible to backup my
    > Linux File Server to a Hard drive on the same Server.


    You don't need any "software". Just use tar. Same as always.


    Peter

  3. Re: Incremental backups?

    Peter T. Breuer wrote:

    > In comp.os.linux.help Ryan Ashline wrote:
    >
    >>the server. What software (less than $75.00) is availible to backup my
    >>Linux File Server to a Hard drive on the same Server.

    >
    >
    > You don't need any "software". Just use tar. Same as always.


    And once again Peter gets it dead wrong. Use "rsync": that way, you
    don't wind up duplicating files that are already there, and you can far
    more easily set it more gracefully to ignore certain files, manipulate
    ownership, or delete files on the target system that have been deleted
    locally. You can even bandwidth limit it trivially to reduce load on
    your systems, unlike tar.


  4. Re: Incremental backups?

    In comp.os.linux.help Ryan Ashline wrote:

    >the server. What software (less than $75.00) is availible to backup my
    >Linux File Server to a Hard drive on the same Server.


    I have some home written code that keeps up to ten generations of files.
    It writes to a local disk and will backup an entire system. I have been using it
    for some time to backup my NT and Linux systems. If you want to try it, it's
    available at http://www.onevista.com/delayed_mirror.c

    It's not a real-time mirror. It runs from a cron job. Every two hours for
    /home and /local. Every 24 hours for the rest of the system.

    Documentation? Look at the source, try the -h option or send mail.

    Of course, nothing beats tape. I use that as well but having online copies of
    files that I have just destroyed is a life saver.

    johna

  5. Re: Incremental backups?

    In comp.os.linux.help Nico Kadel-Garcia wrote:
    > Peter T. Breuer wrote:


    > > In comp.os.linux.help Ryan Ashline wrote:
    > >
    > >>the server. What software (less than $75.00) is availible to backup my
    > >>Linux File Server to a Hard drive on the same Server.

    > >
    > >
    > > You don't need any "software". Just use tar. Same as always.


    > And once again Peter gets it dead wrong. Use "rsync": that way, you


    Look, kook, I don't go insulting you. Simply keep your insults to
    yourself.

    Yes, *I* use rsync. It certainly updates only an increment each time,
    but it doesn't *produce* incremental backups, which is what the guy
    wants, it produces a mirror. In particular, you can't rollback to a
    given day with rsync.

    So it is not what the guy means by "backup", and hence not the answer
    for him.


    Peter

  6. Re: Incremental backups?

    Peter T. Breuer wrote:
    > In comp.os.linux.help Nico Kadel-Garcia wrote:
    >
    >>Peter T. Breuer wrote:

    >
    >
    >>>In comp.os.linux.help Ryan Ashline wrote:
    >>>
    >>>
    >>>>the server. What software (less than $75.00) is availible to backup my
    >>>>Linux File Server to a Hard drive on the same Server.
    >>>
    >>>
    >>>You don't need any "software". Just use tar. Same as always.

    >
    >
    >>And once again Peter gets it dead wrong. Use "rsync": that way, you

    >
    >
    > Look, kook, I don't go insulting you. Simply keep your insults to
    > yourself.


    OK, I'll let you insult yourself. No problem.

    > Yes, *I* use rsync. It certainly updates only an increment each time,
    > but it doesn't *produce* incremental backups, which is what the guy
    > wants, it produces a mirror. In particular, you can't rollback to a
    > given day with rsync.
    >
    > So it is not what the guy means by "backup", and hence not the answer
    > for him.


    Like I said, it only took you a few lines to get it dead wrong. He
    didn't say backup to tape, or to a file: he said backup to disk.

    Using tar for incremental backups does not *delete* files that have been
    removed. Rsync does. If he's backing up to disk, he should use live disk
    to do full file system backukps: that way he has an immediately
    bootable OS image (maybe with an /etc/fstab or grub.conf edit) rather
    than spending the laborious time doing sequential extractions from tar
    archives.

    If he needs sequential recover backups to extract things from the
    archive, there's an old standard trick where you create a full mirror,
    duplicate the mirror using hard links for all the files, and rsync to
    the mirror of the mirror. You can get away with storing quite a few
    incremental mirrors this way without using up significantly more disk
    space, although the extra directories do use up some inodes: it's
    similar to the NetApp trick where they have the ".backup" directory
    hidden in each directory, where incremental read-only backups are
    accessible.


  7. Re: Incremental backups?

    Have you considered using rsync? Check out a great article at
    http://www.mikerubel.org/computers/rsync_snapshots/


    ashline@yahoo.com (Ryan Ashline) wrote in message news:<54109d7d.0309151423.218f41b8@posting.google.com>...
    > I am trying to set up Red Hat Linux to use as a file server. I have
    > used Windows in the past with a program called Dantz Retrospect for
    > incremental backups to another hard drive in the server. Everything I
    > have read so far deals with booting up and making a full back up of
    > the server. What software (less than $75.00) is availible to backup my
    > Linux File Server to a Hard drive on the same Server.
    >
    > Thanks for your help
    > Ryan


  8. Re: Incremental backups?

    Peter T. Breuer wrote:

    > In comp.os.linux.help Nico Kadel-Garcia wrote:
    >> Peter T. Breuer wrote:

    >
    >> > In comp.os.linux.help Ryan Ashline wrote:
    >> >
    >> >>the server. What software (less than $75.00) is availible to backup my
    >> >>Linux File Server to a Hard drive on the same Server.
    >> >
    >> >
    >> > You don't need any "software". Just use tar. Same as always.

    >
    >> And once again Peter gets it dead wrong. Use "rsync": that way, you

    >
    > Look, kook, I don't go insulting you. Simply keep your insults to
    > yourself.
    >
    > Yes, *I* use rsync. It certainly updates only an increment each time,
    > but it doesn't *produce* incremental backups, which is what the guy
    > wants, it produces a mirror. In particular, you can't rollback to a
    > given day with rsync.
    >
    > So it is not what the guy means by "backup", and hence not the answer
    > for him.
    >
    >
    > Peter


    I don't want to enter the insulting match, but, Peter, "Just use 'tar'" is a
    bit vague. Tar has to find what to back up. I use "backerupper," which I
    found on Freshmeat. It does a full backup of user-defined
    directories/files on the first of the month, and incremental backups in
    between. It uses zip as the default, or bzip2 optionally. There is no
    provision as it stands for deleting old backups, but it is just a
    collection of scripts, so anyone with a bit more know-how than me could
    probably arrange this. There is a line for a "final command" where it
    could go. I use the final command to copy the zip file to my Windows
    directory then shut down the computer.

    Doug.
    --
    Registered Linux User No. 277548.
    They say lightning never strikes twice in the same place. My typing is
    about as accurate. Apologies for any typos that slip in. - Doug.

  9. Re: Incremental backups?

    : Doug Laidlaw
    : I don't want to enter the insulting match, but, Peter, "Just use
    : 'tar'" is a bit vague. Tar has to find what to back up. I use
    : "backerupper," which I found on Freshmeat. It does a full backup of
    : user-defined directories/files on the first of the month, and
    : incremental backups in between. It uses zip as the default, or bzip2
    : optionally.

    Yes, tar has to find what to back up. Both classic tar and GNU tar have
    several parameters designed to do just that. I use GNU tar's
    --listed-incremental.

    `--listed-incremental=SNAPSHOT-FILE'
    `-g SNAPSHOT-FILE'
    During a `--create' operation, specifies that the archive that
    `tar' creates is a new GNU-format incremental backup, using
    SNAPSHOT-FILE to determine which files to backup. With other
    operations, informs `tar' that the archive is in incremental
    format.

    along with a cron job to clean up the snapshot so that a full
    backup is done monthly. More normal methods might involve

    `--newer=DATE'
    `--after-date=DATE'
    `-N'
    When creating an archive, `tar' will only add files that have
    changed since DATE. If DATE begins with `/' or `.', it is taken
    to be the name of a file whose last-modified time specifies the
    date.

    and so on and so forth. GNU tar is a normal GNU "kitchen sink"
    tool, and can handle most tasks by careful selection of its
    many options.


    Wayne Throop throopw@sheol.org http://sheol.org/throopw

  10. Re: Incremental backups?

    :: Peter T. Breuer
    :: Yes, *I* use rsync. It certainly updates only an increment each
    :: time, but it doesn't *produce* incremental backups, which is what the
    :: guy wants, it produces a mirror. In particular, you can't rollback
    :: to a given day with rsync. So it is not what the guy means by
    :: "backup", and hence not the answer for him.

    : Nico Kadel-Garcia
    : If he needs sequential recover backups to extract things from the
    : archive, there's an old standard trick where you create a full mirror,
    : duplicate the mirror using hard links for all the files, and rsync to
    : the mirror of the mirror.

    Well... actually you want to rsync to a "latest" mirror, and then
    at convenient intervals, snapshot that mirror with hard links.
    If you next rsync to the duplicate of the mirror, you'll end up
    copying some things that could be links.

    Unless of course I'm misunderstanding what is meant by
    "rsync to the mirror of the mirror".

    But anyways, the basic notion is to use rsync to keep a mirror
    up to date, and then use cp -al to create incremental snapshots.

    It would be possible to use cp -al to make snapshots directly,
    if they were on the same filesystem... but that's not very robust
    against disk failures, and use of cp -al for this purpose depends
    on all updates to the tree being cp -al'ed occur by unlink without
    any overwrites. Fortuitously, rsync always unlinks, so that's
    allright then.

    On the other hand, I use rsync to create a mirror, and tar to
    create a separate incrmental archive. The incremental archives can
    be compressed, and if I got short of space, I could dispense with
    the mirror and still have compressed tarballs.

    On the other hand, use of rsync && cp -al might work well
    with a compressed filesystem; there are several implementations
    of those hanging around.

    So as usual, there are zillions of ways to do it.


    Wayne Throop throopw@sheol.org http://sheol.org/throopw

  11. Re: Incremental backups?

    In article <54109d7d.0309151423.218f41b8@posting.google.com>, Ryan Ashline
    wrote:

    > I am trying to set up Red Hat Linux to use as a file server. I have
    > used Windows in the past with a program called Dantz Retrospect for
    > incremental backups to another hard drive in the server. Everything I
    > have read so far deals with booting up and making a full back up of
    > the server. What software (less than $75.00) is availible to backup my
    > Linux File Server to a Hard drive on the same Server.


    There are lots of good backup scripts that use basic Unix tools, and it's
    not too hard to write another one if you know a bit of bash or perl. I
    like Reoback enough not to bother thinking about it myself:
    http://reoback.sourceforge.net/ . The price is well within your budget

    --
    Juha Siltala
    http://www.edu.helsinki.fi/activity/people/jsiltala/

  12. Re: Incremental backups?

    Wayne Throop wrote:

    > :: Peter T. Breuer
    > :: Yes, *I* use rsync. It certainly updates only an increment each
    > :: time, but it doesn't *produce* incremental backups, which is what the
    > :: guy wants, it produces a mirror. In particular, you can't rollback
    > :: to a given day with rsync. So it is not what the guy means by
    > :: "backup", and hence not the answer for him.
    >
    > : Nico Kadel-Garcia
    > : If he needs sequential recover backups to extract things from the
    > : archive, there's an old standard trick where you create a full mirror,
    > : duplicate the mirror using hard links for all the files, and rsync to
    > : the mirror of the mirror.
    >
    > Well... actually you want to rsync to a "latest" mirror, and then
    > at convenient intervals, snapshot that mirror with hard links.
    > If you next rsync to the duplicate of the mirror, you'll end up
    > copying some things that could be links.
    >
    > Unless of course I'm misunderstanding what is meant by
    > "rsync to the mirror of the mirror".


    You described what I meant.

    > But anyways, the basic notion is to use rsync to keep a mirror
    > up to date, and then use cp -al to create incremental snapshots.


    cp -al is precisely what I meant, thank you.

    > It would be possible to use cp -al to make snapshots directly,
    > if they were on the same filesystem... but that's not very robust
    > against disk failures, and use of cp -al for this purpose depends
    > on all updates to the tree being cp -al'ed occur by unlink without
    > any overwrites. Fortuitously, rsync always unlinks, so that's
    > allright then.


    That's true. If you need *tape* backup, which the original poster did
    not mention, though, this allows you to do the tape and chance of disk
    writes during the backup off of the primary server. This is
    *particularly* useful for systems that run databases: the database can
    be built on the mirror using a specific snapshot data dump of the
    original disk, rather than from a potentially "in the midst of an atomic
    operation" file, as well.

    > On the other hand, I use rsync to create a mirror, and tar to
    > create a separate incrmental archive. The incremental archives can
    > be compressed, and if I got short of space, I could dispense with
    > the mirror and still have compressed tarballs.
    >
    > On the other hand, use of rsync && cp -al might work well
    > with a compressed filesystem; there are several implementations
    > of those hanging around.
    >
    > So as usual, there are zillions of ways to do it.


    Yup. That's why I get so irritated at Peter's snapshot answers: they're
    snappish, often pretend as if the questioner was an idiot for not
    knowing the answer already, and often frankly wrong for the situation
    the questioner described.


  13. Re: Incremental backups?

    Nico Kadel-Garcia wrote:

    > Peter T. Breuer wrote:
    >
    >> In comp.os.linux.help Ryan Ashline wrote:
    >>
    >>>the server. What software (less than $75.00) is availible to backup my
    >>>Linux File Server to a Hard drive on the same Server.

    >>
    >>
    >> You don't need any "software". Just use tar. Same as always.

    >
    > And once again Peter gets it dead wrong. Use "rsync": that way, you
    > don't wind up duplicating files that are already there, and you can far
    > more easily set it more gracefully to ignore certain files, manipulate
    > ownership, or delete files on the target system that have been deleted
    > locally. You can even bandwidth limit it trivially to reduce load on
    > your systems, unlike tar.


    With deference to Peter (and BTW thanks Peter for your explanation of tar)

    I had /usr/src as the only partition on /dev/hdb, and I had space for it on
    /dev/hda. I used rsync to move it, and everything rebooted like a charm.
    The only error message was to the effect of "mount: /usr/src doesn't exist
    as a mount point," but mtab showed that the mount had succeeded. The line
    in fstab was identical to the previous one, with only the partition ID
    changed. It was probably that I was trying to mount /usr/src before /usr,
    but anyway, it worked. The real attraction of rsync was that it preserves
    linking.

    Doug.

    --
    Registered Linux User No. 277548.
    "They" say lightning never strikes twice in the same place. My typing is
    about as accurate. Apologies for any typos that slip in. - Doug.

+ Reply to Thread