moving files - Unix

This is a discussion on moving files - Unix ; When a filesystem is restructured (e.g. new space is made available), and data needs to be moved from an old space to a new space, what's a good plan for doing that? Our sysadmin copies the data (through cp or ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: moving files

  1. moving files

    When a filesystem is restructured (e.g. new space is made available), and
    data needs to be moved from an old space to a new space, what's a good plan
    for doing that?

    Our sysadmin copies the data (through cp or two tars or whatever). And
    later deletes the data. That's fine with me, but he doesn't lock down the
    old area (by e.g. keeping everyone else out while the cp is going on by
    restricting permissions). So, as people keep using the old area, he has no
    idea whether the new area is still in sync with the old area.

    Isn't this a bad idea?



  2. Re: moving files

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    sinister wrote:
    > When a filesystem is restructured (e.g. new space is made available), and
    > data needs to be moved from an old space to a new space, what's a good plan
    > for doing that?
    >
    > Our sysadmin copies the data (through cp or two tars or whatever). And
    > later deletes the data. That's fine with me, but he doesn't lock down the
    > old area (by e.g. keeping everyone else out while the cp is going on by
    > restricting permissions). So, as people keep using the old area, he has no
    > idea whether the new area is still in sync with the old area.
    >
    > Isn't this a bad idea?


    FWIW, this is my opinion only. Take it for what it's worth.

    When making backups or moving directory trees around, unless you are willing
    to live with potential file syncronization errors, the sysadm should take the
    system into single user mode before starting the copy, and revert to multiuser
    mode once the copy (and validation) is complete.

    S/he might risk remaining in multiuser mode /if/ the directory tree is stable
    (no changes to it will be made during the time the backup or move process is
    executing), but for active directory trees (/var or /home, for instance) the
    only /safe/ way to move them is to restrict access to read only or no access
    at all.

    Having said all that, it depends on the environment and the use of the
    machine. Some systems can be permitted to be volatile or unstable, while
    others are required to be available 7/24. In either case, it may be unsuitable
    to restrict write access to directories and files while backing them up or
    moving them. That's a risk that site management must evaluate and deal with.

    - --
    Lew Pitcher

    Master Codewright & JOAT-in-training | GPG public key available on request
    Registered Linux User #112576 (http://counter.li.org/)
    Slackware - Because I know what I'm doing.
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.2.7 (GNU/Linux)

    iD8DBQFDD6RnagVFX4UWr64RAgxRAJ9F+2vOTwQbTBL+5I156g jrsCaZtwCgj0VQ
    S2KU8ccD1/yQHhZ+6d4MhHg=
    =evN9
    -----END PGP SIGNATURE-----

  3. Re: moving files

    Lew Pitcher wrote:
    > sinister wrote:
    >
    > > When a filesystem is restructured (e.g. new space is made available), and
    > > data needs to be moved from an old space to a new space, what's a good plan
    > > for doing that?

    >
    > > Our sysadmin copies the data (through cp or two tars or whatever). And
    > > later deletes the data. That's fine with me, but he doesn't lock down the
    > > old area (by e.g. keeping everyone else out while the cp is going on by
    > > restricting permissions). So, as people keep using the old area, he has no
    > > idea whether the new area is still in sync with the old area.

    >
    > > Isn't this a bad idea?

    >
    > FWIW, this is my opinion only. Take it for what it's worth.
    >
    > When making backups or moving directory trees around, unless you are willing
    > to live with potential file syncronization errors, the sysadm should take the
    > system into single user mode before starting the copy, and revert to multiuser
    > mode once the copy (and validation) is complete.
    >
    > S/he might risk remaining in multiuser mode /if/ the directory tree is stable
    > (no changes to it will be made during the time the backup or move process is
    > executing), but for active directory trees (/var or /home, for instance) the
    > only /safe/ way to move them is to restrict access to read only or no access
    > at all.


    Use logical volume managers and modern filesytem formats
    to grow the storage in place continuing to use the old
    devs.

    Use LVM level mirroring to move the data to the new hardware
    on the same host. Add new devs to group, mirror and sync to
    new devs, break mirror, del old devs, grow LVM and FS to the
    added storage.

    Moving from host to host, that's more of a challenge. For a
    way that works on medium-to-low activity trees like source
    code I've done replacing with symbolic links to the new NFS
    tree as in http://relocation.wopr.net/ Improving this model
    by doing CRC or similar would be even better. With an OS
    like Windows that does mandatory locks on open files this
    strategy could work by only migrating closed files.


  4. Re: moving files


    "Doug Freyburger" wrote in message
    news:1125243797.494039.144410@g14g2000cwa.googlegr oups.com...
    > Lew Pitcher wrote:
    >> sinister wrote:
    >>
    >> > When a filesystem is restructured (e.g. new space is made available),
    >> > and
    >> > data needs to be moved from an old space to a new space, what's a good
    >> > plan
    >> > for doing that?

    >>
    >> > Our sysadmin copies the data (through cp or two tars or whatever). And
    >> > later deletes the data. That's fine with me, but he doesn't lock down
    >> > the
    >> > old area (by e.g. keeping everyone else out while the cp is going on by
    >> > restricting permissions). So, as people keep using the old area, he
    >> > has no
    >> > idea whether the new area is still in sync with the old area.

    >>
    >> > Isn't this a bad idea?

    >>
    >> FWIW, this is my opinion only. Take it for what it's worth.
    >>
    >> When making backups or moving directory trees around, unless you are
    >> willing
    >> to live with potential file syncronization errors, the sysadm should take
    >> the
    >> system into single user mode before starting the copy, and revert to
    >> multiuser
    >> mode once the copy (and validation) is complete.
    >>
    >> S/he might risk remaining in multiuser mode /if/ the directory tree is
    >> stable
    >> (no changes to it will be made during the time the backup or move process
    >> is
    >> executing), but for active directory trees (/var or /home, for instance)
    >> the
    >> only /safe/ way to move them is to restrict access to read only or no
    >> access
    >> at all.

    >
    > Use logical volume managers and modern filesytem formats
    > to grow the storage in place continuing to use the old
    > devs.


    OS is Solaris 9; old filesystem is ufs IIRC. The new system is under
    Veritas, which was pitched as allowing the storage to grow in place. Looks
    like he's using rsync to copy the files over.

    Does that make what he's doing OK?

    At least in the past I've seen him copy data over from one ufs filesystem to
    another, and then after a few days they got out of sync.

    Maybe Veritas plus rsync makes that a moot point.

    Thanks for the reply.

    > Use LVM level mirroring to move the data to the new hardware
    > on the same host. Add new devs to group, mirror and sync to
    > new devs, break mirror, del old devs, grow LVM and FS to the
    > added storage.
    >
    > Moving from host to host, that's more of a challenge. For a
    > way that works on medium-to-low activity trees like source
    > code I've done replacing with symbolic links to the new NFS
    > tree as in http://relocation.wopr.net/ Improving this model
    > by doing CRC or similar would be even better. With an OS
    > like Windows that does mandatory locks on open files this
    > strategy could work by only migrating closed files.
    >




+ Reply to Thread