rsync 3.0.2 with --fileflags on FreeBSD: cannot rsync hardlinkedimmutable files - Tools

This is a discussion on rsync 3.0.2 with --fileflags on FreeBSD: cannot rsync hardlinkedimmutable files - Tools ; Hi *, it seems rsync with --fileflags isn't able to work on (already) hardlinked and immutable ("schg") files on FreeBSD. The following scripts will create a simple example for this behaviour: -------------------------------------------------------------- #! /bin/sh # # set -x DIR="/var/tmp/rsync_$(date +%s)/" ...

+ Reply to Thread
Results 1 to 3 of 3

Thread: rsync 3.0.2 with --fileflags on FreeBSD: cannot rsync hardlinkedimmutable files

  1. rsync 3.0.2 with --fileflags on FreeBSD: cannot rsync hardlinkedimmutable files

    Hi *,

    it seems rsync with --fileflags isn't able to work on (already)
    hardlinked and immutable ("schg") files on FreeBSD. The following
    scripts will create a simple example for this behaviour:

    --------------------------------------------------------------
    #! /bin/sh
    #
    #

    set -x
    DIR="/var/tmp/rsync_$(date +%s)/"
    mkdir "${DIR}/"

    # Preparing dir_A
    mkdir "${DIR}/dir_A/"
    touch "${DIR}/dir_A/file_A"
    ln "${DIR}/dir_A/file_A" "${DIR}/dir_A/file_B"
    chflags schg "${DIR}/dir_A/file_A"
    ls -laio "${DIR}/dir_A/"

    # Try rsync: will fail because of schg'ed hardlinked file
    /usr/local/bin/rsync -avHWx --fileflags "${DIR}/dir_A/" "${DIR}/dir_B/"
    ls -laio "${DIR}/dir_B/"

    # Try cpdup: will work
    /usr/local/bin/cpdup "${DIR}/dir_A/" "${DIR}/dir_C/"
    ls -laio "${DIR}/dir_C/"
    --------------------------------------------------------------

    will output:


    --------------------------------------------------------------
    + date +%s
    + DIR=/var/tmp/rsync_1212259951/
    + mkdir /var/tmp/rsync_1212259951//
    + mkdir /var/tmp/rsync_1212259951//dir_A/
    + touch /var/tmp/rsync_1212259951//dir_A/file_A
    + ln /var/tmp/rsync_1212259951//dir_A/file_A /var/tmp/rsync_1212259951//dir_A/file_B
    + chflags schg /var/tmp/rsync_1212259951//dir_A/file_A
    + ls -laio /var/tmp/rsync_1212259951//dir_A/
    total 4
    188674 drwxr-xr-x 2 root wheel - 512 May 31 20:52 .
    188673 drwxr-xr-x 3 root wheel - 512 May 31 20:52 ..
    188676 -rw-r--r-- 2 root wheel schg 0 May 31 20:52 file_A
    188676 -rw-r--r-- 2 root wheel schg 0 May 31 20:52 file_B
    + /usr/local/bin/rsync -avHWx --fileflags /var/tmp/rsync_1212259951//dir_A//var/tmp/rsync_1212259951//dir_B/
    sending incremental file list
    created directory /var/tmp/rsync_1212259951//dir_B
    ./
    file_B
    rsync: link "/var/tmp/rsync_1212259951/dir_B/file_A" => file_B failed: Operation not permitted (1)

    sent 95 bytes received 34 bytes 258.00 bytes/sec
    total size is 0 speedup is 0.00
    rsync error: some files could not be transferred (code 23) at main.c(1031) [sender=3.0.2]
    + ls -laio /var/tmp/rsync_1212259951//dir_B/
    total 4
    188677 drwxr-xr-x 2 root wheel - 512 May 31 20:52 .
    188673 drwxr-xr-x 4 root wheel - 512 May 31 20:52 ..
    188678 -rw-r--r-- 1 root wheel schg 0 May 31 20:52 file_B
    + /usr/local/bin/cpdup /var/tmp/rsync_1212259951//dir_A/ /var/tmp/rsync_1212259951//dir_C/
    + ls -laio /var/tmp/rsync_1212259951//dir_C/
    total 4
    188679 drwxr-xr-x 2 root wheel - 512 May 31 20:52 .
    188673 drwxr-xr-x 5 root wheel - 512 May 31 20:52 ..
    188680 -rw-r--r-- 2 root wheel schg 0 May 31 20:52 file_A
    188680 -rw-r--r-- 2 root wheel schg 0 May 31 20:52 file_B
    --------------------------------------------------------------

    It seems rsync first sets the immutable-flag ("schg") and try to
    hardlink the second file on the (already) schg'ed file, which fails as
    expected.

    A workaround would be to set alle hardlinks before setting the
    immutable-flags on the files.

    I got a patched patch for "flags.diff" by Rolf Grossmann for rsync
    2.6.9, which handled hardlinks on schg'ed files gracefully (using
    --flags instead of --fileflags), see
    http://rabe.uugrn.org/files/replacem...hes_flags.diff



    --------------------------------------------------------------
    + date +%s
    + DIR=/var/tmp/rsync_1212261208/
    + mkdir /var/tmp/rsync_1212261208//
    + mkdir /var/tmp/rsync_1212261208//dir_A/
    + touch /var/tmp/rsync_1212261208//dir_A/file_A
    + ln /var/tmp/rsync_1212261208//dir_A/file_A /var/tmp/rsync_1212261208//dir_A/file_B
    + chflags schg /var/tmp/rsync_1212261208//dir_A/file_A
    + ls -laio /var/tmp/rsync_1212261208//dir_A/
    total 4
    212060 drwxr-xr-x 2 root wheel - 512 May 31 21:13 .
    212059 drwxr-xr-x 3 root wheel - 512 May 31 21:13 ..
    212061 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_A
    212061 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_B
    + /usr/local/bin/rsync -avHWx --flags /var/tmp/rsync_1212261208//dir_A/ /var/tmp/rsync_1212261208//dir_B/
    building file list ... done
    created directory /var/tmp/rsync_1212261208//dir_B
    ./
    file_B
    file_A => file_B

    sent 148 bytes received 61 bytes 418.00 bytes/sec
    total size is 0 speedup is 0.00
    + ls -laio /var/tmp/rsync_1212261208//dir_B/
    total 4
    212062 drwxr-xr-x 2 root wheel - 512 May 31 21:13 .
    212059 drwxr-xr-x 4 root wheel - 512 May 31 21:13 ..
    212063 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_A
    212063 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_B
    + /usr/local/bin/cpdup /var/tmp/rsync_1212261208//dir_A/ /var/tmp/rsync_1212261208//dir_C/
    + ls -laio /var/tmp/rsync_1212261208//dir_C/
    total 4
    212064 drwxr-xr-x 2 root wheel - 512 May 31 21:13 .
    212059 drwxr-xr-x 5 root wheel - 512 May 31 21:13 ..
    212065 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_A
    212065 -rw-r--r-- 2 root wheel schg 0 May 31 21:13 file_B
    --------------------------------------------------------------
    root@bottom:~# pkg_info | grep rsync
    rsync-2.6.9_2 A network file distribution/synchronization utility
    --------------------------------------------------------------


    I use rsync for lot of things, especially rsyncing FreeBSD jails between
    jailhosts (via ssh) needs the hardlinks on schg'ed files, for
    local-to-local syncing cpdup is doing fine here.

    Regards
    Raphael Becker

    --
    --
    Raphael Becker http://rabe.uugrn.org/
    GnuPG: E7B2 1D66 3AF2 EDC7 9828 6D7A 9CDA 3E7B 10CA 9F2D
    .........|.........|.........|.........|.........| .........|.........|..

    --
    Please use reply-all for most replies to avoid omitting the mailing list.
    To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
    Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.2 (FreeBSD)

    iD8DBQFIQaQbnNo+exDKny0RApxrAJ40Aj/JbU3K3hmMa/+RU1BnN+4pfACg15Gi
    vi1xKzAtgBfkEqD5aS3D02Q=
    =+Hcg
    -----END PGP SIGNATURE-----


  2. Re: rsync 3.0.2 with --fileflags on FreeBSD: cannot rsynchardlinked immutable files

    On Sat, May 31, 2008 at 09:16:43PM +0200, I wrote:
    > Hi *,
    >
    > it seems rsync with --fileflags isn't able to work on (already)
    > hardlinked and immutable ("schg") files on FreeBSD. The following
    > scripts will create a simple example for this behaviour:


    I forgot to check --force-change but this didn't help either.

    I googled around and found http://rsync.samba.org/ftp/rsync/patches/fileflags.diff

    ... which doesn't seem to work on the current(?) sources:

    root@daemon:/usr/ports/net/rsync# make extract
    ===> Vulnerability check disabled, database not found
    ===> Found saved configuration for rsync-3.0.2_1
    => rsync-3.0.2.tar.gz doesn't seem to exist in
    /data/ftp/pub/FreeBSD/distfiles//.
    => Attempting to fetch from http://ftp2.de.freebsd.org/FreeBSD/distfiles//.
    fetch: http://ftp2.de.freebsd.org/FreeBSD/d...-3.0.2.tar.gz: Not Found
    => Attempting to fetch from http://rsync.samba.org/ftp/rsync/.
    rsync-3.0.2.tar.gz 100% of 747 kB 21 MBps
    => rsync-patches-3.0.2.tar.gz doesn't seem to exist in /data/ftp/pub/FreeBSD/distfiles//.
    => Attempting to fetch from http://ftp2.de.freebsd.org/FreeBSD/distfiles//.
    fetch: http://ftp2.de.freebsd.org/FreeBSD/d...-3.0.2.tar.gz: Not Found
    => Attempting to fetch from http://rsync.samba.org/ftp/rsync/.
    rsync-patches-3.0.2.tar.gz 100% of 150 kB 17 MBps
    ===> Extracting for rsync-3.0.2_1
    => MD5 Checksum OK for rsync-3.0.2.tar.gz.
    => SHA256 Checksum OK for rsync-3.0.2.tar.gz.
    => MD5 Checksum OK for rsync-patches-3.0.2.tar.gz.
    => SHA256 Checksum OK for rsync-patches-3.0.2.tar.gz.
    ===> rsync-3.0.2_1 depends on file: /usr/local/bin/perl5.8.8 - found

    Overriding the patch from the distfile with the patch from the web:
    root@daemon:/usr/ports/net/rsync# wget -O /data/tmp/usr/ports/net/rsync/work/rsync-3.0.2/patches/fileflags.diff http://rsync.samba.org/ftp/rsync/patches/fileflags.diff

    But this patch doesn't match the current(?) sources:

    root@daemon:/usr/ports/net/rsync# make patch
    ===> Patching for rsync-3.0.2_1
    ===> rsync-3.0.2_1 depends on file: /usr/local/bin/perl5.8.8 - found
    ===> Applying extra patch
    /data/tmp/usr/ports/net/rsync/work/rsync-3.0.2/patches/fileflags.diff
    1 out of 2 hunks failed--saving rejects to t_stub.c.rej
    1 out of 4 hunks failed--saving rejects to util.c.rej
    *** Error code 2

    Stop in /usr/ports/net/rsync.
    *** Error code 1

    Stop in /usr/ports/net/rsync.

    The two distfiles are
    MD5 (rsync-3.0.2.tar.gz) = fd4c5d77d8cb7bb86ab209076fa214d9
    MD5 (rsync-patches-3.0.2.tar.gz) = c076661c447ddc165f4ff427e7264a31

    Does anyone know if there's a way to rsync hardlinks on immutable files?
    Wayne Davison? Rolf Grossmann? Anyone else?

    Help would be greatly appreciated.

    Regards
    Raphael Becker

    --
    Raphael Becker http://rabe.uugrn.org/
    GnuPG: E7B2 1D66 3AF2 EDC7 9828 6D7A 9CDA 3E7B 10CA 9F2D
    .........|.........|.........|.........|.........| .........|.........|..

    --
    Please use reply-all for most replies to avoid omitting the mailing list.
    To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
    Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.2 (FreeBSD)

    iD8DBQFISxHLnNo+exDKny0RAk8vAKCHg9F/r1JIE0Y9xCvqAE0aSkdrhQCeOuMM
    GKveOVbIGPFfL/w/IXWgZwQ=
    =k43C
    -----END PGP SIGNATURE-----


  3. RE: rsync 3.0.2 with --fileflags on FreeBSD: cannot rsynchardlinkedimmutable files

    > -----Original Message-----
    > From: rsync-bounces+jcall=verio.net@lists.samba.org [mailto:rsync-
    > bounces+jcall=verio.net@lists.samba.org] On Behalf Of Raphael Becker
    > Sent: Saturday, June 07, 2008 4:55 PM
    > To: rsync@lists.samba.org
    > Subject: Re: rsync 3.0.2 with --fileflags on FreeBSD: cannot
    > rsynchardlinked immutable files
    >
    > On Sat, May 31, 2008 at 09:16:43PM +0200, I wrote:
    > > Hi *,
    > >
    > > it seems rsync with --fileflags isn't able to work on (already)
    > > hardlinked and immutable ("schg") files on FreeBSD. The following
    > > scripts will create a simple example for this behaviour:

    >
    > I forgot to check --force-change but this didn't help either.
    >
    > I googled around and found
    > http://rsync.samba.org/ftp/rsync/patches/fileflags.diff
    >
    > ... which doesn't seem to work on the current(?) sources:
    >
    > root@daemon:/usr/ports/net/rsync# make extract
    > ===> Vulnerability check disabled, database not found
    > ===> Found saved configuration for rsync-3.0.2_1
    > => rsync-3.0.2.tar.gz doesn't seem to exist in
    > /data/ftp/pub/FreeBSD/distfiles//.
    > => Attempting to fetch from
    > http://ftp2.de.freebsd.org/FreeBSD/distfiles//.
    > fetch:

    http://ftp2.de.freebsd.org/FreeBSD/d...-3.0.2.tar.gz:
    > Not Found
    > => Attempting to fetch from http://rsync.samba.org/ftp/rsync/.
    > rsync-3.0.2.tar.gz 100% of 747 kB 21

    MBps
    > => rsync-patches-3.0.2.tar.gz doesn't seem to exist in
    > /data/ftp/pub/FreeBSD/distfiles//.
    > => Attempting to fetch from
    > http://ftp2.de.freebsd.org/FreeBSD/distfiles//.
    > fetch: http://ftp2.de.freebsd.org/FreeBSD/d...rsync-patches-
    > 3.0.2.tar.gz: Not Found
    > => Attempting to fetch from http://rsync.samba.org/ftp/rsync/.
    > rsync-patches-3.0.2.tar.gz 100% of 150 kB 17

    MBps
    > ===> Extracting for rsync-3.0.2_1
    > => MD5 Checksum OK for rsync-3.0.2.tar.gz.
    > => SHA256 Checksum OK for rsync-3.0.2.tar.gz.
    > => MD5 Checksum OK for rsync-patches-3.0.2.tar.gz.
    > => SHA256 Checksum OK for rsync-patches-3.0.2.tar.gz.
    > ===> rsync-3.0.2_1 depends on file: /usr/local/bin/perl5.8.8 - found
    >
    > Overriding the patch from the distfile with the patch from the web:
    > root@daemon:/usr/ports/net/rsync# wget -O
    > /data/tmp/usr/ports/net/rsync/work/rsync-3.0.2/patches/fileflags.diff
    > http://rsync.samba.org/ftp/rsync/patches/fileflags.diff
    >
    > But this patch doesn't match the current(?) sources:
    >
    > root@daemon:/usr/ports/net/rsync# make patch
    > ===> Patching for rsync-3.0.2_1
    > ===> rsync-3.0.2_1 depends on file: /usr/local/bin/perl5.8.8 - found
    > ===> Applying extra patch
    > /data/tmp/usr/ports/net/rsync/work/rsync-3.0.2/patches/fileflags.diff
    > 1 out of 2 hunks failed--saving rejects to t_stub.c.rej
    > 1 out of 4 hunks failed--saving rejects to util.c.rej
    > *** Error code 2
    >
    > Stop in /usr/ports/net/rsync.
    > *** Error code 1
    >
    > Stop in /usr/ports/net/rsync.
    >
    > The two distfiles are
    > MD5 (rsync-3.0.2.tar.gz) = fd4c5d77d8cb7bb86ab209076fa214d9
    > MD5 (rsync-patches-3.0.2.tar.gz) = c076661c447ddc165f4ff427e7264a31
    >
    > Does anyone know if there's a way to rsync hardlinks on immutable

    files?
    > Wayne Davison? Rolf Grossmann? Anyone else?
    >
    > Help would be greatly appreciated.
    >
    > Regards
    > Raphael Becker
    >


    This impacts my systems as well so I would also appreciate a
    fix/solution.

    Jonathan


    This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you.
    --
    Please use reply-all for most replies to avoid omitting the mailing list.
    To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
    Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


+ Reply to Thread