rsync hangs during --remove-source-files - Tools

This is a discussion on rsync hangs during --remove-source-files - Tools ; Hi, I'm using the following command to move files from a to b: rsync -avvvv --remove-source-files --ignore-existing /path/to/dir1/ /path/to/dir2 >> logfile dir2 isn't empty so I only move files from dir1 to dir2 that don't exist already. Every so often ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: rsync hangs during --remove-source-files

  1. rsync hangs during --remove-source-files

    Hi,

    I'm using the following command to move files from a to b:
    rsync -avvvv --remove-source-files --ignore-existing /path/to/dir1/
    /path/to/dir2 >> logfile
    dir2 isn't empty so I only move files from dir1 to dir2 that don't exist
    already.
    Every so often rsync stops in the middle and doesn't continue with the
    result that I have to kill it.

    The last time it happened, rsync had successfully moved 1320 files,
    there were 2943 files in dir2 and 1101 files in dir1.
    lsof showed no open files inside dir1 or dir2, the log file was opened 4
    times, dir1 itself showed up and dir2 twice. This is on a dual core CPU
    with rsync version 3.0.4 protocol version 30.

    The log file shows no errors. Last 4 lines say:
    renaming dir/res/.svn/text-base/.somefile.svn-base.mDBdUb to
    dir/res/.svn/text-base/somefile.svn-base
    sender removed dir/.svn/text-base/somefile.svn-base
    touch_up_dirs: dir/.svn/prop-base (84)
    set modtime of dir/res/.svn/prop-base to (1223212308) Sun Oct 5
    21:11:48 2008

    Any suggestions how to debug this so that I can find out what's going
    on? Unfortunately I haven't found a way to reproduce it. Every time I
    kill and execute the same command again, it works.

    Cheers,
    Tom


    --
    Please use reply-all for most replies to avoid omitting the mailing list.
    To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
    Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


  2. Re: rsync hangs during --remove-source-files

    On Mon, 2008-10-13 at 21:19 +0800, Thomas Gutzler wrote:
    > I'm using the following command to move files from a to b:
    > rsync -avvvv --remove-source-files --ignore-existing /path/to/dir1/
    > /path/to/dir2 >> logfile
    > dir2 isn't empty so I only move files from dir1 to dir2 that don't exist
    > already.
    > Every so often rsync stops in the middle and doesn't continue with the
    > result that I have to kill it.


    > Any suggestions how to debug this so that I can find out what's going
    > on? Unfortunately I haven't found a way to reproduce it. Every time I
    > kill and execute the same command again, it works.


    The next time rsync hangs, before you kill it, attach gdb to each of the
    three rsync processes ("gdb /PATH/TO/rsync PID") and get a stack trace
    ("bt"). Wayne should have more suggestions for what information would
    be helpful for debugging.

    Matt

    --
    Please use reply-all for most replies to avoid omitting the mailing list.
    To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
    Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


  3. Re: rsync hangs during --remove-source-files

    On Fri, Oct 17, 2008 at 03:34:03PM +0800, Thomas Gutzler wrote:
    > I had another rsync hang. I've attached a text file with the stack
    > traces you asked for and some other info.


    The generator backtrace shows that it is waiting for the receiver to
    finish some files before going on. This attempts to sprinkle the source
    removes more evenly during the transfer instead of having them all get
    bunched up at the end. So, the strange thing is that the receiver is
    waiting around for more files to process, which raises the question of
    why the generator doesn't know that the files it initiated are all done
    (or make me wonder if some pipe data didn't get delivered, but that is
    less likely). Did your transfer report any errors?

    ...wayne..
    --
    Please use reply-all for most replies to avoid omitting the mailing list.
    To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
    Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


  4. Re: rsync hangs during --remove-source-files

    Wayne Davison wrote:
    > On Fri, Oct 17, 2008 at 03:34:03PM +0800, Thomas Gutzler wrote:
    >> I had another rsync hang. I've attached a text file with the stack
    >> traces you asked for and some other info.

    >
    > [...] So, the strange thing is that the receiver is
    > waiting around for more files to process, which raises the question of
    > why the generator doesn't know that the files it initiated are all done
    > (or make me wonder if some pipe data didn't get delivered, but that is
    > less likely). Did your transfer report any errors?


    No, there are no errors in the log file expect the one when I killed the
    process.

    I've just switched back to rsync version 2.6.9 - see if that works.
    Happy to test/debug stuff, you just have to tell me what to do.

    Tom
    --
    Please use reply-all for most replies to avoid omitting the mailing list.
    To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
    Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


+ Reply to Thread