rsync hangs during --remove-source-files - Tools
This is a discussion on rsync hangs during --remove-source-files - Tools ; Hi,
I'm using the following command to move files from a to b:
rsync -avvvv --remove-source-files --ignore-existing /path/to/dir1/
/path/to/dir2 >> logfile
dir2 isn't empty so I only move files from dir1 to dir2 that don't exist
already.
Every so often ...
-
rsync hangs during --remove-source-files
Hi,
I'm using the following command to move files from a to b:
rsync -avvvv --remove-source-files --ignore-existing /path/to/dir1/
/path/to/dir2 >> logfile
dir2 isn't empty so I only move files from dir1 to dir2 that don't exist
already.
Every so often rsync stops in the middle and doesn't continue with the
result that I have to kill it.
The last time it happened, rsync had successfully moved 1320 files,
there were 2943 files in dir2 and 1101 files in dir1.
lsof showed no open files inside dir1 or dir2, the log file was opened 4
times, dir1 itself showed up and dir2 twice. This is on a dual core CPU
with rsync version 3.0.4 protocol version 30.
The log file shows no errors. Last 4 lines say:
renaming dir/res/.svn/text-base/.somefile.svn-base.mDBdUb to
dir/res/.svn/text-base/somefile.svn-base
sender removed dir/.svn/text-base/somefile.svn-base
touch_up_dirs: dir/.svn/prop-base (84)
set modtime of dir/res/.svn/prop-base to (1223212308) Sun Oct 5
21:11:48 2008
Any suggestions how to debug this so that I can find out what's going
on? Unfortunately I haven't found a way to reproduce it. Every time I
kill and execute the same command again, it works.
Cheers,
Tom
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
-
Re: rsync hangs during --remove-source-files
On Mon, 2008-10-13 at 21:19 +0800, Thomas Gutzler wrote:
> I'm using the following command to move files from a to b:
> rsync -avvvv --remove-source-files --ignore-existing /path/to/dir1/
> /path/to/dir2 >> logfile
> dir2 isn't empty so I only move files from dir1 to dir2 that don't exist
> already.
> Every so often rsync stops in the middle and doesn't continue with the
> result that I have to kill it.
> Any suggestions how to debug this so that I can find out what's going
> on? Unfortunately I haven't found a way to reproduce it. Every time I
> kill and execute the same command again, it works.
The next time rsync hangs, before you kill it, attach gdb to each of the
three rsync processes ("gdb /PATH/TO/rsync PID") and get a stack trace
("bt"). Wayne should have more suggestions for what information would
be helpful for debugging.
Matt
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
-
Re: rsync hangs during --remove-source-files
On Fri, Oct 17, 2008 at 03:34:03PM +0800, Thomas Gutzler wrote:
> I had another rsync hang. I've attached a text file with the stack
> traces you asked for and some other info.
The generator backtrace shows that it is waiting for the receiver to
finish some files before going on. This attempts to sprinkle the source
removes more evenly during the transfer instead of having them all get
bunched up at the end. So, the strange thing is that the receiver is
waiting around for more files to process, which raises the question of
why the generator doesn't know that the files it initiated are all done
(or make me wonder if some pipe data didn't get delivered, but that is
less likely). Did your transfer report any errors?
...wayne..
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
-
Re: rsync hangs during --remove-source-files
Wayne Davison wrote:
> On Fri, Oct 17, 2008 at 03:34:03PM +0800, Thomas Gutzler wrote:
>> I had another rsync hang. I've attached a text file with the stack
>> traces you asked for and some other info.
>
> [...] So, the strange thing is that the receiver is
> waiting around for more files to process, which raises the question of
> why the generator doesn't know that the files it initiated are all done
> (or make me wonder if some pipe data didn't get delivered, but that is
> less likely). Did your transfer report any errors?
No, there are no errors in the log file expect the one when I killed the
process.
I've just switched back to rsync version 2.6.9 - see if that works.
Happy to test/debug stuff, you just have to tell me what to do.
Tom
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html