DO NOT REPLY [Bug 5728] New: Rsync --min-size and --max-size filtertoo late in the process. - Tools

This is a discussion on DO NOT REPLY [Bug 5728] New: Rsync --min-size and --max-size filtertoo late in the process. - Tools ; https://bugzilla.samba.org/show_bug.cgi?id=5728 Summary: Rsync --min-size and --max-size filter too late in the process. Product: rsync Version: 2.6.9 Platform: x86 OS/Version: Linux Status: NEW Severity: minor Priority: P3 Component: core AssignedTo: wayned@samba.org ReportedBy: r.e.wolff@bitwizard.nl QAContact: rsync-qa@samba.org To allow me to copy a ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: DO NOT REPLY [Bug 5728] New: Rsync --min-size and --max-size filtertoo late in the process.

  1. DO NOT REPLY [Bug 5728] New: Rsync --min-size and --max-size filtertoo late in the process.

    https://bugzilla.samba.org/show_bug.cgi?id=5728

    Summary: Rsync --min-size and --max-size filter too late in the
    process.
    Product: rsync
    Version: 2.6.9
    Platform: x86
    OS/Version: Linux
    Status: NEW
    Severity: minor
    Priority: P3
    Component: core
    AssignedTo: wayned@samba.org
    ReportedBy: r.e.wolff@bitwizard.nl
    QAContact: rsync-qa@samba.org


    To allow me to copy a very large directory that cannot be copied in one go (see
    bug 5727) I restricted the sizes of the files to part of the range of
    filesizes.

    However, it turns out that rsync still loads all files into the in-memory list
    of files to copy. This means that rsync still crashes even when I restrict the
    number of files-to-copy by 95%.

    (Copying by subdirectory doesn't work for me: there are loads of hardlinks, and
    because I know that files of size XX won't be hardlinked to files of size YY, I
    can copy in stages by limiting the filesizes).


    --
    Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
    ------- You are receiving this mail because: -------
    You are the QA contact for the bug, or are watching the QA contact.
    --
    Please use reply-all for most replies to avoid omitting the mailing list.
    To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
    Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


  2. DO NOT REPLY [Bug 5728] Rsync --min-size and --max-size filter toolate in the process.

    https://bugzilla.samba.org/show_bug.cgi?id=5728


    wayned@samba.org changed:

    What |Removed |Added
    ----------------------------------------------------------------------------
    Status|NEW |RESOLVED
    Resolution| |WONTFIX
    Summary|Rsync --min-size and --max- |Rsync --min-size and --max-
    |size filter too late in the |size filter too late in the
    |process. |process.




    ------- Comment #1 from wayned@samba.org 2008-08-31 12:29 CST -------
    Those options just affect what files in the file-list may be transferred. If
    they actually excluded files from the file-list, the receiving side would not
    be able to correctly handle a --delete option.

    You should instead sub-set the directory using an exclude, such as
    --exclude='dir/[a-m]*' for one run, and --exclude='dir/[n-z]*' for the other
    (or something similar).


    --
    Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
    ------- You are receiving this mail because: -------
    You are the QA contact for the bug, or are watching the QA contact.
    --
    Please use reply-all for most replies to avoid omitting the mailing list.
    To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
    Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


  3. DO NOT REPLY [Bug 5728] Change --min-size and --max-size to filterthe file lists, like excludes

    https://bugzilla.samba.org/show_bug.cgi?id=5728


    wayned@samba.org changed:

    What |Removed |Added
    ----------------------------------------------------------------------------
    Severity|minor |enhancement
    Status|RESOLVED |REOPENED
    Resolution|WONTFIX |
    Summary|Rsync --min-size and --max- |Change --min-size and --max-
    |size filter too late in the |size to filter the file
    |process. |lists, like excludes
    Version|2.6.9 |3.1.0




    ------- Comment #2 from wayned@samba.org 2008-08-31 13:31 CST -------
    Actually, I was too hasty in my closure. Filtering the file lists would work
    OK as long as the receiver was doing a similar filter. Delete would behave
    differently than with the current options, but not-deleting certain
    out-of-range files would be comparable to the exclusions that happen for the
    current filter rules.


    --
    Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
    ------- You are receiving this mail because: -------
    You are the QA contact for the bug, or are watching the QA contact.
    --
    Please use reply-all for most replies to avoid omitting the mailing list.
    To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
    Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


  4. DO NOT REPLY [Bug 5728] Change --min-size and --max-size to filterthe file lists, like excludes

    https://bugzilla.samba.org/show_bug.cgi?id=5728





    ------- Comment #3 from r.e.wolff@bitwizard.nl 2008-08-31 16:18 CST -------
    I do not want to delete files on the recieving side. So there is no need to
    "prepare" for that possibility.

    And as I said, there are enormous amounts of hardlinks between the directories,
    that would be broken if I copied per-directory.

    There could be something like
    if (otherside_needs_full_list || filter (curpath))
    send_this_file (curpath);
    in the code that sends the file list?


    --
    Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
    ------- You are receiving this mail because: -------
    You are the QA contact for the bug, or are watching the QA contact.
    --
    Please use reply-all for most replies to avoid omitting the mailing list.
    To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
    Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


+ Reply to Thread