Yea old defrag - Kernel

This is a discussion on Yea old defrag - Kernel ; I'm writing this to Theodore Ts'o because he has worked on the package in the past, and currently maintains e2fsprogs. I am CCing lkml since there are likely to be other interested parties, and there does not seem to be ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: Yea old defrag

  1. Yea old defrag

    I'm writing this to Theodore Ts'o because he has worked on the package
    in the past, and currently maintains e2fsprogs. I am CCing lkml since
    there are likely to be other interested parties, and there does not seem
    to be a separate list for e2fsprogs.

    The old defrag package for doing offline defragmentation of ext2 has not
    been maintained in many years. About two years ago I fixed a few bugs
    that were causing it to not work on ext3 and placed the changes in
    Ubuntu. I would like to see this program continue to live since it
    still serves a purpose, but because it has had no upstream maintainer in
    so long, Debian and thus Ubuntu have dropped the package. I am trying
    to resurrect it.

    I was wondering why it was never integrated into e2fsprogs, and if this
    could now be done. It seems to me that e2fsprogs would be the proper
    home for it, and a neutral upstream for everyone. If not, then I will
    try to maintain it as an Ubuntu native package in a bazaar branch on
    launchpad.
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  2. Re: Yea old defrag

    On Wed, Oct 22, 2008 at 12:10:23PM -0400, Phillip Susi wrote:
    > I'm writing this to Theodore Ts'o because he has worked on the package
    > in the past, and currently maintains e2fsprogs. I am CCing lkml since
    > there are likely to be other interested parties, and there does not seem
    > to be a separate list for e2fsprogs.
    >
    > The old defrag package for doing offline defragmentation of ext2 has not
    > been maintained in many years. About two years ago I fixed a few bugs
    > that were causing it to not work on ext3 and placed the changes in
    > Ubuntu. I would like to see this program continue to live since it
    > still serves a purpose, but because it has had no upstream maintainer in
    > so long, Debian and thus Ubuntu have dropped the package. I am trying
    > to resurrect it.
    >
    > I was wondering why it was never integrated into e2fsprogs, and if this
    > could now be done. It seems to me that e2fsprogs would be the proper
    > home for it, and a neutral upstream for everyone. If not, then I will
    > try to maintain it as an Ubuntu native package in a bazaar branch on
    > launchpad.


    The problem with the defrag code has always been that I haven't had
    time to give it proper love and attention, and it needs a *lot* of
    work before I would be willing to stand behind it as a useful and
    "safe" tool to use. Basically, I don't want to get the hate mail from
    people who lose their data due to bugs or other potential failures
    from e2defrag.

    I have a vague memory it doesn't even work if the filesystem blocksize
    is greater than 1k, but I'm not 100% certain that is still true.

    There is also an on-line defragmetter that requires kernel support
    that some folks from NEC are working on for ext4 (which can also
    support ext3 filesystems). My thinking is that it's more worthwhile
    to focus my attentions on helping Akira Fujita and Takashi Sato finish
    theiron-line ext4 defragmentation patches than to worry about
    e2defrag. Of course, if you'd like to give some love and attention to
    e2defrag, that's great. But e2defrag doesn't use the ext2fs
    libraries, and it would be a huge amount of work to make sure it
    supports things like extended attributes and other newer format
    enhancements that have been made to the ext3 filesystem.

    Regards,

    - Ted

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  3. Re: Yea old defrag

    Theodore Tso wrote:
    > The problem with the defrag code has always been that I haven't had
    > time to give it proper love and attention, and it needs a *lot* of
    > work before I would be willing to stand behind it as a useful and
    > "safe" tool to use. Basically, I don't want to get the hate mail from
    > people who lose their data due to bugs or other potential failures
    > from e2defrag.


    Indeed, it rightly should be covered with warnings about it possibly
    munching your data and should only be used on fully backed up
    filesystems. As far as I know currently though, it works properly and
    I'm willing to use what free time I have to fix any bugs that are found,
    but I need to decide where to house the source in a proper VCS instead
    of just continuing to add dpatches to the ubuntu source package.

    > I have a vague memory it doesn't even work if the filesystem blocksize
    > is greater than 1k, but I'm not 100% certain that is still true.


    I have an active 110 gig ext3 root fs ( that is only about 8% used )
    that I tested on with no errors ( after removing one disk from the
    mirror and using that to test with ). It uses 4k blocks, has_journal,
    resize_inode, dir_index, filetype, sparse_super, large_file. I think it
    won't support blocks larger than 4k, but I don't think the kernel will
    either. I also verified the md5sums of all files on the disk after the
    defrag.

    > There is also an on-line defragmetter that requires kernel support
    > that some folks from NEC are working on for ext4 (which can also
    > support ext3 filesystems). My thinking is that it's more worthwhile
    > to focus my attentions on helping Akira Fujita and Takashi Sato finish
    > theiron-line ext4 defragmentation patches than to worry about
    > e2defrag. Of course, if you'd like to give some love and attention to
    > e2defrag, that's great. But e2defrag doesn't use the ext2fs
    > libraries, and it would be a huge amount of work to make sure it
    > supports things like extended attributes and other newer format
    > enhancements that have been made to the ext3 filesystem.


    I've been reading a lot in the last few hours about the proposed online
    defragmenter, and while it seems quite interesting, it appears to not be
    ready yet, whereas e2defrag is readily available. It will be
    interesting to compare the two approaches.

    I have been trying to identify any new features that cause problems and
    fix any that arise. Aren't extended attributes stored in the extra
    space of large inodes? If so, I would need to format a new fs using
    large inodes to test. I don't think e2defrag currently supports large
    inodes, but it shouldn't be too hard to add support. Other than being
    aware of the correct size of the inodes, I don't see anything special it
    would need to do to support extended attributes if they are just extra
    data in the inode, or can they be stored in data blocks and so it would
    need to understand the block pointers in the inode?

    Are there any other ext3 features that might cause trouble? I know
    there are plenty in ext4, but I want to make sure everything works well
    in ext3 first.
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

  4. Re: Yea old defrag

    On Wed, Oct 22, 2008 at 04:01:48PM -0400, Phillip Susi wrote:
    >
    > I have been trying to identify any new features that cause problems and
    > fix any that arise. Aren't extended attributes stored in the extra
    > space of large inodes? If so, I would need to format a new fs using
    > large inodes to test.


    Extended attributes *can* be stored in the extra space of large
    inodes, but they can also be stored in separate, stand-alone data
    blocks --- and just to make life entertaining, if multiple files have
    exactly the same set of extended attributes (for example, if all of
    the files inone directory have the same ACL and/or SELinux property),
    an extended attribute block can be shared across multiple inodes for
    space efficiency.

    > Are there any other ext3 features that might cause trouble? I know
    > there are plenty in ext4, but I want to make sure everything works well
    > in ext3 first.


    That's probably the main one; and as you say, there are plenty of new
    features that will require special handling for e2defrag, especially
    since it doesn't use the standard libext2fs library.

    - Ted

    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

+ Reply to Thread