47,900 MP3's - How To Organize Hard Discs? - Storage

This is a discussion on 47,900 MP3's - How To Organize Hard Discs? - Storage ; On Mon, 10 Sep 2007 09:34:21 +0000, Andre Majorel wrote: > On what OS ? For Unix filesystems, the bigger the filesystem the > farther away a directory entry will be from the inode on > average. That's no good. ...

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2
Results 21 to 22 of 22

Thread: 47,900 MP3's - How To Organize Hard Discs?

  1. Re: 47,900 MP3's - How To Organize Hard Discs?

    On Mon, 10 Sep 2007 09:34:21 +0000, Andre Majorel wrote:


    > On what OS ? For Unix filesystems, the bigger the filesystem the
    > farther away a directory entry will be from the inode on
    > average. That's no good.
    >
    > Directory fragmentation will slow things down, as Arno pointed
    > out. On the other hand, if you create subdirectories to fight
    > that, fewer directories will be fragmented but there will be
    > more of them so the total seek time may actually increase.
    >
    > There are good reasons for avoiding directories with large
    > number of entries, though (CPU).
    >


    Yea, I used to put all my mp3's in one folder but doing that made opening
    the folder take a long time so now I sub categorize the folders to cut
    down on the amount of files in one folder. With a good media player like
    MediaMonkey it doesn't matter as it sees all the files in sub folders and
    has it's own categories to sort by.

  2. Re: 47,900 MP3's - How To Organize Hard Discs?

    Andre Majorel wrote
    > Rod Speed wrote
    >> Andre Majorel wrote
    >>> matthewhargrove@msn.com wrote


    >>>> I have defragged and have an organized directory tree set-up
    >>>> (each letter of alphabet --> band name --> album --> song files),


    >>> So you have about 4000 directories with a dozen of entries each.
    >>> Directories with so few entries are unlikely to become fragmented.


    >> It aint the directory that gets fragmented, its the files.


    > If I'm not mistaken the OP has not been complaining about file I/O.


    Irrelevant to YOUR claims about fragmentation.

    >>> On the other hand, that makes a lot of them.


    >>> Scanning that tree means opening 4000 directories. That's
    >>> expensive. Unless NTFS wildly differs from the file systems I
    >>> know, opening a directory has a significant fixed cost. Assuming
    >>> no fragmentation, listing 10 directories with 10 entries each
    >>> takes more time than listing 1 directory with 100 entries.


    >> Fragmentation has no effect on listing directorys.


    > Unless your disk drive has zero seek time, it has.


    Doesnt need zero seek time, file fragmentation has no effect on listing directorys.

    >>> You could make fewer directories with more entries. For example,
    >>> one directory per artist. But that won't solve everything because
    >>> scanning the tree still means stating nearly 50,000 files.


    >> But you only need to scan the directory of the artist you are looking at.


    > I assume the OP is scanning all (or a significant portion of) the tree.


    Bad assumption.

    > Otherwise it wouldn't be slow enough to notice.


    Irrelevant to your claim about 'scanning the tree still means stating nearly 50,000 files'

    >>> Alternatively, you could have a cron job (or whatever periodic
    >>> tasks are called on Windows) slowly scan the tree every few
    >>> minutes. A simple "DIR D:\ E:\ F:\ ... /-O /-P /S >NUL" would be
    >>> a good start. That would help the directories and file metadata
    >>> remain in the cache.


    >> No it wouldnt with that many files.


    > $ time find / | head -n 50000 | wc -l
    > 50000


    > real 0m7.794s
    > user 0m0.169s
    > sys 0m0.291s
    > $ time find / | head -n 50000 | wc -l
    > 50000
    >
    > real 0m0.301s
    > user 0m0.134s
    > sys 0m0.166s


    > Works for me, on Ext3 at least.


    You dont even know that he's using that.

    >>>> I am trying to decide if I should have just a few large
    >>>> partitions or a bunch or smaller ones to organize the
    >>>> hard disks for speed of accessing the data.


    >>> Your many-small-partitions scheme makes day-to-day management
    >>> more difficult but I don't think it makes things slower.


    >> It can do.


    > In the OP's situation ? How ?


    If its being done manually, you have to fart around keeping
    track of what is in each partition. You're unlikely to be able
    to have something logical like say one partition per artist etc.

    >>> In fact, it probably makes them a bit faster by improving locality.


    >> Pity about farting around changing the current partition.


    > That's Windows for you. :->


    It aint just windows.



+ Reply to Thread
Page 2 of 2 FirstFirst 1 2