Maximum Files in a Directory - Microsoft Windows

This is a discussion on Maximum Files in a Directory - Microsoft Windows ; P> Is there a maximum # of files a Windows directory can hold? DJC> It depends upon the media and format. True. DJC> FAT32 and NTFS are unlimited except by total sectors available. False. FAT32 is limited, for non-zero-sized files, ...

+ Reply to Thread
Results 1 to 5 of 5

Thread: Maximum Files in a Directory

  1. Maximum Files in a Directory

    P> Is there a maximum # of files a Windows directory can hold?

    DJC> It depends upon the media and format.

    True.

    DJC> FAT32 and NTFS are unlimited except by total sectors available.

    False.

    FAT32 is limited, for non-zero-sized files, by the number of clusters
    in the volume, which is a fraction of the number of sectors. There
    are also limits imposed by the bitness of the filesystem drivers,
    albeit not by the on-disc filesystem format itself. Some drivers, for
    example, use 16-bit numbers for directory entry indices, limiting each
    directory to no more than 65536 entries -- a limit that is further
    reduced by the facts that long filenames occupy multiple entries and
    the "." and ".." entries always consume two entries.


  2. Re: Maximum Files in a Directory

    J de Boyne Pollard wrote:

    >FAT32 is limited, for non-zero-sized files, by the number of clusters
    >in the volume, which is a fraction of the number of sectors. There
    >are also limits imposed by the bitness of the filesystem drivers,
    >albeit not by the on-disc filesystem format itself. Some drivers, for
    >example, use 16-bit numbers for directory entry indices, limiting each
    >directory to no more than 65536 entries -- a limit that is further
    >reduced by the facts that long filenames occupy multiple entries and
    >the "." and ".." entries always consume two entries.


    The Fat32 spec says that any conforming driver *must* limit
    directories to 65,536 entries. This has nothing to do with whether the
    driver is 16- or 32-bit code. To conform with the spec, the limit must
    be there. I think the reason is that, as you have said, FAT32 doesn't
    scale well to huge directories. To find a file in a FAT32 directory
    you must do a sequential search of directory entries. NTFS uses a
    btree structure, so a search for a single file is quick. Getting a
    list of all filenames in the directory can take longer in NTFS than in
    FAT32, especially for huge directories.

    And yes, file names that don't fit the DOS 8.3 standard will take from
    2 to 13 entries in a FAT32 directory, so the directory will fill up
    before you expect it to.

    --
    Tim Slattery
    MS MVP(Shell/User)
    Slattery_T@bls.gov
    http://members.cox.net/slatteryt

  3. Maximum Files in a Directory

    JdeBP> There are also limits imposed by the bitness of the
    JdeBP> filesystem drivers, albeit not by the on-disc filesystem
    JdeBP> format itself. Some drivers, for example, use 16-bit
    JdeBP> numbers for directory entry indices, limiting each
    JdeBP> directory to no more than 65536 entries -- a limit
    JdeBP> that is further reduced by the facts that long
    JdeBP> filenames occupy multiple entries and the
    JdeBP> "." and ".." entries always consume two entries.

    TS> The Fat32 spec says that any conforming driver *must*
    TS> limit directories to 65,536 entries.

    It does? So it does.

    TS> This has nothing to do with whether the driver is
    TS> 16- or 32-bit code.

    Who said anything about _code_? The aforementioned bitness is in the
    _data_.

    TS> To conform with the spec, the limit must be there.
    TS> I think the reason is that, as you have said, FAT32
    TS> doesn't scale well to huge directories. [...]

    Actually, the specification itself states the reasons. The first
    reason is, indeed, the problem with the inefficient on-disc data
    structures. The second reason is the same as I gave above -- namely
    that some filesystem drivers and utility softwares (Microsoft says
    "many", "including Microsoft's".) use 16-bit numbers for directory
    entry indices.

    I have to revisit some code and make a decision, I see. Personally, I
    aim not to hardwire any 16-bit limits into my own softwares. This is
    a good example of why one should always endeavour not to. It's an
    example of how one curtails people decades later by doing so. This
    limit was not originally part of the FAT on-disc data structures, but
    has become so entrenched in driver and utility softwares that
    Microsoft has formalized it in a specification and _made_ it a limit
    on the on-disc data structures.

    I suspect that the same will happen with the 32-bit cluster number
    limit for NTFS.


  4. Re: Maximum Files in a Directory

    J de Boyne Pollard wrote:

    >JdeBP> There are also limits imposed by the bitness of the
    >JdeBP> filesystem drivers, albeit not by the on-disc filesystem
    >JdeBP> format itself. Some drivers, for example, use 16-bit
    >JdeBP> numbers for directory entry indices, limiting each
    >JdeBP> directory to no more than 65536 entries -- a limit
    >JdeBP> that is further reduced by the facts that long
    >JdeBP> filenames occupy multiple entries and the
    >JdeBP> "." and ".." entries always consume two entries.
    >
    >TS> The Fat32 spec says that any conforming driver *must*
    >TS> limit directories to 65,536 entries.
    >
    >It does? So it does.
    >
    >TS> This has nothing to do with whether the driver is
    >TS> 16- or 32-bit code.
    >
    >Who said anything about _code_? The aforementioned bitness is in the
    >_data_.


    That makes no difference at all to the file system. It doesn't know
    what's being stored, other than a bunch of bytes. Might be a string,
    might be integers, might be floating point, the file system doesn't
    know and doesn't care.

    --
    Tim Slattery
    MS MVP(Shell/User)
    Slattery_T@bls.gov
    http://members.cox.net/slatteryt

  5. Maximum Files in a Directory

    JdeBP> There are also limits imposed by the bitness of the
    JdeBP> filesystem drivers, albeit not by the on-disc filesystem
    JdeBP> format itself. Some drivers, for example, use 16-bit
    JdeBP> numbers for directory entry indices, limiting each
    JdeBP> directory to no more than 65536 entries -- a limit
    JdeBP> that is further reduced by the facts that long
    JdeBP> filenames occupy multiple entries and the
    JdeBP> "." and ".." entries always consume two entries.

    TS> The Fat32 spec says that any conforming driver *must*
    TS> limit directories to 65,536 entries.

    JdeBP> It does? So it does.

    TS> This has nothing to do with whether the driver is
    TS> 16- or 32-bit code.

    JdeBP> Who said anything about _code_? The aforementioned
    JdeBP> bitness is in the _data_.

    TS> That makes no difference at all to the file system. It
    TS> doesn't know what's being stored, other than a bunch
    TS> of bytes. [...]

    Who said anything about what is being stored? The aforementioned
    data, with the aforementioned bitness, are _in the driver_. Please re-
    read the first and second sentences of the very first quotation given
    above.


+ Reply to Thread