J de Boyne Pollard wrote:

>FAT32 is limited, for non-zero-sized files, by the number of clusters
>in the volume, which is a fraction of the number of sectors. There
>are also limits imposed by the bitness of the filesystem drivers,
>albeit not by the on-disc filesystem format itself. Some drivers, for
>example, use 16-bit numbers for directory entry indices, limiting each
>directory to no more than 65536 entries -- a limit that is further
>reduced by the facts that long filenames occupy multiple entries and
>the "." and ".." entries always consume two entries.

The Fat32 spec says that any conforming driver *must* limit
directories to 65,536 entries. This has nothing to do with whether the
driver is 16- or 32-bit code. To conform with the spec, the limit must
be there. I think the reason is that, as you have said, FAT32 doesn't
scale well to huge directories. To find a file in a FAT32 directory
you must do a sequential search of directory entries. NTFS uses a
btree structure, so a search for a single file is quick. Getting a
list of all filenames in the directory can take longer in NTFS than in
FAT32, especially for huge directories.

And yes, file names that don't fit the DOS 8.3 standard will take from
2 to 13 entries in a FAT32 directory, so the directory will fill up
before you expect it to.

--
Tim Slattery
MS MVP(Shell/User)
Slattery_T@bls.gov
http://members.cox.net/slatteryt

JdeBP> There are also limits imposed by the bitness of the
JdeBP> filesystem drivers, albeit not by the on-disc filesystem
JdeBP> format itself. Some drivers, for example, use 16-bit
JdeBP> numbers for directory entry indices, limiting each
JdeBP> directory to no more than 65536 entries -- a limit
JdeBP> that is further reduced by the facts that long
JdeBP> filenames occupy multiple entries and the
JdeBP> "." and ".." entries always consume two entries.

TS> The Fat32 spec says that any conforming driver *must*
TS> limit directories to 65,536 entries.

It does? So it does.

TS> This has nothing to do with whether the driver is
TS> 16- or 32-bit code.

Who said anything about _code_? The aforementioned bitness is in the
_data_.

TS> To conform with the spec, the limit must be there.
TS> I think the reason is that, as you have said, FAT32
TS> doesn't scale well to huge directories. [...]

Actually, the specification itself states the reasons. The first
reason is, indeed, the problem with the inefficient on-disc data
structures. The second reason is the same as I gave above -- namely
that some filesystem drivers and utility softwares (Microsoft says
"many", "including Microsoft's".) use 16-bit numbers for directory
entry indices.

I have to revisit some code and make a decision, I see. Personally, I
aim not to hardwire any 16-bit limits into my own softwares. This is
a good example of why one should always endeavour not to. It's an
example of how one curtails people decades later by doing so. This
limit was not originally part of the FAT on-disc data structures, but
has become so entrenched in driver and utility softwares that
Microsoft has formalized it in a specification and _made_ it a limit
on the on-disc data structures.

I suspect that the same will happen with the 32-bit cluster number
limit for NTFS.

J de Boyne Pollard wrote:

>JdeBP> There are also limits imposed by the bitness of the
>JdeBP> filesystem drivers, albeit not by the on-disc filesystem
>JdeBP> format itself. Some drivers, for example, use 16-bit
>JdeBP> numbers for directory entry indices, limiting each
>JdeBP> directory to no more than 65536 entries -- a limit
>JdeBP> that is further reduced by the facts that long
>JdeBP> filenames occupy multiple entries and the
>JdeBP> "." and ".." entries always consume two entries.
>
>TS> The Fat32 spec says that any conforming driver *must*
>TS> limit directories to 65,536 entries.
>
>It does? So it does.
>
>TS> This has nothing to do with whether the driver is
>TS> 16- or 32-bit code.
>
>Who said anything about _code_? The aforementioned bitness is in the
>_data_.

That makes no difference at all to the file system. It doesn't know
what's being stored, other than a bunch of bytes. Might be a string,
might be integers, might be floating point, the file system doesn't
know and doesn't care.

--
Tim Slattery
MS MVP(Shell/User)
Slattery_T@bls.gov
http://members.cox.net/slatteryt

JdeBP> There are also limits imposed by the bitness of the
JdeBP> filesystem drivers, albeit not by the on-disc filesystem
JdeBP> format itself. Some drivers, for example, use 16-bit
JdeBP> numbers for directory entry indices, limiting each
JdeBP> directory to no more than 65536 entries -- a limit
JdeBP> that is further reduced by the facts that long
JdeBP> filenames occupy multiple entries and the
JdeBP> "." and ".." entries always consume two entries.

TS> The Fat32 spec says that any conforming driver *must*
TS> limit directories to 65,536 entries.

JdeBP> It does? So it does.

TS> This has nothing to do with whether the driver is
TS> 16- or 32-bit code.

JdeBP> Who said anything about _code_? The aforementioned
JdeBP> bitness is in the _data_.

TS> That makes no difference at all to the file system. It
TS> doesn't know what's being stored, other than a bunch
TS> of bytes. [...]

Who said anything about what is being stored? The aforementioned
data, with the aforementioned bitness, are _in the driver_. Please re-
read the first and second sentences of the very first quotation given
above.