Directory Search Time Question - Unix

This is a discussion on Directory Search Time Question - Unix ; I'm deciding how to lay out a directory (or directory with subdirectories) for PHP include files. If I have a large number of files in a single directory (1000, say) will it take a relatively long time to find and ...

+ Reply to Thread
Results 1 to 3 of 3

Thread: Directory Search Time Question

  1. Directory Search Time Question

    I'm deciding how to lay out a directory (or directory with subdirectories)
    for PHP include files.

    If I have a large number of files in a single directory (1000, say) will it
    take a relatively long time to find and open a file?

    Would breaking the 1000 files into subdirectories help?

    I'm curious about the speed when the PHP interpreter encounters a statement
    such as:

    include("md5.inc");

    versus something like:

    include("crypto_hashes/md5.inc");

    If there are 999 files along with "md5.inc" in a single directory, how much
    does speed suffer?

    Does it get any better with the second form (fewer files per directory)?

    Thanks.
    --
    David T. Ashley (dta@e3ft.com)
    http://www.e3ft.com (Consulting Home Page)
    http://www.dtashley.com (Personal Home Page)
    http://gpl.e3ft.com (GPL Publications and Projects)



  2. Re: Directory Search Time Question

    "David T. Ashley" wrote in message
    news:qIydnT2bPNgtTJzbnZ2dnUVZ_tmknZ2d@giganews.com ...
    > I'm deciding how to lay out a directory (or directory with subdirectories)
    > for PHP include files.
    >
    > If I have a large number of files in a single directory (1000, say) will
    > it take a relatively long time to find and open a file?
    >
    > Would breaking the 1000 files into subdirectories help?
    >
    > I'm curious about the speed when the PHP interpreter encounters a
    > statement such as:
    >
    > include("md5.inc");
    >
    > versus something like:
    >
    > include("crypto_hashes/md5.inc");
    >
    > If there are 999 files along with "md5.inc" in a single directory, how
    > much does speed suffer?
    >
    > Does it get any better with the second form (fewer files per directory)?


    Sorry, some qualifiers I should have added:

    a)Linux system.

    b)This isn't a PHP question ... this is about OS internals on an open() or
    fopen() call ...

    Thanks.

    --
    David T. Ashley (dta@e3ft.com)
    http://www.e3ft.com (Consulting Home Page)
    http://www.dtashley.com (Personal Home Page)
    http://gpl.e3ft.com (GPL Publications and Projects)



  3. Re: Directory Search Time Question

    In article ,
    "David T. Ashley" wrote:

    > I'm deciding how to lay out a directory (or directory with subdirectories)
    > for PHP include files.
    >
    > If I have a large number of files in a single directory (1000, say) will it
    > take a relatively long time to find and open a file?


    1,000 probably isn't too bad on modern hardware, but you might notice
    some slowdown with 10,000. Most Unix filesystems perform linear search
    through directories.

    >
    > Would breaking the 1000 files into subdirectories help?


    Yes.

    >
    > I'm curious about the speed when the PHP interpreter encounters a statement
    > such as:
    >
    > include("md5.inc");
    >
    > versus something like:
    >
    > include("crypto_hashes/md5.inc");
    >
    > If there are 999 files along with "md5.inc" in a single directory, how much
    > does speed suffer?
    >
    > Does it get any better with the second form (fewer files per directory)?


    Since it's a linear search, the average time to open a file is usually a
    linear function of the number of files in the directory.

    And if the directory gets VERY large there can be a sudden increase in
    the lookup time, because the special file may need to start using
    indirect blocks.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

+ Reply to Thread