everything is a directory - Plan9

This is a discussion on everything is a directory - Plan9 ; I'm not sure if this is the right place to post this, but it seems like a good fit. What better forum for deep thought on the meaning of files and directories than the Plan 9 news group? There would ...

+ Reply to Thread
Page 1 of 4 1 2 3 ... LastLast
Results 1 to 20 of 73

Thread: everything is a directory

  1. everything is a directory

    I'm not sure if this is the right place to post this, but it seems
    like a good fit. What better forum for deep thought on the meaning of
    files and directories than the Plan 9 news group?

    There would be great utility in merging files and directories into a
    composite content/container object that respond 'read' and 'write' for
    file ops and 'list', 'add', 'delete' for directory ops. For example, a
    disk drive could respond to 'read' with a bunch of stuff on the disk,
    and respond to 'list' with a listing of it's hardware settings, which
    could be set with a 'write'. Merged file/directories also make a lot
    of sense when you think about languages with hierarchical modules --
    instead of having naming conventions to find a sub-module, you just
    look it up and read it. Similarly, hierarchical documents map straight
    on to the mixed file/folder -- you put the intro in the head and its
    components under the head.

    I'm sure this idea has come up in the past; many of my ideas are like
    that. The 'everything is a file' model is proverbial, but it was not
    so once upon a time. I'm sure the 'everything is a directory' model
    had its proponents in days gone by, just as functional languages did
    (and will again!). In fact, 'everything is a directory' is the man
    behind the curtain in LDAP.

    In the considered opinion of the list, is "everything is a directory"
    a big mess, a resource wasting fantasy, an idea whose time has come?

  2. Re: [9fans] everything is a directory

    We already have composite container / content objects, they are called
    directories

    "list" gives you a list of their files

    reading the files gives you the data available

    new semantics like yours will need a new raft of tools, protocols and
    all the ****e that goes with it



    > is "everything is a directory"
    > a big mess, a resource wasting fantasy, an idea whose time has come?
    >
    >


    just the first two

  3. Re: [9fans] everything is a directory

    >I'm sure this idea has come up in the past; many of my ideas are like

    i'm afraid that directory-with-contents does indeed reappear from time to time.
    it's not obvious (at least to me) that it ever made a big difference,
    but it does add ambiguity. a version and configuration system made use of one implementation,
    so that directory "xyz" might contain the portable source of something, and "xyz/special-variant"
    might have a modified version or a machine-specific part of the general
    "xyz" (eg, "console", "console/dgnova", "console/ti990").
    there was a bit more to it than that, but in the end on other systems "xyz" was
    a directory, "xyz/port" had the general version, and other substructure gave the variants.
    so the directory-with-content didn't really seem to make things simpler.

    at the moment in Plan 9, /dev/sdC0/data has the drive's data and /dev/sdC0/ctl
    shows you its attributes and writes control them,
    and ls /dev/sdC0 shows you ctl, data and others.
    /dev/sdC0 refers to that set (as distinct from the data, which is in "data").
    this seems clear and adequate. i'd have said it was a simpler model.

  4. Re: [9fans] everything is a directory

    In the first edition of Plan B we had "boxes". They are generic containers that
    behave either as files or as directories depending on what op. you
    use. In general,
    a box is a typed container that has inner boxes. After having then
    working, the lack
    of applications made us drop the idea and switch back to files. A
    brief description is
    at "The Box, a replacement for files" paper in lsub.org. Also, I think
    I still have the source
    of that plan b, with a prefix mount table included and a box library,
    but would have to
    dig in the worm to re-locate it.

    On 8/16/07, jsnx wrote:
    > I'm not sure if this is the right place to post this, but it seems
    > like a good fit. What better forum for deep thought on the meaning of
    > files and directories than the Plan 9 news group?
    >
    > There would be great utility in merging files and directories into a
    > composite content/container object that respond 'read' and 'write' for
    > file ops and 'list', 'add', 'delete' for directory ops. For example, a
    > disk drive could respond to 'read' with a bunch of stuff on the disk,
    > and respond to 'list' with a listing of it's hardware settings, which
    > could be set with a 'write'. Merged file/directories also make a lot
    > of sense when you think about languages with hierarchical modules --
    > instead of having naming conventions to find a sub-module, you just
    > look it up and read it. Similarly, hierarchical documents map straight
    > on to the mixed file/folder -- you put the intro in the head and its
    > components under the head.
    >
    > I'm sure this idea has come up in the past; many of my ideas are like
    > that. The 'everything is a file' model is proverbial, but it was not
    > so once upon a time. I'm sure the 'everything is a directory' model
    > had its proponents in days gone by, just as functional languages did
    > (and will again!). In fact, 'everything is a directory' is the man
    > behind the curtain in LDAP.
    >
    > In the considered opinion of the list, is "everything is a directory"
    > a big mess, a resource wasting fantasy, an idea whose time has come?
    >


  5. Re: [9fans] everything is a directory


    > Unix already has this, and their called extended attributes, and I
    > hate them with a burning passion. Rsync, cp, any tool that
    > manipulates files (tar for example) has to be able to capture this
    > data, and just reading the file won't do it anymore.
    >
    > Oh and to make things more "fun" FreeBSD, Linux and Mac OS X at least
    > have different ways of dealing with them, and max size limits etc.


    you forgot NTFS's version

    http://www.wikistc.org/wiki/Alternate_data_streams

    echo plain > t.xt
    echo hidden > t.txt:hidden.txt

    %type t.txt
    plain
    % type t.txt:hidden.txt
    The filename, directory name, or volume label syntax is incorrect.

    % notepad t.txt:hidden.txt
    hidden







  6. Re: [9fans] everything is a directory

    On 8/16/07, maht wrote:
    >
    >
    > > Unix already has this, and their called extended attributes, and I
    > > hate them with a burning passion. Rsync, cp, any tool that
    > > manipulates files (tar for example) has to be able to capture this
    > > data, and just reading the file won't do it anymore.
    > >
    > > Oh and to make things more "fun" FreeBSD, Linux and Mac OS X at least
    > > have different ways of dealing with them, and max size limits etc.

    >
    > you forgot NTFS's version
    >
    > http://www.wikistc.org/wiki/Alternate_data_streams
    >
    > echo plain > t.xt
    > echo hidden > t.txt:hidden.txt
    >
    > %type t.txt
    > plain
    > % type t.txt:hidden.txt
    > The filename, directory name, or volume label syntax is incorrect.
    >
    > % notepad t.txt:hidden.txt
    > hidden



    So there we have it... all the major OSes I know of have turned files into
    directories already... *sigh*


  7. Re: [9fans] everything is a directory

    On 8/16/07, David Leimbach wrote:

    > So there we have it... all the major OSes I know of have turned files into
    > directories already... *sigh*


    yes, it's a measure of how bad a given os is ... a simple metric.

    ron

  8. Re: [9fans] everything is a directory

    I'm confused. Is it the concept that's awful, or the implimentation? Is the
    problem that visibiliby of the attributes to tools is inconsistant and
    they're managed with entirely different symantics? Or is the whole idea of
    arbitrary file attributes a tool of the devil?

    Certainly, some file formats (I'm thinking of MP3) use file attributes to
    great effect, though of course they're specially implemented in the context
    of the files' use case. Is it appropriate to implement a general file
    tagging capability that is file independant and managed by the OS?


  9. Re: [9fans] everything is a directory

    On 8/16/07, Robert Sherwood wrote:
    >
    > I'm confused. Is it the concept that's awful, or the implimentation? Is
    > the problem that visibiliby of the attributes to tools is inconsistant and
    > they're managed with entirely different symantics? Or is the whole idea of
    > arbitrary file attributes a tool of the devil?
    >


    Both?

    There's no standardized interface across the board that's guaranteed to work
    for all platforms that are supposed to be more or less compatible. And the
    fact that the implementation exists means my fundamental abstractions that
    made Unix Unix to begin with are changing underneath the tools and
    applications I have to work with them.

    I think the root of the issue, for me, is that we had a definition for what
    a file was in an OS and the world was for the most part simple. People
    stored things like a tar table of contents in the file and defined formats
    on a per-file basis, and people realized that if they wanted to manipulate
    such files, that all that was needed was to use some well defined structures
    and APIs to manage them. For things that were common to "files" we could
    copy them, delete them, rename them with no expectations of side-effects or
    missed metadata.

    At a high level, copying a file is defined as copying the whole file - all
    the data on the disk associated with that particular blob, because that's
    what a file is, a representation of the data for humans to deal with, not
    just a portion of the data. When things like extended attributes were
    added, that was no longer the case (and even earlier in things like Mac OS
    Resource Forks). Now I've lost my original abstraction that I loved so
    much, because someone didn't figure out another way to store an icon for my
    program that didn't break the semantics of keeping all the file's data in
    one abstracted "place".

    Sure people fixed tar, rsync and (hopefully) all the standard faire to deal
    with these new and wonderful attributes, but couldn't we have found a better
    way? For some things, like access control lists, maybe not (I'm still
    unconvinced that it had to be done with extended attributes). But let's
    just say it doesn't always feel that it's worth the pain involved to break
    fundamental abstractions like "file" and "directory" to get these new
    features.


    One could argue if we must break those abstractions, that maybe it's time
    for a new OS.

    Dave


  10. Re: [9fans] everything is a directory

    if the goal of the extra metadata is to somehow help in the selection
    of the data (i.e. file), then something like what abhey presented at
    iwp9 will fit the problem.

    http://www.cs.bell-labs.com/sources/...yshahplan9.pdf

    > I'm confused. Is it the concept that's awful, or the implimentation? Is the
    > problem that visibiliby of the attributes to tools is inconsistant and
    > they're managed with entirely different symantics? Or is the whole idea of
    > arbitrary file attributes a tool of the devil?
    >
    > Certainly, some file formats (I'm thinking of MP3) use file attributes to
    > great effect, though of course they're specially implemented in the context
    > of the files' use case. Is it appropriate to implement a general file
    > tagging capability that is file independant and managed by the OS?



  11. Re: everything is a directory

    On Aug 16, 5:40 am, n...@lsub.org (Francisco J Ballesteros) wrote:
    > In the first edition of Plan B we had "boxes".


    After reading through the paper, I realize that my directories-as-
    files idea gets a little strange when you have union mounts. Just
    joining putting the bytes in row one after the other is liable to
    result in an invalid file for many file types.

  12. Re: everything is a directory

    On Aug 16, 4:48 am, fors...@terzarima.net (Charles Forsyth) wrote:
    > >I'm sure this idea has come up in the past; many of my ideas are like

    >
    > i'm afraid that directory-with-contents does indeed reappear from time to time.
    > it's not obvious (at least to me) that it ever made a big difference,
    > but it does add ambiguity.


    How does it add ambiguity? The example you give is really no different
    from putting the portable code for a foo in foo/code.c and the arch
    specific versions in foo/i386/code.c, except for it doesn't have the
    baggage of a file naming convention.


  13. Re: everything is a directory

    On Aug 16, 10:46 am, leim...@gmail.com (David Leimbach) wrote:
    > So there we have it... all the major OSes I know of have turned files into
    > directories already... *sigh*


    But what about making directories into files? We should get rid of one
    or the other, is my point. Give all filesystem objects children (which
    are other filesystem objects) and then as many 'channels' as they
    like, for app and OS specific things (ACL channel, owner channel, byte
    channel, diff channel...).

  14. Re: [9fans] everything is a directory

    What do you mean by "extended attributes"?
    I haven't noticed them on the Unix systems I use.

  15. Re: [9fans] everything is a directory

    You can't argue against the principle, we take files as directories all
    the time and one level or another.
    So, for me, it's botched implementation. Just wait for extended
    attribute directories to be added on with something like
    file.txt:meta.zip (but not zip files, something new [that's like zip but
    slightly different - maybe even encrypted])
    They already have synthetic file systems built into NT!






  16. Re: [9fans] everything is a directory

    > You can't argue against the principle, we take files as directories all
    > the time and one level or another.
    > So, for me, it's botched implementation. Just wait for extended
    > attribute directories to be added on with something like
    > file.txt:meta.zip (but not zip files, something new [that's like zip but
    > slightly different - maybe even encrypted])
    > They already have synthetic file systems built into NT!


    i think it's worse thank that. attributes and their ilk essentially
    add methods to a filesystem.

    if you want an object system, you probablly don't want plan 9,
    windows or linux.

    - erik


  17. Re: [9fans] everything is a directory

    On 8/17/07, Douglas A. Gwyn wrote:
    > What do you mean by "extended attributes"?
    > I haven't noticed them on the Unix systems I use.
    >

    maybe I'm missing the question, but on my linux:

    man -k extended | grep attrib | wc
    21 183 1417
    libattr (rpm) - Dynamic library for extended attribute support
    fgetxattr [getxattr] (2) - retrieve an extended attribute value
    flistxattr[listxattr] (2) - list extended attribute names
    (etc. It's a lot like read but, as usual, it needs a new set of
    system calls ... 5 of them.
    And ls can't show them, but ... we have getfattr/setfattr
    The output format of getfattr -d is as follows:
    1: # file: somedir/
    2: user.name0="value0"
    3: user.name1="value1"
    4: user.name2="value2"
    5: ...


    it's everywhere. But it was too hard to put in in some normal way, so
    it went in from the side.

    ron

  18. Re: [9fans] everything is a directory

    hey, how do you grep for stuff in extended attributes?

    - erik


    > On 8/17/07, Douglas A. Gwyn wrote:
    >> What do you mean by "extended attributes"?
    >> I haven't noticed them on the Unix systems I use.
    >>

    > maybe I'm missing the question, but on my linux:
    >
    > man -k extended | grep attrib | wc
    > 21 183 1417
    > libattr (rpm) - Dynamic library for extended attribute support
    > fgetxattr [getxattr] (2) - retrieve an extended attribute value
    > flistxattr[listxattr] (2) - list extended attribute names
    > (etc. It's a lot like read but, as usual, it needs a new set of
    > system calls ... 5 of them.
    > And ls can't show them, but ... we have getfattr/setfattr
    > The output format of getfattr -d is as follows:
    > 1: # file: somedir/
    > 2: user.name0="value0"
    > 3: user.name1="value1"
    > 4: user.name2="value2"
    > 5: ...
    >
    >
    > it's everywhere. But it was too hard to put in in some normal way, so
    > it went in from the side.
    >
    > ron



  19. Re: [9fans] Re: everything is a directory

    > But what about making directories into files? We should get rid of one
    > or the other, is my point. Give all filesystem objects children (which
    > are other filesystem objects) and then as many 'channels' as they
    > like, for app and OS specific things (ACL channel, owner channel, byte
    > channel, diff channel...).


    of course directories are files. the whole file system is a file. filesystems
    role in life is to turn a linear array of blocks into something with heirarchy.
    often that linear array of blocks is known as a disk drive.

    (mbox format is a directory masquerading as a file and so is /adm/keys.)

    - erik


  20. Re: [9fans] everything is a directory

    On Fri, Aug 17, 2007 at 09:24:26AM -0400, erik quanstrom wrote:
    > ron wrote:
    >> On 8/17/07, Douglas A. Gwyn wrote:
    >>> What do you mean by "extended attributes"?
    >>> I haven't noticed them on the Unix systems I use.
    >>>

    >> maybe I'm missing the question, but on my linux:
    >>
    >> man -k extended | grep attrib | wc
    >> 21 183 1417
    >> libattr (rpm) - Dynamic library for extended attribute support
    >> fgetxattr [getxattr] (2) - retrieve an extended attribute value
    >> flistxattr[listxattr] (2) - list extended attribute names
    >> (etc. It's a lot like read but, as usual, it needs a new set of
    >> system calls ... 5 of them.
    >> And ls can't show them, but ... we have getfattr/setfattr
    >> The output format of getfattr -d is as follows:
    >> 1: # file: somedir/
    >> 2: user.name0="value0"
    >> 3: user.name1="value1"
    >> 4: user.name2="value2"
    >> 5: ...
    >>
    >>
    >> it's everywhere. But it was too hard to put in in some normal way, so
    >> it went in from the side.
    >>
    >> ron

    >
    > hey, how do you grep for stuff in extended attributes?
    >
    > - erik


    grepfattr, I should think. (Now with ANSI color support!)

+ Reply to Thread
Page 1 of 4 1 2 3 ... LastLast