Data as Files - Linux

This is a discussion on Data as Files - Linux ; Why the concept of "file" is such a big success? What made UNIX based operating system to consider everything as a file? Was/Is there any competitive entity for "file"? Do you see, any such entity will emerge? Why can't we ...

+ Reply to Thread
Results 1 to 10 of 10

Thread: Data as Files

  1. Data as Files

    Why the concept of "file" is such a big success? What made UNIX
    based operating system to consider everything as a file?

    Was/Is there any competitive entity for "file"? Do you see, any
    such entity will emerge?

    Why can't we store data into a relational data base table? Like,

    Filename | user | content
    ---------+------+--------
    x.y | me | nothin


  2. Re: Data as Files

    "v4vijayakumar" said:
    >Why the concept of "file" is such a big success? What made UNIX
    >based operating system to consider everything as a file?
    >
    >Was/Is there any competitive entity for "file"? Do you see, any
    >such entity will emerge?


    Well, let's leave at least those homework questions for yourself
    to research.

    >Why can't we store data into a relational data base table? Like,
    >
    >Filename | user | content
    >---------+------+--------
    >x.y | me | nothin


    .... which would necessitate writing the db server into the operating
    system kernel, no? Otherwise, in which kind of repository would the
    db server reside, as it would have to be loaded somewhat?

    Also, that example is somewhat stretching the issue of
    relational/hierarchical, as at least your file naming in the above
    appears to be hierarchical -- so, after all you're not even proposing
    to leave behind the file naming hierarchy. Instead, you're just
    moving the file metadata (and contents) into relational form
    (which actually is just a change of representation, not a change in
    functionality - unless you propose that users would be free to add
    columns to the metadata structure).

    As a historical point-of-view, relational db theory was very new (or
    non-existent) when Unix was originally written; what is, to my
    knowledge, the initial paper on relational data storage by E.F.Codd,
    was published on June 1970 in Communications of the ACM. The paper
    had been circulating within IBM already before that, but the
    June 1970 CACM publication was what brought the research available
    for a larger audience.
    --
    Wolf a.k.a. Juha Laiho Espoo, Finland
    (GC 3.0) GIT d- s+: a C++ ULSH++++$ P++@ L+++ E- W+$@ N++ !K w !O !M V
    PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++
    "...cancel my subscription to the resurrection!" (Jim Morrison)

  3. Re: Data as Files

    "v4vijayakumar" writes:

    > Why the concept of "file" is such a big success?


    For the same reasons OO is such a big success.

    There are two things.

    First, naming scheme, based on hierarchical "directories".
    The main reason of success of this naming scheme is it's orthogonality
    and ease of use.

    Secondly, considering everything (or a lot of things) as files, or
    actually as some kind of files, allows to reuse common operations,
    like open, read, write and close. So we have some classes and a
    virtual "inheritance", and operator overloading.

    abstract-file
    ^
    |
    +---- regular-file
    | ^
    | |
    | +-------- directory-file
    |
    +---- device
    |
    +---- pipe
    |
    ... (virtual files like /proc, etc)

    Some operations, like fcntl, or ioctl can't be applied on all
    abstract-file instance, but open, read, write and close usually can be
    applied on any abstract-file instance.

    This simplifies a lot the API (and even implementation of the kernel),
    so it has been successful.


    > What made UNIX based operating system to consider everything as a file?


    Simplicity of the concept.


    > Was/Is there any competitive entity for "file"?


    Yes. The "Object", notably, the "Persistent Object".
    Have a look at http://www.eros-os.org/
    Notably: http://www.eros-os.org/papers/storedesign2002.pdf

    You might do some research in computer history to see how data has
    been stored in the various systems:

    - plain data on hard disk (you can still do that using directly the
    disk device on unix, which is sometimes done to store databases),

    - disk slices (ranges manually allocated for the "files"),

    - "objects" in archives/libraries, that was often done on mainframes,
    and applications like emacs can still do it, for example with tar
    archives. This is like accessing files in looped file systems,
    but with a simplier "file system".

    - objects stored in OO databases.

    - objects or entities stored in relationational or other databases.

    - etc.



    > Do you see, any such entity will emerge?


    With the big^W enormous address spaces that are becoming available
    with 64-bit processors, persistent object stores may become
    perponderant, since they allow to avoid entirely any I/O at the
    application level: you only have pointers to the objects, and the OS
    VM does all the I/O work.


    > Why can't we store data into a relational data base table?


    無! We _can_ store the data into a relational or other kind of data
    base instead of files.


    --
    __Pascal Bourguignon__ http://www.informatimago.com/

    Nobody can fix the economy. Nobody can be trusted with their finger
    on the button. Nobody's perfect. VOTE FOR NOBODY.

  4. Re: Data as Files

    v4vijayakumar wrote:
    > Why the concept of "file" is such a big success? What made UNIX
    > based operating system to consider everything as a file?
    >
    > Was/Is there any competitive entity for "file"? Do you see, any
    > such entity will emerge?
    >
    > Why can't we store data into a relational data base table? Like,
    >
    > Filename | user | content
    > ---------+------+--------
    > x.y | me | nothin
    >


    Where does a hierarchical file system differ from a database?
    It has a method of storing and retrieving data associated with a key.
    Using links, you can even associate several keys with a single data item.

    A big computer company in the northwest of the USA has tried in vain to
    replace traditional filesystems with a database to store data.
    Until now, the idea has not taken off.

    From the end-user point of view, storing his pictures, videos, music,
    documents, etc in a huge database would not make very much of a
    difference from a handling point of view, maybe one day we'll see it happen.

    OTOH thrid party software developers and even a lot of users are used to
    the concept of a hierarchical file system. So it may very well be that
    it's just convenience that keeps file systems as we know them alive.
    --
    Josef Mllers (Pinguinpfleger bei FSC)
    If failure had no penalty success would not be a prize
    -- T. Pratchett


  5. Re: Data as Files

    v4vijayakumar wrote:
    > Why the concept of "file" is such a big success? What made UNIX
    > based operating system to consider everything as a file?
    >
    > Was/Is there any competitive entity for "file"? Do you see, any
    > such entity will emerge?
    >
    > Why can't we store data into a relational data base table? Like,


    Where will you store the table? In a raw disk partition? More likely in
    a file in a filesystem.

    Even database systems that can use raw disk partitions still create a
    filesystem for themselves.

    Disk partitioning itself is a kind of filesystem: there is a table
    which says that the disk is carved into identifiable objects, which can
    be perceived as files.

    > Filename | user | content
    > ---------+------+--------
    > x.y | me | nothin


    That looks to me like a filesystem's directory containing a file.


  6. Re: Data as Files

    v4vijayakumar wrote:
    > Why can't we store data into a relational data base table?


    You can - if what I got told by a friend is correct, on the IBM
    AS/400 under OS/400 all data are stored in a database and there
    is no file system in the classical sense. But then this system
    came out only nearly 20 years after Unix was developed...

    Regards, Jens
    --
    \ Jens Thoms Toerring ___ jt@toerring.de
    \__________________________ http://toerring.de

  7. Re: Data as Files

    Juha Laiho wrote:
    >"v4vijayakumar" said:
    >>Why the concept of "file" is such a big success? What made UNIX
    >>based operating system to consider everything as a file?
    >>...
    >>Why can't we store data into a relational data base table? Like,
    >>
    >>Filename | user | content
    >>---------+------+--------
    >>x.y | me | nothin

    >
    >... which would necessitate writing the db server into the operating
    >system kernel, no? Otherwise, in which kind of repository would the
    >db server reside, as it would have to be loaded somewhat?


    Right. Those of you who follow the world of the Evil Empire recognize that
    this is EXACTLY the architecture of Microsoft's so-called Cairo File
    System, originally scheduled for Vista but dropped last last year. The
    file system driver would be a slimmed down version of SQL Server.

    If you can get past the nutty anti-Microsoft religious arguments, this is a
    very interesting idea. The SQL Server folks are very, very good at
    searches, query optimization, page management, and journalling --
    specialties that the operating system folks wouldn't necessarily have. It
    opens up the possibility for some very "rich" search operations.
    --
    Tim Roberts, timr@probo.com
    Providenza & Boekelheide, Inc.

  8. Re: Data as Files

    Tim Roberts writes:
    > Juha Laiho wrote:
    >>"v4vijayakumar" said:
    >>>Why the concept of "file" is such a big success? What made UNIX
    >>>based operating system to consider everything as a file?
    >>>...
    >>>Why can't we store data into a relational data base table? Like,
    >>>
    >>>Filename | user | content
    >>>---------+------+--------
    >>>x.y | me | nothin

    >>
    >>... which would necessitate writing the db server into the operating
    >>system kernel, no? Otherwise, in which kind of repository would the
    >>db server reside, as it would have to be loaded somewhat?

    >
    > Right. Those of you who follow the world of the Evil Empire recognize that
    > this is EXACTLY the architecture of Microsoft's so-called Cairo File
    > System, originally scheduled for Vista but dropped last last year.


    As far as I remember, this has been 'originally scheduled' for each NT
    release since 4.0 (I actually remember this only back to w2k, but have
    heard that it is still older).

    > If you can get past the nutty anti-Microsoft religious arguments,
    > this is a very interesting idea.


    Without looking at the 'interesting idea' itself: That a company with
    the ressources and knowledge of Microsoft apparently cannot implement
    this with a shippable quality within more than six years is a strong
    hint that the idea is not that good. What it could do is much less
    interesting than what it can do.

    > The SQL Server folks are very, very good at searches, query
    > optimization, page management, and journalling -- specialties that
    > the operating system folks wouldn't necessarily have.


    One would expect that people who do filesystems have the last two and
    people who don't do 'large [unstructured] databases' don't need the
    first.

    And there is one usual counter argument: That UNIX(*) has sucessfully
    demonstrated that something different than the traditional approach
    (files are entities whose structure is determined by the OS and not by
    each application) does work is not an argument in favor of the
    traditional approach.

  9. Re: Data as Files

    in 139654 20061110 100040 Rainer Weikusat wrote:
    >Tim Roberts writes:
    >> Juha Laiho wrote:
    >>>"v4vijayakumar" said:
    >>>>Why the concept of "file" is such a big success? What made UNIX
    >>>>based operating system to consider everything as a file?
    >>>>...
    >>>>Why can't we store data into a relational data base table? Like,
    >>>>
    >>>>Filename | user | content
    >>>>---------+------+--------
    >>>>x.y | me | nothin
    >>>
    >>>... which would necessitate writing the db server into the operating
    >>>system kernel, no? Otherwise, in which kind of repository would the
    >>>db server reside, as it would have to be loaded somewhat?

    >>
    >> Right. Those of you who follow the world of the Evil Empire recognize that
    >> this is EXACTLY the architecture of Microsoft's so-called Cairo File
    >> System, originally scheduled for Vista but dropped last last year.

    >
    >As far as I remember, this has been 'originally scheduled' for each NT
    >release since 4.0 (I actually remember this only back to w2k, but have
    >heard that it is still older).
    >
    >> If you can get past the nutty anti-Microsoft religious arguments,
    >> this is a very interesting idea.


    It doesn't sound new, it sounds like IBM's Shared File System in VM/CMS
    and that is at least 15 years old.

  10. Re: Data as Files

    On Fri, 10 Nov 2006 11:00:40 +0100 Rainer Weikusat wrote:

    | Without looking at the 'interesting idea' itself: That a company with
    | the ressources and knowledge of Microsoft apparently cannot implement
    | this with a shippable quality within more than six years is a strong
    | hint that the idea is not that good. What it could do is much less
    | interesting than what it can do.

    Or it shows that such a company is nor correctly making use of resources
    and knowledge it has, and/or has a rather distorted concept of "shippable
    quality". This "interesting idea" has actually been discussed for many
    years even outside of Sasquatchsoft.

    --
    |---------------------------------------/----------------------------------|
    | Phil Howard KA9WGN (ka9wgn.ham.org) / Do not send to the address below |
    | first name lower case at ipal.net / spamtrap-2006-11-11-1318@ipal.net |
    |------------------------------------/-------------------------------------|

+ Reply to Thread