what happen when we append a file in a disk - Linux

This is a discussion on what happen when we append a file in a disk - Linux ; Hi, Assume that a user wants to append a data to a file in the Hard Drive disk. I am just wondering about the steps that are undertaken to complete this operation. My understanding is that first the OS has ...

+ Reply to Thread
Results 1 to 14 of 14

Thread: what happen when we append a file in a disk

  1. what happen when we append a file in a disk

    Hi,
    Assume that a user wants to append a data to a file in the Hard Drive
    disk. I am just wondering about the steps that are undertaken to
    complete this operation. My understanding is that first the OS has to
    locate the file in the disk to retrieve its physical location and its
    size. By knowing its size, the location of the last file block
    (cluster) can be deduced. This block is then READ from the disk platter
    to the disk buffer. the data that we want to append to the file is then
    appended in the disk buffer to the block which has just been read from
    the platters. Once this operation is completed the newly block is
    written from disk buffer to the disk platter.The above low operations
    are executed by the disk controller which runs the disk device driver

    Am I right?


    A sideline question, each file has a number of blocks/cluster. each a
    cluster is a set of consecutive sectors

    Thanks for your help


  2. Re: what happen when we append a file in a disk

    xu_feng_xu@yahoo.com wrote:
    > Hi,
    > Assume that a user wants to append a data to a file in the Hard Drive
    > disk. I am just wondering about the steps that are undertaken to
    > complete this operation. My understanding is that first the OS has to
    > locate the file in the disk to retrieve its physical location and its
    > size.


    It should be able to find this from the i-node or its extensions.

    > By knowing its size, the location of the last file block
    > (cluster) can be deduced.


    Not really. I.e., you cannot deduce the location of the last file block from
    the file size and starting point. You have to look it up in tables that are
    on the disk itself.

    > This block is then READ from the disk platter
    > to the disk buffer. the data that we want to append to the file is then
    > appended in the disk buffer to the block which has just been read from
    > the platters. Once this operation is completed the newly block is
    > written from disk buffer to the disk platter.The above low operations
    > are executed by the disk controller which runs the disk device driver


    The disk device driver runs the disk controller, not the other way around.
    Depending on the controller and the disk drive itself, chances are pretty
    good that the disk drive will read the entire track from the disk, not just
    the specified block, keeping it in its cache memory (typically 8 MBytes
    these days). Then the drive and the controller will read back the blocks
    requested by the OS. Even the OS may request the controller to read ahead if
    it notices you are reading sequentially.
    >
    > Am I right?
    >
    >
    > A sideline question, each file has a number of blocks/cluster. each a
    > cluster is a set of consecutive sectors
    >
    > Thanks for your help
    >



    --
    .~. Jean-David Beyer Registered Linux User 85642.
    /V\ PGP-Key: 9A2FC99A Registered Machine 241939.
    /( )\ Shrewsbury, New Jersey http://counter.li.org
    ^^-^^ 07:35:01 up 21 days, 10:25, 3 users, load average: 4.54, 4.39, 4.22

  3. Re: what happen when we append a file in a disk

    xu_feng_xu@yahoo.com wrote:
    > Hi,
    > Assume that a user wants to append a data to a file in the Hard Drive
    > disk. I am just wondering about the steps that are undertaken to
    > complete this operation. My understanding is that first the OS has to
    > locate the file in the disk to retrieve its physical location and its
    > size. By knowing its size, the location of the last file block
    > (cluster) can be deduced. This block is then READ from the disk
    > platter to the disk buffer. the data that we want to append to the
    > file is then appended in the disk buffer to the block which has just
    > been read from the platters. Once this operation is completed the
    > newly block is written from disk buffer to the disk platter.The above
    > low operations are executed by the disk controller which runs the
    > disk device driver
    >
    > Am I right?


    There appears to be a bug in what you said. You didn't think about the case
    when the last cluster/block is full.

    > A sideline question, each file has a number of blocks/cluster. each a
    > cluster is a set of consecutive sectors


    That's a statement, not a question. And it seems OK.

    Alex


  4. Re: what happen when we append a file in a disk

    > > A sideline question, each file has a number of blocks/cluster. each a
    > > cluster is a set of consecutive sectors

    >
    > That's a statement, not a question. And it seems OK.


    That's an important feature of newsgroups, you see. :-) You don't need
    to ask questions. You just need to make statements, and if they don't
    "seem OK" even slightly to anyone for any reason, you are bound to get
    a good flogging about it, otherwise everyone usually just stays quiet
    about it.


  5. Re: what happen when we append a file in a disk

    microx wrote:
    >
    >>> A sideline question, each file has a number of blocks/cluster.
    >>> each a cluster is a set of consecutive sectors

    >>
    >> That's a statement, not a question. And it seems OK.

    >
    > That's an important feature of newsgroups, you see. :-) You don't
    > need to ask questions. You just need to make statements, and if
    > they don't "seem OK" even slightly to anyone for any reason, you
    > are bound to get a good flogging about it, otherwise everyone
    > usually just stays quiet about it.


    To illustrate which, there exist file systems that have no blocks
    nor clusters. So the statement is not OK. Consider it flogged.

    Please don't strip attribution lines for material you quote. Those
    are the "xxx wrote:" lines at the beginning. Doing so invokes more
    strokes of the cat than the faulty statement.

    --
    Some informative links:







  6. Re: what happen when we append a file in a disk

    CBFalconer writes:

    >microx wrote:
    >>
    >>>> A sideline question, each file has a number of blocks/cluster.
    >>>> each a cluster is a set of consecutive sectors
    >>>
    >>> That's a statement, not a question. And it seems OK.

    >>
    >> That's an important feature of newsgroups, you see. :-) You don't
    >> need to ask questions. You just need to make statements, and if
    >> they don't "seem OK" even slightly to anyone for any reason, you
    >> are bound to get a good flogging about it, otherwise everyone
    >> usually just stays quiet about it.


    >To illustrate which, there exist file systems that have no blocks
    >nor clusters. So the statement is not OK. Consider it flogged.


    There may exist filesystems without the concept of blocks/clusters, but the
    hard drive itself has those.



  7. Re: what happen when we append a file in a disk

    Unruh wrote:
    > CBFalconer writes:
    >

    .... snip ...
    >
    >> To illustrate which, there exist file systems that have no blocks
    >> nor clusters. So the statement is not OK. Consider it flogged.

    >
    > There may exist filesystems without the concept of blocks/clusters,
    > but the hard drive itself has those.


    Oh? Ultimately the drive consists of nothing more than a more or
    less continuous magnetic (or other) media, possibly broken into
    tracks (but not necessarily). Various formatting operations may
    add in such conveniences as sectors with address fields, and even
    translation tables to replace faulty sectors. The rest belongs to
    the file system.

    --
    Some informative links:








  8. Re: what happen when we append a file in a disk

    CBFalconer wrote:
    > Oh? Ultimately the drive consists of nothing more than a more or
    > less continuous magnetic (or other) media, possibly broken into
    > tracks (but not necessarily). Various formatting operations may
    > add in such conveniences as sectors with address fields, and even
    > translation tables to replace faulty sectors. The rest belongs to
    > the file system.


    No.. That's only how it can be made to appear. What I learned is that
    when the drive is accessed, even if you ask the driver to give you a
    random chunk in some linear "continuous" address speace, then at some
    level of the driver, it has to specify to the hardware which sector
    number and track number you want to read, and this sector number X
    track number constitutes a block which has to be read in as a unit into
    some buffer, then you can select the exact bytes you want out of it
    based on your logical linear address. That's of course assuming we are
    talking about block devices, like disk drives, and not tapes for
    example.

    Left the attribution in there for you. ;-)


  9. Re: what happen when we append a file in a disk

    Unruh wrote:
    > CBFalconer writes:
    >
    >> microx wrote:
    >>>>> A sideline question, each file has a number of blocks/cluster.
    >>>>> each a cluster is a set of consecutive sectors
    >>>> That's a statement, not a question. And it seems OK.
    >>> That's an important feature of newsgroups, you see. :-) You don't
    >>> need to ask questions. You just need to make statements, and if
    >>> they don't "seem OK" even slightly to anyone for any reason, you
    >>> are bound to get a good flogging about it, otherwise everyone
    >>> usually just stays quiet about it.

    >
    >> To illustrate which, there exist file systems that have no blocks
    >> nor clusters. So the statement is not OK. Consider it flogged.

    >
    > There may exist filesystems without the concept of blocks/clusters, but the
    > hard drive itself has those.


    But you don't have to have a hard drive to have a file system. It is
    painful, but you could have a file system placed on a tape device.
    You'll have blocks then but no clusters. You could conceivably even
    place a filesystem on a byte-addressable device, although you could
    argue then that you have a block size equal to one byte.

  10. Re: what happen when we append a file in a disk

    Douglas O'Neal wrote:
    > Unruh wrote:
    >> CBFalconer writes:
    >>
    >>> microx wrote:
    >>>>>> A sideline question, each file has a number of blocks/cluster.
    >>>>>> each a cluster is a set of consecutive sectors
    >>>>> That's a statement, not a question. And it seems OK.
    >>>> That's an important feature of newsgroups, you see. :-) You don't
    >>>> need to ask questions. You just need to make statements, and if
    >>>> they don't "seem OK" even slightly to anyone for any reason, you
    >>>> are bound to get a good flogging about it, otherwise everyone
    >>>> usually just stays quiet about it.

    >>
    >>> To illustrate which, there exist file systems that have no blocks
    >>> nor clusters. So the statement is not OK. Consider it flogged.

    >>
    >> There may exist filesystems without the concept of blocks/clusters,
    >> but the
    >> hard drive itself has those.

    >
    > But you don't have to have a hard drive to have a file system. It is
    > painful, but you could have a file system placed on a tape device.
    > You'll have blocks then but no clusters. You could conceivably even
    > place a filesystem on a byte-addressable device, although you could
    > argue then that you have a block size equal to one byte.


    A filesystem argues at least sone kind of random access, and random
    access is faster if you can split the media into blocks with some being
    address structures pointing to other blocks.

    I have implemented a ram disk in my time, and it is simply easier and
    more efficient to read and write it in large chunks...and you need some
    sort of directory structure to tell you where data is. I.e. a filesystem
    is an indirect mechanism of indices and data blocks. If its un-indexed
    and byte sized (RAM) the first thing you do is use some of it to keep
    track of the rest of it.


  11. Re: what happen when we append a file in a disk

    The Natural Philosopher wrote:
    > Douglas O'Neal wrote:
    >> Unruh wrote:
    >>> CBFalconer writes:
    >>>
    >>>> microx wrote:
    >>>>>>> A sideline question, each file has a number of blocks/cluster.
    >>>>>>> each a cluster is a set of consecutive sectors
    >>>>>> That's a statement, not a question. And it seems OK.
    >>>>> That's an important feature of newsgroups, you see. :-) You don't
    >>>>> need to ask questions. You just need to make statements, and if
    >>>>> they don't "seem OK" even slightly to anyone for any reason, you
    >>>>> are bound to get a good flogging about it, otherwise everyone
    >>>>> usually just stays quiet about it.
    >>>
    >>>> To illustrate which, there exist file systems that have no blocks
    >>>> nor clusters. So the statement is not OK. Consider it flogged.
    >>>
    >>> There may exist filesystems without the concept of blocks/clusters,
    >>> but the
    >>> hard drive itself has those.

    >>
    >> But you don't have to have a hard drive to have a file system. It is
    >> painful, but you could have a file system placed on a tape device.
    >> You'll have blocks then but no clusters. You could conceivably even
    >> place a filesystem on a byte-addressable device, although you could
    >> argue then that you have a block size equal to one byte.

    >
    > A filesystem argues at least sone kind of random access, and random
    > access is faster if you can split the media into blocks with some being
    > address structures pointing to other blocks.
    >
    > I have implemented a ram disk in my time, and it is simply easier and
    > more efficient to read and write it in large chunks...and you need some
    > sort of directory structure to tell you where data is. I.e. a filesystem
    > is an indirect mechanism of indices and data blocks. If its un-indexed
    > and byte sized (RAM) the first thing you do is use some of it to keep
    > track of the rest of it.


    An efficient file system argues for some kind of random access. If you
    don't mind waiting for long periods of time while reading a tape you can
    put a filesystem on a tape device. I have used these on both PDP Unix
    systems and on VAX-based Ultrix systems. A decent explanation of a tape
    file system is at http://en.wikipedia.org/wiki/LINC

    A PDP tribute page at http://hampage.hu/pdp-11/periferia.html also
    mentions using a TU55 1" tape drive as a disk with a normal file system.

    Doug

  12. Re: what happen when we append a file in a disk

    Douglas O'Neal writes:

    >The Natural Philosopher wrote:
    >> Douglas O'Neal wrote:
    >>> Unruh wrote:
    >>>> CBFalconer writes:
    >>>>
    >>>>> microx wrote:
    >>>>>>>> A sideline question, each file has a number of blocks/cluster.
    >>>>>>>> each a cluster is a set of consecutive sectors
    >>>>>>> That's a statement, not a question. And it seems OK.
    >>>>>> That's an important feature of newsgroups, you see. :-) You don't
    >>>>>> need to ask questions. You just need to make statements, and if
    >>>>>> they don't "seem OK" even slightly to anyone for any reason, you
    >>>>>> are bound to get a good flogging about it, otherwise everyone
    >>>>>> usually just stays quiet about it.
    >>>>
    >>>>> To illustrate which, there exist file systems that have no blocks
    >>>>> nor clusters. So the statement is not OK. Consider it flogged.
    >>>>
    >>>> There may exist filesystems without the concept of blocks/clusters,
    >>>> but the
    >>>> hard drive itself has those.
    >>>
    >>> But you don't have to have a hard drive to have a file system. It is
    >>> painful, but you could have a file system placed on a tape device.
    >>> You'll have blocks then but no clusters. You could conceivably even
    >>> place a filesystem on a byte-addressable device, although you could
    >>> argue then that you have a block size equal to one byte.


    I'm sorry, but I wonder if you have looked at the title of this thread.
    We can of course generalise the idea of what a filesystem is and what a
    storage medium is until no statement at all is universally true. HOwever
    the OP had a question which was more specific.


    >>
    >> A filesystem argues at least sone kind of random access, and random
    >> access is faster if you can split the media into blocks with some being
    >> address structures pointing to other blocks.
    >>
    >> I have implemented a ram disk in my time, and it is simply easier and
    >> more efficient to read and write it in large chunks...and you need some
    >> sort of directory structure to tell you where data is. I.e. a filesystem
    >> is an indirect mechanism of indices and data blocks. If its un-indexed
    >> and byte sized (RAM) the first thing you do is use some of it to keep
    >> track of the rest of it.


    >An efficient file system argues for some kind of random access. If you
    >don't mind waiting for long periods of time while reading a tape you can
    >put a filesystem on a tape device. I have used these on both PDP Unix
    >systems and on VAX-based Ultrix systems. A decent explanation of a tape
    >file system is at http://en.wikipedia.org/wiki/LINC


    >A PDP tribute page at http://hampage.hu/pdp-11/periferia.html also
    >mentions using a TU55 1" tape drive as a disk with a normal file system.



  13. Re: what happen when we append a file in a disk

    > (cluster) can be deduced. This block is then READ from the disk platter
    > to the disk buffer.


    No need in read if your appended portion covers the whole block. Just allocate
    the empty block (full of junk), fill it with data and insert it in the cache.

    Modern OSes usually use the page-based cache. You see, to support mmap(), you
    need to have a _set of physical pages tied to a file and containing the file's
    data_. The same set is re-usable for a cache.

    --
    Maxim Shatskih, Windows DDK MVP
    StorageCraft Corporation
    maxim@storagecraft.com
    http://www.storagecraft.com


  14. Re: what happen when we append a file in a disk

    > level of the driver, it has to specify to the hardware which sector
    > number and track number you want to read, and this sector number X
    > track number constitutes a block which has to be read in as a unit into


    No. The drive's hardware protocol works in sector numbers only - Linear Block
    Addresses. You specify the sector number + sector count for your IO.

    Ancient IDE drives used the CHS concept - cylinder/head/sector - to specify the
    starting sector. It is also supported on modern IDE drives as a fake to be
    compatible with ancient stuff.

    SCSI drives always used LBA only, so are USB flash drives and such.

    --
    Maxim Shatskih, Windows DDK MVP
    StorageCraft Corporation
    maxim@storagecraft.com
    http://www.storagecraft.com


+ Reply to Thread