what happen when we append a file in a disk

Hi,
Assume that a user wants to append a data to a file in the Hard Drive
disk. I am just wondering about the steps that are undertaken to
complete this operation. My understanding is that first the OS has to
locate the file in the disk to retrieve its physical location and its
size. By knowing its size, the location of the last file block
(cluster) can be deduced. This block is then READ from the disk platter
to the disk buffer. the data that we want to append to the file is then
appended in the disk buffer to the block which has just been read from
the platters. Once this operation is completed the newly block is
written from disk buffer to the disk platter.The above low operations
are executed by the disk controller which runs the disk device driver

Am I right?


A sideline question, each file has a number of blocks/cluster. each a
cluster is a set of consecutive sectors

Thanks for your help

[email]xu_feng_xu@yahoo.com[/email] wrote:[color=blue]
> Hi,
> Assume that a user wants to append a data to a file in the Hard Drive
> disk. I am just wondering about the steps that are undertaken to
> complete this operation. My understanding is that first the OS has to
> locate the file in the disk to retrieve its physical location and its
> size.[/color]

It should be able to find this from the i-node or its extensions.
[color=blue]
> By knowing its size, the location of the last file block
> (cluster) can be deduced.[/color]

Not really. I.e., you cannot deduce the location of the last file block from
the file size and starting point. You have to look it up in tables that are
on the disk itself.
[color=blue]
> This block is then READ from the disk platter
> to the disk buffer. the data that we want to append to the file is then
> appended in the disk buffer to the block which has just been read from
> the platters. Once this operation is completed the newly block is
> written from disk buffer to the disk platter.The above low operations
> are executed by the disk controller which runs the disk device driver[/color]

The disk device driver runs the disk controller, not the other way around.
Depending on the controller and the disk drive itself, chances are pretty
good that the disk drive will read the entire track from the disk, not just
the specified block, keeping it in its cache memory (typically 8 MBytes
these days). Then the drive and the controller will read back the blocks
requested by the OS. Even the OS may request the controller to read ahead if
it notices you are reading sequentially.[color=blue]
>
> Am I right?
>
>
> A sideline question, each file has a number of blocks/cluster. each a
> cluster is a set of consecutive sectors
>
> Thanks for your help
>[/color]


--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey [url]http://counter.li.org[/url]
^^-^^ 07:35:01 up 21 days, 10:25, 3 users, load average: 4.54, 4.39, 4.22

[email]xu_feng_xu@yahoo.com[/email] wrote:[color=blue]
> Hi,
> Assume that a user wants to append a data to a file in the Hard Drive
> disk. I am just wondering about the steps that are undertaken to
> complete this operation. My understanding is that first the OS has to
> locate the file in the disk to retrieve its physical location and its
> size. By knowing its size, the location of the last file block
> (cluster) can be deduced. This block is then READ from the disk
> platter to the disk buffer. the data that we want to append to the
> file is then appended in the disk buffer to the block which has just
> been read from the platters. Once this operation is completed the
> newly block is written from disk buffer to the disk platter.The above
> low operations are executed by the disk controller which runs the
> disk device driver
>
> Am I right?[/color]

There appears to be a bug in what you said. You didn't think about the case
when the last cluster/block is full. :)
[color=blue]
> A sideline question, each file has a number of blocks/cluster. each a
> cluster is a set of consecutive sectors[/color]

That's a statement, not a question. And it seems OK.

Alex

> > A sideline question, each file has a number of blocks/cluster. each a[color=blue][color=green]
> > cluster is a set of consecutive sectors[/color]
>
> That's a statement, not a question. And it seems OK.[/color]

That's an important feature of newsgroups, you see. :-) You don't need
to ask questions. You just need to make statements, and if they don't
"seem OK" even slightly to anyone for any reason, you are bound to get
a good flogging about it, otherwise everyone usually just stays quiet
about it.

microx wrote:[color=blue]
>[color=green][color=darkred]
>>> A sideline question, each file has a number of blocks/cluster.
>>> each a cluster is a set of consecutive sectors[/color]
>>
>> That's a statement, not a question. And it seems OK.[/color]
>
> That's an important feature of newsgroups, you see. :-) You don't
> need to ask questions. You just need to make statements, and if
> they don't "seem OK" even slightly to anyone for any reason, you
> are bound to get a good flogging about it, otherwise everyone
> usually just stays quiet about it.[/color]

To illustrate which, there exist file systems that have no blocks
nor clusters. So the statement is not OK. Consider it flogged.

Please don't strip attribution lines for material you quote. Those
are the "xxx wrote:" lines at the beginning. Doing so invokes more
strokes of the cat than the faulty statement.

--
Some informative links:
<news:news.announce.newusers
<http://www.geocities.com/nnqweb/>
<http://www.catb.org/~esr/faqs/smart-questions.html>
<http://www.caliburn.nl/topposting.html>
<http://www.netmeister.org/news/learn2quote.html>
<http://cfaj.freeshell.org/google/>

CBFalconer <cbfalconer@yahoo.com> writes:
[color=blue]
>microx wrote:[color=green]
>>[color=darkred]
>>>> A sideline question, each file has a number of blocks/cluster.
>>>> each a cluster is a set of consecutive sectors
>>>
>>> That's a statement, not a question. And it seems OK.[/color]
>>
>> That's an important feature of newsgroups, you see. :-) You don't
>> need to ask questions. You just need to make statements, and if
>> they don't "seem OK" even slightly to anyone for any reason, you
>> are bound to get a good flogging about it, otherwise everyone
>> usually just stays quiet about it.[/color][/color]
[color=blue]
>To illustrate which, there exist file systems that have no blocks
>nor clusters. So the statement is not OK. Consider it flogged.[/color]

There may exist filesystems without the concept of blocks/clusters, but the
hard drive itself has those.

Unruh wrote:[color=blue]
> CBFalconer <cbfalconer@yahoo.com> writes:
>[/color]
.... snip ...[color=blue]
>[color=green]
>> To illustrate which, there exist file systems that have no blocks
>> nor clusters. So the statement is not OK. Consider it flogged.[/color]
>
> There may exist filesystems without the concept of blocks/clusters,
> but the hard drive itself has those.[/color]

Oh? Ultimately the drive consists of nothing more than a more or
less continuous magnetic (or other) media, possibly broken into
tracks (but not necessarily). Various formatting operations may
add in such conveniences as sectors with address fields, and even
translation tables to replace faulty sectors. The rest belongs to
the file system.

--
Some informative links:
<news:news.announce.newusers
<http://www.geocities.com/nnqweb/>
<http://www.catb.org/~esr/faqs/smart-questions.html>
<http://www.caliburn.nl/topposting.html>
<http://www.netmeister.org/news/learn2quote.html>
<http://cfaj.freeshell.org/google/>

CBFalconer wrote:[color=blue]
> Oh? Ultimately the drive consists of nothing more than a more or
> less continuous magnetic (or other) media, possibly broken into
> tracks (but not necessarily). Various formatting operations may
> add in such conveniences as sectors with address fields, and even
> translation tables to replace faulty sectors. The rest belongs to
> the file system.[/color]

No.. That's only how it can be made to appear. What I learned is that
when the drive is accessed, even if you ask the driver to give you a
random chunk in some linear "continuous" address speace, then at some
level of the driver, it has to specify to the hardware which sector
number and track number you want to read, and this sector number X
track number constitutes a block which has to be read in as a unit into
some buffer, then you can select the exact bytes you want out of it
based on your logical linear address. That's of course assuming we are
talking about block devices, like disk drives, and not tapes for
example.

Left the attribution in there for you. ;-)

Unruh wrote:[color=blue]
> CBFalconer <cbfalconer@yahoo.com> writes:
>[color=green]
>> microx wrote:[color=darkred]
>>>>> A sideline question, each file has a number of blocks/cluster.
>>>>> each a cluster is a set of consecutive sectors
>>>> That's a statement, not a question. And it seems OK.
>>> That's an important feature of newsgroups, you see. :-) You don't
>>> need to ask questions. You just need to make statements, and if
>>> they don't "seem OK" even slightly to anyone for any reason, you
>>> are bound to get a good flogging about it, otherwise everyone
>>> usually just stays quiet about it.[/color][/color]
>[color=green]
>> To illustrate which, there exist file systems that have no blocks
>> nor clusters. So the statement is not OK. Consider it flogged.[/color]
>
> There may exist filesystems without the concept of blocks/clusters, but the
> hard drive itself has those.[/color]

But you don't have to have a hard drive to have a file system. It is
painful, but you could have a file system placed on a tape device.
You'll have blocks then but no clusters. You could conceivably even
place a filesystem on a byte-addressable device, although you could
argue then that you have a block size equal to one byte.

Douglas O'Neal wrote:[color=blue]
> Unruh wrote:[color=green]
>> CBFalconer <cbfalconer@yahoo.com> writes:
>>[color=darkred]
>>> microx wrote:
>>>>>> A sideline question, each file has a number of blocks/cluster.
>>>>>> each a cluster is a set of consecutive sectors
>>>>> That's a statement, not a question. And it seems OK.
>>>> That's an important feature of newsgroups, you see. :-) You don't
>>>> need to ask questions. You just need to make statements, and if
>>>> they don't "seem OK" even slightly to anyone for any reason, you
>>>> are bound to get a good flogging about it, otherwise everyone
>>>> usually just stays quiet about it.[/color]
>>[color=darkred]
>>> To illustrate which, there exist file systems that have no blocks
>>> nor clusters. So the statement is not OK. Consider it flogged.[/color]
>>
>> There may exist filesystems without the concept of blocks/clusters,
>> but the
>> hard drive itself has those.[/color]
>
> But you don't have to have a hard drive to have a file system. It is
> painful, but you could have a file system placed on a tape device.
> You'll have blocks then but no clusters. You could conceivably even
> place a filesystem on a byte-addressable device, although you could
> argue then that you have a block size equal to one byte.[/color]

A filesystem argues at least sone kind of random access, and random
access is faster if you can split the media into blocks with some being
address structures pointing to other blocks.

I have implemented a ram disk in my time, and it is simply easier and
more efficient to read and write it in large chunks...and you need some
sort of directory structure to tell you where data is. I.e. a filesystem
is an indirect mechanism of indices and data blocks. If its un-indexed
and byte sized (RAM) the first thing you do is use some of it to keep
track of the rest of it.

The Natural Philosopher wrote:[color=blue]
> Douglas O'Neal wrote:[color=green]
>> Unruh wrote:[color=darkred]
>>> CBFalconer <cbfalconer@yahoo.com> writes:
>>>
>>>> microx wrote:
>>>>>>> A sideline question, each file has a number of blocks/cluster.
>>>>>>> each a cluster is a set of consecutive sectors
>>>>>> That's a statement, not a question. And it seems OK.
>>>>> That's an important feature of newsgroups, you see. :-) You don't
>>>>> need to ask questions. You just need to make statements, and if
>>>>> they don't "seem OK" even slightly to anyone for any reason, you
>>>>> are bound to get a good flogging about it, otherwise everyone
>>>>> usually just stays quiet about it.
>>>
>>>> To illustrate which, there exist file systems that have no blocks
>>>> nor clusters. So the statement is not OK. Consider it flogged.
>>>
>>> There may exist filesystems without the concept of blocks/clusters,
>>> but the
>>> hard drive itself has those.[/color]
>>
>> But you don't have to have a hard drive to have a file system. It is
>> painful, but you could have a file system placed on a tape device.
>> You'll have blocks then but no clusters. You could conceivably even
>> place a filesystem on a byte-addressable device, although you could
>> argue then that you have a block size equal to one byte.[/color]
>
> A filesystem argues at least sone kind of random access, and random
> access is faster if you can split the media into blocks with some being
> address structures pointing to other blocks.
>
> I have implemented a ram disk in my time, and it is simply easier and
> more efficient to read and write it in large chunks...and you need some
> sort of directory structure to tell you where data is. I.e. a filesystem
> is an indirect mechanism of indices and data blocks. If its un-indexed
> and byte sized (RAM) the first thing you do is use some of it to keep
> track of the rest of it.[/color]

An efficient file system argues for some kind of random access. If you
don't mind waiting for long periods of time while reading a tape you can
put a filesystem on a tape device. I have used these on both PDP Unix
systems and on VAX-based Ultrix systems. A decent explanation of a tape
file system is at [url]http://en.wikipedia.org/wiki/LINC[/url]

A PDP tribute page at [url]http://hampage.hu/pdp-11/periferia.html[/url] also
mentions using a TU55 1" tape drive as a disk with a normal file system.

Doug

Douglas O'Neal <oneal@dbi.udel.edu> writes:
[color=blue]
>The Natural Philosopher wrote:[color=green]
>> Douglas O'Neal wrote:[color=darkred]
>>> Unruh wrote:
>>>> CBFalconer <cbfalconer@yahoo.com> writes:
>>>>
>>>>> microx wrote:
>>>>>>>> A sideline question, each file has a number of blocks/cluster.
>>>>>>>> each a cluster is a set of consecutive sectors
>>>>>>> That's a statement, not a question. And it seems OK.
>>>>>> That's an important feature of newsgroups, you see. :-) You don't
>>>>>> need to ask questions. You just need to make statements, and if
>>>>>> they don't "seem OK" even slightly to anyone for any reason, you
>>>>>> are bound to get a good flogging about it, otherwise everyone
>>>>>> usually just stays quiet about it.
>>>>
>>>>> To illustrate which, there exist file systems that have no blocks
>>>>> nor clusters. So the statement is not OK. Consider it flogged.
>>>>
>>>> There may exist filesystems without the concept of blocks/clusters,
>>>> but the
>>>> hard drive itself has those.
>>>
>>> But you don't have to have a hard drive to have a file system. It is
>>> painful, but you could have a file system placed on a tape device.
>>> You'll have blocks then but no clusters. You could conceivably even
>>> place a filesystem on a byte-addressable device, although you could
>>> argue then that you have a block size equal to one byte.[/color][/color][/color]

I'm sorry, but I wonder if you have looked at the title of this thread.
We can of course generalise the idea of what a filesystem is and what a
storage medium is until no statement at all is universally true. HOwever
the OP had a question which was more specific.

[color=blue][color=green]
>>
>> A filesystem argues at least sone kind of random access, and random
>> access is faster if you can split the media into blocks with some being
>> address structures pointing to other blocks.
>>
>> I have implemented a ram disk in my time, and it is simply easier and
>> more efficient to read and write it in large chunks...and you need some
>> sort of directory structure to tell you where data is. I.e. a filesystem
>> is an indirect mechanism of indices and data blocks. If its un-indexed
>> and byte sized (RAM) the first thing you do is use some of it to keep
>> track of the rest of it.[/color][/color]
[color=blue]
>An efficient file system argues for some kind of random access. If you
>don't mind waiting for long periods of time while reading a tape you can
>put a filesystem on a tape device. I have used these on both PDP Unix
>systems and on VAX-based Ultrix systems. A decent explanation of a tape
>file system is at [url]http://en.wikipedia.org/wiki/LINC[/url][/color]
[color=blue]
>A PDP tribute page at [url]http://hampage.hu/pdp-11/periferia.html[/url] also
>mentions using a TU55 1" tape drive as a disk with a normal file system.[/color]

> (cluster) can be deduced. This block is then READ from the disk platter[color=blue]
> to the disk buffer.[/color]

No need in read if your appended portion covers the whole block. Just allocate
the empty block (full of junk), fill it with data and insert it in the cache.

Modern OSes usually use the page-based cache. You see, to support mmap(), you
need to have a _set of physical pages tied to a file and containing the file's
data_. The same set is re-usable for a cache.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
[email]maxim@storagecraft.com[/email]
[url]http://www.storagecraft.com[/url]

> level of the driver, it has to specify to the hardware which sector[color=blue]
> number and track number you want to read, and this sector number X
> track number constitutes a block which has to be read in as a unit into[/color]

No. The drive's hardware protocol works in sector numbers only - Linear Block
Addresses. You specify the sector number + sector count for your IO.

Ancient IDE drives used the CHS concept - cylinder/head/sector - to specify the
starting sector. It is also supported on modern IDE drives as a fake to be
compatible with ancient stuff.

SCSI drives always used LBA only, so are USB flash drives and such.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
[email]maxim@storagecraft.com[/email]
[url]http://www.storagecraft.com[/url]