Printable View
Hi,
I have built a prototype disk imaging tool for making disk images for
the Dave Dunfield Horizon simulator using a generic PC with a
Catweasel. The tool is now working but is not ready for release. If
you are interested in testing or developing please contact me. Here
is a sample of output produced by the tool:
[url]http://www.geocities.com/lynchaj/latest.zip[/url]
Thanks!
Andrew Lynch
Hi CP/M users
For some time now I have been trying to use a Compact Flash as a hard drive
with my Interak computer, using CP/M 2.2
The hardware works fine i.e. I can read and write data to the CF. The
difficult bit has been integrating the CP/M file system to accept the CF as
part of the computer working with the 2 floppy disks.
Initially I could not even get a C: prompt to appear on the screen. At
present I can now store files on the CF and use NewSweep to copy files from
floppy to CF and back again. Also disk utility programs can read sectors and
tracks OK.
The problem I have been struggling with for some time is - When a file is
written to disk the de-blocking works until the end of the file. At this
point any data in the host buffer ( which holds 512 bytes - 4 x 128 byte
CP/M sectors ) is lost when the directory entry is written.
When reading flies the correct size is read back which means that the last
128 to 512 bytes are corrupt. Depending where the file has ended either
1,2,3 or 4 blocks of 128 bytes are still in the host buffer.
It seems that while writing a file the first indication that the end-of-file
has occurred is when a write type 1 is sent (write to directory). At this
point the directory has already been copied into the host buffer -
overwriting the final data.
I have several CP/M books but I cannot find sufficient information regarding
this.
One method, which I think may be the best way to go, is to read File Control
Block 2 before BDOS starts writing. This has record size information which
could be compared to the current record as it is written. Then when a match
occurs the host buffer could be immediately written out to the CF.
Am I on the right track here ? Or is there a better way to go ?
Any help greatly appreciated, Alan
Alan wrote:[color=blue]
>[...]
> It seems that while writing a file the first indication that the end-of-file
> has occurred is when a write type 1 is sent (write to directory). At this
> point the directory has already been copied into the host buffer -
> overwriting the final data.[/color]
The host buffer is owned by you, not the application, right? So, when the
application calls 'write block', you're copying the data into your own buffer,
and then only flushing it to disk when you have a complete buffer-full?
That being the case, then there should be no situation where data gets copied
into the buffer without your code first checking to see if the buffer contains
data that needs to get flushed. i.e.:
to write sector n:
is n == oldn?
copy data into buffer at (n % 4)
return
else
does buffer need flushing?
write buffer to disk at block (oldn / 4)
set oldn = n
reload buffer (n / 4) from disk
copy data into buffer at (n % 4)
return
(If you're writing at the end of file you might be able to get away without
reloading the buffer, if you're clever.)
(Don't forget that you also need to flush buffers on disk logoff. Probably on
application exit, too.)
--
┌── ｄｇ*ｃｏｗｌａｒｋ．ｃｏｍ ─── [url]http://www.cowlark.com[/url] ───────────────────
│
│ "There does not now, nor will there ever, exist a programming language in
│ which it is the least bit hard to write bad programs." --- Flon's Axiom
That being the case, then there should be no situation where data gets
copied
into the buffer without your code first checking to see if the buffer
contains
data that needs to get flushed. i.e.:
Hi David
Thanks for your reply. Also apologies to Andrew for somehow getting into
his thread. I sent my query to the group as a new thread.
As I see it there could be a situation where a block of 128bytes is written
into the buffer (which already contains say 128 bytes) this would make the
512-byte buffer half full - therefore code checks it - no need to flush it
yet. But this 128byte block is the final one.
to write sector n:
is n == oldn?
copy data into buffer at (n % 4)
return
else
does buffer need flushing?
write buffer to disk at block (oldn / 4)
set oldn = n
reload buffer (n / 4) from disk
copy data into buffer at (n % 4)
return
Sorry I do not fully understand this code (looks like C) as I tend to use
Z80 machine code.
Alan
On Sun, 09 Sep 2007 16:20:03 GMT, "Alan" <z378reply@blueyonder.co.uk>
wrote:
[color=blue]
>Hi CP/M users
>For some time now I have been trying to use a Compact Flash as a hard drive
>with my Interak computer, using CP/M 2.2
>The hardware works fine i.e. I can read and write data to the CF. The
>difficult bit has been integrating the CP/M file system to accept the CF as
>part of the computer working with the 2 floppy disks.
>Initially I could not even get a C: prompt to appear on the screen. At
>present I can now store files on the CF and use NewSweep to copy files from
>floppy to CF and back again. Also disk utility programs can read sectors and
>tracks OK.
>
>The problem I have been struggling with for some time is - When a file is
>written to disk the de-blocking works until the end of the file. At this
>point any data in the host buffer ( which holds 512 bytes - 4 x 128 byte
>CP/M sectors ) is lost when the directory entry is written.
>When reading flies the correct size is read back which means that the last
>128 to 512 bytes are corrupt. Depending where the file has ended either
>1,2,3 or 4 blocks of 128 bytes are still in the host buffer.
>It seems that while writing a file the first indication that the end-of-file
>has occurred is when a write type 1 is sent (write to directory). At this
>point the directory has already been copied into the host buffer -
>overwriting the final data.
>
>I have several CP/M books but I cannot find sufficient information regarding
>this.
>One method, which I think may be the best way to go, is to read File Control
>Block 2 before BDOS starts writing. This has record size information which
>could be compared to the current record as it is written. Then when a match
>occurs the host buffer could be immediately written out to the CF.
>
>Am I on the right track here ? Or is there a better way to go ?
>
>Any help greatly appreciated, Alan[/color]
First are you reading Andy Johnson-Laird, The programmer CP/M
handbook? IF not get it..
CP/M provides a few signals of when to write a block (flush the cache)
and when not to. These are available to the bios and should be used.
If you using an application to read and write media (any!) then it's
the responseability of that program to manage caching and writing
as it's likely not doing BDOS calls for that (direct BIOS IO).
If all else fails and yo want it to run failsafe but slow every write
has a read and you then lay in the logical sector to the physical
sector buffer and write it back. Not elegant but it always works
and when to write is implied always.
Allison
On Sun, 09 Sep 2007 16:51:03 GMT, "Alan" <z378reply@blueyonder.co.uk>
wrote:
[color=blue]
>
>That being the case, then there should be no situation where data gets
>copied
>into the buffer without your code first checking to see if the buffer
>contains
>data that needs to get flushed. i.e.:
>
>Hi David
>Thanks for your reply. Also apologies to Andrew for somehow getting into
>his thread. I sent my query to the group as a new thread.
>
>As I see it there could be a situation where a block of 128bytes is written
>into the buffer (which already contains say 128 bytes) this would make the
>512-byte buffer half full - therefore code checks it - no need to flush it
>yet. But this 128byte block is the final one.[/color]
At the BIOS level...
CP/M has three cases for write:
1) Write to directory (always pre read and write back for directory
security)
2) Write to allocated block (pre read is required as there may be
data that that is to be preserved.
3) Write to unallocated block (no data has been stored in this block
that needs to be saved. usually this is the write to the first
logical sector of an allocation block.
When I say pre-read I mean the BIOS must track if the current logical
sector to be written is in the physical sector buffer. The BDOS when
it calls the BIOS provides information on what of the three cases it
will be for write. Generally when deblocking is in the BIOS you
have a lot of information to let you know when two write back...
Are you writing a different physical sector?
Are you writing to a different track than last time?
Are you reading the directory? {non obvious but every time
a directory read occurs you are changing track and sector
and likely need to write back the buffers before doing the
directory read.
This is covered in the CP/M alteration guide section 6.12 "Sector
Blocking and deblocking". It's light on the "howto" but the core
infor is there. AJL The Programmers CP/M handbook coveres
it better and provides example code.
Biggest thing is when you read or write is keeping track of when
to write back if you are not doing the slower read-modify-write
always. If removeable media is used then you have to insure
buffers are written before removing or very bad things happen.
[color=blue]
>
>to write sector n:
> is n == oldn?
> copy data into buffer at (n % 4)
> return
> else
> does buffer need flushing?
> write buffer to disk at block (oldn / 4)
> set oldn = n
> reload buffer (n / 4) from disk
> copy data into buffer at (n % 4)
> return
>
>Sorry I do not fully understand this code (looks like C) as I tend to use
>Z80 machine code.[/color]
Looks like pseudo code, sort of plain language for procedural
description.
Allison
On 2007-09-09, Alan <z378reply@blueyonder.co.uk> wrote:[color=blue]
> It seems that while writing a file the first indication that the end-of-file
> has occurred is when a write type 1 is sent (write to directory). At this
> point the directory has already been copied into the host buffer -
> overwriting the final data.[/color]
I was under the impression that CP/M always called HOME before manipulating
the directory, but I sure don't see it in the manual (which probably
means I'm mistaken again).
--
roger ivie
[email]rivie@ridgenet.net[/email]
On Sun, 09 Sep 2007 22:02:41 GMT, Roger Ivie <rivie@ridgenet.net>
wrote:
[color=blue]
>On 2007-09-09, Alan <z378reply@blueyonder.co.uk> wrote:[color=green]
>> It seems that while writing a file the first indication that the end-of-file
>> has occurred is when a write type 1 is sent (write to directory). At this
>> point the directory has already been copied into the host buffer -
>> overwriting the final data.[/color]
>
>I was under the impression that CP/M always called HOME before manipulating
>the directory, but I sure don't see it in the manual (which probably
>means I'm mistaken again).[/color]
Generally thats true save for the directory is not at track 0000 and
most system use that to load a zero into the storage location called
track. Otherwise in most moden (post V1.4) home is a do mostly
nothing.
Allison
Alan wrote:
[...][color=blue]
> As I see it there could be a situation where a block of 128bytes is written
> into the buffer (which already contains say 128 bytes) this would make the
> 512-byte buffer half full - therefore code checks it - no need to flu****
> yet. But this 128byte block is the final one.[/color]
Sure, but it doesn't know that until the *next* write comes along --- which is
to a completely different disk block. Therefore your code knows that it needs
to flush the block out to disk before it can copy the next sector into the buffer.
[color=blue]
> Sorry I do not fully understand this code (looks like C) as I tend to use
> Z80 machine code.[/color]
It's pseudocode --- just a plain English description of an algorithm.
--
┌── ｄｇ*ｃｏｗｌａｒｋ．ｃｏｍ ─── [url]http://www.cowlark.com[/url] ───────────────────
│
│ "There does not now, nor will there ever, exist a programming language in
│ which it is the least bit hard to write bad programs." --- Flon's Axiom
On Mon, 10 Sep 2007 21:07:41 +0100, David Given <dg@cowlark.com>
wrote:
[color=blue]
>Alan wrote:
>[...][color=green]
>> As I see it there could be a situation where a block of 128bytes is written
>> into the buffer (which already contains say 128 bytes) this would make the
>> 512-byte buffer half full - therefore code checks it - no need to flush it
>> yet. But this 128byte block is the final one.[/color]
>
>Sure, but it doesn't know that until the *next* write comes along --- which is
>to a completely different disk block. Therefore your code knows that it needs
>to flush the block out to disk before it can copy the next sector into the buffer.[/color]
There are two cases. User application that does direct bios calls to
read/write disk and BDOS calls. The first case the user must take
care to not forget to flush the buffer. For the second case a
properly written BIOS gets signals from the BDOS to tell it to preread
and also when to write..
I've been using various floppies and disks for years with sector
deblocking without issues and it really makes no difference what the
media really is only that the code is correct. Despite claims about a
deblocking bug I've used the DRI code with no problems though I prefer
my own as it's easier for me to understand.
Allison
[color=blue][color=green]
>> Sorry I do not fully understand this code (looks like C) as I tend to use
>> Z80 machine code.[/color]
>
>It's pseudocode --- just a plain English description of an algorithm.[/color]
Hi Thanks for the mass of good information. I will sift carefully through
it.
The floppy drives are using 512-byte sectors (same as the CF) so this helps.
But I am keeping individual host buffers and de-blocking for floppy's and CF
as I assumed that this would be necessary when copying files.
I have a copy of the Programmers CP/M Handbook which has been very useful
and I have also 'borrowed' some of John Baker's code which was written for
an IDE interface.
I realise that there are quite a few things to check when reading or writing
takes place but I did not want to slow the process down too much with
unnecessary code.
[color=blue]
>If all else fails and yo want it to run failsafe but slow every write
>has a read and you then lay in the logical sector to the physical
>sector buffer and write it back. Not elegant but it always works
>and when to write is implied always.[/color]
[color=blue]
>Allison[/color]
I guess this would mean writing the 512-byte sector to disk every time the
BDOS sends a 128-byte sector. As the Compact Flash is fast (the slowest
point here is reading/writing the floppies) would this be noticable? Also
would it notice much when writing files directly to the CF?
I like the idea of being failsafe.
Thanks, Alan
On Tue, 11 Sep 2007 09:17:44 GMT, "Alan" <z378reply@blueyonder.co.uk>
wrote:
[color=blue]
>Hi Thanks for the mass of good information. I will sift carefully through
>it.
>The floppy drives are using 512-byte sectors (same as the CF) so this helps.
>But I am keeping individual host buffers and de-blocking for floppy's and CF
>as I assumed that this would be necessary when copying files.
>
>I have a copy of the Programmers CP/M Handbook which has been very useful
>and I have also 'borrowed' some of John Baker's code which was written for
>an IDE interface.
>I realise that there are quite a few things to check when reading or writing
>takes place but I did not want to slow the process down too much with
>unnecessary code.[/color]
the amouunt of code to do all that is actually small. Additional
buffers take space though but again thats not all that much.
FYI: multiple buffers can be faster!
[color=blue][color=green]
>>If all else fails and yo want it to run failsafe but slow every write
>>has a read and you then lay in the logical sector to the physical
>>sector buffer and write it back. Not elegant but it always works
>>and when to write is implied always.[/color]
>[color=green]
>>Allison[/color]
>
>I guess this would mean writing the 512-byte sector to disk every time the
>BDOS sends a 128-byte sector. As the Compact Flash is fast (the slowest
>point here is reading/writing the floppies) would this be noticable? Also
>would it notice much when writing files directly to the CF?
>I like the idea of being failsafe.[/color]
For CF I'd not do that only because it has a limited (though large)
number of write cycles. The real issue is the file close problem,
that is insuring the file is closed and buffers are flushed. There
may be a secondary "what if the power fails" issue but that is a
call for a UPS.
Allison
[color=blue]
>
>Thanks, Alan
>
>[/color]
[email]no.spam@no.uce.bellatlantic.net[/email] wrote:
(snip)
[color=blue]
> For CF I'd not do that only because it has a limited (though large)
> number of write cycles. The real issue is the file close problem,
> that is insuring the file is closed and buffers are flushed. There
> may be a secondary "what if the power fails" issue but that is a
> call for a UPS.[/color]
I wonder how many have IDE drives on their CP/M-80 systems.
It seems that while IDE originally included the option of
eight bit I/O, many drives only implement 16 bit I/O.
This requires extra latches for eight bit processors.
I now wonder if CF, which has a similar interface to IDE,
has non-optional eight bit I/O.
I am considering building an 8080 system that should be
able to run CP/M, and the IDE interface is fairly simple.
-- glen
On 2007-09-11, glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote:[color=blue]
> I now wonder if CF, which has a similar interface to IDE,
> has non-optional eight bit I/O.[/color]
CF does, indeed, have eight bit I/O. It's often used in embedded
applications, where the 8-bit interface is handy.
The CF specs are quickly and easily available on-line.
--
roger ivie
[email]rivie@ridgenet.net[/email]
On Tue, 11 Sep 2007 19:50:48 GMT, Roger Ivie <rivie@ridgenet.net>
wrote:
[color=blue]
>On 2007-09-11, glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote:[color=green]
>> I now wonder if CF, which has a similar interface to IDE,
>> has non-optional eight bit I/O.[/color]
>
>CF does, indeed, have eight bit I/O. It's often used in embedded
>applications, where the 8-bit interface is handy.
>
>The CF specs are quickly and easily available on-line.[/color]
True, and handy too.
However, being one that enjoys a good crufty hack. A IDE hard disk
does not require a 16 bit interface. There are caveats like you loose
half the storage and then it's only 256bytes/sector. The answer is
simply, yes. But the interface as a result is so minimal its easier
than talking to an 8255 PPI! As to the storage loss, even a 40MB
drive is plenty huge even if only 20mb is "used". When you consider
the likely size fo easy to find older sub 1GB drives is around
400-500mb the unused space is a good trade.
For the truely insane you can do the above and even skip deblocking
and only use the first 128bytes of a sector. Horror of horrors!
Wasting 3/4 of the drives storage capacity. Well with 1GB drives
that remaining 256MB is still over the upper limit (128mb) for sixteen
8MB logical CP/M 2.2 drives A-P. CP/M isn't PC and code bloat
doesn't live there so even a few 8MB logical drives is luxury.
For those that do not have a few CF drives, or the nasty tiny
connectors to fit and none of those IDE to CF adaptors the IDE
hard disk is easy to live with and can be dirt simple. As to the
power difference, last I checked most of my CP/M systems have
power sources that are well oversized and have enough excess
+12 and +5 for even the oldest IDE hard disks and the later ones
or the smaller 2.5" ones are remarkably low power.
Just a few ideas!
Allison
Roger Ivie wrote:[color=blue]
> On 2007-09-11, glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote:[/color]
[color=blue][color=green]
>>I now wonder if CF, which has a similar interface to IDE,
>>has non-optional eight bit I/O.[/color][/color]
[color=blue]
> CF does, indeed, have eight bit I/O. It's often used in embedded
> applications, where the 8-bit interface is handy.[/color]
[color=blue]
> The CF specs are quickly and easily available on-line.[/color]
I have already downloaded some version, but I wanted to get a
discussion started. (or continue the existing one)
As I understand it, IDE also has an eight bit interface, but
most don't implement it. Maybe the early drives (that one might
want to attach to a PC/XT) did.
I am following the discussion on reblocking. I remember that SCSI
allows one to reformat a disk for a different block size. I don't
remember if IDE has that option.
I have ordered the previously mentioned CP/M book but it
hasn't come yet.
thanks for all the discussions,
-- glen
On Tue, 11 Sep 2007 14:47:01 -0800, glen herrmannsfeldt
<gah@ugcs.caltech.edu> wrote:
[color=blue]
>Roger Ivie wrote:[color=green]
>> On 2007-09-11, glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote:[/color]
>[color=green][color=darkred]
>>>I now wonder if CF, which has a similar interface to IDE,
>>>has non-optional eight bit I/O.[/color][/color]
>[color=green]
>> CF does, indeed, have eight bit I/O. It's often used in embedded
>> applications, where the 8-bit interface is handy.[/color]
>[color=green]
>> The CF specs are quickly and easily available on-line.[/color]
>
>I have already downloaded some version, but I wanted to get a
>discussion started. (or continue the existing one)
>
>As I understand it, IDE also has an eight bit interface, but
>most don't implement it. Maybe the early drives (that one might
>want to attach to a PC/XT) did.[/color]
I have a few really old IDE as in 10mb with steppers and non do 8bit.
[color=blue]
>I am following the discussion on reblocking. I remember that SCSI
>allows one to reformat a disk for a different block size. I don't
>remember if IDE has that option.[/color]
None of the IDE specs I've seen allow that.
Allison
On 2007-09-11, [email]no.spam@no.uce.bellatlantic.net[/email] <no.spam@no.uce.bellatlantic.net> wrote:[color=blue]
> On Tue, 11 Sep 2007 14:47:01 -0800, glen herrmannsfeldt[color=green]
>>As I understand it, IDE also has an eight bit interface, but
>>most don't implement it. Maybe the early drives (that one might
>>want to attach to a PC/XT) did.[/color]
>
> I have a few really old IDE as in 10mb with steppers and non do 8bit.[/color]
I've seen a grand total of one drive that supports 8 bit IDE. It was a
40MB drive made by, IIRC, WD.
[color=blue][color=green]
>>I am following the discussion on reblocking. I remember that SCSI
>>allows one to reformat a disk for a different block size. I don't
>>remember if IDE has that option.[/color]
>
> None of the IDE specs I've seen allow that.[/color]
The IDE spec original required the bits in the 1010 register file that
specify sector size to be fixed at the 512 byte code. Since then, I
think at least one bit has been taken over for one uses; IIRC, one
of them switches the addresses from CHS to LBA.
I once wrote the firmware for an AT-compatible disk controller, so I
used to know this stuff cold. The bits, they leak away over time...
--
roger ivie
[email]rivie@ridgenet.net[/email]
On Sep 11, 7:41 pm, Roger Ivie <ri...@ridgenet.net> wrote:[color=blue]
> On 2007-09-11, no.s...@no.uce.bellatlantic.net <no.s...@no.uce.bellatlantic.net> wrote:
>[color=green]
> > On Tue, 11 Sep 2007 14:47:01 -0800, glen herrmannsfeldt[color=darkred]
> >>As I understand it, IDE also has an eight bit interface, but
> >>most don't implement it. Maybe the early drives (that one might
> >>want to attach to a PC/XT) did.[/color][/color]
>[color=green]
> > I have a few really old IDE as in 10mb with steppers and non do 8bit.[/color]
>
> I've seen a grand total of one drive that supports 8 bit IDE. It was a
> 40MB drive made by, IIRC, WD.
>[/color]
I use an old laptop IDE in my xt clone, 256mb. The hard part to find
was the controller. I think it came out of a Tandy.
[color=blue][color=green][color=darkred]
> >>I am following the discussion on reblocking. I remember that SCSI
> >>allows one to reformat a disk for a different block size. I don't
> >>remember if IDE has that option.[/color][/color]
>[color=green]
> > None of the IDE specs I've seen allow that.[/color]
>
> The IDE spec original required the bits in the 1010 register file that
> specify sector size to be fixed at the 512 byte code. Since then, I
> think at least one bit has been taken over for one uses; IIRC, one
> of them switches the addresses from CHS to LBA.
>
> I once wrote the firmware for an AT-compatible disk controller, so I
> used to know this stuff cold. The bits, they leak away over time...
> --[/color]
I hear that! Wasn't it the IDE that shouldn't be low level formatted?
Steve
[color=blue]
> roger ivie
> ri...@ridgenet.net[/color]
On 2007-09-12, [email]s_dubrovich@yahoo.com[/email] <s_dubrovich@yahoo.com> wrote:[color=blue]
> I hear that! Wasn't it the IDE that shouldn't be low level formatted?[/color]
Indeed. IDE and anything newer. Somewhere I have an old TURBO Pascal
program I wrote to format an AT disk at 2:1 interleave. It does,
of course, assume MFM 'cause it was written back in the day...
--
roger ivie
[email]rivie@ridgenet.net[/email]