Catweasel NorthStar disk imaging tool - CP/M

This is a discussion on Catweasel NorthStar disk imaging tool - CP/M ; Hi, I have built a prototype disk imaging tool for making disk images for the Dave Dunfield Horizon simulator using a generic PC with a Catweasel. The tool is now working but is not ready for release. If you are ...

+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 26

Thread: Catweasel NorthStar disk imaging tool

  1. Catweasel NorthStar disk imaging tool

    Hi,

    I have built a prototype disk imaging tool for making disk images for
    the Dave Dunfield Horizon simulator using a generic PC with a
    Catweasel. The tool is now working but is not ready for release. If
    you are interested in testing or developing please contact me. Here
    is a sample of output produced by the tool:

    http://www.geocities.com/lynchaj/latest.zip

    Thanks!

    Andrew Lynch


  2. BDOS/BIOS problem

    Hi CP/M users
    For some time now I have been trying to use a Compact Flash as a hard drive
    with my Interak computer, using CP/M 2.2
    The hardware works fine i.e. I can read and write data to the CF. The
    difficult bit has been integrating the CP/M file system to accept the CF as
    part of the computer working with the 2 floppy disks.
    Initially I could not even get a C: prompt to appear on the screen. At
    present I can now store files on the CF and use NewSweep to copy files from
    floppy to CF and back again. Also disk utility programs can read sectors and
    tracks OK.

    The problem I have been struggling with for some time is - When a file is
    written to disk the de-blocking works until the end of the file. At this
    point any data in the host buffer ( which holds 512 bytes - 4 x 128 byte
    CP/M sectors ) is lost when the directory entry is written.
    When reading flies the correct size is read back which means that the last
    128 to 512 bytes are corrupt. Depending where the file has ended either
    1,2,3 or 4 blocks of 128 bytes are still in the host buffer.
    It seems that while writing a file the first indication that the end-of-file
    has occurred is when a write type 1 is sent (write to directory). At this
    point the directory has already been copied into the host buffer -
    overwriting the final data.

    I have several CP/M books but I cannot find sufficient information regarding
    this.
    One method, which I think may be the best way to go, is to read File Control
    Block 2 before BDOS starts writing. This has record size information which
    could be compared to the current record as it is written. Then when a match
    occurs the host buffer could be immediately written out to the CF.

    Am I on the right track here ? Or is there a better way to go ?

    Any help greatly appreciated, Alan





  3. Re: BDOS/BIOS problem

    Alan wrote:
    >[...]
    > It seems that while writing a file the first indication that the end-of-file
    > has occurred is when a write type 1 is sent (write to directory). At this
    > point the directory has already been copied into the host buffer -
    > overwriting the final data.


    The host buffer is owned by you, not the application, right? So, when the
    application calls 'write block', you're copying the data into your own buffer,
    and then only flushing it to disk when you have a complete buffer-full?

    That being the case, then there should be no situation where data gets copied
    into the buffer without your code first checking to see if the buffer contains
    data that needs to get flushed. i.e.:


    to write sector n:
    is n == oldn?
    copy data into buffer at (n % 4)
    return
    else
    does buffer need flushing?
    write buffer to disk at block (oldn / 4)
    set oldn = n
    reload buffer (n / 4) from disk
    copy data into buffer at (n % 4)
    return

    (If you're writing at the end of file you might be able to get away without
    reloading the buffer, if you're clever.)

    (Don't forget that you also need to flush buffers on disk logoff. Probably on
    application exit, too.)

    --
    ┌── dg*cowlark.com ─── http://www.cowlark.com ──────────────── ──

    │ "There does not now, nor will there ever, exist a programming language in
    │ which it is the least bit hard to write bad programs." --- Flon's Axiom

  4. Re: BDOS/BIOS problem


    That being the case, then there should be no situation where data gets
    copied
    into the buffer without your code first checking to see if the buffer
    contains
    data that needs to get flushed. i.e.:

    Hi David
    Thanks for your reply. Also apologies to Andrew for somehow getting into
    his thread. I sent my query to the group as a new thread.

    As I see it there could be a situation where a block of 128bytes is written
    into the buffer (which already contains say 128 bytes) this would make the
    512-byte buffer half full - therefore code checks it - no need to flush it
    yet. But this 128byte block is the final one.

    to write sector n:
    is n == oldn?
    copy data into buffer at (n % 4)
    return
    else
    does buffer need flushing?
    write buffer to disk at block (oldn / 4)
    set oldn = n
    reload buffer (n / 4) from disk
    copy data into buffer at (n % 4)
    return

    Sorry I do not fully understand this code (looks like C) as I tend to use
    Z80 machine code.
    Alan




  5. Re: BDOS/BIOS problem

    On Sun, 09 Sep 2007 16:20:03 GMT, "Alan"
    wrote:

    >Hi CP/M users
    >For some time now I have been trying to use a Compact Flash as a hard drive
    >with my Interak computer, using CP/M 2.2
    >The hardware works fine i.e. I can read and write data to the CF. The
    >difficult bit has been integrating the CP/M file system to accept the CF as
    >part of the computer working with the 2 floppy disks.
    >Initially I could not even get a C: prompt to appear on the screen. At
    >present I can now store files on the CF and use NewSweep to copy files from
    >floppy to CF and back again. Also disk utility programs can read sectors and
    >tracks OK.
    >
    >The problem I have been struggling with for some time is - When a file is
    >written to disk the de-blocking works until the end of the file. At this
    >point any data in the host buffer ( which holds 512 bytes - 4 x 128 byte
    >CP/M sectors ) is lost when the directory entry is written.
    >When reading flies the correct size is read back which means that the last
    >128 to 512 bytes are corrupt. Depending where the file has ended either
    >1,2,3 or 4 blocks of 128 bytes are still in the host buffer.
    >It seems that while writing a file the first indication that the end-of-file
    >has occurred is when a write type 1 is sent (write to directory). At this
    >point the directory has already been copied into the host buffer -
    >overwriting the final data.
    >
    >I have several CP/M books but I cannot find sufficient information regarding
    >this.
    >One method, which I think may be the best way to go, is to read File Control
    >Block 2 before BDOS starts writing. This has record size information which
    >could be compared to the current record as it is written. Then when a match
    >occurs the host buffer could be immediately written out to the CF.
    >
    >Am I on the right track here ? Or is there a better way to go ?
    >
    >Any help greatly appreciated, Alan



    First are you reading Andy Johnson-Laird, The programmer CP/M
    handbook? IF not get it..

    CP/M provides a few signals of when to write a block (flush the cache)
    and when not to. These are available to the bios and should be used.
    If you using an application to read and write media (any!) then it's
    the responseability of that program to manage caching and writing
    as it's likely not doing BDOS calls for that (direct BIOS IO).

    If all else fails and yo want it to run failsafe but slow every write
    has a read and you then lay in the logical sector to the physical
    sector buffer and write it back. Not elegant but it always works
    and when to write is implied always.

    Allison

  6. Re: BDOS/BIOS problem

    On Sun, 09 Sep 2007 16:51:03 GMT, "Alan"
    wrote:

    >
    >That being the case, then there should be no situation where data gets
    >copied
    >into the buffer without your code first checking to see if the buffer
    >contains
    >data that needs to get flushed. i.e.:
    >
    >Hi David
    >Thanks for your reply. Also apologies to Andrew for somehow getting into
    >his thread. I sent my query to the group as a new thread.
    >
    >As I see it there could be a situation where a block of 128bytes is written
    >into the buffer (which already contains say 128 bytes) this would make the
    >512-byte buffer half full - therefore code checks it - no need to flush it
    >yet. But this 128byte block is the final one.


    At the BIOS level...

    CP/M has three cases for write:
    1) Write to directory (always pre read and write back for directory
    security)
    2) Write to allocated block (pre read is required as there may be
    data that that is to be preserved.
    3) Write to unallocated block (no data has been stored in this block
    that needs to be saved. usually this is the write to the first
    logical sector of an allocation block.

    When I say pre-read I mean the BIOS must track if the current logical
    sector to be written is in the physical sector buffer. The BDOS when
    it calls the BIOS provides information on what of the three cases it
    will be for write. Generally when deblocking is in the BIOS you
    have a lot of information to let you know when two write back...

    Are you writing a different physical sector?
    Are you writing to a different track than last time?
    Are you reading the directory? {non obvious but every time
    a directory read occurs you are changing track and sector
    and likely need to write back the buffers before doing the
    directory read.

    This is covered in the CP/M alteration guide section 6.12 "Sector
    Blocking and deblocking". It's light on the "howto" but the core
    infor is there. AJL The Programmers CP/M handbook coveres
    it better and provides example code.

    Biggest thing is when you read or write is keeping track of when
    to write back if you are not doing the slower read-modify-write
    always. If removeable media is used then you have to insure
    buffers are written before removing or very bad things happen.

    >
    >to write sector n:
    > is n == oldn?
    > copy data into buffer at (n % 4)
    > return
    > else
    > does buffer need flushing?
    > write buffer to disk at block (oldn / 4)
    > set oldn = n
    > reload buffer (n / 4) from disk
    > copy data into buffer at (n % 4)
    > return
    >
    >Sorry I do not fully understand this code (looks like C) as I tend to use
    >Z80 machine code.


    Looks like pseudo code, sort of plain language for procedural
    description.

    Allison



  7. Re: BDOS/BIOS problem

    On 2007-09-09, Alan wrote:
    > It seems that while writing a file the first indication that the end-of-file
    > has occurred is when a write type 1 is sent (write to directory). At this
    > point the directory has already been copied into the host buffer -
    > overwriting the final data.


    I was under the impression that CP/M always called HOME before manipulating
    the directory, but I sure don't see it in the manual (which probably
    means I'm mistaken again).
    --
    roger ivie
    rivie@ridgenet.net

  8. Re: BDOS/BIOS problem

    On Sun, 09 Sep 2007 22:02:41 GMT, Roger Ivie
    wrote:

    >On 2007-09-09, Alan wrote:
    >> It seems that while writing a file the first indication that the end-of-file
    >> has occurred is when a write type 1 is sent (write to directory). At this
    >> point the directory has already been copied into the host buffer -
    >> overwriting the final data.

    >
    >I was under the impression that CP/M always called HOME before manipulating
    >the directory, but I sure don't see it in the manual (which probably
    >means I'm mistaken again).


    Generally thats true save for the directory is not at track 0000 and
    most system use that to load a zero into the storage location called
    track. Otherwise in most moden (post V1.4) home is a do mostly
    nothing.

    Allison


  9. Re: BDOS/BIOS problem

    Alan wrote:
    [...]
    > As I see it there could be a situation where a block of 128bytes is written
    > into the buffer (which already contains say 128 bytes) this would make the
    > 512-byte buffer half full - therefore code checks it - no need to flu****
    > yet. But this 128byte block is the final one.


    Sure, but it doesn't know that until the *next* write comes along --- which is
    to a completely different disk block. Therefore your code knows that it needs
    to flush the block out to disk before it can copy the next sector into the buffer.

    > Sorry I do not fully understand this code (looks like C) as I tend to use
    > Z80 machine code.


    It's pseudocode --- just a plain English description of an algorithm.

    --
    ┌── dg*cowlark.com ─── http://www.cowlark.com ──────────────── ──

    │ "There does not now, nor will there ever, exist a programming language in
    │ which it is the least bit hard to write bad programs." --- Flon's Axiom

  10. Re: BDOS/BIOS problem

    On Mon, 10 Sep 2007 21:07:41 +0100, David Given
    wrote:

    >Alan wrote:
    >[...]
    >> As I see it there could be a situation where a block of 128bytes is written
    >> into the buffer (which already contains say 128 bytes) this would make the
    >> 512-byte buffer half full - therefore code checks it - no need to flush it
    >> yet. But this 128byte block is the final one.

    >
    >Sure, but it doesn't know that until the *next* write comes along --- which is
    >to a completely different disk block. Therefore your code knows that it needs
    >to flush the block out to disk before it can copy the next sector into the buffer.


    There are two cases. User application that does direct bios calls to
    read/write disk and BDOS calls. The first case the user must take
    care to not forget to flush the buffer. For the second case a
    properly written BIOS gets signals from the BDOS to tell it to preread
    and also when to write..

    I've been using various floppies and disks for years with sector
    deblocking without issues and it really makes no difference what the
    media really is only that the code is correct. Despite claims about a
    deblocking bug I've used the DRI code with no problems though I prefer
    my own as it's easier for me to understand.

    Allison

    >> Sorry I do not fully understand this code (looks like C) as I tend to use
    >> Z80 machine code.

    >
    >It's pseudocode --- just a plain English description of an algorithm.



  11. Re: BDOS/BIOS problem

    Hi Thanks for the mass of good information. I will sift carefully through
    it.
    The floppy drives are using 512-byte sectors (same as the CF) so this helps.
    But I am keeping individual host buffers and de-blocking for floppy's and CF
    as I assumed that this would be necessary when copying files.

    I have a copy of the Programmers CP/M Handbook which has been very useful
    and I have also 'borrowed' some of John Baker's code which was written for
    an IDE interface.
    I realise that there are quite a few things to check when reading or writing
    takes place but I did not want to slow the process down too much with
    unnecessary code.

    >If all else fails and yo want it to run failsafe but slow every write
    >has a read and you then lay in the logical sector to the physical
    >sector buffer and write it back. Not elegant but it always works
    >and when to write is implied always.


    >Allison


    I guess this would mean writing the 512-byte sector to disk every time the
    BDOS sends a 128-byte sector. As the Compact Flash is fast (the slowest
    point here is reading/writing the floppies) would this be noticable? Also
    would it notice much when writing files directly to the CF?
    I like the idea of being failsafe.

    Thanks, Alan




  12. Re: BDOS/BIOS problem

    On Tue, 11 Sep 2007 09:17:44 GMT, "Alan"
    wrote:

    >Hi Thanks for the mass of good information. I will sift carefully through
    >it.
    >The floppy drives are using 512-byte sectors (same as the CF) so this helps.
    >But I am keeping individual host buffers and de-blocking for floppy's and CF
    >as I assumed that this would be necessary when copying files.
    >
    >I have a copy of the Programmers CP/M Handbook which has been very useful
    >and I have also 'borrowed' some of John Baker's code which was written for
    >an IDE interface.
    >I realise that there are quite a few things to check when reading or writing
    >takes place but I did not want to slow the process down too much with
    >unnecessary code.


    the amouunt of code to do all that is actually small. Additional
    buffers take space though but again thats not all that much.
    FYI: multiple buffers can be faster!

    >>If all else fails and yo want it to run failsafe but slow every write
    >>has a read and you then lay in the logical sector to the physical
    >>sector buffer and write it back. Not elegant but it always works
    >>and when to write is implied always.

    >
    >>Allison

    >
    >I guess this would mean writing the 512-byte sector to disk every time the
    >BDOS sends a 128-byte sector. As the Compact Flash is fast (the slowest
    >point here is reading/writing the floppies) would this be noticable? Also
    >would it notice much when writing files directly to the CF?
    >I like the idea of being failsafe.


    For CF I'd not do that only because it has a limited (though large)
    number of write cycles. The real issue is the file close problem,
    that is insuring the file is closed and buffers are flushed. There
    may be a secondary "what if the power fails" issue but that is a
    call for a UPS.

    Allison

    >
    >Thanks, Alan
    >
    >



  13. Re: BDOS/BIOS problem

    no.spam@no.uce.bellatlantic.net wrote:

    (snip)

    > For CF I'd not do that only because it has a limited (though large)
    > number of write cycles. The real issue is the file close problem,
    > that is insuring the file is closed and buffers are flushed. There
    > may be a secondary "what if the power fails" issue but that is a
    > call for a UPS.


    I wonder how many have IDE drives on their CP/M-80 systems.
    It seems that while IDE originally included the option of
    eight bit I/O, many drives only implement 16 bit I/O.
    This requires extra latches for eight bit processors.

    I now wonder if CF, which has a similar interface to IDE,
    has non-optional eight bit I/O.

    I am considering building an 8080 system that should be
    able to run CP/M, and the IDE interface is fairly simple.

    -- glen


  14. Re: BDOS/BIOS problem

    On 2007-09-11, glen herrmannsfeldt wrote:
    > I now wonder if CF, which has a similar interface to IDE,
    > has non-optional eight bit I/O.


    CF does, indeed, have eight bit I/O. It's often used in embedded
    applications, where the 8-bit interface is handy.

    The CF specs are quickly and easily available on-line.
    --
    roger ivie
    rivie@ridgenet.net

  15. Re: BDOS/BIOS problem

    On Tue, 11 Sep 2007 19:50:48 GMT, Roger Ivie
    wrote:

    >On 2007-09-11, glen herrmannsfeldt wrote:
    >> I now wonder if CF, which has a similar interface to IDE,
    >> has non-optional eight bit I/O.

    >
    >CF does, indeed, have eight bit I/O. It's often used in embedded
    >applications, where the 8-bit interface is handy.
    >
    >The CF specs are quickly and easily available on-line.


    True, and handy too.

    However, being one that enjoys a good crufty hack. A IDE hard disk
    does not require a 16 bit interface. There are caveats like you loose
    half the storage and then it's only 256bytes/sector. The answer is
    simply, yes. But the interface as a result is so minimal its easier
    than talking to an 8255 PPI! As to the storage loss, even a 40MB
    drive is plenty huge even if only 20mb is "used". When you consider
    the likely size fo easy to find older sub 1GB drives is around
    400-500mb the unused space is a good trade.

    For the truely insane you can do the above and even skip deblocking
    and only use the first 128bytes of a sector. Horror of horrors!
    Wasting 3/4 of the drives storage capacity. Well with 1GB drives
    that remaining 256MB is still over the upper limit (128mb) for sixteen
    8MB logical CP/M 2.2 drives A-P. CP/M isn't PC and code bloat
    doesn't live there so even a few 8MB logical drives is luxury.

    For those that do not have a few CF drives, or the nasty tiny
    connectors to fit and none of those IDE to CF adaptors the IDE
    hard disk is easy to live with and can be dirt simple. As to the
    power difference, last I checked most of my CP/M systems have
    power sources that are well oversized and have enough excess
    +12 and +5 for even the oldest IDE hard disks and the later ones
    or the smaller 2.5" ones are remarkably low power.

    Just a few ideas!

    Allison


  16. Re: BDOS/BIOS problem

    Roger Ivie wrote:
    > On 2007-09-11, glen herrmannsfeldt wrote:


    >>I now wonder if CF, which has a similar interface to IDE,
    >>has non-optional eight bit I/O.


    > CF does, indeed, have eight bit I/O. It's often used in embedded
    > applications, where the 8-bit interface is handy.


    > The CF specs are quickly and easily available on-line.


    I have already downloaded some version, but I wanted to get a
    discussion started. (or continue the existing one)

    As I understand it, IDE also has an eight bit interface, but
    most don't implement it. Maybe the early drives (that one might
    want to attach to a PC/XT) did.

    I am following the discussion on reblocking. I remember that SCSI
    allows one to reformat a disk for a different block size. I don't
    remember if IDE has that option.

    I have ordered the previously mentioned CP/M book but it
    hasn't come yet.

    thanks for all the discussions,

    -- glen


  17. Re: BDOS/BIOS problem

    On Tue, 11 Sep 2007 14:47:01 -0800, glen herrmannsfeldt
    wrote:

    >Roger Ivie wrote:
    >> On 2007-09-11, glen herrmannsfeldt wrote:

    >
    >>>I now wonder if CF, which has a similar interface to IDE,
    >>>has non-optional eight bit I/O.

    >
    >> CF does, indeed, have eight bit I/O. It's often used in embedded
    >> applications, where the 8-bit interface is handy.

    >
    >> The CF specs are quickly and easily available on-line.

    >
    >I have already downloaded some version, but I wanted to get a
    >discussion started. (or continue the existing one)
    >
    >As I understand it, IDE also has an eight bit interface, but
    >most don't implement it. Maybe the early drives (that one might
    >want to attach to a PC/XT) did.


    I have a few really old IDE as in 10mb with steppers and non do 8bit.

    >I am following the discussion on reblocking. I remember that SCSI
    >allows one to reformat a disk for a different block size. I don't
    >remember if IDE has that option.


    None of the IDE specs I've seen allow that.


    Allison


  18. Re: BDOS/BIOS problem

    On 2007-09-11, no.spam@no.uce.bellatlantic.net wrote:
    > On Tue, 11 Sep 2007 14:47:01 -0800, glen herrmannsfeldt
    >>As I understand it, IDE also has an eight bit interface, but
    >>most don't implement it. Maybe the early drives (that one might
    >>want to attach to a PC/XT) did.

    >
    > I have a few really old IDE as in 10mb with steppers and non do 8bit.


    I've seen a grand total of one drive that supports 8 bit IDE. It was a
    40MB drive made by, IIRC, WD.

    >>I am following the discussion on reblocking. I remember that SCSI
    >>allows one to reformat a disk for a different block size. I don't
    >>remember if IDE has that option.

    >
    > None of the IDE specs I've seen allow that.


    The IDE spec original required the bits in the 1010 register file that
    specify sector size to be fixed at the 512 byte code. Since then, I
    think at least one bit has been taken over for one uses; IIRC, one
    of them switches the addresses from CHS to LBA.

    I once wrote the firmware for an AT-compatible disk controller, so I
    used to know this stuff cold. The bits, they leak away over time...
    --
    roger ivie
    rivie@ridgenet.net

  19. Re: BDOS/BIOS problem

    On Sep 11, 7:41 pm, Roger Ivie wrote:
    > On 2007-09-11, no.s...@no.uce.bellatlantic.net wrote:
    >
    > > On Tue, 11 Sep 2007 14:47:01 -0800, glen herrmannsfeldt
    > >>As I understand it, IDE also has an eight bit interface, but
    > >>most don't implement it. Maybe the early drives (that one might
    > >>want to attach to a PC/XT) did.

    >
    > > I have a few really old IDE as in 10mb with steppers and non do 8bit.

    >
    > I've seen a grand total of one drive that supports 8 bit IDE. It was a
    > 40MB drive made by, IIRC, WD.
    >

    I use an old laptop IDE in my xt clone, 256mb. The hard part to find
    was the controller. I think it came out of a Tandy.

    > >>I am following the discussion on reblocking. I remember that SCSI
    > >>allows one to reformat a disk for a different block size. I don't
    > >>remember if IDE has that option.

    >
    > > None of the IDE specs I've seen allow that.

    >
    > The IDE spec original required the bits in the 1010 register file that
    > specify sector size to be fixed at the 512 byte code. Since then, I
    > think at least one bit has been taken over for one uses; IIRC, one
    > of them switches the addresses from CHS to LBA.
    >
    > I once wrote the firmware for an AT-compatible disk controller, so I
    > used to know this stuff cold. The bits, they leak away over time...
    > --

    I hear that! Wasn't it the IDE that shouldn't be low level formatted?

    Steve

    > roger ivie
    > ri...@ridgenet.net




  20. Re: BDOS/BIOS problem

    On 2007-09-12, s_dubrovich@yahoo.com wrote:
    > I hear that! Wasn't it the IDE that shouldn't be low level formatted?


    Indeed. IDE and anything newer. Somewhere I have an old TURBO Pascal
    program I wrote to format an AT disk at 2:1 interleave. It does,
    of course, assume MFM 'cause it was written back in the day...
    --
    roger ivie
    rivie@ridgenet.net

+ Reply to Thread
Page 1 of 2 1 2 LastLast