CP/M filesystem quesions - CP/M

This is a discussion on CP/M filesystem quesions - CP/M ; I've been doing some data recovery from CDOS (CP/M compatible) disks for someone, and in the process I've written a utility to extract files from ImageDisk CP/M images - Now I'm looking into expanding this into a general purpose utility ...

+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 26

Thread: CP/M filesystem quesions

  1. CP/M filesystem quesions

    I've been doing some data recovery from CDOS (CP/M compatible)
    disks for someone, and in the process I've written a utility to
    extract files from ImageDisk CP/M images - Now I'm looking into
    expanding this into a general purpose utility to read or write
    files in any (at least most) CP/M images, and a few questions
    have come up:

    1) Byte 0 in directory entry:
    This contains E5 for unallocated entries, or the "user
    number" on allocated entry.
    - Do any versions of CP/M use more than 16 user numbers?
    - I note that Cromemco/CDOS uses a value of 81 for a volume
    label:
    - Is this standard (or semi-standard) in CP/M?
    - Are there any other "special" values?

    2) Does CP/M rearrange directory entries so that extents are
    in ascending order?
    - In my program I have taken the safe approach and will find
    extents no matter what order they occur in, however I want
    to know if when writing the disk I should insure that the
    extent entries are in order (ie: if I allocate a directory
    entry occuring physically BEFORE an entry contining a lower
    extent number, will CP/M have problems)?

    3) So far, I've needed to define the following information to
    access a CP/M disk:
    - Size of physical sector
    - Mapping of logical sectors to physical sectors
    (ie: Cylinder/Head mapping and sector skew)
    - Number of reserved (system) tracks (offset to user area)
    - Number of 128 byte records in an allocation unit
    - Number of directory entries
    I can get physical sector size and some mapping information
    (# sides, # sectors/track etc.) from the .IMD file, however
    other mapping information and the filesystem parameters must
    be configured independantly...
    - Can any of this information be reliably (or even unreliably)
    gleaned from the content of the disk?

    Dave


    --
    dave06a@ Low-cost firmware development tools: www.dunfield.com
    dunfield. Classic computer collection: www.classiccmp.org/dunfield
    com Some stuff I have for sale: www.dunfield.com/sale


  2. Re: CP/M filesystem quesions

    On Sat, 13 Oct 2007 10:18:26 +0000, Dave Dunfield wrote:

    > I've been doing some data recovery from CDOS (CP/M compatible)
    > disks for someone, and in the process I've written a utility to
    > extract files from ImageDisk CP/M images - Now I'm looking into
    > expanding this into a general purpose utility to read or write
    > files in any (at least most) CP/M images, and a few questions
    > have come up:


    Do you know about Michael's cpmtools?
    http://www.moria.de/~michael/cpmtools/

    Most of your questions probably can be answered by reading the source
    of this tools.

    ....
    > I can get physical sector size and some mapping information
    > (# sides, # sectors/track etc.) from the .IMD file, however
    > other mapping information and the filesystem parameters must
    > be configured independantly...
    > - Can any of this information be reliably (or even unreliably)
    > gleaned from the content of the disk?


    No, one could make a guess from the size of the disk images.
    Instead of making this unreliable cpmtools use a disk definition file and
    the definitions look like this:

    diskdef ibm-3740
    seclen 128
    tracks 77
    sectrk 26
    blocksize 1024
    maxdir 64
    skew 6
    boottrk 2
    os p2dos
    end

    diskdef hd
    seclen 128
    tracks 255
    sectrk 128
    blocksize 2048
    maxdir 1024
    skew 0
    boottrk 0
    os 2.2
    end

    Udo Munk
    --
    The real fun is building it and then using it...


  3. Re: CP/M filesystem quesions

    I've been following you CDOS saga.

    As far as I know, the 1st byte in CP/M is either E5 or 0. It only takes
    on a "user" value in memory, not on the disk (I think).

    CP/M has no standard volume label.

    CP/M does not rearrange directory entries for extents to always and
    necessarily be in ascending order. They usually are, but it's not a
    requirement, and it's entirely legitimate for them to be non-ascending.



    Dave Dunfield wrote:
    > I've been doing some data recovery from CDOS (CP/M compatible)
    > disks for someone, and in the process I've written a utility to
    > extract files from ImageDisk CP/M images - Now I'm looking into
    > expanding this into a general purpose utility to read or write
    > files in any (at least most) CP/M images, and a few questions
    > have come up:
    >
    > 1) Byte 0 in directory entry:
    > This contains E5 for unallocated entries, or the "user
    > number" on allocated entry.
    > - Do any versions of CP/M use more than 16 user numbers?
    > - I note that Cromemco/CDOS uses a value of 81 for a volume
    > label:
    > - Is this standard (or semi-standard) in CP/M?
    > - Are there any other "special" values?
    >
    > 2) Does CP/M rearrange directory entries so that extents are
    > in ascending order?
    > - In my program I have taken the safe approach and will find
    > extents no matter what order they occur in, however I want
    > to know if when writing the disk I should insure that the
    > extent entries are in order (ie: if I allocate a directory
    > entry occuring physically BEFORE an entry contining a lower
    > extent number, will CP/M have problems)?
    >
    > 3) So far, I've needed to define the following information to
    > access a CP/M disk:
    > - Size of physical sector
    > - Mapping of logical sectors to physical sectors
    > (ie: Cylinder/Head mapping and sector skew)
    > - Number of reserved (system) tracks (offset to user area)
    > - Number of 128 byte records in an allocation unit
    > - Number of directory entries
    > I can get physical sector size and some mapping information
    > (# sides, # sectors/track etc.) from the .IMD file, however
    > other mapping information and the filesystem parameters must
    > be configured independantly...
    > - Can any of this information be reliably (or even unreliably)
    > gleaned from the content of the disk?
    >
    > Dave
    >
    >
    > --
    > dave06a@ Low-cost firmware development tools: www.dunfield.com
    > dunfield. Classic computer collection: www.classiccmp.org/dunfield
    > com Some stuff I have for sale: www.dunfield.com/sale
    >


  4. Re: CP/M filesystem quesions

    On Sat, 13 Oct 2007 16:57:14 -0400, Barry Watzman wrote:

    > As far as I know, the 1st byte in CP/M is either E5 or 0. It only takes
    > on a "user" value in memory, not on the disk (I think).


    No on disk. If you put files into various user areas, a dir command will
    only show the files for the current user, so it has to know the user the
    files belong to.

    Udo Munk
    --
    The real fun is building it and then using it...


  5. Re: CP/M filesystem quesions

    [I haven't seen the original post, so am piggybacking. And have rearranged
    into sane order, and trimmed quotes.]

    Barry Watzman wrote:
    : Dave Dunfield wrote:
    :> - Do any versions of CP/M use more than 16 user numbers?

    Some Z80 BDOSes for CP/M 2 allowed 32 users.

    :> - I note that Cromemco/CDOS uses a value of 81 for a volume
    :> label:
    :> - Is this standard (or semi-standard) in CP/M?

    No.

    :> - Are there any other "special" values?

    Under CP/M Plus, 10h-1Fh are for passwords, 20h is for the volume label
    and 21h is for timestamps.

    :> 2) Does CP/M rearrange directory entries so that extents are
    :> in ascending order?

    No.

    :> 3) So far, I've needed to define the following information to
    :> access a CP/M disk:
    :> - Can any of this information be reliably (or even unreliably)
    :> gleaned from the content of the disk?

    No. There is no standard superblock; a few BIOSes stored filesystem
    parameters in the boot sector (with no two doing it the same way, of course),
    but most just hardcoded the parameters they required.

    : As far as I know, the 1st byte in CP/M is either E5 or 0. It only takes
    : on a "user" value in memory, not on the disk (I think).

    Wrong. It's an easy test to do with cpmtools:

    % mkfs.cpm -f cf2dd tmp.bin
    % cpmcp -f cf2dd tmp.bin zx.bdf 2:
    % hexedit tmp.bin

    [many lines omitted]
    00001200 20 55 4E 4C 41 42 45 4C 45 44 20 20 11 00 00 00 UNLABELED ....
    00001210 00 00 00 00 00 00 00 00 7E 2A 23 26 7E 2A 23 26 ........~*#&~*#&
    00001220 02 5A 58 20 20 20 20 20 20 42 44 46 00 00 00 80 .ZX BDF....
    00001230 04 00 05 00 06 00 07 00 08 00 09 00 0A 00 0B 00 ................
    00001240 02 5A 58 20 20 20 20 20 20 42 44 46 01 2F 00 53 .ZX BDF./.S
    00001250 0C 00 0D 00 0E 00 0F 00 10 00 11 00 00 00 00 00 ................
    00001260 21 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 7E 2A 23 26 7E !..........~*#&~
    00001270 2A 23 26 E5 E5 7E 2A 23 26 7E 2A 23 26 E5 E5 E5 *#&..~*#&~*#&...
    [many more lines omitted]

    : CP/M has no standard volume label.

    CP/M Plus does. It's a directory entry with the first byte set to 20h. See
    the first entry in the above hex dump.

    --
    John Elliott

  6. Re: CP/M filesystem quesions

    On Sat, 13 Oct 2007 10:18:26 GMT,
    Dave.Dunfield@use.techsupport.link.on.my.website (Dave Dunfield)
    wrote:

    >I've been doing some data recovery from CDOS (CP/M compatible)
    >disks for someone, and in the process I've written a utility to
    >extract files from ImageDisk CP/M images - Now I'm looking into
    >expanding this into a general purpose utility to read or write
    >files in any (at least most) CP/M images, and a few questions
    >have come up:
    >
    >1) Byte 0 in directory entry:
    > This contains E5 for unallocated entries, or the "user
    > number" on allocated entry.
    > - Do any versions of CP/M use more than 16 user numbers?


    Zcpr some versions allowed 32 The OS really doesn't seem to
    care.

    > - I note that Cromemco/CDOS uses a value of 81 for a volume
    > label:


    CDOS is not CP/M, similar though.

    > - Is this standard (or semi-standard) in CP/M?


    E5h is an erased disk or file, the only standard for CP/M.

    > - Are there any other "special" values?


    I've seen lower values there but teh OS only looks for E5 or
    not E5 (erased or not) until user numbers are factored in then
    there are a range of unused values possible.

    This is an area where CP/M expect only a few things and
    does not force boundary conditions. STAT, USR and PIP
    however only know 16 users but that limit is in the utilities code
    and not an BDOS enforced limit.

    >2) Does CP/M rearrange directory entries so that extents are
    > in ascending order?


    Generally no. however if you do a file copy disk to disk
    or say A>pip foo1=foo then it will _if_ the disk is not too
    fragmented. If you can generalize CP/M will do things
    sequentually if possible. Obvious ly if the disk is fairly
    full and had many adds and dels fragmentation will
    nominally be the case for extents and files.

    > - In my program I have taken the safe approach and will find
    > extents no matter what order they occur in, however I want
    > to know if when writing the disk I should insure that the
    > extent entries are in order (ie: if I allocate a directory
    > entry occuring physically BEFORE an entry contining a lower
    > extent number, will CP/M have problems)?


    IF image then GIGO. IF Media to file back to media then
    no negative effect. If you write out a file directory as sequential
    and correctly allocated space then the files will also likely be
    sequential on the disk as a matter of course.

    I and done this when creating (ep)ROM disks where I create
    the directory and lay in files then put the files sequentially
    on the files area of the rom.

    One positive effect is any time CP/M has everything sequential
    it's a bit faster as the search is pure sequential. (note: may if not
    all of the improved CP/M work alikes generally has the directory for
    speed.).

    >3) So far, I've needed to define the following information to
    > access a CP/M disk:
    > - Size of physical sector


    unknown to CP/M as BIOS deblock code deals with this.

    > - Mapping of logical sectors to physical sectors
    > (ie: Cylinder/Head mapping and sector skew)


    This is in the BIOS in the form of DPB and DPH and not stored
    on media.

    > - Number of reserved (system) tracks (offset to user area)


    The content of the disk tables tell the BDOS.

    > - Number of 128 byte records in an allocation unit


    DPH, DPB

    > - Number of directory entries


    DPH, DPB

    > I can get physical sector size and some mapping information
    > (# sides, # sectors/track etc.) from the .IMD file, however
    > other mapping information and the filesystem parameters must
    > be configured independantly...
    > - Can any of this information be reliably (or even unreliably)
    > gleaned from the content of the disk?


    Guessed at yes, accuracy poor. Anything I pointed to DPH and DPB
    is from the sperspective of looking at unknown disk hidden as it's in
    the source systems BIOS.

    You may get lukcy and have a copy of the bios on the disk (system
    tracks) but you still need to know in advance how to read them.
    If you do the DPH, DPB are findable as the SELDISK call will
    return the headers that have pointers to the tables.

    Allison

  7. Re: CP/M filesystem quesions


    On Oct 13, 4:18 am, Dave.Dunfi...@use.techsupport.link.on.my.website
    (Dave Dunfield) wrote:
    > I've been doing some data recovery from CDOS (CP/M compatible)
    > disks for someone, and in the process I've written a utility to
    > extract files from ImageDisk CP/M images - Now I'm looking into
    > expanding this into a general purpose utility to read or write
    > files in any (at least most) CP/M images, and a few questions
    > have come up:
    >
    > 1) Byte 0 in directory entry:
    > This contains E5 for unallocated entries, or the "user
    > number" on allocated entry.
    > - Do any versions of CP/M use more than 16 user numbers?
    > - I note that Cromemco/CDOS uses a value of 81 for a volume
    > label:
    > - Is this standard (or semi-standard) in CP/M?
    > - Are there any other "special" values?
    >

    John's has covered this well. In addition, I've seen some diskette
    volume labeling utility use the first directory entry as a volume
    label by making a zero length file entry.

    > 2) Does CP/M rearrange directory entries so that extents are
    > in ascending order?


    No rearranging, however the extents will be in ascending order until a
    file delete operation. After which, a reclaiming of a deleted fcb
    could change the ascending order. In other words, an out of order
    diskette would be put into ascending order when wildcard filecopying
    to a freshly formatted diskette in the other drive.
    The directory scan fn 17 'search for first directory entry for
    filespec' finds for the first extent. fn 17 must be called first as it
    takes an offset parameter of a fcb to match for. fn 18 'seach for
    next' is used for subsequent search efforts. If the fcb is setup to
    wildcard match, i.e. : ? ???????? ??? ? , then every directory entry
    is a match. ( I've added spaces for readability ) , curiously, the
    last '?' is the extent field, so can one search directly for an extent
    of a file?

    > - In my program I have taken the safe approach and will find
    > extents no matter what order they occur in, however I want
    > to know if when writing the disk I should insure that the
    > extent entries are in order (ie: if I allocate a directory
    > entry occuring physically BEFORE an entry contining a lower
    > extent number, will CP/M have problems)?
    >

    No, CP/M should find it. As far as extents go, be aware that one
    directory entry can represent more than one extent of 16k, depending
    on the block storage size.

    > 3) So far, I've needed to define the following information to
    > access a CP/M disk:
    > - Size of physical sector
    > - Mapping of logical sectors to physical sectors
    > (ie: Cylinder/Head mapping and sector skew)
    > - Number of reserved (system) tracks (offset to user area)
    > - Number of 128 byte records in an allocation unit
    > - Number of directory entries
    > I can get physical sector size and some mapping information
    > (# sides, # sectors/track etc.) from the .IMD file, however
    > other mapping information and the filesystem parameters must
    > be configured independantly...
    > - Can any of this information be reliably (or even unreliably)
    > gleaned from the content of the disk?


    The only constant is that the logical storage Block Zero holds the
    beginning of the Directory. Everything splinters from there, the
    directory can be, often is, more than one block in length. I imagine
    finding the first sector holding directory records would begin the
    process of deconstructing a disk format.

    Steve
    >
    > Dave
    >
    > --
    > dave06a@ Low-cost firmware development tools: www.dunfield.com
    > dunfield. Classic computer collection: www.classiccmp.org/dunfield
    > com Some stuff I have for sale: www.dunfield.com/sale




  8. Re: CP/M filesystem quesions

    Barry Watzman wrote: *** and top-posted - fixed ***
    > Dave Dunfield wrote:
    >
    >> I've been doing some data recovery from CDOS (CP/M compatible)
    >> disks for someone, and in the process I've written a utility to
    >> extract files from ImageDisk CP/M images - Now I'm looking into
    >> expanding this into a general purpose utility to read or write
    >> files in any (at least most) CP/M images, and a few questions
    >> have come up:
    >>
    >> 1) Byte 0 in directory entry:
    >> This contains E5 for unallocated entries, or the "user
    >> number" on allocated entry.
    >> - Do any versions of CP/M use more than 16 user numbers?
    >> - I note that Cromemco/CDOS uses a value of 81 for a volume
    >> label:
    >> - Is this standard (or semi-standard) in CP/M?
    >> - Are there any other "special" values?
    >>

    .... snip ...
    >
    > I've been following you CDOS saga.
    >
    > As far as I know, the 1st byte in CP/M is either E5 or 0. It only
    > takes on a "user" value in memory, not on the disk (I think).


    As Mr. Dunfield mentioned, that first byte is used to hold the user
    number, shifted into the high order bits, in the directory. You
    can find the source for a complete treatment in the source files
    for DOSPLUS, at:



    --
    Chuck F (cbfalconer at maineline dot net)
    Available for consulting/temporary embedded and systems.



    --
    Posted via a free Usenet account from http://www.teranews.com


  9. Re: CP/M filesystem quesions

    >> I've been doing some data recovery from CDOS (CP/M compatible)
    >> disks for someone, and in the process I've written a utility to
    >> extract files from ImageDisk CP/M images - Now I'm looking into
    >> expanding this into a general purpose utility to read or write
    >> files in any (at least most) CP/M images, and a few questions
    >> have come up:



    From Udo:

    >Do you know about Michael's cpmtools?
    >http://www.moria.de/~michael/cpmtools/
    >
    >Most of your questions probably can be answered by reading the source
    >of this tools.


    I've heard of them, however I've not looked into them. I don't really
    want to clone someone elses code. In this case I already have my read
    code working to suit my purposes, and a few specific questions came up
    in thinking about updating it to write the disks - I was hoping that
    some of the very knowlegable people here might have the answers rather
    than spend hours wading through someone elses code in the hopes that
    I can deduce the information.


    >> - Can any of this information be reliably (or even unreliably)
    >> gleaned from the content of the disk?

    >
    >No, one could make a guess from the size of the disk images.
    >Instead of making this unreliable cpmtools use a disk definition file and
    >the definitions look like this:
    >
    >diskdef ibm-3740
    > seclen 128
    > tracks 77
    > sectrk 26
    > blocksize 1024
    > maxdir 64
    > skew 6
    > boottrk 2
    > os p2dos
    >end
    >
    >diskdef hd
    > seclen 128
    > tracks 255
    > sectrk 128
    > blocksize 2048
    > maxdir 1024
    > skew 0
    > boottrk 0
    > os 2.2
    >end


    Thats basically what I am doing, although the above is missing some
    critical information (8 or 16 bit block numbers, # reserved blocks,
    handling of disks with side1 a logical extension of side2 [kaypro
    etc.]... I expect I'll have to add to mine as "new" twists unfold.


    From Barry:

    >As far as I know, the 1st byte in CP/M is either E5 or 0. It only takes
    >on a "user" value in memory, not on the disk (I think).


    No, files are assigned to user number - as far as I can tell, the
    first byte of the entry contains E5 if unused, otherwise the user
    number and what appears to be system-specific special values.

    >CP/M has no standard volume label.


    Thanks - What I'm seeing must be a Cromemco extension.

    >CP/M does not rearrange directory entries for extents to always and
    >necessarily be in ascending order. They usually are, but it's not a
    >requirement, and it's entirely legitimate for them to be non-ascending.


    Ok - good, what I will do is make new entries/extents sequential if
    possible, but I will allow non-sequential if I can't fit it without
    rearranging the disk,


    From Allison:

    >> - Do any versions of CP/M use more than 16 user numbers?

    >
    >Zcpr some versions allowed 32 The OS really doesn't seem to
    >care.


    Humm ... This could be tricky - someone else mentioned 10 as a
    volume label in another variation ... The thing I am trying to
    do is figure out which entries represent actual files. Here we
    already have a conflict - 10 could be an extended user number,
    or another special value.

    So far all the special values I've seen which don't point to
    files have had 00's in the blocks fields ( which would never
    be a valid file since 00 would point back to the directory),
    and indicate 0 records.
    I also note that 22disk does not have any configuration parameters
    for this ... So there must be a reliable way to tell.

    >> - I note that Cromemco/CDOS uses a value of 81 for a volume
    >> label:

    >
    >CDOS is not CP/M, similar though.


    From what I can tell, the file system is virtually identical.

    >> - Are there any other "special" values?

    >
    >I've seen lower values there but the OS only looks for E5 or
    >not E5 (erased or not) until user numbers are factored in then
    > there are a range of unused values possible.
    >
    >This is an area where CP/M expect only a few things and
    >does not force boundary conditions. STAT, USR and PIP
    >however only know 16 users but that limit is in the utilities code
    >and not an BDOS enforced limit.


    I think we are running into extensions here - special values that
    do not indicate files would have to be accomdated by the OS (and
    utilities).

    >> - In my program I have taken the safe approach and will find
    >> extents no matter what order they occur in, however I want
    >> to know if when writing the disk I should insure that the
    >> extent entries are in order (ie: if I allocate a directory
    >> entry occuring physically BEFORE an entry contining a lower
    >> extent number, will CP/M have problems)?

    >
    >IF image then GIGO. IF Media to file back to media then
    >no negative effect. If you write out a file directory as sequential
    >and correctly allocated space then the files will also likely be
    >sequential on the disk as a matter of course.


    I'm not reading the image sequentially (for file access purposes),
    I'm accessing it at the sector level like a disk, so in that respect it's
    media. I scan and update the directly exactly as you would for any
    file system on disk.

    I'm thinking specifically of cases where earlier files have been
    deleted, and you need to to use those extents when appending to
    another file occuring later on the disk - From what I understand,
    CP/M will not move the later entries "down" to keep them in
    sequential order, it will simply allocate the next extent from
    the earlier (and now free) position in the directory.

    >One positive effect is any time CP/M has everything sequential
    >it's a bit faster as the search is pure sequential. (note: may if not
    >all of the improved CP/M work alikes generally has the directory for
    >speed.).


    Agreed - If I were making this for daily use I might consider building
    in optimizations that CP/M didn't do (such as rearranging the directory
    for sequential extent access), however It's main use is to transfer
    files on/off CP/M systems via ImageDisk images, so I think "what CP/M
    does" is good enough.

    >>3) So far, I've needed to define the following information to
    >> access a CP/M disk:
    >> - Size of physical sector

    >
    >unknown to CP/M as BIOS deblock code deals with this.


    In my case, the information is available in the ImageDisk track
    header.


    >> - Mapping of logical sectors to physical sectors
    >> (ie: Cylinder/Head mapping and sector skew)

    >
    >This is in the BIOS in the form of DPB and DPH and not stored
    >on media.


    It's more than DPB/DPH - some disks are just laid out differently.
    (for example, some disks use side2 as a logical extension of
    side1 - sectors are marked as head0 and sector # increase on
    the other side - others use up all of side0 before going to
    side1) ... this is buried in the code of the BIOS and I don't
    think it can be determined other than a fixed configuration
    file.


    >> - Number of reserved (system) tracks (offset to user area)

    >
    >The content of the disk tables tell the BDOS.


    But .. is there any even remotely reliable way to find the disk
    tables on a disk of unknown origin - what I'm trying to do is
    find ways to make it easier to write a disk definition - ie: find
    as much information automatically as possible.

    >> - Number of 128 byte records in an allocation unit

    >DPH, DPB
    >> - Number of directory entries

    >DPH, DPB


    Yes, but same question - any way to locate the disk table? I
    don't think so - the bottom line is that this information is
    buried in the BIOS code, and is not at any specific location
    on the disk right? (in fact the BIOS code does not even have
    to be on the disk).

    >Guessed at yes, accuracy poor. Anything I pointed to DPH and DPB
    >is from the sperspective of looking at unknown disk hidden as it's in
    >the source systems BIOS.
    >
    >You may get lukcy and have a copy of the bios on the disk (system
    >tracks) but you still need to know in advance how to read them.
    >If you do the DPH, DPB are findable as the SELDISK call will
    >return the headers that have pointers to the tables.


    Yes, if you have a running system, this is not too hard to find,
    but creating disk definitions for disk images that you have
    without a working system is going to be tricky...

    Thanks to everyone for the help.

    Dave


    --
    dave06a@ Low-cost firmware development tools: www.dunfield.com
    dunfield. Classic computer collection: www.classiccmp.org/dunfield
    com Some stuff I have for sale: www.dunfield.com/sale


  10. Re: CP/M filesystem quesions

    Dave Dunfield wrote:
    (snip)

    > Yes, if you have a running system, this is not too hard to find,
    > but creating disk definitions for disk images that you have
    > without a working system is going to be tricky...


    Especially sector interleave.

    Many later systems that I know about write the sectors interleaved,
    but with the appropriate sector number in the physical sector header.

    CP/M, it seems, uses a lookup table in the BIOS, but doesn't change
    the headers on disk. If you can recognize directory blocks you
    might be able to figure out the interleave.

    -- glen


  11. Re: CP/M filesystem quesions

    >From Allison:
    >
    >>> - Do any versions of CP/M use more than 16 user numbers?

    >>
    >>Zcpr some versions allowed 32 The OS really doesn't seem to
    >>care.

    >
    >Humm ... This could be tricky - someone else mentioned 10 as a
    >volume label in another variation ... The thing I am trying to
    >do is figure out which entries represent actual files. Here we
    >already have a conflict - 10 could be an extended user number,
    >or another special value.


    If you know its CDOS then treat it as CDOS and apply rules
    as it fits CDOS. IF Vanilla CP/M then rules are more fluid..

    However, What VERSION of CP/M I assumed V2.x for this
    and NOT V1.4 or CP/M PLUS (V3) or MP/M.

    IF CP/M is using ZCPR (CCP upgrade only) then User
    numbers can be affected (extended). This includes
    datestamping.

    IF CP/M is one of the improved versions ( P2DOC, NovaDOS, ZRdos
    DOSplus) then each has it's uses for user numbers and added "stuff"
    in the directory like date stamps and named userspaces.

    >So far all the special values I've seen which don't point to
    >files have had 00's in the blocks fields ( which would never
    >be a valid file since 00 would point back to the directory),
    >and indicate 0 records.
    >I also note that 22disk does not have any configuration parameters
    >for this ... So there must be a reliable way to tell.


    No vertain way. The big thing is IF you find something unsual it may
    be easier then based on whats found to decide what to do. It's
    helpful if the program "knows" from user query that the OS was
    CDOS or other known entity.

    >>> - I note that Cromemco/CDOS uses a value of 81 for a volume
    >>> label:

    >>
    >>CDOS is not CP/M, similar though.

    >
    >From what I can tell, the file system is virtually identical.


    The filesystem is, but if it uses 10 or 81 for some special purpose
    that CP/M does not even do then.. it is different if only that it
    uses unspecified or nonreserved values.

    >>> - Are there any other "special" values?

    >>
    >>I've seen lower values there but the OS only looks for E5 or
    >>not E5 (erased or not) until user numbers are factored in then
    >> there are a range of unused values possible.
    >>
    >>This is an area where CP/M expect only a few things and
    >>does not force boundary conditions. STAT, USR and PIP
    >>however only know 16 users but that limit is in the utilities code
    >>and not an BDOS enforced limit.

    >
    >I think we are running into extensions here - special values that
    >do not indicate files would have to be accomdated by the OS (and
    >utilities).


    Most likely but it may still be special to the tools.

    >
    >>> - In my program I have taken the safe approach and will find
    >>> extents no matter what order they occur in, however I want
    >>> to know if when writing the disk I should insure that the
    >>> extent entries are in order (ie: if I allocate a directory
    >>> entry occuring physically BEFORE an entry contining a lower
    >>> extent number, will CP/M have problems)?

    >>
    >>IF image then GIGO. IF Media to file back to media then
    >>no negative effect. If you write out a file directory as sequential
    >>and correctly allocated space then the files will also likely be
    >>sequential on the disk as a matter of course.

    >
    >I'm not reading the image sequentially (for file access purposes),
    >I'm accessing it at the sector level like a disk, so in that respect it's
    >media. I scan and update the directly exactly as you would for any
    >file system on disk.


    Unsure what that means. Your either reading a file at a time or
    your copying the disk as an image? Is there a third choice?

    >I'm thinking specifically of cases where earlier files have been
    >deleted, and you need to to use those extents when appending to
    >another file occuring later on the disk - From what I understand,
    >CP/M will not move the later entries "down" to keep them in
    >sequential order, it will simply allocate the next extent from
    >the earlier (and now free) position in the directory.


    Right. But the assumption here is the destination disk is a "used"
    cp/m disk not blank media.

    For example If I have two disks that have been in use and I copy
    file of size sufficiently harger than one extent zxd.foo to the second
    if the second is already fragmented and has deletions and the like
    then zxd.foo at the directory level and likely the file level could be
    fragmented.

    IF However, the second disk is freshly formatted and never used
    it's safe bet it will be completely sequentially laid down in both
    directory and file space. Any succeeding files will as well until
    ther is a deletion that leaves a hole that can be filled (directory or
    file space).

    CP/M (others may differ in directory and file space use) will
    always lay thing in to the media in ascending order until it
    cannot, it will then seek the next place in ascending order.
    (I leave out skewing from this).

    >>One positive effect is any time CP/M has everything sequential
    >>it's a bit faster as the search is pure sequential. (note: may if not
    >>all of the improved CP/M work alikes generally has the directory for
    >>speed.).

    >
    >Agreed - If I were making this for daily use I might consider building
    >in optimizations that CP/M didn't do (such as rearranging the directory
    >for sequential extent access), however It's main use is to transfer
    >files on/off CP/M systems via ImageDisk images, so I think "what CP/M
    >does" is good enough.


    Generrally it is. However for large media with more tha 1024
    directory entries it makes a big difference. This is only important
    to those who wonder why I tend to say CP/M and improved versions.

    >
    >>>3) So far, I've needed to define the following information to
    >>> access a CP/M disk:
    >>> - Size of physical sector

    >>
    >>unknown to CP/M as BIOS deblock code deals with this.

    >
    >In my case, the information is available in the ImageDisk track
    >header.


    Thats at the image level rather than CP/M internal file data.

    >
    >>> - Mapping of logical sectors to physical sectors
    >>> (ie: Cylinder/Head mapping and sector skew)

    >>
    >>This is in the BIOS in the form of DPB and DPH and not stored
    >>on media.

    >
    >It's more than DPB/DPH - some disks are just laid out differently.
    >(for example, some disks use side2 as a logical extension of
    >side1 - sectors are marked as head0 and sector # increase on
    >the other side - others use up all of side0 before going to
    >side1) ... this is buried in the code of the BIOS and I don't
    >think it can be determined other than a fixed configuration
    >file.


    Two possible cases at least. 80 track floppy two sided.
    Side 0 can be 0-79 and side 1 can be 80-159, Side 0 can be
    0,2,4,6,8,..(even side) and side 1 can be 1,3,5,7,9..(odd side).
    This also does not consider sector order.

    Bit the distinction in CP/M systems is limited to what in the bios
    and it's unique coding. If you do not know that first it's hard to
    tell if the bios treated side 0 as track 0 and side one as track 2
    from those that did all side 0 as tracks 0 to n and side 1 as
    starting as track n and continuing to ascend. I've done both
    and there are file and performance differences and the
    DPB/DPH do not have any info as to whuich was done.

    >>> - Number of reserved (system) tracks (offset to user area)

    >>
    >>The content of the disk tables tell the BDOS.

    >
    >But .. is there any even remotely reliable way to find the disk
    >tables on a disk of unknown origin - what I'm trying to do is
    >find ways to make it easier to write a disk definition - ie: find
    >as much information automatically as possible.


    You into the real of performing a huristic program. IF the BIOS
    is on disk (not always) then you can search for the signature of
    the bios jump table and basically using that find seldisk and pares
    seldisk for the pointer to DPH and from that infer where the DPB is.
    A running CP/M program you can do that by doing a call to seldisk
    and seeing what is returned ( Valid disk signal and a pointer)
    the using that you can determine the disk called exists and then get
    the characteristics for that drive.

    >>> - Number of 128 byte records in an allocation unit

    >>DPH, DPB
    >>> - Number of directory entries

    >>DPH, DPB

    >
    >Yes, but same question - any way to locate the disk table? I
    >don't think so - the bottom line is that this information is
    >buried in the BIOS code, and is not at any specific location
    >on the disk right? (in fact the BIOS code does not even have
    >to be on the disk).


    See above but, that does not tell you how the bios does deblocking.
    There is no deblock table in the bios unless a programmer did it that
    way.

    >>Guessed at yes, accuracy poor. Anything I pointed to DPH and DPB
    >>is from the sperspective of looking at unknown disk hidden as it's in
    >>the source systems BIOS.
    >>
    >>You may get lukcy and have a copy of the bios on the disk (system
    >>tracks) but you still need to know in advance how to read them.
    >>If you do the DPH, DPB are findable as the SELDISK call will
    >>return the headers that have pointers to the tables.

    >
    >Yes, if you have a running system, this is not too hard to find,
    >but creating disk definitions for disk images that you have
    >without a working system is going to be tricky...


    Exactly, unless you know something about the system.
    Even then some of my dense QD floppies from _my_ systems
    that use my bios code will leave you in fits. With a rom bios
    and no system on the disk and no system tracks reserved
    the only hope you have then is pattern matching a known
    file like PIP or STAT and reading the disk while building a
    template of where all the peices of STAT were and by reverse
    inference get thigs like sector ordering, deblocking order, maybe
    even directory size.

    Directory size may be easiest as it's structure is predictable
    and E5s generally fill the unused areas.

    Definately a challenge.

    Allison


  12. Re: CP/M filesystem quesions

    On Sun, 14 Oct 2007 19:27:02 GMT, no.spam@no.uce.bellatlantic.net
    wrote:

    >>From Allison:
    >>

    >
    >Exactly, unless you know something about the system.
    >Even then some of my dense QD floppies from _my_ systems
    >that use my bios code will leave you in fits. With a rom bios
    >and no system on the disk and no system tracks reserved
    >the only hope you have then is pattern matching a known
    >file like PIP or STAT and reading the disk while building a
    >template of where all the peices of STAT were and by reverse
    >inference get thigs like sector ordering, deblocking order, maybe
    >even directory size.
    >
    >Directory size may be easiest as it's structure is predictable
    >and E5s generally fill the unused areas.
    >
    >Definately a challenge.


    Another item is allocation block size. for floppies this will
    generally be 1k for under 256k disks and most likely 2K
    for larger and I havent seen 4K used as it's too granualar.
    Since the largest floppy flying is around 1.6M it's possible
    that 4k might have been used but unlikely. The reason for that
    is CP/M files tend to be small and storage efficientcy is often
    a ccritical item for floppies especially those in the 5.25" class
    where common storage sizes ran from a puny 80k to as much
    as 780k but early systems running under 360k (DD 2side 40tr)
    are most common .

    That info also yeilds possible info on possible directory layouts
    for the allocation field. If the disks is known to be small usually
    single density and under256k then the allocation size is usually
    1K and the directory allocation files is a BYTE value (16 allocations
    per field). If the allocation size is 2K or larger the allocation
    field is a word (2 bytes) value and only 8 allocations per field.

    So if you find a file in the directory area with 1 allocation block
    and you can find all the peices if it's 1k or smaller or 1k or larger
    it may be possible to assume a logical allocations size then try
    that against a larger file.

    While it's possible to automate some or all of it knowing the
    possible source system or the FDC Vendor used makes it
    easier. Knowing the FDC vendor used is handy as many
    boards were used in multiple systems with a bios that only
    required the terminal drivers and printer drivers to be altered
    for the spcific user system. Examples of this were the Tarbell
    and NS* controllers but there were meany more like the Jade
    Doubler.

    Some hints on how you might proceed.

    Allison


    >Allison



  13. Re: CP/M filesystem quesions

    no.spam@no.uce.bellatlantic.net wrote:
    : Another item is allocation block size. for floppies this will
    : generally be 1k for under 256k disks and most likely 2K
    : for larger and I havent seen 4K used as it's too granualar.

    I have, on a 784k format used by Amstrad PCWs. The reason being that the
    allocation vector can only hold data for 360 blocks, so for formats above
    720k you need to go to 4k blocks.

    --
    John Elliott

  14. Re: CP/M filesystem quesions

    On Mon, 15 Oct 2007 19:14:25 +0100, John Elliott
    wrote:

    >no.spam@no.uce.bellatlantic.net wrote:
    >: Another item is allocation block size. for floppies this will
    >: generally be 1k for under 256k disks and most likely 2K
    >: for larger and I havent seen 4K used as it's too granualar.


    > I have, on a 784k format used by Amstrad PCWs. The reason being that the
    >allocation vector can only hold data for 360 blocks, so for formats above
    >720k you need to go to 4k blocks.


    I'd call that a featured bug. If it was useless it's a bug but since
    it has some useful function it's a feature.

    While I did say it was possible that someone would do that, the reason
    you just gave is true. Amstrands around here (NE USA) are scarce
    to rare. I did make it clear anything is possible but general cases
    are most likely. The PCW is anything but general case, maybe there
    it might be.

    The yabut is yes if the system did not allocate enough space for the
    allocation vector in memory (bit map of used sectors) and the check
    vector (a checksum used for disk change checking) then you might be
    forced to do that.

    However if you use the average size for CP/M programs the storage
    space increase is poor as you a hve a lot more partially filled
    sectors. If the files are large it works.

    Allison

  15. Re: CP/M filesystem quesions

    John Elliott wrote:

    > no.spam@no.uce.bellatlantic.net wrote:
    > : Another item is allocation block size. for floppies this will
    > : generally be 1k for under 256k disks and most likely 2K
    > : for larger and I havent seen 4K used as it's too granualar.


    > I have, on a 784k format used by Amstrad PCWs. The reason being that the
    > allocation vector can only hold data for 360 blocks, so for formats above
    > 720k you need to go to 4k blocks.


    So that would be true for DS/DD 8 inch floppies. I would have thought
    someone would use them for CP/M.

    I don't suppose ED floppies (2880K) were ever used with CP/M.
    I have one disk and a few drives.

    -- glen


  16. Re: CP/M filesystem quesions

    On Mon, 15 Oct 2007 16:15:05 -0800, glen herrmannsfeldt
    wrote:

    >John Elliott wrote:
    >
    >> no.spam@no.uce.bellatlantic.net wrote:
    >> : Another item is allocation block size. for floppies this will
    >> : generally be 1k for under 256k disks and most likely 2K
    >> : for larger and I havent seen 4K used as it's too granualar.

    >
    >> I have, on a 784k format used by Amstrad PCWs. The reason being that the
    >> allocation vector can only hold data for 360 blocks, so for formats above
    >> 720k you need to go to 4k blocks.

    >
    >So that would be true for DS/DD 8 inch floppies. I would have thought
    >someone would use them for CP/M.


    Indeed they were. I did it back in 82 with NEC1152s (two sided 8")
    for 1Mb/drive. Worked well and the allocation size was 2K.

    My current system has a heath Zenith H207 drives running at 1mb
    (pair of SA851s) also using 2k allocation.

    The 3.5" floppies on my systems are DD so I get 800k but used
    2k allocation size. I've also done the same format for 5.25" QD.
    Parameters were:

    5 spt
    1024 bytes/sector
    2k allocation
    128 checked directory entries, 2 sectors for directory
    1 reserved tracks

    The one system that can do faster data rates (has DMA)
    is 1.6mb formatted for CP/M file system. Again I used 2k
    allocation size.

    >I don't suppose ED floppies (2880K) were ever used with CP/M.
    >I have one disk and a few drives.


    None I know of. However it's no different than a 5mb hard disk
    and it's a preference call for 2k or 4K allocation blocks as both
    work.

    For CP/M if you use 1K allocation and the standard DRI table of
    values the largest drive is 256K. For 2k or larger allocation size
    that jumps to 8mb. The reason is the mask and shift values
    decide if the 16 bytes of allocation space will be 16 byte values
    (256 allocation blocks max) or 8 two byte values for up to
    a maximum of 8mb. The 8MB is a limit due to 16bit truncation
    in the math performed. For those Improved CP/M work alikes
    that is fixed and you can have up to 65535* allocation size
    or 1GB.

    So the answer is for a floppy you can have any allocation size but
    smaller is better and 2K for disks over 256k is the usual but theres
    nothing to stop the implementer for going larger beyond storage
    inefficientcy for many small files.

    The problem is for larger disks the Allocation space in ram
    grows and a 1.6MB floppy with 2k allocation size wants 100bytes
    and another 50 for the check vector. If you use a 4k allocation
    schedule those become 50 and 25 bytes. So the problemm is with
    deblocking buffers and multiple disks with large allocation spaces
    you use up valuable ram in the BIOS. This problem makes bank
    switching or paging look handy!

    However none of this helps Dave with figuring out what was done to
    the unmarked disk with files on it.


    Allison


  17. Re: CP/M filesystem quesions

    no.spam@no.uce.bellatlantic.net wrote:

    (snip)

    > However none of this helps Dave with figuring out what was done to
    > the unmarked disk with files on it.


    I have some unmarked disks with CP/M files on them that I might get
    around to looking at. I believe they are 8in SS disks from an
    Apple II CP/M system. I still have the disk controller but no
    Apple II. I have an 8 in DS drive, too.

    -- glen


  18. Re: CP/M filesystem quesions

    no.spam@no.uce.bellatlantic.net wrote:
    > On Mon, 15 Oct 2007 19:14:25 +0100, John Elliott
    > wrote:
    >
    >> no.spam@no.uce.bellatlantic.net wrote:
    >> : Another item is allocation block size. for floppies this will
    >> : generally be 1k for under 256k disks and most likely 2K
    >> : for larger and I havent seen 4K used as it's too granualar.

    >
    >> I have, on a 784k format used by Amstrad PCWs. The reason being that the
    >> allocation vector can only hold data for 360 blocks, so for formats above
    >> 720k you need to go to 4k blocks.

    >
    > I'd call that a featured bug. If it was useless it's a bug but since
    > it has some useful function it's a feature.
    >


    Hi Allison,

    what would you say about the Commodore (64) CP/M format ? It's not
    exotic, it's almost chaotic, see also my documents and my disk reader
    program at http://www.z80.eu/c64.html ... sector numbers per track are
    changing, a non data track in the middle (which has to be skipped), and
    also an unusual booting method ;-)

    Regards
    Peter

  19. Re: CP/M filesystem quesions

    On Tue, 16 Oct 2007 10:06:42 +0200, "Peter Dassow (remove the NOSPAM.
    for direct answer)" wrote:

    >no.spam@no.uce.bellatlantic.net wrote:
    >> On Mon, 15 Oct 2007 19:14:25 +0100, John Elliott
    >> wrote:
    >>
    >>> no.spam@no.uce.bellatlantic.net wrote:
    >>> : Another item is allocation block size. for floppies this will
    >>> : generally be 1k for under 256k disks and most likely 2K
    >>> : for larger and I havent seen 4K used as it's too granualar.

    >>
    >>> I have, on a 784k format used by Amstrad PCWs. The reason being that the
    >>> allocation vector can only hold data for 360 blocks, so for formats above
    >>> 720k you need to go to 4k blocks.

    >>
    >> I'd call that a featured bug. If it was useless it's a bug but since
    >> it has some useful function it's a feature.
    >>

    >
    >Hi Allison,
    >
    >what would you say about the Commodore (64) CP/M format ? It's not
    >exotic, it's almost chaotic, see also my documents and my disk reader
    >program at http://www.z80.eu/c64.html ... sector numbers per track are
    >changing, a non data track in the middle (which has to be skipped), and
    >also an unusual booting method ;-)
    >
    >Regards
    > Peter


    Commies are well, interesting. I bet theres a reason too.
    Must be the 6502, it makes people do crazy things.

    Actually directory in the middle tracks or even tracks closer to
    center hub are not unusual. Also TRSDOS did different things.

    Theres nothing other than convention to say CP/M had it more right
    than any other scheme.


    Allison

  20. Re: CP/M filesystem quesions

    no.spam@no.uce.bellatlantic.net wrote:
    > On Tue, 16 Oct 2007 10:06:42 +0200, "Peter Dassow (remove the NOSPAM.
    > for direct answer)" wrote:
    >
    >> no.spam@no.uce.bellatlantic.net wrote:
    >>> On Mon, 15 Oct 2007 19:14:25 +0100, John Elliott
    >>> wrote:
    >>>
    >>>> no.spam@no.uce.bellatlantic.net wrote:
    >>>> : Another item is allocation block size. for floppies this will
    >>>> : generally be 1k for under 256k disks and most likely 2K
    >>>> : for larger and I havent seen 4K used as it's too granualar.
    >>>> I have, on a 784k format used by Amstrad PCWs. The reason being that the
    >>>> allocation vector can only hold data for 360 blocks, so for formats above
    >>>> 720k you need to go to 4k blocks.
    >>> I'd call that a featured bug. If it was useless it's a bug but since
    >>> it has some useful function it's a feature.
    >>>

    >> Hi Allison,
    >>
    >> what would you say about the Commodore (64) CP/M format ? It's not
    >> exotic, it's almost chaotic, see also my documents and my disk reader
    >> program at http://www.z80.eu/c64.html ... sector numbers per track are
    >> changing, a non data track in the middle (which has to be skipped), and
    >> also an unusual booting method ;-)
    >>
    >> Regards
    >> Peter

    >
    > Commies are well, interesting. I bet theres a reason too.
    > Must be the 6502, it makes people do crazy things.
    >
    > Actually directory in the middle tracks or even tracks closer to
    > center hub are not unusual. Also TRSDOS did different things.
    >
    > Theres nothing other than convention to say CP/M had it more right
    > than any other scheme.
    >
    >
    > Allison


    CBM was more or less bleeding edge at the time... let me see here...
    S-VIDEO
    ZBR (Zone bit recording) for increased reliability depending on BPI...
    Butterfly disk storage for lower access times...

    None of these are crazy at all. ZBR is used in all hard disks these
    days, although because of the translation, you never see that...

    Alot of modern FSs are going back to butterfly storage mixed with binary
    trees....

    CBM's largest mistake as far as the disk goes IMHO was using GCR, a weak
    checksum, and the linked list of sectors in the data stream, making
    random seeks a nightmare (REL files are a bad idea), but they had the
    other points right, including the name system...

+ Reply to Thread
Page 1 of 2 1 2 LastLast