DDS-4 tape drive compatiblity

I was given two used SCSI DDS-4 tape drives, and I am testing them for
use in making backups.

One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
SDT-10000 (EOD006, .

I find that each drive can read the tapes that it writes, but that
neither drive can read the tapes written by the other.

I have put a cleaning tape through each drive, and the firmware of each
drive is up-to-date. Both drives pass the diagnostics in the HP Library
& Tape Tool.

Shouldn't these drives be able to read each other's tapes? Do I need to
adjust some parameters, or should I assume that at least one of them is
defective?

Are you using the same software to test each?

"Matt" <matt@themattfella.xxxyyz.com> wrote in message
news:4KWXj.234$Vf.88@fe093.usenetserver.com...[color=blue]
>I was given two used SCSI DDS-4 tape drives, and I am testing them for use
>in making backups.
>
> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
> SDT-10000 (EOD006, .
>
> I find that each drive can read the tapes that it writes, but that neither
> drive can read the tapes written by the other.
>
> I have put a cleaning tape through each drive, and the firmware of each
> drive is up-to-date. Both drives pass the diagnostics in the HP Library &
> Tape Tool.
>
> Shouldn't these drives be able to read each other's tapes? Do I need to
> adjust some parameters, or should I assume that at least one of them is
> defective?[/color]

I'm testing using tar and diff for both. Also using the same SCSI
adapter and the same computer for both. I test with only one tape drive
installed at a time.

I test whether drive 1 can read the tape it has written, then I shut
down, swap drives, and try to read the same tape with drive 2 and find
that it can't read it.

Bill wrote:[color=blue]
> Are you using the same software to test each?
>
> "Matt" <matt@themattfella.xxxyyz.com> wrote in message
> news:4KWXj.234$Vf.88@fe093.usenetserver.com...[color=green]
>> I was given two used SCSI DDS-4 tape drives, and I am testing them for use
>> in making backups.
>>
>> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
>> SDT-10000 (EOD006, .
>>
>> I find that each drive can read the tapes that it writes, but that neither
>> drive can read the tapes written by the other.
>>
>> I have put a cleaning tape through each drive, and the firmware of each
>> drive is up-to-date. Both drives pass the diagnostics in the HP Library &
>> Tape Tool.
>>
>> Shouldn't these drives be able to read each other's tapes? Do I need to
>> adjust some parameters, or should I assume that at least one of them is
>> defective?[/color][/color]

My bet is the HP is not compatible with anything else.

"Matt" <matt@themattfella.xxxyyz.com> wrote in message
news:EFYXj.1378$ZB5.382@fe087.usenetserver.com...[color=blue]
> I'm testing using tar and diff for both. Also using the same SCSI adapter
> and the same computer for both. I test with only one tape drive installed
> at a time.
>
> I test whether drive 1 can read the tape it has written, then I shut down,
> swap drives, and try to read the same tape with drive 2 and find that it
> can't read it.
>
> Bill wrote:[color=green]
>> Are you using the same software to test each?
>>
>> "Matt" <matt@themattfella.xxxyyz.com> wrote in message
>> news:4KWXj.234$Vf.88@fe093.usenetserver.com...[color=darkred]
>>> I was given two used SCSI DDS-4 tape drives, and I am testing them for
>>> use in making backups.
>>>
>>> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
>>> SDT-10000 (EOD006, .
>>>
>>> I find that each drive can read the tapes that it writes, but that
>>> neither drive can read the tapes written by the other.
>>>
>>> I have put a cleaning tape through each drive, and the firmware of each
>>> drive is up-to-date. Both drives pass the diagnostics in the HP Library
>>> & Tape Tool.
>>>
>>> Shouldn't these drives be able to read each other's tapes? Do I need to
>>> adjust some parameters, or should I assume that at least one of them is
>>> defective?[/color][/color][/color]

Matt <matt@themattfella.xxxyyz.com> writes:
[color=blue]
> I was given two used SCSI DDS-4 tape drives, and I am testing them for
> use in making backups.
>
> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
> SDT-10000 (EOD006, .
>
> I find that each drive can read the tapes that it writes, but that
> neither drive can read the tapes written by the other.
>
> I have put a cleaning tape through each drive, and the firmware of
> each drive is up-to-date. Both drives pass the diagnostics in the HP
> Library & Tape Tool.
>
> Shouldn't these drives be able to read each other's tapes? Do I need
> to adjust some parameters, or should I assume that at least one of
> them is defective?[/color]

DDS-4 is DDS-4, but I'm not certain that they would use the same
hardware compression technique, if you have that enabled via the
jumpers on the drives. You should be able to disable compression
using "mt compression 0".

Scott
--
Scott Hemphill [email]hemphill@alumni.caltech.edu[/email]
"This isn't flying. This is falling, with style." -- Buzz Lightyear

Scott Hemphill wrote:[color=blue]
> Matt <matt@themattfella.xxxyyz.com> writes:
>[color=green]
>> I was given two used SCSI DDS-4 tape drives, and I am testing them for
>> use in making backups.
>>
>> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
>> SDT-10000 (EOD006, .
>>
>> I find that each drive can read the tapes that it writes, but that
>> neither drive can read the tapes written by the other.
>>
>> I have put a cleaning tape through each drive, and the firmware of
>> each drive is up-to-date. Both drives pass the diagnostics in the HP
>> Library & Tape Tool.
>>
>> Shouldn't these drives be able to read each other's tapes? Do I need
>> to adjust some parameters, or should I assume that at least one of
>> them is defective?[/color]
>
> DDS-4 is DDS-4,[/color]


I was guessing something like that. I think you are saying that the
format is as standardize as say ext3 or fat16.

[color=blue]
> but I'm not certain that they would use the same
> hardware compression technique, if you have that enabled via the
> jumpers on the drives.[/color]


Thanks, Scott, I'll try that. So we are expecting/hoping to be able to
interchange any non-compressed DDS-4 tapes among DDS-4 drives.

I found the following doc which gives a lot of technical detail on the
Compaq (rebadged Sony) drive.
[color=blue]
> PRODUCT DESCRIPTION MANUAL
> DDS-4 Tape Drive
> Model : SDT-10000
> : SDT-11000
> Ver. 1.3 March, 2006
> Sony Corporation[/color]
[color=blue]
> Data Compression For Information Interchange
> Adaptive Coding with Embedded Dictionary, DCLZ Algorithm, June 1991
> European Computer Manufacturers Association (ECMA-151)1[/color]
[color=blue]
> 4. 6 DATA COMPRESSION
> The tape capacity is increased by compressing data prior to writing it to the tape. Data compression is a well established
> technology for reducing the number of bits used to represent data in order to improve data transfer rate as well as reduce
> the amount of storage space consumed by the data.
> The SDT-10000/SDT-11000 uses the AK 8320 Data Compression IC from ASAHI KASEI MICROSYSTEMS. This
> chip provides a powerful data compression algorithm in a very small package. The data compression used by the chip is
> the DCLZ algorithm. DCLZ has been standardized (or is in the process of standardization) with the ANSI, ECMA and
> ISO standards organizations. The DDS Manufacturers Group (made up of representatives from active DDS Format
> Licensees) has agreed upon the DCLZ algorithm as the standard data compression algorithm for data interchange be-
> tween DDS format drives.
> The DC control page allows the host computer to enable data compression and also configure the way in which the drive
> responds to compressed/uncompressed data boundaries on the tape.
> Note: The DDS format allows both compressed and uncompressed data to reside on the same tape.
> SDT-10000/SDT-11000 has a Dip switch to disable the Data Compression.
> After power-on reset with this jumper set, data compression is disabled. However, a MODE SELECT command can
> override the setting of this jumper.
> After power-on reset without this Dip switch set, both data compression and data decompression are enabled. (See
> 3.1.5)
> The more random the data is, the less compression is possible. This is due to the fact that data compression operates on
> the principle of reducing the redundancy in the data string and random data has very little redundancy.
> Data compression is a very powerful and reliable method increasing data capacity and transfer rate without compromis-
> ing data reliability.[/color]
[color=blue]
> Compression Algorithm:
> The compression algorithm field indicates the compression algorithm the drive will use to process data sent to it by
> the initiator (if the DCE bit is one).
> The SDT-10000/SDT-11000 supports the DCLZ data compression algorithm which is identified by the value: 00 00
> 00 20h in the compression algorithm field. A value of zero shall indicate that no compression algorithm is currently
> selected. Any other values in this field will cause the drive to return a CHECK CONDITION status the sense key
> shall be set to ILLEGAL REQUEST.[/color]
[color=blue]
> Decompression Algorithm:
> For MODE SELECT the decompression algorithm field indicates the decompression algorithm selected by the
> initiator for use in subsequent decompression of data encountered on the medium.
> The SDT-10000/SDT-11000 can decompress data recorded with the DCLZ algorithm therefore this field can be set
> to 00 00 00 20h. However, the SDT-10000/SDT-11000 is capable of automatic recognition of the compression
> algorithm used to process the data encountered on the medium. Therefore, the drive will override the value in the
> decompression field (if is set to zero) for a subsequent read operation when DCLZ compressed data is detected on
> the media.[/color]
[color=blue]
> Note: A CHECK CONDITION will occur on the transition from uncompress to compressed if RED = 10b.
> For the MODE SENSE command, the decompression algorithm field reflects either the algorithm selected
> by the initiator or compression algorithm which was used to process the data most recently encountered on
> the medium, during a read operation.
> A value of zero shall indicate that the data encountered on the medium during the most recent read operation
> was uncompressed.[/color]


I haven't found anything definite yet about DCLZ for the DAT40i. More
about that later.

Matt wrote:[color=blue]
> Scott Hemphill wrote:[color=green]
>> Matt <matt@themattfella.xxxyyz.com> writes:
>>[color=darkred]
>>> I was given two used SCSI DDS-4 tape drives, and I am testing them for
>>> use in making backups.
>>>
>>> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
>>> SDT-10000 (EOD006, .
>>>
>>> I find that each drive can read the tapes that it writes, but that
>>> neither drive can read the tapes written by the other.
>>>
>>> I have put a cleaning tape through each drive, and the firmware of
>>> each drive is up-to-date. Both drives pass the diagnostics in the HP
>>> Library & Tape Tool.
>>>
>>> Shouldn't these drives be able to read each other's tapes? Do I need
>>> to adjust some parameters, or should I assume that at least one of
>>> them is defective?[/color]
>>
>> DDS-4 is DDS-4,[/color]
>
>
> I was guessing something like that. I think you are saying that the
> format is as standardize as say ext3 or fat16.
>
>[color=green]
>> but I'm not certain that they would use the same
>> hardware compression technique, if you have that enabled via the
>> jumpers on the drives.[/color]
>
>
> Thanks, Scott, I'll try that. So we are expecting/hoping to be able to
> interchange any non-compressed DDS-4 tapes among DDS-4 drives.
>
> I found the following doc which gives a lot of technical detail on the
> Compaq (rebadged Sony) drive.
>[color=green]
>> PRODUCT DESCRIPTION MANUAL
>> DDS-4 Tape Drive
>> Model : SDT-10000
>> : SDT-11000
>> Ver. 1.3 March, 2006
>> Sony Corporation[/color]
>[color=green]
>> Data Compression For Information Interchange
>> Adaptive Coding with Embedded Dictionary, DCLZ Algorithm, June 1991
>> European Computer Manufacturers Association (ECMA-151)1[/color]
>[color=green]
>> 4. 6 DATA COMPRESSION
>> The tape capacity is increased by compressing data prior to writing it
>> to the tape. Data compression is a well established
>> technology for reducing the number of bits used to represent data in
>> order to improve data transfer rate as well as reduce
>> the amount of storage space consumed by the data.
>> The SDT-10000/SDT-11000 uses the AK 8320 Data Compression IC from
>> ASAHI KASEI MICROSYSTEMS. This
>> chip provides a powerful data compression algorithm in a very small
>> package. The data compression used by the chip is
>> the DCLZ algorithm. DCLZ has been standardized (or is in the process
>> of standardization) with the ANSI, ECMA and
>> ISO standards organizations. The DDS Manufacturers Group (made up of
>> representatives from active DDS Format
>> Licensees) has agreed upon the DCLZ algorithm as the standard data
>> compression algorithm for data interchange be-
>> tween DDS format drives.
>> The DC control page allows the host computer to enable data
>> compression and also configure the way in which the drive
>> responds to compressed/uncompressed data boundaries on the tape.
>> Note: The DDS format allows both compressed and uncompressed data to
>> reside on the same tape.
>> SDT-10000/SDT-11000 has a Dip switch to disable the Data Compression.
>> After power-on reset with this jumper set, data compression is
>> disabled. However, a MODE SELECT command can
>> override the setting of this jumper.
>> After power-on reset without this Dip switch set, both data
>> compression and data decompression are enabled. (See
>> 3.1.5)
>> The more random the data is, the less compression is possible. This is
>> due to the fact that data compression operates on
>> the principle of reducing the redundancy in the data string and random
>> data has very little redundancy.
>> Data compression is a very powerful and reliable method increasing
>> data capacity and transfer rate without compromis-
>> ing data reliability.[/color]
>[color=green]
>> Compression Algorithm:
>> The compression algorithm field indicates the compression algorithm
>> the drive will use to process data sent to it by
>> the initiator (if the DCE bit is one).
>> The SDT-10000/SDT-11000 supports the DCLZ data compression
>> algorithm which is identified by the value: 00 00
>> 00 20h in the compression algorithm field. A value of zero shall
>> indicate that no compression algorithm is currently
>> selected. Any other values in this field will cause the drive to
>> return a CHECK CONDITION status the sense key
>> shall be set to ILLEGAL REQUEST.[/color]
>[color=green]
>> Decompression Algorithm:
>> For MODE SELECT the decompression algorithm field indicates the
>> decompression algorithm selected by the
>> initiator for use in subsequent decompression of data encountered
>> on the medium.
>> The SDT-10000/SDT-11000 can decompress data recorded with the DCLZ
>> algorithm therefore this field can be set
>> to 00 00 00 20h. However, the SDT-10000/SDT-11000 is capable of
>> automatic recognition of the compression
>> algorithm used to process the data encountered on the medium.
>> Therefore, the drive will override the value in the
>> decompression field (if is set to zero) for a subsequent read
>> operation when DCLZ compressed data is detected on
>> the media.[/color]
>[color=green]
>> Note: A CHECK CONDITION will occur on the transition from uncompress
>> to compressed if RED = 10b.
>> For the MODE SENSE command, the decompression algorithm field
>> reflects either the algorithm selected
>> by the initiator or compression algorithm which was used to
>> process the data most recently encountered on
>> the medium, during a read operation.
>> A value of zero shall indicate that the data encountered on the
>> medium during the most recent read operation
>> was uncompressed.[/color]
>
>
> I haven't found anything definite yet about DCLZ for the DAT40i. More
> about that later.
>[/color]

That's the culprit - HW compression chips differ in the compression
algorithm used.

Set the drives jumpers to NOT use HW compression, then use software like
the commercial BackupEdge ([url]www.microlite.com[/url]) with software compression
enabled, or pipe the backup through gzip or bzip2.

--
----------------------------------------------------
Pat Welch, UBB Computer Services, a WCS Affiliate
SCO Authorized Partner
Microlite BackupEdge Certified Reseller
Unix/Linux/Windows/Hardware Sales/Support
(209) 745-1401 Cell: (209) 251-9120
E-mail: [email]patubb@inreach.com[/email]
----------------------------------------------------

Pat Welch wrote:[color=blue]
> Matt wrote:[color=green]
>> Scott Hemphill wrote:[color=darkred]
>>> Matt <matt@themattfella.xxxyyz.com> writes:[/color][/color][/color]
[color=blue][color=green][color=darkred]
>>>> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
>>>> SDT-10000 (EOD006, .
>>>>
>>>> I find that each drive can read the tapes that it writes, but that
>>>> neither drive can read the tapes written by the other.[/color][/color][/color]
[color=blue][color=green][color=darkred]
>>> DDS-4 is DDS-4,[/color][/color][/color]
[color=blue][color=green][color=darkred]
>>> but I'm not certain that they would use the same
>>> hardware compression technique, if you have that enabled via the
>>> jumpers on the drives.[/color][/color][/color]
[color=blue]
> That's the culprit - HW compression chips differ in the compression
> algorithm used.
>
> Set the drives jumpers to NOT use HW compression,[/color]

Thank you, Pat.

I have docs for the Compaq drive describing how to do that, but I
haven't been able to find good docs on the jumpers (DIP switches
actually) for the HP DAT40i. Any specific information about that would
be a big help.

Matt wrote:[color=blue]
> Pat Welch wrote:[color=green]
>> Matt wrote:[color=darkred]
>>> Scott Hemphill wrote:
>>>> Matt <matt@themattfella.xxxyyz.com> writes:[/color][/color]
>[color=green][color=darkred]
>>>>> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
>>>>> SDT-10000 (EOD006, .
>>>>>
>>>>> I find that each drive can read the tapes that it writes, but that
>>>>> neither drive can read the tapes written by the other.[/color][/color]
>[color=green][color=darkred]
>>>> DDS-4 is DDS-4,[/color][/color]
>[color=green][color=darkred]
>>>> but I'm not certain that they would use the same
>>>> hardware compression technique, if you have that enabled via the
>>>> jumpers on the drives.[/color][/color]
>[color=green]
>> That's the culprit - HW compression chips differ in the compression
>> algorithm used.
>>
>> Set the drives jumpers to NOT use HW compression,[/color]
>
> Thank you, Pat.
>
> I have docs for the Compaq drive describing how to do that, but I
> haven't been able to find good docs on the jumpers (DIP switches
> actually) for the HP DAT40i. Any specific information about that would
> be a big help.
>[/color]

See if this helps (took me a while to find it on HP's site):

Data compression - Switches 1 and 2
Switches 1 and 2 are used to configure the way in which data compression
is set for the drive. The following table shows the available options:

Switch 1 Switch 2 Meaning
On On Compression enabled at power-on, with host control

On Off Compression enabled at power-on, no host control

Off On Compression disabled at power-on; the host is allowed
to control compression

Off Off Compression disabled at power-on, no host control

*
If Switch 1 is ON, data written to the tape will be compressed
without the knowledge of the host.
*
If Switch 2 is ON, data compression can be controlled by the
operating system if supported.
*
By default, the drive will decompress data when reading a
compressed tape, regardless of the settings of Switches 1 and 2.
Decompression can be turned off through choice of device file, and can
be controlled by the operating system if supported.

Enjoy. :)

--
----------------------------------------------------
Pat Welch, UBB Computer Services, a WCS Affiliate
SCO Authorized Partner
Microlite BackupEdge Certified Reseller
Unix/Linux/Windows/Hardware Sales/Support
(209) 745-1401 Cell: (209) 251-9120
E-mail: [email]patubb@inreach.com[/email]
----------------------------------------------------

Bill wrote:[color=blue]
> My bet is the HP is not compatible with anything else.
>
> "Matt" <matt@themattfella.xxxyyz.com> wrote in message
> news:EFYXj.1378$ZB5.382@fe087.usenetserver.com...[color=green]
>> I'm testing using tar and diff for both. Also using the same SCSI adapter
>> and the same computer for both. I test with only one tape drive installed
>> at a time.
>>
>> I test whether drive 1 can read the tape it has written, then I shut down,
>> swap drives, and try to read the same tape with drive 2 and find that it
>> can't read it.
>>
>> Bill wrote:[color=darkred]
>>> Are you using the same software to test each?
>>>
>>> "Matt" <matt@themattfella.xxxyyz.com> wrote in message
>>> news:4KWXj.234$Vf.88@fe093.usenetserver.com...
>>>> I was given two used SCSI DDS-4 tape drives, and I am testing them for
>>>> use in making backups.
>>>>
>>>> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
>>>> SDT-10000 (EOD006, .
>>>>
>>>> I find that each drive can read the tapes that it writes, but that
>>>> neither drive can read the tapes written by the other.
>>>>
>>>> I have put a cleaning tape through each drive, and the firmware of each
>>>> drive is up-to-date. Both drives pass the diagnostics in the HP Library
>>>> & Tape Tool.
>>>>
>>>> Shouldn't these drives be able to read each other's tapes? Do I need to
>>>> adjust some parameters, or should I assume that at least one of them is
>>>> defective?[/color][/color]
>
>[/color]
Is there a hardware compression built-in to one or both that can be
defeated?

Kirk C Aune wrote:[color=blue]
> Bill wrote:[color=green]
>> My bet is the HP is not compatible with anything else.
>>
>> "Matt" <matt@themattfella.xxxyyz.com> wrote in message
>> news:EFYXj.1378$ZB5.382@fe087.usenetserver.com...[color=darkred]
>>> I'm testing using tar and diff for both. Also using the same SCSI
>>> adapter and the same computer for both. I test with only one tape
>>> drive installed at a time.
>>>
>>> I test whether drive 1 can read the tape it has written, then I shut
>>> down, swap drives, and try to read the same tape with drive 2 and
>>> find that it can't read it.
>>>
>>> Bill wrote:
>>>> Are you using the same software to test each?
>>>>
>>>> "Matt" <matt@themattfella.xxxyyz.com> wrote in message
>>>> news:4KWXj.234$Vf.88@fe093.usenetserver.com...
>>>>> I was given two used SCSI DDS-4 tape drives, and I am testing them
>>>>> for use in making backups.
>>>>>
>>>>> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
>>>>> SDT-10000 (EOD006, .
>>>>>
>>>>> I find that each drive can read the tapes that it writes, but that
>>>>> neither drive can read the tapes written by the other.
>>>>>
>>>>> I have put a cleaning tape through each drive, and the firmware of
>>>>> each drive is up-to-date. Both drives pass the diagnostics in the
>>>>> HP Library & Tape Tool.
>>>>>
>>>>> Shouldn't these drives be able to read each other's tapes? Do I
>>>>> need to adjust some parameters, or should I assume that at least
>>>>> one of them is defective?[/color]
>>
>>[/color]
> Is there a hardware compression built-in to one or both that can be
> defeated?[/color]

I do not know about the DDS-4 drives, but my VXA (scsi) drives have hardware
compression that can be defeated with the _mt_ command.

mt -f /dev/st0 compression

compression
(SCSI tapes) The compression within the drive can be switched on
or off using the MTCOMPRESSION ioctl. Note that this method is
not supported by all drives implementing compression. For
instance, the Exabyte 8 mm drives use density codes to select
compression.

MTCOMPRESSION ioctl is described in the st man page.

I am not sure this is true for all Exabyte 8mm drives as they have several
product lines. (I always use compression, so I have never tried this.)

Exabyte say:

9.7 DATA COMPRESSION PAGE (PAGE CODE 0Fh)
The Data Compression page enables you to turn data compression on or off at
any position on the tape. To turn compression off, send this page with the
DCE bit set to 0. To turn compression back on, send this page with the DCE
bit set to 1.
Bit 7 6 5 4 3 2 1 0
Byte
00 RSVD | Page Code
01 Page Length
02 DCE | DCC | Reserved
03 DDE RED | Reserved
04 (MSB)
Compression Algorithm
...
07 (LSB)
08 (MSB)
Decompression Algorithm
...
11 (LSB)
12
Reserved
...
15

Byte 00, Bits 5 through 0 Â* Page Code
The value for this field is 0Fh, identifying the current page as the Data
Compression page.
Byte 01 Â*Length This field indicates the number of bytes in the Data
Compression page that follow this byte. The valid value for this field is
0Eh (14 bytes).
Byte 02, Bit 7 Â* DCE (Data Compression Enable This field enables or disables
data compression, as follows:
0 Data compression is disabled
1 Data compression is enabled (default setting)


--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey [url]http://counter.li.org[/url]
^^-^^ 17:40:01 up 7:37, 4 users, load average: 4.54, 4.58, 4.43

Pat Welch wrote:[color=blue]
> Matt wrote:[color=green]
>> Pat Welch wrote:[color=darkred]
>>> Matt wrote:
>>>> Scott Hemphill wrote:
>>>>> Matt <matt@themattfella.xxxyyz.com> writes:[/color]
>>[color=darkred]
>>>>>> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
>>>>>> SDT-10000 (EOD006, .
>>>>>>
>>>>>> I find that each drive can read the tapes that it writes, but that
>>>>>> neither drive can read the tapes written by the other.[/color]
>>[color=darkred]
>>>>> DDS-4 is DDS-4,[/color]
>>[color=darkred]
>>>>> but I'm not certain that they would use the same
>>>>> hardware compression technique, if you have that enabled via the
>>>>> jumpers on the drives.[/color]
>>[color=darkred]
>>> That's the culprit - HW compression chips differ in the compression
>>> algorithm used.
>>>
>>> Set the drives jumpers to NOT use HW compression,[/color]
>>
>> Thank you, Pat.
>>
>> I have docs for the Compaq drive describing how to do that, but I
>> haven't been able to find good docs on the jumpers (DIP switches
>> actually) for the HP DAT40i. Any specific information about that
>> would be a big help.
>>[/color]
>
> See if this helps (took me a while to find it on HP's site):
>
> Data compression - Switches 1 and 2
> Switches 1 and 2 are used to configure the way in which data compression
> is set for the drive. The following table shows the available options:
>
> Switch 1 Switch 2 Meaning
> On On Compression enabled at power-on, with host control
>
> On Off Compression enabled at power-on, no host control
>
> Off On Compression disabled at power-on; the host is
> allowed to control compression
>
> Off Off Compression disabled at power-on, no host control
>
> *
> If Switch 1 is ON, data written to the tape will be compressed
> without the knowledge of the host.
> *
> If Switch 2 is ON, data compression can be controlled by the
> operating system if supported.
> *
> By default, the drive will decompress data when reading a
> compressed tape, regardless of the settings of Switches 1 and 2.
> Decompression can be turned off through choice of device file, and can
> be controlled by the operating system if supported.
>
> Enjoy. :)
>[/color]

Thanks. I googled for some of the text you quoted and was able to find
on the HP site only a manual for an 8GB DAT drive. However I found "HP
DDS Drives, Edition 8, December 1999, P/N C1534-90911 at
[url]http://download.transtec.de/doit/loadva/software/w3/CD_STOR_HTML/backup/t71/HP_DDS_auto_eng.pdf[/url]
which gave settings for model C5383A, which is not the model number
stamped on my drive, but is the model number reported by my drive's BIOS.

Scott Hemphill wrote:[color=blue]
> Matt <matt@themattfella.xxxyyz.com> writes:
>[color=green]
>> I was given two used SCSI DDS-4 tape drives, and I am testing them for
>> use in making backups.
>>
>> One is an HP DAT40i (aka C5686A, C5683A) and the other is a Compaq
>> SDT-10000 (EOD006, .
>>
>> I find that each drive can read the tapes that it writes, but that
>> neither drive can read the tapes written by the other.
>>
>> I have put a cleaning tape through each drive, and the firmware of
>> each drive is up-to-date. Both drives pass the diagnostics in the HP
>> Library & Tape Tool.
>>
>> Shouldn't these drives be able to read each other's tapes? Do I need
>> to adjust some parameters, or should I assume that at least one of
>> them is defective?[/color]
>
> DDS-4 is DDS-4, but I'm not certain that they would use the same
> hardware compression technique, if you have that enabled via the
> jumpers on the drives. You should be able to disable compression
> using "mt compression 0".
>
> Scott[/color]


Thanks to everybody for your help so far.

I set DIP switches on the drive to disable compression.

I find that the HP DDS Drives User Manual that I mention elsewhere on
the thread states that the DAT 40GB drive uses DCLZ compression. The
Sony docs for the Compaq drive say that too.

I find that both drives can read each other's tapes, but I think it was
probably not fixed by turning off compression. It gets tedious to
experiment, as I shut down the system when I change drives, but I will
settle that question after a while.

To fill in some details: Each drive reads its own tapes with ease. It
seems the HP can read and write about 1/3 faster than the Compaq. The
Compaq reads the HP's tapes pretty well, but the HP makes a lot of
awful noises as if doing some kind of head-loading or positioning
operation. I know these noises are normal for this drive, but they
shouldn't be so frequent. Pretty clearly it is retrying a failed
operation. In fact when I use du to look at what tar is writing on the
disk during extraction, I can see the output files fail to grow while
there is a lot of noise from the drive. It always succeeds eventually,
and the data comes out exactly correct each time---for instance, I made
the HP read and verify a tape of ten 100MB archives made by the Compaq.
But it takes the HP a lot of noise and something like five times as
much time to read the Compaq's tapes as it takes to read its own tapes.

Maybe it was the compression, but I also think it may have been that the
drives were dirty. I have used a cleaning tape on each one about five
times.

Another change I made was to install the mt-st package, which apparently
has its own device drivers.

I also have been more patient. Rather than killing the job when the
drive makes so much noise, I am now virtually plugging my ears and
letting it run.

I found this text at
[url]http://grox.net/doc/linux/howto-OLD-VERSIONS/SCSI-HOWTO-html/SCSI-HOWTO-8.html[/url]

[color=blue]
> Problems taking tapes to/from other systems
>
> You can't read a tape made with another operating system or another operating system can't read a tape written in Linux.
>
> Different systems often use different block sizes. On a tape device using a fixed blocksize, you will get errors when reading blocks written using a different block size.
>
> To read these tapes, you must set the blocksize of the tape driver to match the blocksize used when the tape was written, or to variable.
>
> NOTE : this is the hardware block size, not the blocking factor used with tar, dump, etc.
>
> You can do this with the mt command -
>
> mt setblk <size>
>
> or
>
> mt setblk 0
>
> to get variable block length support.
>
> Note that these mt flags are NOT supported under the GNU version of mt which is included with some Linux distributions. Instead, you must use the BSD derived Linux SCSI mt command. Source should be available from
>
> tsx-11.mit.edu:/pub/linux/ALPHA/scsi
>
> Also note that by default, ST_BUFFER_BLOCKS (defined in /usr/src/linux/drivers/scsi/st_options.h in newer kernels, st.c in older kernels) is set to allow for a 32K maximum buffer size; you'll need to edit the source to use larger blocks.[/color]


Here is the output of mt-st -f /dev/nst0 status:

SCSI 2 tape drive:
File number=10, block number=0, partition=0.
Tape block size 0 bytes. Density code 0x26 (DDS-4 or QIC-4GB).
Soft error count since last status=0
General status bits on (81010000):
EOF ONLINE IM_REP_EN

Can anybody offer any wisdom about changing the block size? I'm
thinking maybe the two drives are using two different block sizes and
that the Compaq firmware copes better with the difference than the HP does.

Baron wrote:[color=blue]
> Matt wrote:
>[color=green]
>> John F. Morse wrote:[color=darkred]
>>> [Crossposts removed and Followup set to a.o.l.u]
>>>
>>> Matt wrote:
>>>> I find that each drive can read the tapes that it writes, but that
>>>> neither drive can read the tapes written by the other.[/color][/color]
>
> Are you surprised ? Each manufacturer uses its own data format and
> track layout. I've been in situations where a manufacturer has used a
> different format on different models !
>
> Its called "Lock In" ![/color]


Do you have experience using DDS-4 tape drives?

Matt wrote:
[color=blue]
> Baron wrote:[color=green]
>> Matt wrote:
>>[color=darkred]
>>> John F. Morse wrote:
>>>> [Crossposts removed and Followup set to a.o.l.u]
>>>>
>>>> Matt wrote:
>>>>> I find that each drive can read the tapes that it writes, but that
>>>>> neither drive can read the tapes written by the other.[/color]
>>
>> Are you surprised ? Each manufacturer uses its own data format and
>> track layout. I've been in situations where a manufacturer has used
>> a different format on different models !
>>
>> Its called "Lock In" ![/color]
>
>
> Do you have experience using DDS-4 tape drives?[/color]

Depends upon what you mean "experience" !

In general I do whatever the client requires. Tape & tape hardware
tends to be a long term thing ! The problems start when replacements
can't be found and repairs become uneconomical. That is when
businesses find that their archives are just so much junk. In terms of
backup systems you wouldn't belive how people rely on them without
testing that they actually work.

--
Best Regards:
Baron.

In article <4KWXj.234$Vf.88@fe093.usenetserver.com>,
Matt <matt@themattfella.xxxyyz.com> wrote:
[color=blue]
> Shouldn't these drives be able to read each other's tapes? Do I need to
> adjust some parameters, or should I assume that at least one of them is
> defective?[/color]

In general, yes ... however even if you do everything correctly, it's
still possible to produce tapes on one DDS drive that are not readable
on another.

I have had this issue in the past. I think the problem has to do with
head alignment. I've heard of others having the same issue, so I don't
think it's particularly uncommon. (In my case, I had two basically
identical HP-12000s that would never read each other's tapes perfectly.)

Anyway, it sounds like maybe there's an issue with the hardware
compression also, but I just wanted to toss out there that even if you
get everything else resolved, you still may not have perfect
compatibility between devices.

It's one of the reasons I finally gave up on DDS autoloaders and moved
to HD-based backups.

Good luck!
-Kadin.

Kadin2048 wrote:[color=blue]
> In article <4KWXj.234$Vf.88@fe093.usenetserver.com>,
> Matt <matt@themattfella.xxxyyz.com> wrote:
>[color=green]
>> Shouldn't these drives be able to read each other's tapes? Do I need to
>> adjust some parameters, or should I assume that at least one of them is
>> defective?[/color]
>
> In general, yes ... however even if you do everything correctly, it's
> still possible to produce tapes on one DDS drive that are not readable
> on another.
>
> I have had this issue in the past. I think the problem has to do with
> head alignment. I've heard of others having the same issue, so I don't
> think it's particularly uncommon. (In my case, I had two basically
> identical HP-12000s that would never read each other's tapes perfectly.)[/color]

Head alignment used to be a problem with tape drives. And the greater the
density written on the tapes, the greater the need for correct alignment.

Ecrix designed a tape drive that worked somewhat differently from other tape
drives, so alignment is much less critical. Ecrix was bought out by Exabyte
who, in turn, has been bought out by Tandberg. I have two of their drives,
one on each of my two computers. I can easily read tapes written on one and
played on the other. This was not true with DDS-2 tapes that I frequently
could not read even on the machine that write them. I also used some other
tapes that used a floppy controller to run them (I think they were called
Travan, or something like that). They were not too bad, but they were too
slow and too small even then; I think I could get around a gigabyte on
them). The DDS-2 were too small by todays standards; my RAM on my present
computer could fill one of those when using compressed mode.[color=blue]
>
> Anyway, it sounds like maybe there's an issue with the hardware
> compression also, but I just wanted to toss out there that even if you
> get everything else resolved, you still may not have perfect
> compatibility between devices.
>
> It's one of the reasons I finally gave up on DDS autoloaders and moved
> to HD-based backups.[/color]

I think it depends on your requirements whether disk or tape is the best for
any individual. It is a trade-off, and YMWCV -- Your mileage will certainly
vary.[color=blue]
>
> Good luck!
> -Kadin.[/color]


--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey [url]http://counter.li.org[/url]
^^-^^ 15:55:01 up 4 days, 5:52, 4 users, load average: 4.31, 4.20, 4.11

Matt <matt@themattfella.xxxyyz.com> writes:

[snip]
[color=blue]
> Thanks to everybody for your help so far.
>
> I set DIP switches on the drive to disable compression.
>
> I find that the HP DDS Drives User Manual that I mention elsewhere on
> the thread states that the DAT 40GB drive uses DCLZ compression. The
> Sony docs for the Compaq drive say that too.
>
> I find that both drives can read each other's tapes, but I think it
> was probably not fixed by turning off compression. It gets tedious to
> experiment, as I shut down the system when I change drives, but I will
> settle that question after a while.
>
> To fill in some details: Each drive reads its own tapes with ease. It
> seems the HP can read and write about 1/3 faster than the Compaq. The
> Compaq reads the HP's tapes pretty well, but the HP makes a lot of
> awful noises as if doing some kind of head-loading or positioning
> operation. I know these noises are normal for this drive, but they
> shouldn't be so frequent. Pretty clearly it is retrying a failed
> operation. In fact when I use du to look at what tar is writing on
> the disk during extraction, I can see the output files fail to grow
> while there is a lot of noise from the drive. It always succeeds
> eventually, and the data comes out exactly correct each time---for
> instance, I made the HP read and verify a tape of ten 100MB archives
> made by the Compaq. But it takes the HP a lot of noise and something
> like five times as much time to read the Compaq's tapes as it takes to
> read its own tapes.[/color]

What you are describing is probably not bad bits on the tape, but loss
of streaming. DDS drives try to keep the tape moving even if another
read request hasn't come in yet. If there is a big enough delay
before the read request finally comes, then it will have to rewind the
tape and try again.
[color=blue]
> Maybe it was the compression, but I also think it may have been that
> the drives were dirty. I have used a cleaning tape on each one about
> five times.
>
> Another change I made was to install the mt-st package, which
> apparently has its own device drivers.[/color]

Good! This package is highly useful. No, it doesn't have device
drivers.
[color=blue]
> I also have been more patient. Rather than killing the job when the
> drive makes so much noise, I am now virtually plugging my ears and
> letting it run.
>
> I found this text at
> [url]http://grox.net/doc/linux/howto-OLD-VERSIONS/SCSI-HOWTO-html/SCSI-HOWTO-8.html[/url]
>
>[color=green]
>> Problems taking tapes to/from other systems
>>
>> You can't read a tape made with another operating system or another operating system can't read a tape written in Linux.
>>
>> Different systems often use different block sizes. On a tape device using a fixed blocksize, you will get errors when reading blocks written using a different block size.
>>
>> To read these tapes, you must set the blocksize of the tape driver to match the blocksize used when the tape was written, or to variable.
>>
>> NOTE : this is the hardware block size, not the blocking factor used with tar, dump, etc.
>>
>> You can do this with the mt command -
>>
>> mt setblk <size>
>>
>> or
>>
>> mt setblk 0
>>
>> to get variable block length support.
>>
>> Note that these mt flags are NOT supported under the GNU version of mt which is included with some Linux distributions. Instead, you must use the BSD derived Linux SCSI mt command. Source should be available from
>>
>> tsx-11.mit.edu:/pub/linux/ALPHA/scsi
>>
>> Also note that by default, ST_BUFFER_BLOCKS (defined in /usr/src/linux/drivers/scsi/st_options.h in newer kernels, st.c in older kernels) is set to allow for a 32K maximum buffer size; you'll need to edit the source to use larger blocks.[/color][/color]

The above information you've quoted is old. The default maximum block
size is still 32K, but with current 2.6 kernels, you can increase that
by putting "options st buffer_kbs=nnn" in your /etc/modprobe.conf.
(Replace the "nnn" with the number of K bytes you want.) It used to be
that there was a limit of just under 128K, but I just tried loading
the module with buffer_kbs set to 256, and it didn't complain. You
can change the number, then "rmmod st" and "modprobe st" to reload the
kernel module with a different buffer size--no need to reboot. Look
at /var/log/messages to see what's happening when the module is
loaded.
[color=blue]
> Here is the output of mt-st -f /dev/nst0 status:
>
> SCSI 2 tape drive:
> File number=10, block number=0, partition=0.
> Tape block size 0 bytes. Density code 0x26 (DDS-4 or QIC-4GB).
> Soft error count since last status=0
> General status bits on (81010000):
> EOF ONLINE IM_REP_EN
>
> Can anybody offer any wisdom about changing the block size? I'm
> thinking maybe the two drives are using two different block sizes and
> that the Compaq firmware copes better with the difference than the HP
> does.[/color]

Notice that this tape drive is set to a block size of 0. That means
the block size is variable. The size of the block will be determined
by the size of the data buffer in the write call given by the
application. If you are using tar, look at the "-b" option in its
man page. This shows that with the default "-b 20" you should be
getting 10K blocks. On reading, a block up to the size of the kernel
module's buffer can be read and handed to the application. I don't
know if a fixed block size will improve the read performance of your
HP drive, but it's worth a shot. I would definitely increase the
block size, allowing you to fit more data on the tape. I've used 32K
in the past with DDS-3 drives, but you needn't feel constrained to
that figure just because it's the default.

If you want to use software compression, use bzip2 instead of gzip
(tar's "-j" option instead of "-z"). The main reason is that bzip2
can recover from a data error. If there is a data error in gzip'd
data, you lose the rest of the tape file. Also, bzip2 compresses
better. If you use software compression, you should also use tar's
"--block-compress" option.

Scott
--
Scott Hemphill [email]hemphill@alumni.caltech.edu[/email]
"This isn't flying. This is falling, with style." -- Buzz Lightyear

Scott Hemphill wrote (in part):
[color=blue]
> What you are describing is probably not bad bits on the tape, but loss of
> streaming. DDS drives try to keep the tape moving even if another read
> request hasn't come in yet. If there is a big enough delay before the
> read request finally comes, then it will have to rewind the tape and try
> again.
>[/color]
This illustrates one of the advantages of VXA tape drives over DDS. The VXA
drives are able to slow down the speed of the tape dynamically if the
reading or writing process cannot keep up with the speed of the drive. So it
is not necessary to stop the drive, reverse it, stop it, and then restart it
when the speed of the computer and the speed of the drive do not match. This
saves time, tape, noise, and trouble.

You may not have the option of switching tape drives, but if you do, you may
wish to examine these drives:

[url]http://www.exabyte.com/products/prodviewstype.cfm?v=drive[/url]

I use both VXA-1 and VXA-2 drives on SCSI controllers.

The VXA-2 has twice the speed and capacity of a VXA-1.
The VXA-320 has twice the speed and capacity of a VXA-2.

--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey [url]http://counter.li.org[/url]
^^-^^ 16:00:01 up 6 days, 5:57, 4 users, load average: 4.40, 4.41, 4.43