This is a discussion on Secure backups with GPG: local SCSI tape with DUMP, two remote streaming backups using MKISOFS and TAR - BSD ; I have tested using GnuPG to encrypt and decrypt a filesystem backup using DUMP, TAR, and MKISOFS. The first test went to a locally attached SCSI-2 (Conner) tape drive, and the latter two were cross system from Solaris to FBSD ...
I have tested using GnuPG to encrypt and decrypt a filesystem backup using
DUMP, TAR, and MKISOFS. The first test went to a locally attached SCSI-2
(Conner) tape drive, and the latter two were cross system from Solaris to
FBSD SCSI0-2 tape/CD-ROM. I have also tested using Solaris
USFDUMP/UFSRESTORE with a "GnuPG, DD, FreeBSD" pipline, but the
INTERACTIVE restore function always failed due to the way that UFSRESTORE
handles stdout, stdin, and stderr. Where file/system backups don't already
have an (automatic) method of encrypting/decrypting backup data, these
tests may provide some feasible possibilities. Note: blocksizes used come
from either older versions of Solaris UFSDUMP or a "relative guess" for
DD. Don't view these values as the "best", "correct", or "expected" values
to use. YMMV. Last, but not least - choices for keys (types, passphrases,
lengths, etc.) and alternative methods are left to the reader.
GPG Encrypted System Backup To Local SCSI-2 Tape Drive
It is possible to use GnuPG to encrypt an entire filesystem backup (ie;
Unix DUMP) and later decrypt and restore this encrypted backup. The
(BSD) Unix RESTORE can even be done in interactive mode.
The following example uses (BSD) Unix DUMP and RESTORE commands, GnuPG,
and Unix DD command, with a (true) SCSI2 (Conner, 4GB, QIC-3080 Wide)
tape drive. Block size values are not well defined so a "random" choice
of 4096 was used. In normal DUMP/RESTORE use, the density value is set
at 32760. I suspect, but have not yet proven, that block size on both
sides of the GPG pipe should be the same, though it may not be necessary
to use the BS parameter in the second DD command. It also may not have
been necessary to use the DD "conv=notrunc conv=sync" parameters, but
not using them in this experiment has not been tested yet.
The reason that DD was used is that GPG doesn't work well writing
directly to tape. At EOF, it issues several error messages, possibly due
to some unanticipated interaction from the tape drive, and fails to
write out its encryption block marker, thereby making the backup
Note that BSD Unix DUMP has the "-P" parameter, but other Unix's do not.
Also note that manual tape rewinds were (sometimes) necessary even
though the "automatic tape rewind" device was chosen.
1) dump -0ad 32760 -P /root/gpg_dd_to_tape /dev/ad1s1d.eli
DUMP: WARNING: should use -L when dumping live read-write filesystems!
DUMP: Date of this level 0 dump: Tue Sep 12 16:15:07 2006
DUMP: Date of last level 0 dump: the epoch
DUMP: Dumping /dev/ad1s1d.eli to child pipeline process
DUMP: mapping (Pass I) [regular files]
DUMP: mapping (Pass II) [directories]
DUMP: estimated 282 tape blocks.
DUMP: dumping (Pass III) [directories]
DUMP: dumping (Pass IV) [regular files]
DUMP: DUMP: 310 tape blocks on 1 volume
DUMP: finished in less than a second
45+1 records in
46+0 records out
188416 bytes transferred in 7.012598 secs (26868 bytes/sec)
DUMP: Closing child pipeline process
DUMP: DUMP IS DONE
2) cat /root/gpg_dd_to_tape
gpg --quiet --batch --armor --recipient
| dd bs=4096 conv=sync conv=notrunc of=/dev/sa0
3) restore -i -P /root/dd_fromtape_togpg
46+0 records in
46+0 records out
188416 bytes transferred in 3.130139 secs (60194 bytes/sec)
restore > ls
AppleTalk.zones GnuPG_manual.pdf news5pm.space
restore > ?
Available commands are:
ls [arg] - list directory
cd arg - change directory
pwd - print current directory
add [arg] - add `arg' to list of files to be extracted
delete [arg] - delete `arg' from list of files to be extracted
extract - extract requested files
setmodes - set modes of requested directories
quit - immediately exit program
what - list dump header information
verbose - toggle verbose flag (useful with ``ls'')
help or `?' - print this list
If no `arg' is supplied, the current directory is used
restore > add *
restore > ls
*AppleTalk.zones *GnuPG_manual.pdf *news5pm.space
restore > verbose
verbose mode on
restore > extract
Extract requested files
You have not read any tapes yet.
If you are extracting just a few files, start with the last volume
and work towards the first; restore can quickly skip tapes that
have no further files to extract. Otherwise, begin with volume 1.
Specify next volume #: 1
extract file ./AppleTalk.zones
extract file ./news5pm.space
extract file ./GnuPG_manual.pdf
Set directory mode, owner, and times.
set owner/mode for '.'? [yn] n
restore > q
4) cat /root/dd_fromtape_togpg
dd if=/dev/sa0 bs=4096 conv=sync conv=notrunc | gpg --quiet --batch
Use popen(3) to execute the sh(1) script string defined by
pipecommand for the output device of each volume. This child
pipeline's stdin (/dev/fd/0) is redirected from the dump output
stream, and the environment variable DUMP_VOLUME is set to the
current volume number being written. After every volume, the
writer side of the pipe is closed and pipecommand is executed
again. Subject to the media size specified by -B, each volume is
written in this manner as if the output were a tape drive.
Use popen(3) to execute the sh(1) script string defined by
pipecommand as the input for every volume in the backup. This
child pipeline's stdout (/dev/fd/1) is redirected to the restore
input stream, and the environment variable RESTORE_VOLUME is set
to the current volume number being read. The pipecommand script
is started each time a volume is loaded, as if it were a tape
GPG Encrypted Remote System Backup To Local SCSI-2 Tape Drive Using TAR
I have not yet been able to successfully restore an encrypted Solaris
UFSDUMP with a "dd-gpg-UFSRETORE" pipe. Solaris UFSRESTORE opens up
another file descriptor when it prompts the user to set permission bits,
and I haven't figured out how to get EXPECT to deal with that (yet).
The GOOD news is that I have successfully used a "tar-gpg-dd/dd-gpg-tar"
encrypted backup and restore successfully on Solaris Sparc(9) with "no runs,
no drips, and no errors". I had to adjust the DD parameters as it was
chopping off the last partial block, and a larger (4096 vs default of 512)
block size meant faster dumps and restores. Here is what I have tested so
/usr/bin/tar cf - /var/stage | /usr/bin/dd obs=4096 conv=notrunc |
/usr/local/bin/gpg --armor --encrypt --batch --output=- --recipient
| /usr/bin/ssh "dd ibs=4096 obs=4096
/bin/dd bs=4096 if=/dev/sa0 |
/usr/local/bin/gpg --batch --quiet --decrypt --output=- --recipient
--passphrase |/usr/bin/tar tvf -
GPG Encrypted Remote System Backup To Local CD-ROM Using MKISOFS
I have successfully tested a secure "streaming" backup to a remote system
which used MKISOFS to create an ISO image file on-the-fly. No intermediate
files are used. The following MKISOFS (Solaris and BSD, at least) parameter
allows MKISOFS to read a datastream from STDIN. One caveat about "media size"
- it appears to be 2KB "sectors".
My first tests that used the same number of 512-byte blocks, as returned by
DU -S, created an ISO image file that was about 5x the size of the original
data. I did this because multiple references to "sectors" in the MKISOFS
MANual page mentioned "512 byte sector", though they were mostly talking
about BOOT sectors. Attempting to burn a 1+GB "CD" failed, duh! Looking at
the BURNCD (CDRECORD) command output about the data actually burned and the
reported "blocks" lead to the 2KB block size estimation.
This test is slightly flawed in that my CD burner is scratching the crap out
of the CD's and fails to fixate them. Nevertheless, the ISO image file was
mounted on the remote system and read from the "restore pipe". A locally
mounted CD would use a slightly modified "restore pipe" than was used in this
test. Nevertheless, this test does provide a "proof of concept".
Select streaming operation and set the media size to #
sectors. This allows you to pipe the output of the tar
program into mkisofs and to create a iso9660 filesystem
without the need of an intermediate tar archive file.
If this option has been specified, mkisofs reads from
stdin and creates a file with the name STREAM.IMG. The
maximum size of the file (with padding) is 200 sectors
less than the specified media size. If -no-pad has been
specified, the file size is 50 sectors less than the
specified media size. If the file is smaller, then
mkisofs will write padding. This may take a while.
The option -stream-media-size creates simple iso9660
filesystems only and may not used together with multi-
session or hybrid filesystem options.
du -sk /var/stage
mkisofs "disk image size" = # of 2048 byte "sectors"
(438063KB * 1024) / 2048 = 219031.5
/usr/bin/tar cf - /var/stage | dd obs=2048 conv=notrunc | /usr/local/bin/gpg
--armor --encrypt --batch --output=- --recipient
-stream-media-size 219032 -o /var/tmp/iso.toburn"
"/bin/dd if=/mnt/stream.img" | /usr/local/bin/gpg
--batch --decrypt --output=- --recipient
| tar -tvf - > /opt/tmp/secure.tar.iso.restore.output
"/bin/dd if=/mnt/stream.img" |
/usr/local/bin/gpg --batch --decrypt --output=- --recipient
--passphrase | tar xvf -
The output was the exact same output as the previous "tar-gpg-dd-tape" secure
backup and restore output. Also, a (real) file extraction from the secure
backup ISO image was successfully restored.
Note that all of these tests (with/without tape) used the fact of being able
to read the PGP (OpenPGP) header and trailer "tags" in the "dump file" after
the dump as verification that the dump was successful, as well as, obviously,
a later restore.
OBVIOUSLY - one must choose a filesystem, in this methodology, that will fit
on the remote media (CD or DVD), or find a way to limit the amount of files
and directories that are included in the dump datastream. TAR exclusion
parameters might work well enough for this.