CCD mirroring setup difficulties: eventual success & micro HOWTO - BSD

This is a discussion on CCD mirroring setup difficulties: eventual success & micro HOWTO - BSD ; I was trying to set up a simple mirror (ie, software raid 1) under OpenBSD 3.8 using CCD and the generic kernel, mirroring the entirety of two one-partition disks. This should have been bone-simple but it was pretty frustrating. The ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: CCD mirroring setup difficulties: eventual success & micro HOWTO

  1. CCD mirroring setup difficulties: eventual success & micro HOWTO

    I was trying to set up a simple mirror (ie, software raid 1) under
    OpenBSD 3.8 using CCD and the generic kernel, mirroring the entirety of
    two one-partition disks. This should have been bone-simple but it was
    pretty frustrating. The various net HOWTOs on how to set up a mirror
    with ccd/ccdconfig were less than fully informative (and in some places
    just plain wrong). And it doesn't help when mis-specifying the
    parameters to simple commands like newfs will give you instant kernel
    panics and crash your system hard...

    I'm not sure why this area of the system's not as robust as everything
    else in OpenBSD, but in case it helps others, here's what I found I had
    to do to not get the usual failures and error messages and to make it
    work. (Google bait follows: " inappropriate file type or format /
    CCDIOCSET / error 22 on component 0 / device not configured / write
    error ")

    My scheme:

    0) Obviously, get the bios to recognize the two disks (my fault in this
    case, since I had to find the semi-hidden setting to turn on the
    secondary IDE channel in the BIOS). They should both be seen by the
    kernel at boot. FWIW, I've got two Seagate 40GB ide drives, wd0 and
    wd1.

    1) fdisk -i wd0 (ditto for wd1)
    Writes a default partition table on each drive with one slice taking up
    the whole drive.

    2) disklabel -e -d wd0 (ditto for wd1)
    This gets you an "a" partition on each diskthat takes up the whole disk
    (minus the first cylinder). Change the "fstype" field in this first
    partition from "unused" to "4.2BSD". Ignore the "c" partition that
    represents the whole physical disk.

    3) newfs wd0a (ditto for wd1a)
    Make an identical file system on each drive *separately*, not on the
    combined mirror. (Don't ask me why, all I know is this what made it
    work for me.) Use whatever newfs options appeal to you, as long as both
    drives are configured identically.

    4) edit /etc/ccd.conf:
    Mine reads:
    ccd0 16 CCDF_MIRROR /dev/wd0a /dev/wd1a
    (No, I don't know what the interleave factor of "16" does for a mirror
    set, either. I just left it alone.)

    5) ccdconfig -C
    This creates your mirror. Check it's there with ccdconfig -v -g.
    In my case, I got a new disk, /dev/ccd0c, for my troubles.

    6) mount the mirror set and edit /etc/fstab so it's permanent:
    For me, this was done with "mount /dev/ccd0c /archive". My final
    /etc/fstab entry looks like:
    /dev/ccd0c /samba ffs rw,nodev,nosuid,noexec 1 2
    (of course you may have different mount options than these...this is my
    archive backup disk set so I don't expect to be putting executables
    there)

    Hope this helps, but if not post your questions to the group, not to me
    - all I know about the subject I've just written here. Good luck.

    -jon-


  2. Re: CCD mirroring setup difficulties: eventual success & micro HOWTO

    Jonathan Rogers wrote:
    > I was trying to set up a simple mirror (ie, software raid 1) under
    > OpenBSD 3.8 using CCD and the generic kernel, mirroring the entirety of
    > two one-partition disks. This should have been bone-simple but it was
    > pretty frustrating. The various net HOWTOs on how to set up a mirror
    > with ccd/ccdconfig were less than fully informative (and in some places
    > just plain wrong). And it doesn't help when mis-specifying the
    > parameters to simple commands like newfs will give you instant kernel
    > panics and crash your system hard...


    Indeed, OpenBSD is not forgiving when it comes to messing around with
    filesystems and drives.

    > I'm not sure why this area of the system's not as robust as everything
    > else in OpenBSD, but in case it helps others, here's what I found I had
    > to do to not get the usual failures and error messages and to make it
    > work. (Google bait follows: " inappropriate file type or format /
    > CCDIOCSET / error 22 on component 0 / device not configured / write
    > error ")
    >
    > My scheme:
    >
    > 0) Obviously, get the bios to recognize the two disks (my fault in this
    > case, since I had to find the semi-hidden setting to turn on the
    > secondary IDE channel in the BIOS). They should both be seen by the
    > kernel at boot. FWIW, I've got two Seagate 40GB ide drives, wd0 and
    > wd1.
    >
    > 1) fdisk -i wd0 (ditto for wd1)
    > Writes a default partition table on each drive with one slice taking up
    > the whole drive.


    Okay, better make sure there's nothing on it.

    > 2) disklabel -e -d wd0 (ditto for wd1)
    > This gets you an "a" partition on each diskthat takes up the whole disk
    > (minus the first cylinder). Change the "fstype" field in this first
    > partition from "unused" to "4.2BSD". Ignore the "c" partition that
    > represents the whole physical disk.


    You should be using ccd for ccd devices, see ccd(4) (on a 3.8 system, I
    assume it'll be similar for older systems).

    Okay, it will not help much - but it's cleaner, and better for forward
    compatibility.

    > 3) newfs wd0a (ditto for wd1a)
    > Make an identical file system on each drive *separately*, not on the
    > combined mirror. (Don't ask me why, all I know is this what made it
    > work for me.) Use whatever newfs options appeal to you, as long as both
    > drives are configured identically.


    Should not be necessary. Looks like voodoo to me.

    > 4) edit /etc/ccd.conf:
    > Mine reads:
    > ccd0 16 CCDF_MIRROR /dev/wd0a /dev/wd1a
    > (No, I don't know what the interleave factor of "16" does for a mirror
    > set, either. I just left it alone.)
    >
    > 5) ccdconfig -C
    > This creates your mirror. Check it's there with ccdconfig -v -g.
    > In my case, I got a new disk, /dev/ccd0c, for my troubles.
    >
    > 6) mount the mirror set and edit /etc/fstab so it's permanent:
    > For me, this was done with "mount /dev/ccd0c /archive". My final
    > /etc/fstab entry looks like:
    > /dev/ccd0c /samba ffs rw,nodev,nosuid,noexec 1 2
    > (of course you may have different mount options than these...this is my
    > archive backup disk set so I don't expect to be putting executables
    > there)
    >
    > Hope this helps, but if not post your questions to the group, not to me
    > - all I know about the subject I've just written here. Good luck.


    Now, there's just been a lengthy thread on misc@openbsd.org where some
    very knowledgeable fellows, including the RAIDframe maintainer, told
    someone who had also just set up a CCD mirror and posted instructions
    similar to yours that a) HOWTOs are evil and b) ccd mirroring will eat
    your data (so use RAIDframe).

    I tend to agree with a) here - your instructions are not wrong, but not
    always correct either, and may tempt other to just type them without
    understanding them. Additionally, b) is very, very important if your
    data is important enough to mirror - it's likely to be important enough
    to mirror right, too.

    Joachim

  3. Re: CCD mirroring setup difficulties: eventual success & micro HOWTO

    >> 3) newfs wd0a (ditto for wd1a)
    > >Make an identical file system on each drive *separately*, not on the
    > >combined mirror. (Don't ask me why, all I know is this what made it work for me.)


    >Should not be necessary. Looks like voodoo to me.


    I agree it's voodoo - but without setting up the file systems in
    advance on the separate drives, when I tried to do the newfs on the ccd
    mirrored drive all I got was "error 22 ioctl (CCDIOCSET) on component
    0". This was the only way I found to make it work.

    >someone who had also just set up a CCD mirror and posted instructions
    >similar to yours that a) HOWTOs are evil and b) ccd mirroring will eat
    >your data (so use RAIDframe).
    >...if your data is important enough to mirror - it's likely to be important enough
    >to mirror right, too.


    No argument there. I was avoiding raidframe only because I'm an OpenBSD
    newbie (w/lots of Linux experience) and I was trying to find a solution
    that would still allow me to use the stock/generic kernel for now.
    After some more experience I'll look at switching to raidframe.

    Certainly the kernel panics I got in the initial failed attempts to
    make ccd work don't promote a sense of high confidence and
    well-being....

    thanks =jon=


  4. Re: CCD mirroring setup difficulties: eventual success & micro HOWTO

    Jonathan Rogers wrote:
    > Joachim wrote:
    >>someone who had also just set up a CCD mirror and posted instructions
    >>similar to yours that a) HOWTOs are evil and b) ccd mirroring will eat
    >>your data (so use RAIDframe). ...if your data is important enough to
    >>mirror - it's likely to be important enough to mirror right, too.

    >
    > No argument there. I was avoiding raidframe only because I'm an OpenBSD
    > newbie (w/lots of Linux experience) and I was trying to find a solution
    > that would still allow me to use the stock/generic kernel for now.
    > After some more experience I'll look at switching to raidframe.
    >
    > Certainly the kernel panics I got in the initial failed attempts to
    > make ccd work don't promote a sense of high confidence and
    > well-being....


    Yes, I have observed the same. RAIDframe is not much better there.

    That said, RAIDframe has saved my behind at least once when a disk
    crashed. Recovery was not totally painless (the disk confused the heck
    out of the IDE bus, taking down the other attached disk as well, so it
    was not immediately obvious which one had failed), but ultimately very
    succesful.

    Also, it's very stable once you set it up and stop poking it - I've had
    quite a few problems and panics setting it up, though.

    That being said, raid(4) has instructions for creating a custom kernel.
    I use this one for single-processor PCs with RAID:

    include "arch/i386/conf/GENERIC"
    pseudo-device raid 4 # RAIDframe
    option RAID_AUTOCONFIG

    Where 'option RAID_AUTOCONFIG' is only necessary if you want root on
    RAID.

    Instructions for compiling a new kernel are in the FAQ, 5.3. Take the
    opportunity to upgrade to -stable while you're there. It's very
    painless, though it's best to do it while not using the machine or to
    run it with nice, as it produces quite a bit of system load.

    Joachim

+ Reply to Thread