Bug#497110: boot loader installation failed when dmraid=true - Debian

This is a discussion on Bug#497110: boot loader installation failed when dmraid=true - Debian ; On Sunday 07 September 2008, Giuseppe Iuculano wrote: > Giuseppe Iuculano ha scritto: > > Approximately "dmraid -rD" generates three files, from this we can > > extrapolate metadata, and with a hex editor we can add the fake > ...

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2
Results 21 to 38 of 38

Thread: Bug#497110: boot loader installation failed when dmraid=true

  1. Re: Bug#497110: improved dmraid support in D-I

    On Sunday 07 September 2008, Giuseppe Iuculano wrote:
    > Giuseppe Iuculano ha scritto:
    > > Approximately "dmraid -rD" generates three files, from this we can
    > > extrapolate metadata, and with a hex editor we can add the fake
    > > signature to the qemu/virtualbox drive


    Let's take this out of the BTS as it is unrelated to the BR.

    > Ok,this procedure seems to work for me:
    > dmraid -rD and i got:

    [...]

    *.size is the disk size in sectors.
    hda_{0,1,2,3}_sil.dat are all identical, but there are some differences
    between .dat files for hda and hdb.
    *.offset files are numbers close to the end of the disk, so apparently the
    SATA RAID signature is saved 4 times near the end of the disk for
    redundancy.

    The offsets are probably some fixed distance from the end of the disk.

    Question is if the disk size is encoded in the .dat files. IMO the main
    reason to use the procedure you devised would be to generate disk images
    of a different size (80GB was a bit of a pain for the conversion from
    vbox to VirtualBox format for example).


    --
    To UNSUBSCRIBE, email to debian-boot-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  2. Re: Bug#497110: improved dmraid support in D-I

    Frans Pop ha scritto:

    > *.size is the disk size in sectors.
    > hda_{0,1,2,3}_sil.dat are all identical, but there are some differences
    > between .dat files for hda and hdb.
    > *.offset files are numbers close to the end of the disk, so apparently the
    > SATA RAID signature is saved 4 times near the end of the disk for
    > redundancy.


    In this case yes (Silicon Image fakeraid), but this isn't true always.


    > Question is if the disk size is encoded in the .dat files. IMO the main


    Yes, you can find Silicon Image metadata layout on dmraid source,
    1.0.0.rc14/lib/format/ataraid/sil.h

    > reason to use the procedure you devised would be to generate disk images
    > of a different size (80GB was a bit of a pain for the conversion from
    > vbox to VirtualBox format for example).


    Uhmm, there are some unknown metadata sectors, I think this isn't possible.


    Giuseppe.


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (GNU/Linux)

    iEYEARECAAYFAkjELhMACgkQNxpp46476arAggCfdmzntPG6sZ LH9C5fFLt9BB9a
    kaAAn2UxGHR3X4IJ2MbRd4DxULi148rm
    =adho
    -----END PGP SIGNATURE-----


  3. Re: Bug#497110: improved dmraid support in D-I

    On Sunday 07 September 2008, Frans Pop wrote:
    > The offsets are probably some fixed distance from the end of the disk.


    The offsets appear to be:
    for i in 0 1 2 3; do
    echo $(( ($(cat hda_sil.size) - ($i * 512 + 1)) * 512 ))
    done

    fdisk -ul /dev/hda reports:
    80026361856 bytes; 156301488 sectors

    fdisk -ul /dev/mapper/sil_* reports:
    80025313280 bytes; 156299440 sectors

    So that is 2048 sectors less, which matches 4 * 512 in which this metadata
    lives.

    In the .dat files:
    - the value 156299440 (b0f05009) can be found at offset 108
    - the value 156301488 (b0f85009) can be found at offsets 120 and 200


    --
    To UNSUBSCRIBE, email to debian-boot-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  4. Re: Bug#497110: improved dmraid support in D-I

    Giuseppe Iuculano ha scritto:

    >> reason to use the procedure you devised would be to generate disk images
    >> of a different size (80GB was a bit of a pain for the conversion from
    >> vbox to VirtualBox format for example).

    >
    > Uhmm, there are some unknown metadata sectors, I think this isn't possible.



    Ok, with Highpoint 45X ATARAID this is simple.

    we have 11 blocks, so for example for a 4GB disk image:

    dd if=/dev/null of=hda.dsk seek=8388608
    dd if=/dev/null of=hdb.dsk seek=8388608

    8388608-11=8388597 , so:

    dd if=hpa.dat of=hda.dsk conv=notrunc seek=8388597
    dd if=hpa.dat of=hdb.dsk conv=notrunc seek=8388597


    (sorry for an eventual duplicate mail, but it seems that my previous mailwas
    dropped due to .dat attachment)

    Giuseppe


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (GNU/Linux)

    iEUEARECAAYFAkjEOZ8ACgkQNxpp46476arJbACgjnF2KuJSP7 3WbcNTgpo7Xesl
    PVYAmJ3uQ5XtA2Nz5gPwjF4KbnPVxMw=
    =jWCX
    -----END PGP SIGNATURE-----


  5. Re: Bug#497110: improved dmraid support in D-I

    On Sunday 07 September 2008, Giuseppe Iuculano wrote:
    > Ok, with Highpoint 45X ATARAID this is simple.


    Thanks for that! It's close but does not quite work for me yet.
    I get 4GB raw images and can convert those to vdi images for VirtualBox
    (using the same conversion I used for the sil images).

    The physical disks show as 4GB, but the (correctly detected) dmraid device
    is only 100.8MB in size, which is a bit small for actual use :-P

    I get the same if I use the raw images with qemu.

    Any ideas?


    --
    To UNSUBSCRIBE, email to debian-boot-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  6. Bug#497110: improved dmraid support in D-I

    Frans Pop writes:

    > - get an OK for the change in parted from RT (and maybe check that the
    > patch will also get accepted upstream?)


    ..oO(With GNU Parted Release Manager Hat)

    For the patch to be accepted in upstream I see no problem but it will
    only be done after few steps has been done:

    - we do unit tests to reproduce the current issue;
    - Luk (Ubuntu guy) signs the required papers to pass the changes
    copyright to FSF

    I'll contact him to see if he is willing to contribute with us to get
    this done and also look in the rest of Ubuntu changes to see if we can
    get other fixes from it for next release.

    --
    O T A V I O S A L V A D O R
    ---------------------------------------------
    E-mail: otavio@debian.org UIN: 5906116
    GNU/Linux User: 239058 GPG ID: 49A5F855
    Home Page: http://otavio.ossystems.com.br
    ---------------------------------------------
    "Microsoft sells you Windows ... Linux gives
    you the whole house."



    --
    To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  7. Bug#497110: improved dmraid support in D-I

    Greetings all,
    As previously noted in this bug, I was the person who implemented the bulk of this work in Ubuntu. Granted, as already discussed, there are bits that need work to be cleared up, but I was likely to get to those in the near future for Ubuntu's bug fixing work for intrepid, due in October.

    I apologise for not immediately sending my changes back to Debian. Feature freeze for Ubuntu was on August 28, and I was working up till that day to get everything at a point where it worked. Unfortunately that Thursday and Friday were spent on other important Ubuntu pieces, and I have been away from the computer this last week, so I haven't had a chance to follow up with Debian.

    Having said that, I am very happy to see that this is being considered for Lenny, and am willing to work with you all to make sure the code being added is understood, reviewed, and uploaded, as well as making any changes I make for Ubuntu available for Debian now and in the future.

    Feel free to ask me any questions you may have about any code I may have written, particularly related to parted itself, as that patch took a while to work out, due to dmraid's design. I will be responding in turn to other messages in this bug relating to dmraid metadata and its collection, as well as other caviats I've noticed in my testing.

    I am also happy to test on real hardware, as I have a few PCI cards, as well as a recent intel ICH9 board that I can test such configurations as RAID 5, and possibly RAID10/01 if desired.

    Looking forward to working with you all on this project further.

    Regards
    Luke Yelavich
    Ubuntu dmraid maintainer.

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFIxLSrjVefwtBjIM4RAqhTAKDPWz38CE1cyrBVVX0P7N TxY9kI+gCgh6KO
    5moMPY/dJTdWCZ1rzTglmbQ=
    =L0ln
    -----END PGP SIGNATURE-----


  8. Re: Bug#497110: improved dmraid support in D-I

    Frans Pop ha scritto:
    > The physical disks show as 4GB, but the (correctly detected) dmraid device
    > is only 100.8MB in size, which is a bit small for actual use :-P
    >
    > I get the same if I use the raw images with qemu.
    >
    > Any ideas?


    You are right, also for hpt45x disk size is encoded in the .dat file.
    At this point the dat file needs to be modified
    (1.0.0.rc14/lib/format/ataraid/hpt45x.h in dmraid source helps on this) with an
    hex editor, I don't know if I am able to make this, but I will try.


    Giuseppe.


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (GNU/Linux)

    iEYEARECAAYFAkjE3hwACgkQNxpp46476aqD1gCdGH5dhcUvhU Ou3wtFIjlAYmFZ
    04QAnidmiUtq1XNx8Lu/0AjZAIBYvbh4
    =3inB
    -----END PGP SIGNATURE-----


  9. Re: Bug#497110: improved dmraid support in D-I

    On Monday 08 September 2008, Giuseppe Iuculano wrote:
    > Frans Pop ha scritto:
    > > The physical disks show as 4GB, but the (correctly detected) dmraid
    > > device is only 100.8MB in size, which is a bit small for actual use.

    >
    > You are right, also for hpt45x disk size is encoded in the .dat file.
    > At this point the dat file needs to be modified
    > (1.0.0.rc14/lib/format/ataraid/hpt45x.h in dmraid source helps on this)
    > with an hex editor, I don't know if I am able to make this, but I will
    > try.


    Got it. After changing 3 bytes starting at offset 12 to '000080', which is
    corresponds to 8388608 sectors, everything works perfectly.
    4GB is much more convenient for testing in an emulator. Thanks!

    New hpa.dat attached for anybody playing along.

    To create ~4GB VirtualBox disks use these commands:
    $ dd if=/dev/null of=dmraid_hpa-1.raw seek=8388608
    $ dd if=~/tmp/dmraid/hpa.dat of=dmraid_hpa-1.raw conv=notrunc seek=8388597
    $ VBoxManage convertdd dmraid_hpa-1.raw dmraid_hpa-1.vdi
    $ VBoxManage modifyvdi //dmraid_hpa-1.vdi compact
    $ mv dmraid_hpa-1.raw dmraid_hpa-2.raw
    $ VBoxManage convertdd dmraid_hpa-2.raw dmraid_hpa-2.vdi
    $ VBoxManage modifyvdi //dmraid_hpa-2.vdi compact



  10. Bug#497110: improved dmraid support in D-I

    Luke Yelavich writes:

    > Having said that, I am very happy to see that this is being
    > considered for Lenny, and am willing to work with you all to make
    > sure the code being added is understood, reviewed, and uploaded, as
    > well as making any changes I make for Ubuntu available for Debian
    > now and in the future.


    It is nice to have your support on that. Please take a look in
    Giuseppe patches and see if you can spot any missing stuff.

    Specifically about parted, the easiest way to get it merged is start a
    thread on the parted-devel@lists.alioth.debian.org mailing list and
    attach your proposed patch.

    Please note that for parted we use GIT as SCM and you should base your
    work on master branch.

    We can get it furter once you've sent your mail to the mailing list
    and we start the review process there.

    Thanks a lot for all your help and work.

    --
    O T A V I O S A L V A D O R
    ---------------------------------------------
    E-mail: otavio@debian.org UIN: 5906116
    GNU/Linux User: 239058 GPG ID: 49A5F855
    Home Page: http://otavio.ossystems.com.br
    ---------------------------------------------
    "Microsoft sells you Windows ... Linux gives
    you the whole house."



    --
    To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  11. Bug#497110: improved dmraid support in D-I

    Hello Luke,

    Thank you for your mail.

    I implemented the current support in the installer during DebConf last
    year, more out of annoyance with the continuing stream of requests to
    support dmraid than out of any personal interest. By chance someone could
    make some hardware available to me for the duration of the conference.
    Yes, it was a huge hack (and I've always announced and documented it as
    such), but at least it made installing to dmraid an option and it
    provided a basis to improve support.

    Basically I am quite happy to see dmraid supported properly now, at least
    for partitioning. That still leaves bootloader support and bootloader
    installation essentially a hack, or do you have improvements for that
    too?

    I was a bit miffed at first when Giuseppe came up with your patches, but
    that lessened somewhat when I saw they were recent. However, given that
    Debian/I had done the initial support a mail to our list even just to
    announce that you were working on this would have been very much
    appreciated. I could possibly even have helped or simplified things for
    you in a few cases as obviously I know the partman side quite well.

    We're currently still waiting for Otavio to discuss the changes with the
    release team and give the go-ahead. After that getting the changes into
    SVN and uploaded for the installer should be very quick.

    On Monday 08 September 2008, Luke Yelavich wrote:
    > as well as making any changes I make for Ubuntu available for Debian now
    > and in the future.


    Please consider - at least when it comes to changes in the installer
    itself - to work directly in the Debian SVN repository. After all,
    keeping the diff between Debian/D-I and Ubuntu/D-I small only reduces
    effort when Ubuntu rebases on unstable for a new release.

    I'm quite sad to see the diff growing now that Colin Watson is less active
    when it comes to D-I.

    > I am also happy to test on real hardware, as I have a few PCI cards, as
    > well as a recent intel ICH9 board that I can test such configurations
    > as RAID 5, and possibly RAID10/01 if desired.


    That would be great.

    Question: how does booting from fakeraid work with e.g. RAID5? Reading the
    bootsector will work fine, but I don't see how grub can read its stage
    files from a RAID5 array.

    Cheers,
    FJP

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (GNU/Linux)

    iEYEABECAAYFAkjJEa8ACgkQgm/Kwh6ICoT97gCdGzmKTDTdjIfqmrFiVY1XhHY1
    dQgAnj7K9rpDLMQr7hF2D9z3AjHelWiZ
    =SYkk
    -----END PGP SIGNATURE-----


  12. Bug#497110: improved dmraid support in D-I

    On Fri, Sep 12, 2008 at 03:38:47PM EST, Frans Pop wrote:
    > On Friday 12 September 2008, Luke Yelavich wrote:
    > > So far, I have a patch to allow os-prober to find other OSs on dmraid
    > > devices, but nothing for grub-installer yet, as I have been trying to
    > > track down why grub-installer wasn't using os-prober's output for other
    > > OSs.

    >
    > Main reason is that I'm not sure that, even though I'd expect os-prober to
    > detect other OSes correctly, I doubt that grub-installer would create
    > correct entries for them.
    > If you can test that, then adding os-prober support is of course fine.
    > As you can see in the history of the BR, that is the one part of your
    > changes I was unsure about.


    Right. I noticed some debconf templates related to sataraid in grub-installer, but I am not sure what those are for. To try and figure this one out, Iwas thinking of adding a set -x command right after at the point where os-prober's data is fetched, and follow things from there to see why it doesn't use os-prober's output.

    > > Ok, I'll go into detail where necessary later in this mail, but I will
    > > shortly file bugs against all d-i packages I've touched, with patches
    > > attached.

    >
    > That's not needed. Giuseppe has already taken your changes from the diffs
    > between Debian and Ubuntu sources and I already have commits to our SVN
    > repository prepared based on those (with some mostly mostly minor changes
    > and some additions).
    >
    > What would be useful is if you could check those, which is probably
    > easiest to do after they've been committed.


    I've downloaded the respecitve svn branches, so I'll keep an eye on them.


    > > > Question: how does booting from fakeraid work with e.g. RAID5?
    > > > Reading the bootsector will work fine, but I don't see how grub can
    > > > read its stage files from a RAID5 array.

    > >
    > > As long as the fakeRAID BIOS loads, and grub uses BIOS calls, all will
    > > work. The BIOS sets everything up so that grub simply sees things as an
    > > ordinary disk. No different to how booting from raid0 on fakeraid
    > > works.

    >
    > OK, so that is a case where having real hardware _does_ make a difference
    > and an emulator without the BIOS support cannot be used for testing.


    You could still test them in an emulator, however you can't boot from them.So you could have 3 disks for RAID 5, and a 4th with an install on it to boot from.

    Luke



    --
    To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  13. Bug#497110: marked as done (boot loader installation failed when dmraid=true)


    Your message dated Sat, 13 Sep 2008 15:17:06 +0000
    with message-id
    and subject line Bug#497110: fixed in parted 1.8.8.git.2008.03.24-10
    has caused the Debian Bug report #497110,
    regarding boot loader installation failed when dmraid=true
    to be marked as done.

    This means that you claim that the problem has been dealt with.
    If this is not the case it is now your responsibility to reopen the
    Bug report if necessary, and/or fix the problem forthwith.

    (NB: If you are a system administrator and have no idea what this
    message is talking about, this may indicate a serious mail system
    misconfiguration somewhere. Please contact owner@bugs.debian.org
    immediately.)


    --
    497110: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=497110
    Debian Bug Tracking System
    Contact owner@bugs.debian.org with problems


  14. Bug#498846: marked as done (OOM if number of Choices and Choices-C are not equal)


    Your message dated Sun, 14 Sep 2008 20:47:10 +0000
    with message-id
    and subject line Bug#498846: fixed in cdebconf 0.135
    has caused the Debian Bug report #498846,
    regarding OOM if number of Choices and Choices-C are not equal
    to be marked as done.

    This means that you claim that the problem has been dealt with.
    If this is not the case it is now your responsibility to reopen the
    Bug report if necessary, and/or fix the problem forthwith.

    (NB: If you are a system administrator and have no idea what this
    message is talking about, this may indicate a serious mail system
    misconfiguration somewhere. Please contact owner@bugs.debian.org
    immediately.)


    --
    498846: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=498846
    Debian Bug Tracking System
    Contact owner@bugs.debian.org with problems


  15. Bug#497110: improved dmraid support in D-I

    On Friday 12 September 2008, Luke Yelavich wrote:
    > On Fri, Sep 12, 2008 at 03:38:47PM EST, Frans Pop wrote:
    > > On Friday 12 September 2008, Luke Yelavich wrote:
    > > > Ok, I'll go into detail where necessary later in this mail, but I
    > > > will shortly file bugs against all d-i packages I've touched, with
    > > > patches attached.

    > >
    > > That's not needed. Giuseppe has already taken your changes from the
    > > diffs between Debian and Ubuntu sources and I already have commits to
    > > our SVN repository prepared based on those (with some mostly mostly
    > > minor changes and some additions).
    > >
    > > What would be useful is if you could check those, which is probably
    > > easiest to do after they've been committed.


    The changes have now been committed as revisions r55850 to r55856. The
    first commit is a semi-related bug fix. The second and last commits are
    basically your work with a few minor cleanups. The other commits are
    improvements by me created while I was testing the changes.

    > I've downloaded the respecitve svn branches, so I'll keep an eye on
    > them.


    For preparing complex changes we've found a git-svn checkout to be really
    helpful. You can find info about it at:
    http://wiki.debian.org/DebianInstaller/git-svn

    Note that Joey Hess' repository was outdated for some time. No idea if
    that's still the case.

    Cheers,
    FJP

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (GNU/Linux)

    iEYEABECAAYFAkjOSDcACgkQgm/Kwh6ICoTt4gCffjj1jcXyiCGCFGlUkOn6yP3d
    g78An0QpVRXIYB8zk7DAIfW4hS8u7twv
    =pUoX
    -----END PGP SIGNATURE-----


  16. Bug#498845: marked as done (Causes unmatched Choices when first item in question is a divider)


    Your message dated Mon, 22 Sep 2008 01:17:11 +0000
    with message-id
    and subject line Bug#498845: fixed in partman-base 126
    has caused the Debian Bug report #498845,
    regarding Causes unmatched Choices when first item in question is a divider
    to be marked as done.

    This means that you claim that the problem has been dealt with.
    If this is not the case it is now your responsibility to reopen the
    Bug report if necessary, and/or fix the problem forthwith.

    (NB: If you are a system administrator and have no idea what this
    message is talking about, this may indicate a serious mail system
    misconfiguration somewhere. Please contact owner@bugs.debian.org
    immediately.)


    --
    498845: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=498845
    Debian Bug Tracking System
    Contact owner@bugs.debian.org with problems


  17. Bug#497110: improved dmraid support in D-I

    Frans Pop ha scritto:

    > I've not tested os-prober functionality for dmraid.


    Tested, unfortunately root is empty:

    # This entry automatically added by the Debian installer for an existing
    # linux installation on /dev/mapper/sil_aiahbgbgaaaj1.
    title Debian GNU/Linux, kernel 2.6.26-1-686 (on
    /dev/mapper/sil_aiahbgbgaaaj1)
    root
    kernel /boot/vmlinuz-2.6.26-1-686 root=/dev/mapper/sil_aiahbgbgaaaj1 ro
    initrd /boot/initrd.img-2.6.26-1-686
    savedefault
    boot


    # This entry automatically added by the Debian installer for an existing
    # linux installation on /dev/mapper/sil_aiahbgbgaaaj1.
    title Debian GNU/Linux, kernel 2.6.26-1-686 (single-user mode) (on
    /dev/mapper/sil_aiahbgbgaaaj1)
    root
    kernel /boot/vmlinuz-2.6.26-1-686 root=/dev/mapper/sil_aiahbgbgaaaj1 ro
    single
    initrd /boot/initrd.img-2.6.26-1-686
    savedefault
    boot


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (GNU/Linux)

    iEYEARECAAYFAkjYysYACgkQNxpp46476apbigCgnQ8IEMeHQB GgCoyM7F3zIbXV
    j0UAnjdPBZxLpJT1+63/6gzYkkkVdZqY
    =pjgO
    -----END PGP SIGNATURE-----


  18. Bug#500079: Bug#497110: Missing root option for other detected Linux OS on dmraid

    On Thursday 25 September 2008, Frans Pop wrote:
    > First of all: can this not just be done in a similar way to what's
    > already been done for multipath in lines 103 and 109? If not, why?


    Hmm. I can probably answer that myself: because the devmapper device names
    are not fixed but depend on the name of the SATA driver.

    Still, I wonder if it would not be possible to determine the relevant
    devmapper device name _first_ and then just follow the normal code path.



    --
    To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2