Bug#465122: dd uses a lot of memory - Debian

This is a discussion on Bug#465122: dd uses a lot of memory - Debian ; Package: flash-kernel Version: 1.6 Severity: normal Since dd is run with ibs= the full image size, quite a lot of ram is needed to run flash-kernel. I just had it die when running it from within d-i without swap mounted. ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: Bug#465122: dd uses a lot of memory

  1. Bug#465122: dd uses a lot of memory

    Package: flash-kernel
    Version: 1.6
    Severity: normal

    Since dd is run with ibs= the full image size, quite a lot of ram is
    needed to run flash-kernel. I just had it die when running it from
    within d-i without swap mounted. I think that using a smaller input
    buffer size would be a generally good idea, it might even make
    flash-kernel perform better on systems with swap, since they won't need
    to swap out 3.4 mb of stuff.

    --
    see shy jo

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFHr1asd8HHehbQuO8RAmWEAJ4+n/dXY8qKDtVvs8U3cSt4lgtJbQCbBpUV
    ezZxJVG1Xtmgfw4prdTrbK8=
    =Ht/Y
    -----END PGP SIGNATURE-----


  2. Bug#465122: dd uses a lot of memory

    * Joey Hess [2008-02-10 14:55]:
    > Since dd is run with ibs= the full image size, quite a lot of ram is
    > needed to run flash-kernel. I just had it die when running it from
    > within d-i without swap mounted. I think that using a smaller input
    > buffer size would be a generally good idea, it might even make
    > flash-kernel perform better on systems with swap, since they won't
    > need to swap out 3.4 mb of stuff.


    Do you have a patch?
    --
    Martin Michlmayr
    http://www.cyrius.com/



    --
    To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  3. Bug#465122: dd uses a lot of memory

    Martin Michlmayr wrote:
    > * Joey Hess [2008-02-10 14:55]:
    > > Since dd is run with ibs= the full image size, quite a lot of ram is
    > > needed to run flash-kernel. I just had it die when running it from
    > > within d-i without swap mounted. I think that using a smaller input
    > > buffer size would be a generally good idea, it might even make
    > > flash-kernel perform better on systems with swap, since they won't
    > > need to swap out 3.4 mb of stuff.

    >
    > Do you have a patch?


    Not yet. Do you remember why it uses ibs?

    --
    see shy jo

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFHr3kKd8HHehbQuO8RAsT1AKDmiZj//xGEgJ7lg7/OJQnBRQx3TgCfSE8T
    Qj4bDlg43Kcg6o+xRw/YULU=
    =WOjx
    -----END PGP SIGNATURE-----


  4. Bug#465122: dd uses a lot of memory

    * Joey Hess [2008-02-10 17:22]:
    > > Do you have a patch?

    > Not yet. Do you remember why it uses ibs?


    Here's what svn log says, but I really don't understand dd very well.
    This was based on a comment on a mailing list or something.

    Index: debian/changelog
    ================================================== =================
    --- debian/changelog (revision 42551)
    +++ debian/changelog (revision 42553)
    @@ -1,3 +1,10 @@
    +flash-kernel (0.8) unstable; urgency=low
    +
    + * Speed up flash-kernel on NSLU2 by reading the second part of the
    + kernel in one go rather than byte by byte.
    +
    + -- Martin Michlmayr Fri, 10 Nov 2006 08:12:16 +0000
    +
    flash-kernel (0.7) unstable; urgency=high

    * Copy the ixp400 modules to target if they exist. This code got lost
    Index: flash-kernel
    ================================================== =================
    --- flash-kernel (revision 42551)
    +++ flash-kernel (revision 42553)
    @@ -115,7 +115,6 @@
    boundary=1441792 # 0x00160000
    ksize=$(wc -c $kfile | awk '{print $1}')
    ksize1=$(expr $boundary - $offset - 16)
    - ksize2=$(expr $ksize - $ksize1)
    tmp=$(tempfile)
    printf "Flashing kernel: " >&2
    (
    @@ -123,7 +122,7 @@
    dd if=$kfile of=$tmp bs=$ksize1 count=1 2>/dev/null
    nslu2_swap $tmp
    sercomm_header 131072
    - dd if=$kfile of=$tmp bs=1 skip=$ksize1 count=$ksize2 2>/dev/null
    + dd if=$kfile of=$tmp ibs=$ksize1 skip=1 2>/dev/null
    nslu2_swap $tmp
    rm -f $tmp
    ) > "$mtdkernel" || error "failed."

    --
    Martin Michlmayr
    http://www.cyrius.com/



    --
    To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  5. Bug#465122: dd uses a lot of memory

    Martin Michlmayr wrote:
    > * Joey Hess [2008-02-10 17:22]:
    > > > Do you have a patch?

    > > Not yet. Do you remember why it uses ibs?

    >
    > Here's what svn log says, but I really don't understand dd very well.
    > This was based on a comment on a mailing list or something.


    It does makes sense that a larger buffer size than the default 512
    bytes (not 1 byte AFAIK) would speed it up somewhat, by saving syscall
    overhead.

    If there's a common divisor, like a block size, and it's reasonably big
    (a few K), it'd make sense to use that.

    --
    see shy jo

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFHr4GUd8HHehbQuO8RAtmZAJ46ka0oXh/4/30iReAgblPUagrIQQCguqIw
    TpgtZxm+fdEo3NTvTy3g5ds=
    =kdlR
    -----END PGP SIGNATURE-----


  6. Bug#465122: dd uses a lot of memory

    * Joey Hess [2008-02-10 17:58]:
    > It does makes sense that a larger buffer size than the default 512
    > bytes (not 1 byte AFAIK) would speed it up somewhat, by saving syscall
    > overhead.
    >
    > If there's a common divisor, like a block size, and it's reasonably big
    > (a few K), it'd make sense to use that.


    Uh, that sounds pretty complicated. Is it worth the hassle? How much
    RAM is it using at the moment anyway?
    --
    Martin Michlmayr
    http://www.cyrius.com/



    --
    To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  7. Bug#465122: dd uses a lot of memory

    Martin Michlmayr wrote:
    > * Joey Hess [2008-02-10 17:58]:
    > > It does makes sense that a larger buffer size than the default 512
    > > bytes (not 1 byte AFAIK) would speed it up somewhat, by saving syscall
    > > overhead.
    > >
    > > If there's a common divisor, like a block size, and it's reasonably big
    > > (a few K), it'd make sense to use that.

    >
    > Uh, that sounds pretty complicated. Is it worth the hassle? How much
    > RAM is it using at the moment anyway?


    I assume it's using up to the size of the initramfs, so about 3.5 mb.

    --
    see shy jo

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFHr+old8HHehbQuO8RArU0AKCB6XQBTWlE1wDngIwwqN InIgOxfQCdFvg4
    YfAnfzUzF6O/2VqoITox+bc=
    =1fkY
    -----END PGP SIGNATURE-----


  8. Bug#465122: dd uses a lot of memory

    tags 465122 + help
    thanks

    Joey, I'm tagging this help. Maybe you can help with it.
    --
    Martin Michlmayr
    http://www.cyrius.com/



    --
    To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
    with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

  9. Bug#465122: dd uses a lot of memory

    I thought that maybe the initramfs flashing code could be modified to
    use less memory, more easily than the kernel code. Since it uses 2
    or 3 times as much memory.

    - dd if=$ifile of=$tmp ibs=$(($imtdsize - 16)) conv=sync
    + dd if=$ifile of=$tmp ibs=$(($imtdsize - 16)/8) conv=sync count=8

    But, that won't work, the ibs needs to be the full size of the partition
    so that conv=sync will cause dd to pad fron the end of the file with
    nulls to fill up the partition.

    I don't see a good way to do it, unless something other than dd is used.

    --
    see shy jo

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFIKKw3d8HHehbQuO8RAl4TAKDChz0GUZW/ckcdE9cvCHQIvkMJDwCcD2pu
    SKYq8Ta4B3yN2L/cW48ytHQ=
    =9HHq
    -----END PGP SIGNATURE-----


+ Reply to Thread