Reliable storage on an unreliable embedded box? - Embedded

This is a discussion on Reliable storage on an unreliable embedded box? - Embedded ; Hi folks, I'm planning to make a small embedded gnu/linux box based on a Soekris board, but I'm not sure how to best protect against data loss. (Power may be lost without notice) The box will be used for: - ...

+ Reply to Thread
Results 1 to 8 of 8

Thread: Reliable storage on an unreliable embedded box?

  1. Reliable storage on an unreliable embedded box?

    Hi folks,

    I'm planning to make a small embedded gnu/linux box based on a Soekris
    board, but I'm not sure how to best protect against data loss. (Power
    may be lost without notice)

    The box will be used for:
    - collecting mail with POP3, running an IMAP server
    - storing music, available via NFS
    - running slimserver (www.slimdevices.com)
    - storing other data, available via NFS

    Are there any major gaps in the following plan?
    - read-only root fs on a compactflash card, data on a large 2.5" hdd
    - use reiserfs for the "maildir" partition
    - use ext3 for "music and other data" partition, using "data=journal"
    - mount all partitions with the "sync" option
    - turn off the hdd write cache with "hdparm -W0" (hmmm, hdparm docs say
    this is "dangerous"!)
    - have the hdd spin-down after a few minutes to reduce power
    consumption and noise
    - regularly backup to someplace else with rsync

    Ideally, I'd like to be able to yank the power cord without a clean
    shutdown and still be confident that the data is sound.

    Thanks in advance for any input,
    Matthew


  2. Re: Reliable storage on an unreliable embedded box?

    On 16 Apr 2005 05:07:21 -0700, Matthew Flint
    wrote:

    > Ideally, I'd like to be able to yank the power cord without a clean
    > shutdown and still be confident that the data is sound.


    possibly a small capacitor or battery bank that would supply the system
    long enough to trigger a safe shutdown? a small ups with a direct dc
    connection to the board would do it, besides a mains sensor.

    --
    Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

  3. Re: Reliable storage on an unreliable embedded box?

    Matthew Flint wrote:

    >
    > Ideally, I'd like to be able to yank the power cord without a clean
    > shutdown and still be confident that the data is sound.
    >


    you can use filesystem like reiserFS or jffs2, but that still doesn't
    mean that when power loss occurs your block of data will be valid,
    so I agree with jim, you definetly need some way to maintain power
    long enough to make sure you don't loose any data.

    best regards,
    Mario


  4. Re: Reliable storage on an unreliable embedded box?

    Here we discussed already several times that there is a possibility that
    a Flash card is destroyed (getting completely unaccessible and needs to
    be sent to the manufacturer for internal reformatting) when power goes
    down while a write process is in progress.

    The flash manufacturers don't publish any data about ho long power needs
    to be provided after a write command (that does terminate seemingly
    correctly, but the card still does a lot of internal work).

    But I suppose this usually takes not more than some seconds.

    So the only way to securely handle a flash card is to use hardware that
    provides power for some seconds after the last write to the card has
    happened.

    Of course _additionally_ the Linux system needs to be made write all
    remaining data from the file system cache to the card before.

    -Michael

  5. Re: Reliable storage on an unreliable embedded box?

    On Mon, 18 Apr 2005 08:50:47 +0200, Mile Blenton
    wrote:

    > you can use filesystem like reiserFS or jffs2, but that still doesn't
    > mean that when power loss occurs your block of data will be valid,
    > so I agree with jim, you definetly need some way to maintain power
    > long enough to make sure you don't loose any data.
    >
    > best regards,
    > Mario


    and it doesn't have to be big and ugly, small and elegant is the way you
    go if you want efficiency and possibly the ability to squeeze it into the
    case. no inverter, they suck a lot of power from the battery. a battery
    sized to fit in a floppy or hard disk bay would fix it fine, if you have a
    spare.

    --
    Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

  6. Re: Reliable storage on an unreliable embedded box?

    Jim, Mile: Thanks for your advice.

    I guess I was hoping that the combination of (1) a journalling FS such
    as EXT3, (2) mounting the filesystem with options "sync data=journal"
    and (3) turning off the hdd write cache would ensure that the data is
    either stored properly or stored in the journal. (ie, the data is not
    waiting in the OS filesystem cache or the HDD write cache)

    I'm not sure that a battery is an ideal solution. The whole box
    (http://www.soekris.com/net4801.htm), when the hard-disk is active,
    will draw up to 15W - and I'll struggle to get a battery to source as
    much as an amp while the machine shuts down cleanly. :-/


    Michael: flash writes

    I wasn't intending to ever mount the flash device read-write: this will
    be a read-only root filesystem.


    Matthew


  7. Re: Reliable storage on an unreliable embedded box?

    On 18 Apr 2005 07:21:48 -0700, Matthew Flint
    wrote:

    > I'm not sure that a battery is an ideal solution. The whole box
    > (http://www.soekris.com/net4801.htm), when the hard-disk is active,
    > will draw up to 15W - and I'll struggle to get a battery to source as
    > much as an amp while the machine shuts down cleanly. :-/


    a standard pack of 1400ma nicads that are used by rc racers is about the
    same length of a hard disk, and half as wide,but also half as thick, can
    squeeze in about anywhere, and i believe they're 12V. one thing to do is
    make your own packs, individual cells and tiny buss strips, solder the
    cells together to make 13V, then use a maxim dc/dc converter
    chip(something round 95% efficiency). you can of course use a different
    kind of battery.

    --
    Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

  8. Re: Reliable storage on an unreliable embedded box?


    jim dorey wrote:
    > On 18 Apr 2005 07:21:48 -0700, Matthew Flint
    > wrote:
    >
    > > I'm not sure that a battery is an ideal solution. The whole box
    > > (http://www.soekris.com/net4801.htm), when the hard-disk is active,
    > > will draw up to 15W - and I'll struggle to get a battery to source

    as
    > > much as an amp while the machine shuts down cleanly. :-/


    More than 10W of that would be due to the hard disk. Unfortunately,
    you can't isolate the power for the on-board CF. If you use an
    external flash disk (is 4G big enough for you?), you can keep it in
    stand-by for less than 0.1W (approx 0.5W active). CF based flash disk
    will run in as little as 3.3V and 100 mA. A couple of rechargeable
    batteries will keep it running for hours.

    If you don't want to deal with charging batteries, you can build a
    simple power isolation circuit with a diode and cap. It works well to
    extend power to flash disks for seconds or even minuties.

    ....


+ Reply to Thread