Installing linux on an IDE flash drive. Swap partition question - Embedded

This is a discussion on Installing linux on an IDE flash drive. Swap partition question - Embedded ; Been using a base installation of Debian on a 2GB IDE flash drive plugged into one of these: http://www.ipoxtech.com/ADE-2100.php My question is about the swap partition on the flash drive. Flash eventually wears out. Surely a flash based swap partition ...

+ Reply to Thread
Results 1 to 18 of 18

Thread: Installing linux on an IDE flash drive. Swap partition question

  1. Installing linux on an IDE flash drive. Swap partition question

    Been using a base installation of Debian on a 2GB IDE flash drive
    plugged into one of these:

    http://www.ipoxtech.com/ADE-2100.php

    My question is about the swap partition on the flash drive. Flash
    eventually wears out. Surely a flash based swap partition will wear out
    that part of the flash a lot quicker than the flash used for the rest of
    the Linux installation. Is this considered a problem or just accepted as
    a "feature" of using a flash drive.

    Can I run Linux without a swap partition or base the swap partition in RAM?

    Thanks.

  2. Re: Installing linux on an IDE flash drive. Swap partition question

    Rob Horton wrote:
    > Been using a base installation of Debian on a 2GB IDE flash drive
    > plugged into one of these:
    >
    > http://www.ipoxtech.com/ADE-2100.php
    >
    > My question is about the swap partition on the flash drive. Flash
    > eventually wears out. Surely a flash based swap partition will wear out
    > that part of the flash a lot quicker than the flash used for the rest of
    > the Linux installation. Is this considered a problem or just accepted as
    > a "feature" of using a flash drive.
    >
    > Can I run Linux without a swap partition or base the swap partition in RAM?
    >
    > Thanks.


    You can run Linux without swap (assuming you have enough ram, of
    course!). I've seen a tutorial on the web for setting up a swap
    partition on a ram disk, but I cannot fathom why that might be a useful
    idea (unless you have a system with some fast ram, and some very slow ram).

    It is also quite possible to put swap on a flash drive. It will wear
    out eventually, but how much of an issue that is depends on the type of
    flash, your usage patterns, and what sort of wear levelling you are
    using (a swap file on a jffs2 or yaffs disk will get wear levelling over
    the whole disk, for example, while swap on a partition will have much
    less levelling).

    If you system normally as enough memory for its needs without using
    swap, then it's likely that /tmp will cause more wear than the swap.
    One idea might be to include a swap file (there won't be many writes to
    it if it is not used), and mount /tmp on tmpfs. That way, your swap
    file will be used if needed for space for temporary files, but under
    normal usage small tmp files will not cause any writes at all.

    mvh.,

    David

  3. Re: Installing linux on an IDE flash drive. Swap partition question

    David Brown wrote:

    >> Been using a base installation of Debian on a 2GB IDE flash drive


    > using (a swap file on a jffs2 or yaffs disk will get wear levelling over
    > the whole disk, for example, while swap on a partition will have much
    > less levelling).


    It absolutely doesn't make sense to use a IDE flashdrive or a CF-flashdrive
    with a jffs2 filesystem.
    The internal electronics of such devices already *do* use the whole flash
    memory equally. You don't win anything when using another mechanism
    for 'equalized' writings on top of that.
    Jffs is intended for direct use, such as a microcontroller with direct
    attached flashram where the microcontroller is responible for the complete
    access.

    For the rest, I totally agree: even with 512 megs you don't really need
    swap-space any more, at least if you are running the same software and
    tasks again and again.


    Cheers

  4. Re: Installing linux on an IDE flash drive. Swap partition question

    AFAIK, you can make Linux not use a swap partition. Maybe it's defined
    by a Kernel Parameter when booting.

    -Michael

  5. Re: Installing linux on an IDE flash drive. Swap partition question

    Rob Horton wrote:

    > Been using a base installation of Debian on a 2GB IDE flash drive
    > plugged into one of these:
    >
    > http://www.ipoxtech.com/ADE-2100.php
    >
    > My question is about the swap partition on the flash drive. Flash
    > eventually wears out. Surely a flash based swap partition will wear out
    > that part of the flash a lot quicker than the flash used for the rest of
    > the Linux installation. Is this considered a problem or just accepted as
    > a "feature" of using a flash drive.
    >
    > Can I run Linux without a swap partition or base the swap partition in
    > RAM?
    >
    > Thanks.


    Robert,

    Can I urge you to post your linux install procedure on the flash card? It
    would help me and probably others too.

    Thank you.


    --

    Jerry McBride (jmcbride@mail-on.us)

  6. Re: Installing linux on an IDE flash drive. Swap partition question

    On Wed, 13 Feb 2008 19:11:20 +0100, Jon E. Weingarten wrote:

    > David Brown wrote:
    > It absolutely doesn't make sense to use a IDE flashdrive or a
    > CF-flashdrive with a jffs2 filesystem.
    > The internal electronics of such devices already *do* use the whole
    > flash memory equally. You don't win anything when using another
    > mechanism for 'equalized' writings on top of that. Jffs is intended for
    > direct use, such as a microcontroller with direct attached flashram
    > where the microcontroller is responible for the complete access.


    I would love to actually get a REAL and AUTHORITATIVE answer about the CF
    wear leveling issue. I haven't been able to get the semiconductor
    manufacturers to release details about internal wear leveling, and the
    information on the net is typically sparse, often incorrect, and usually
    from suspect sources.

    I work in embedded systems and have been trying for two years to get CF
    card manufacturers to clarify (on the record) whether they do wear
    leveling, and if so, does their guarantee of 100,000 writes apply to a
    single hardware bit or are they taking wear leveling into account when
    they make those guarantees.

    As for flash options, the CF card is still the best to use simply because
    it has a full IDE compatible parallel interface rather than the serial
    interface of other type chips.

    -Wiseguy


  7. Re: Installing linux on an IDE flash drive. Swap partition question

    Just Me wrote:

    > On Wed, 13 Feb 2008 19:11:20 +0100, Jon E. Weingarten wrote:
    >
    >> David Brown wrote:
    >> It absolutely doesn't make sense to use a IDE flashdrive or a
    >> CF-flashdrive with a jffs2 filesystem.
    >> The internal electronics of such devices already *do* use the whole
    >> flash memory equally. You don't win anything when using another
    >> mechanism for 'equalized' writings on top of that. Jffs is intended for
    >> direct use, such as a microcontroller with direct attached flashram
    >> where the microcontroller is responible for the complete access.

    >
    > I would love to actually get a REAL and AUTHORITATIVE answer about the CF
    > wear leveling issue. I haven't been able to get the semiconductor
    > manufacturers to release details about internal wear leveling, and the
    > information on the net is typically sparse, often incorrect, and usually
    > from suspect sources.


    Maybe this could help. It was one of the posts on LKML in a longer thread
    about the same question:

    On Mon, 11 June 2007 12:13:19 +0200, DervishD wrote:
    >
    > * * I was wondering: is there any reason not to use ext2 on an USB
    > pendrive? Really my question is not only about USB pendrives, but any
    > device whose storage is flash based. Let's assume that the device has a
    > good quality flash memory with wear leveling and the like...


    Reading through the thread I noticed a lot of guesswork and a total lack
    of understanding about how the pendrive actually works. *Everyone treats
    it is a black box that has "wear leveling". *So let me crack the box
    open to some degree.

    Nearly two years ago I have spoken to a person that reverse engineered
    the behaviour of several chips used in pendrives. *At the time that
    reverse engineering apparently covered most of the market. *The details
    were quite lengthy but can be condensed to two words: Smartmedia format.

    Smartmedia is a very simple format and can easily be studied. *Afaik
    access to the specs only requires registration. *Alternatively one can
    study drivers/usb/storage/alauda.c, which implements the format. *I will
    summarize the important bits below, assuming the reader has a good
    understanding about what flash is.


    All flash memory is split into zones of usually 1024(1) erase blocks.
    Within each zone, 1000 logical blocks are mapped to the 1024 physical
    blocks. *That leaves 24 spare one. *Normally the spare blocks are erased
    and contain no data. *The 1000 mapped physical blocks contain the
    associated logical number somewhere in the OOB area.

    Reading from smartmedia first requires reading all logical numbers from
    all physical blocks. *With this information a simple map, essentially a
    1024-entry array is created. *Once the map is set up, logical number get
    translated to physical ones with a simple lookup.

    Writing is slightly more complicated. *First one of the spare blocks is
    selected. *New data is written to the spare block. *If only a partial
    block is written, the remainder has to be copied from the old block.
    After the new block is written, the old block is erased and becomes a
    spare.

    If the chip loses power during a write, several physical block may
    contain the same logical number. *In that case the ECC information is
    used to verify each of the blocks. *If one of the block was not written
    completely, the other is used. *If one block has ECC errors, the other
    is used. *If both are correct, a random one is used.

    That's all there is to smartmedia format. *It is _very_ simple. *It is
    also surprisingly efficient, although it does have some shortcomings.
    So let us look at the problems and how they interact with filesystems.

    1. Write overhead

    If a filesystem only writes a small amount of data, typically 512 or
    4096 bytes, smartmedia has to erase and write a full block. *Most
    flashes used in embedded systems has block sizes of 128KiB or so. *Most
    flashes used for smartmedia have 16KiB. *Writing 16KiB when the
    filesystem only requests writing 4KiB increases the wear 4x and reduces
    performance 4x.

    2. Wear leveling

    Wear leveling happens implicitly by picking a different physical block
    from the spares on each write. *However, some blocks are never used. *If
    a physical block is mapped to a logical block that never gets written,
    it is out of the rotation. *Two seperate 1024-block areas have their
    internal wear leveling each, but nothing is spreading high wear from one
    area to another.

    So if a theoretical filesystem would only ever write to the same logical
    block, smartmedia can spead the wear over 25 blocks or less. *Less, if
    any physical blocks are bad and cannot get used (2).

    More realistically, if Ext3 is having a very hot area - the journal -
    that area is not getting any wear leveling worth mentioning. *Journaling
    filesystems on smartmedia are a bad idea. *Most journals are bigger than
    a smartmedia area, which is usually 1000 * 16KiB. *The round-robin
    access pattern of the journal already provides perfect wear leveling
    _within that area_. *Smartmedia does not add anything.

    3. Hidden caching

    Some chip designers seem to have noticed the 4x write overhead and try
    to outsmart the filesystem. *Their chips start writing the new block,
    but don't copy data from the old block just yet. *Instead they wait for
    further write requests, hoping to write adjacent data to the same block.

    If the power fails while the chip is waiting for further data, smarmedia
    format requires the unfinished block to get erased and its content to
    get discarded. *What happens if the user just typed "sync" and yanks the
    pendrive as soon as the command returns is anyone's guess. *How many
    people ever considered their pendrives to perform caching and require
    proper barriers?

    4. FAT requirement

    When I claimed there was nothing more to smartmedia, I was actually
    lying. *Smartmedia has the odd requirement that only FAT is supported as
    a filesystem. *In fact, the specifications describe FAT in great detail.
    I have already seen FTLs(3) that deliberately look into the FAT and
    pre-erase blocks if the corresponding files have been deleted from the
    FAT. *Let's just hope they always detect non-FAT filesystems.


    After all this, my recommendation for filesystems is to do several
    things:

    a) Do wear leveling!

    Smartmedia wear leveling is limited to within areas. *Any cross-device
    wear leveling must be done by the filesystem. *FAT does that fairly
    well. *The Ext family doesn't.

    b) Write aligned 16KiB blocks

    Anything else will cause write overhead and kill both performance and
    later the complete device. *Most hard disk filesystem do this anyway.
    Any effort to reduce head seek time and rotational delay will also help
    your pendrive.


    (1) Some smaller chips have a single zone of 256 or 512 blocks instead.
    * * Also, the first block in the first zone gets special treatment, so
    * * the first zone effectively only has 1023 blocks.
    (2) Most manufacturers specify that their chips has <3% bad blocks. *24
    * * spare blocks for 1000 logical ones makes 2.4%. *It is conceivable
    * * that the often-quoted <3% number is designed to work with
    * * smartmedia.
    (3) Flash Translation Layers - any format that implements block device
    * * behaviour on flash memory. *Smartmedia is but one example, although
    * * a very popular one.

    Jörn


    > Nearly two years ago I have spoken to a person that reverse engineered
    > the behaviour of several chips used in pendrives. *At the time that
    > reverse engineering apparently covered most of the market. *The details
    > were quite lengthy but can be condensed to two words: Smartmedia format.


    Maybe I should add that things have allegedly improved. *Many devices
    today support both static and dynamic wear leveling. *Dynamic wear
    leveling is what smartmedia does. *It depends on the filesystem writing
    somewhere to move those blocks. *Static wear leveling will also move
    blocks that are not written.

    However, the exact nature of wear leveling is not disclosed. *And I see
    no reason to trust an undisclosed "static and dynamic wear leveling" any
    more than I trust smartmedia.

    I will even go further and claim that nothing short of a filesystem can
    do proper wear leveling across the complete device. *The reason
    smartmedia introduced "areas" was to bound the time until it's map is
    created and the device can get accessed. *If the map spanned a large
    64GB device, access times would go sky-high.

    Any method I can imagine to offer good wear leveling will result in
    either a filesystem or at least a simplified one-file-system with the
    only file being the "block device" exported outward. *So naturally my
    answer to the problem is called LogFS.

  8. Re: Installing linux on an IDE flash drive. Swap partition question

    Rob Horton wrote:

    > Been using a base installation of Debian on a 2GB IDE flash drive
    > plugged into one of these:
    >
    > http://www.ipoxtech.com/ADE-2100.php
    >
    > My question is about the swap partition on the flash drive. Flash
    > eventually wears out. Surely a flash based swap partition will wear out
    > that part of the flash a lot quicker than the flash used for the rest of
    > the Linux installation. Is this considered a problem or just accepted as
    > a "feature" of using a flash drive.
    >
    > Can I run Linux without a swap partition or base the swap partition in
    > RAM?


    Yes you can. Why not? If you have enough usable memory for your application
    there is no reason for swap. You can disable CONFIG_SWAP in your kernel to
    save a few bytes of kernel code. Or keep it enabled then you will be able
    to enable swap at runtime on demand (run swapon/swapoff commands at any
    time).

    JB

  9. Re: Installing linux on an IDE flash drive. Swap partition question

    Michael Schnell wrote:

    > AFAIK, you can make Linux not use a swap partition. Maybe it's defined
    > by a Kernel Parameter when booting.


    AFAIK not. Its done by the swapon/swapoff commands at any time you like.

    JB

  10. Re: Installing linux on an IDE flash drive. Swap partition question

    Jon E. Weingarten wrote:
    > David Brown wrote:
    >
    >>> Been using a base installation of Debian on a 2GB IDE flash drive

    >
    >> using (a swap file on a jffs2 or yaffs disk will get wear levelling over
    >> the whole disk, for example, while swap on a partition will have much
    >> less levelling).

    >
    > It absolutely doesn't make sense to use a IDE flashdrive or a CF-flashdrive
    > with a jffs2 filesystem.
    > The internal electronics of such devices already *do* use the whole flash
    > memory equally. You don't win anything when using another mechanism
    > for 'equalized' writings on top of that.
    > Jffs is intended for direct use, such as a microcontroller with direct
    > attached flashram where the microcontroller is responible for the complete
    > access.
    >


    Sorry - I didn't notice that he was using a drive with wear levelling.
    In that case, a swap partition (if you decide to use swap at all) will
    be levelled just as well as a swap file would be.

    > For the rest, I totally agree: even with 512 megs you don't really need
    > swap-space any more, at least if you are running the same software and
    > tasks again and again.
    >
    >
    > Cheers


  11. Re: Installing linux on an IDE flash drive. Swap partition question

    Juergen Beisert wrote:

    >> AFAIK, you can make Linux not use a swap partition. Maybe it's defined
    >> by a Kernel Parameter when booting.

    >
    > AFAIK not. Its done by the swapon/swapoff commands at any time you like.


    It's even simpler...: If there *IS* *NO* swap partition available at all,
    the kernel runs and doesn't use it....




  12. Re: Installing linux on an IDE flash drive. Swap partition question

    Jon E. Weingarten wrote:

    > Juergen Beisert wrote:
    >
    >>> AFAIK, you can make Linux not use a swap partition. Maybe it's defined
    >>> by a Kernel Parameter when booting.

    >>
    >> AFAIK not. Its done by the swapon/swapoff commands at any time you like.

    >
    > It's even simpler...: If there *IS* *NO* swap partition available at all,
    > the kernel runs and doesn't use it....


    Hmmm, the kernel does *not* detect and use any swap partitions by its own!
    Activating swap is the job of userspace!

    JB


  13. Re: Installing linux on an IDE flash drive. Swap partition question

    Jerry McBride wrote:
    > Rob Horton wrote:
    >
    >> Been using a base installation of Debian on a 2GB IDE flash drive
    >> plugged into one of these:
    >>
    >> http://www.ipoxtech.com/ADE-2100.php
    >>
    >> My question is about the swap partition on the flash drive. Flash
    >> eventually wears out. Surely a flash based swap partition will wear out
    >> that part of the flash a lot quicker than the flash used for the rest of
    >> the Linux installation. Is this considered a problem or just accepted as
    >> a "feature" of using a flash drive.
    >>
    >> Can I run Linux without a swap partition or base the swap partition in
    >> RAM?
    >>
    >> Thanks.

    >
    > Robert,
    >
    > Can I urge you to post your linux install procedure on the flash card? It
    > would help me and probably others too.
    >
    > Thank you.
    >
    >

    The IDE flash drive is a 2GB version of one of these (Embeddisk Module).

    http://www.embeddisk.com/products.htm

    It plugs directly into an IDE socket and can be treated just like an
    ordinary hard disk.

    My installation procedure was to simply pop a cd with an iso image of
    the Debian business card installation into a temporarilty connected USB
    CDROM and perform a base installation. I then installed xserver and
    opera for the kiosk application that I am developing.

    Oh, due to the small size of the drive I only have two partions, one for
    the swap and the other for everything else.


  14. Re: Installing linux on an IDE flash drive. Swap partition question

    Rob Horton wrote:

    > Been using a base installation of Debian on a 2GB IDE flash drive
    > plugged into one of these:
    >
    > http://www.ipoxtech.com/ADE-2100.php
    >
    > My question is about the swap partition on the flash drive. Flash
    > eventually wears out. Surely a flash based swap partition will wear out
    > that part of the flash a lot quicker than the flash used for the rest of
    > the Linux installation. Is this considered a problem or just accepted as
    > a "feature" of using a flash drive.
    >
    > Can I run Linux without a swap partition or base the swap partition in RAM?
    >
    > Thanks.

    Where did you purchase the board? I dont see it actually for sale at the link
    you posted.
    Eric

  15. Re: Installing linux on an IDE flash drive. Swap partition question

    Eric wrote:
    > Rob Horton wrote:
    >
    >> Been using a base installation of Debian on a 2GB IDE flash drive
    >> plugged into one of these:
    >>
    >> http://www.ipoxtech.com/ADE-2100.php
    >>
    >> My question is about the swap partition on the flash drive. Flash
    >> eventually wears out. Surely a flash based swap partition will wear out
    >> that part of the flash a lot quicker than the flash used for the rest of
    >> the Linux installation. Is this considered a problem or just accepted as
    >> a "feature" of using a flash drive.
    >>
    >> Can I run Linux without a swap partition or base the swap partition in RAM?
    >>
    >> Thanks.

    > Where did you purchase the board? I dont see it actually for sale at the link
    > you posted.
    > Eric

    We got the board from a UK supplier, http://www.bvm-store.com
    Here is a direct link to the board
    http://www.bvm-store.com/ProductDeta...dProductId=436

    Hope that helps.

  16. Re: Installing linux on an IDE flash drive. Swap partition question

    Rob Horton wrote:

    > My question is about the swap partition on the flash drive.
    > Flash eventually wears out.


    Yes it does. For this reason Flash memory chips have a so called
    wear balancer, that scatters write operations to the same
    logical address evenly on all physical flash cells. The happens
    transparently and over the whole memory. So this is no problem
    at all.

    > Can I run Linux without a swap partition


    Of course you can. Swap is purely optional for Linux. If you've
    got enough RAM there's no reason for a swap partition at all.

    Wolfgang Draxinger
    --
    E-Mail address works, Jabber: hexarith@jabber.org, ICQ: 134682867


  17. Re: Installing linux on an IDE flash drive. Swap partition question

    Rob Horton wrote:

    > Eric wrote:
    >> Rob Horton wrote:
    >>
    >>> Been using a base installation of Debian on a 2GB IDE flash drive
    >>> plugged into one of these:
    >>>
    >>> http://www.ipoxtech.com/ADE-2100.php
    >>>
    >>> My question is about the swap partition on the flash drive. Flash
    >>> eventually wears out. Surely a flash based swap partition will wear out
    >>> that part of the flash a lot quicker than the flash used for the rest of
    >>> the Linux installation. Is this considered a problem or just accepted as
    >>> a "feature" of using a flash drive.
    >>>
    >>> Can I run Linux without a swap partition or base the swap partition in
    >>> RAM?
    >>>
    >>> Thanks.

    >> Where did you purchase the board? I dont see it actually for sale at the
    >> link you posted.
    >> Eric

    > We got the board from a UK supplier, http://www.bvm-store.com
    > Here is a direct link to the board
    > http://www.bvm-store.com/ProductDeta...dProductId=436
    >
    > Hope that helps.

    sort of - how much was it for a single board?
    Eric

  18. Re: Installing linux on an IDE flash drive. Swap partition question

    Eric wrote:
    > Rob Horton wrote:
    >
    >> Eric wrote:
    >>> Rob Horton wrote:
    >>>
    >>>> Been using a base installation of Debian on a 2GB IDE flash drive
    >>>> plugged into one of these:
    >>>>
    >>>> http://www.ipoxtech.com/ADE-2100.php
    >>>>
    >>>> My question is about the swap partition on the flash drive. Flash
    >>>> eventually wears out. Surely a flash based swap partition will wear out
    >>>> that part of the flash a lot quicker than the flash used for the rest of
    >>>> the Linux installation. Is this considered a problem or just accepted as
    >>>> a "feature" of using a flash drive.
    >>>>
    >>>> Can I run Linux without a swap partition or base the swap partition in
    >>>> RAM?
    >>>>
    >>>> Thanks.
    >>> Where did you purchase the board? I dont see it actually for sale at the
    >>> link you posted.
    >>> Eric

    >> We got the board from a UK supplier, http://www.bvm-store.com
    >> Here is a direct link to the board
    >> http://www.bvm-store.com/ProductDeta...dProductId=436
    >>
    >> Hope that helps.

    > sort of - how much was it for a single board?
    > Eric

    I think the price for an indiviadual board is £150 excluding any tax.

+ Reply to Thread