USB stick as swap? - BSD

This is a discussion on USB stick as swap? - BSD ; Has anyone experimented with configuring a USB flash drive as a swap device and deliberately overallocating memory so that it's actually used? I have a motherboard that is limited to 8GB of physical RAM and I'm curious whether the benefit ...

+ Reply to Thread
Results 1 to 6 of 6

Thread: USB stick as swap?

  1. USB stick as swap?

    Has anyone experimented with configuring a USB flash drive as a swap
    device and deliberately overallocating memory so that it's actually
    used? I have a motherboard that is limited to 8GB of physical RAM and
    I'm curious whether the benefit of the "extra" memory would outweigh
    the fact that swapping is slower.

    Something like Readyboost but for FreeBSD.

  2. Re: USB stick as swap?

    googlegroups@sensation.net.au wrote:
    > Has anyone experimented with configuring a USB flash drive as a swap
    > device and deliberately overallocating memory so that it's actually
    > used? I have a motherboard that is limited to 8GB of physical RAM and
    > I'm curious whether the benefit of the "extra" memory would outweigh
    > the fact that swapping is slower.
    >
    > Something like Readyboost but for FreeBSD.


    How many write cycles before it fails?


    --
    Mike Scott (unet scottsonline.org.uk)
    Harlow Essex England

  3. Re: USB stick as swap?

    On Fri, 02 May 2008 12:04:22 GMT,
    Mike Scott wrote:
    > googlegroups@sensation.net.au wrote:
    >> Has anyone experimented with configuring a USB flash drive as a swap
    >> device and deliberately overallocating memory so that it's actually
    >> used? I have a motherboard that is limited to 8GB of physical RAM and
    >> I'm curious whether the benefit of the "extra" memory would outweigh
    >> the fact that swapping is slower.


    You get 10MB/s read and 3MB/s write from the average cheap usb stick.
    Then there's that USB is a bus, so anything else on that hub might
    interfere. Compare that with the bandwidth, say, memtest86 shows, and I
    start to think that for the average desktop it's at best an interesting
    excercise, but not likely that useful.

    You can speed it all up a bit by getting a solid-state-disk on SATA or
    better (SAS, FC), but it gets a bit pricey with the devices that can
    do 120MB/s read and 90MB/s write. If you're that desperate for fast
    swap you could consider something like [gcrd]. Note that they advertise
    1500Gb/s, so divide by eight and adjust a bit for the usual marketing
    lies.

    If slow swapping isn't a problem, then the obvious question is why a
    normal disk isn't an option.


    > How many write cycles before it fails?


    Something like a million, nowadays. Would be nice to get accurate
    figures on that.


    [gcrd] http://www.gigabyte.de/Products/Stor...ame=GC-RAMDISK

    --
    j p d (at) d s b (dot) t u d e l f t (dot) n l .
    This message was originally posted on Usenet in plain text.
    Any other representation, additions, or changes do not have my
    consent and may be a violation of international copyright law.

  4. Re: USB stick as swap?

    On May 2, 11:00 pm, jpd wrote:
    > On Fri, 02 May 2008 12:04:22 GMT,
    >
    > Mike Scott wrote:
    > > googlegro...@sensation.net.au wrote:
    > >> Has anyone experimented with configuring a USB flash drive as a swap
    > >> device and deliberately overallocating memory so that it's actually
    > >> used? I have a motherboard that is limited to 8GB of physical RAM and
    > >> I'm curious whether the benefit of the "extra" memory would outweigh
    > >> the fact that swapping is slower.

    >
    > You get 10MB/s read and 3MB/s write from the average cheap usb stick.
    > Then there's that USB is a bus, so anything else on that hub might
    > interfere. Compare that with the bandwidth, say, memtest86 shows, and I
    > start to think that for the average desktop it's at best an interesting
    > excercise, but not likely that useful.


    This is for an amd64 machine with 8GB of RAM and a 1.5TB RAID10 array
    running mysql with a decent sized database (about a billion rows in
    total). There's no way that the indexes can fit into RAM (or even RAM
    + flash), but I have noticed that moving from 32 bit and ~1.5GB of RAM
    usable by mysql to 64 bit and 5GB of RAM usable by mysql has made a
    significant difference to performance. (The change from 6.2R to 7.0R
    with SCHED_ULE probably helped too.......)

    As well as a standard 4GB USB cheapie I also have a Sandisk 4GB
    Extreme III CF card which supports DMA, but unfortunately the IDE-CF
    adapter I have doesn't. The other issue is that I cannot seem to get
    mysql to use more than about 4990MB of RAM for its key buffer (even
    though there's still 2GB+ unused according to top); above that figure
    it seems to "wrap" and drops back to less than 1GB used. I don't know
    if this is a mysql or FreeBSD issue.

    In this case it would probably be better if mysql was able to mmap() a
    file on the flash storage and use that as a secondary cache, rather
    than let the OS swap between RAM and "disk" ?

    > You can speed it all up a bit by getting a solid-state-disk on SATA or
    > better (SAS, FC), but it gets a bit pricey with the devices that can
    > do 120MB/s read and 90MB/s write. If you're that desperate for fast
    > swap you could consider something like [gcrd]. Note that they advertise
    > 1500Gb/s, so divide by eight and adjust a bit for the usual marketing
    > lies.


    That definitely looks like an interesting product, and something to
    keep an eye on in the future, but at that level it would probably be
    more sensible to just purchase a motherboard that supports more than
    8GB of RAM...

  5. Re: USB stick as swap?

    On Fri, 2 May 2008 12:03:45 -0700 (PDT),
    googlegroups@sensation.net.au wrote:
    > The other issue is that I cannot seem to get mysql to use more than
    > about 4990MB of RAM for its key buffer (even though there's still 2GB+
    > unused according to top); above that figure it seems to "wrap" and
    > drops back to less than 1GB used. I don't know if this is a mysql or
    > FreeBSD issue.


    Check what the maximum (single) program size is under your current kernel.


    > In this case it would probably be better if mysql was able to mmap() a
    > file on the flash storage and use that as a secondary cache, rather
    > than let the OS swap between RAM and "disk" ?


    I don't really see how mmap() on index files instead of swapped
    in-memory indices is going to make much of a difference. Except WRT
    having to re-gen those indices at boot, but, well. Not that I know much
    about rdbmsen. All I can come up with is ```use pgsql''. ;-)

    Well, not really. You could still add a separate striped array of
    high-speed disks and put the indices on that. But the question really is
    whether you're looking for actual database speed, or just trying to gain
    a bit of extra speed at no cost. I think that the USB stick route is
    pretty much doomed for either, at least for the time being.

    You could also try and ask freebsd-performance@.


    > [...] but at that level it would probably be more sensible to just
    > purchase a motherboard that supports more than 8GB of RAM...


    There is that. You could try and find second-hand unix workstations or
    servers; those tend to be able to take lots of memory. The downside
    is that you'll end up with a cpu with lots less cycles/second, but if
    that's not where the bottleneck lies that's no problem.


    --
    j p d (at) d s b (dot) t u d e l f t (dot) n l .
    This message was originally posted on Usenet in plain text.
    Any other representation, additions, or changes do not have my
    consent and may be a violation of international copyright law.

  6. Re: USB stick as swap?

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    On 2008-05-02, googlegroups@sensation.net.au wrote:
    > On May 2, 11:00 pm, jpd wrote:
    > That definitely looks like an interesting product, and something to
    > keep an eye on in the future, but at that level it would probably be
    > more sensible to just purchase a motherboard that supports more than
    > 8GB of RAM...


    If you're looking at swapping solutions then clearly you need more RAM
    and from what you've described it is you're only solution. You're using
    a low-grade server and asking for enterprise performance. To be honest,
    I can't believe you've even contemplated `swapping to a USB flash drive'.

    - --
    Russell Wood

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2.0.4 (FreeBSD)

    iD8DBQFIHgF/Rn6cGjGf/rsRApWfAJ4gTRKtEy2Fpj79uqVwGLewgvkmGACeMWs1
    WC0hFolPgIOK8g9nOJ32YH8=
    =zK/k
    -----END PGP SIGNATURE-----

+ Reply to Thread