Installing multiple external USB hard drives - Setup

This is a discussion on Installing multiple external USB hard drives - Setup ; Be the first! Start a tradition! Spoken the Pseudelus the slave to his owner Hero. Following the linux threads since I started using external storage I have not seen a thing on this subject. If anyone knows better please let ...

+ Reply to Thread
Results 1 to 3 of 3

Thread: Installing multiple external USB hard drives

  1. Installing multiple external USB hard drives

    Be the first! Start a tradition!

    Spoken the Pseudelus the slave to his owner Hero.

    Following the linux threads since I started using external storage I have not
    seen a thing on this subject. If anyone knows better please let me know so I can
    improve the article.

    Overall, I guess I should add I am happy with the USB storage drives as they do
    as advertised and as expected. And now that I start to explain what I wrote I
    realize I should point out I have several fixed USB ports rather than a hub,
    that I have not tried it with a hub and a couple other things.

    But if anyone is interested in trying it, get one. You will not be disappointed.

    http://www.giwersworld.org/computers...b-drives.phtml

    Installing multiple external USB hard drives
    by Matt Giwer, © 2007 [May]

    Storage increases to fill the space available. It is a fundamental rule of
    nature. Don't try to fight it.

    Why would you want to do a stupid thing like this?
    You are a home user. You only have four slots for drives. At least two are your
    main hard drive and your CD or DVD drive. So you have two empty slots. Unless
    you are richer than the average user you can afford maybe two 500MB internal drives.

    You do not want to do this if the speed of any application depends upon disk
    access time. The boring part
    For the first time since PCs came out there have not gone to machines speed
    increasing as fast as my budget for computer hardware. The speed increasehas
    maxed for two years while Intel and AMD developed multi-core and 64bit CPUs.
    There is no point to switching to those until your most used applicationscan
    make use of them. In the mean time my budget has gone into mass storage.

    It started innocently enough with some unused drives from old machines and
    finding an add-on for an external enclosure that would hold a drive and plug
    into a USB port. These were 40 GB drives and my computer had a 160 GB drive. I
    had the idea that for the first time I could afford to back up all data of any
    serious importance without the hassle of putting it on floppies or CD/DVDs. That
    is a nuisance and I was constantly having to decide what was really, really
    important and keeping track of all the disks and update intervals. You know the
    drill.

    At first I had to switch the two 40s back and forth but it worked. Then Igot a
    sale notice from Tigerdirect on a 250 GB USB drive for something like $110 or
    so. Whatever the price it was the first affordable drive that got me to look
    further. I bought and in one night backed up everything in my home directory and
    had an update scheme in formulated for the future.

    The problem was backup was only 80 GB out of 230 GB available after formatting.
    All that free space ate at me. I found other things to collect from the web. In
    a few weeks I started running out of disk space and bought another and another
    and found a 120GB I had forgotten about to put in the original purchase holder.
    If only I had started with that drive I might never have learned the fundamental
    rule of nature.

    Soon I had hundreds of GB of data and no way to back up all of it. Thingsto be
    stored expand to fill the space available.

    The price of hard drives is dropping rapidly. I assume this is to empty
    inventory while switching over to the denser vertical polarization technology.

    The bottom line is I have a terabyte of external storage for about $US400. The
    original machine came with a 160 GB drive to which I have added a 320 GB drive
    for nearly 1.5 TB total storage. The added internal 320 GB drive was alsoabout
    $US100. So for about $US1000 I have a 3 GHz machine with 1.5 TB of storage.

    End of boring part. Beginning of useful part.

    I have always bought by best price per gigabyte. (I see no reason to consider
    the size of the buffer because of the low data rate through the USB port.) They
    are all different different vendors. They all work with nothing but plugging
    them in. I use Redhat linux FC6 and KDE.

    To satisfy the Microsoft "plug n play" advertising they come with VFAT file
    format. Yes, most Linuxes will read and write that format but it is very
    unreliable and can be wiped out by a simple power glitch. Put a real filesystem
    on it.

    I format them to ext3. Were I to use ext2 the available space would be larger.
    How much larger I have no idea. However, a power glitch can force a filecheck
    with ext2 format. Since a filecheck takes about an hour per 100 GB I'll take the
    hit. With ext3 the advertised size reduces as follows.

    120 GB => 113 GB
    250 GB => 230 GB
    320 GB => 294 GB

    For formatting I use this as root.

    /sbin/mke2fs -c -j -L 320 /dev/sdX1

    The -c does a read-only check for bad sectors. This takes about an hour per 100
    GB on a 3 GHz machine. I have read this isn't really necessary these days. I
    don't know. I am not in a hurry.

    The -j creates an ext3 filesystem. The j is for journaling. The -L is creates a
    label on the disk in this case the label is 320. I have not found a use for this
    yet but if I ever do, there it is. Of course make all the names different.

    There are many other options for mke2fs detailed on the man page. I cannot find
    any that are worth getting a handle on to see if there is significantly more
    usable space. But if you are going to use them for more than offline storage you
    likely do want the defaults without special configuration. But if you do make a
    special configuration, write it down on paper and put that in a safe place. I
    guarantee you a year from now you will not remember what you did.

    The first problem, formatting
    Specifically finding the dev number to use. This is the /dev/sdX1 in the format
    command above. Using KDE when you plug it in a browser window will open and show
    its contents. There will be no contents at this point but the browser window
    will give you how it is mounted showing the sdX1 device name. (I assume
    something similar happens in Gnome.) Write that down, close the browser and go root.

    Just in case as root do

    umount /dev/sdX1

    /sbin/mke2fs -c -j -L 320 /dev/sdX1

    with the right letter for the X. Get a cup of coffee, have lunch, make love to
    your spouse. Whatever you do this will take an hour or three. As this does not
    interfere with anything else you can just get on with whatever it is you do on
    the computer. If you are in a hurry and are confident the bad sector check is
    not needed these days leave out the -c and it will only take a minute or two.

    Instead of using KDE you can read the file /etc/mtab to see what device number
    was assigned to it. It will be the last line of the file.

    The second problem, mounting
    Unless you use it only for occasional backup you want these drives to mount
    themselves and behave just like an internal drive. The problem is when they are
    automatically mounted at boot by fstab they are not given the same drive number
    as when they are plugged into running computer. I am certain there is a way to
    do with without rebooting but I have no idea what it might be.

    KDE to the rescue again. On the desktop there will be an icon for each drive
    after boot. It you gave it a name, that will be the label under the icon
    otherwise it will be the amount of storage on the device. Now you click on the
    icon and that browser will appear again. The navigation window will show you the
    device number it was assigned at boot.

    Here are the relevant entries from my /etc/fstab file

    /dev/hda /media/cdrecorder iso9660 defaults 0 0
    /dev/hdb /media/cdrom iso9660 defaults 0 0

    /dev/sde /media/120 ext3 defaults 0 0

    /dev/sdc1 /media/250 ext3 defaults 0 0
    #/dev/sdd1 /media/250 ext3 defaults 0 0

    /dev/sdb /media/320 ext3 defaults 0 0

    /dev/sdd1 /media/4320 ext3 defaults 0 0
    #/dev/sdc1 /media/4320 ext3 defaults 0 0

    As you can I labeled my drives by the size and also created mountpoints of the
    same name in /media. At first none of the /dev/sdX existed. Once finding their
    device number assigned at boot time I add a line to put it at the right
    mountpoint. (The second 320 is named 4320 simply to make use of tab completion.

    To create the mountpoints you have to be root. But after creating them do

    chown user:user /media/120

    so you can read and write them without being root. There is probably a better
    owner designation. I am the only one using this computer. This works for me.

    Known problem
    Notice from my FSTAB two drives are devices with an appended 1 and two are not.
    When I had only three, only one had an appended 1 and there was no problem. With
    the fourth two have the appended 1 and they occasionally switch at boot time.
    That is not good.

    Here is what happens. If you try to format say /dev/sdc1 as /dev/sdc you get a
    warning that reads dire. I have ignored it and no problems. I assume adding a
    partition to those drives now is impossible without a complete reformat of the
    disk. If that is the worst penalty so be it.

    At my first opportunity meaning when I can offload one of the appended 1 drives
    I will reformat it without the 1 and see if the switching stops.

    In the mean time I have created a file at the top of each drive with its label
    as the file name and then a simple script

    echo
    echo '120'
    ls -lhtd /media/120/120
    echo
    echo '250'
    ls -lhtd /media/250/250
    echo
    echo '320-TV'
    ls -lhtd /media/320/320
    echo
    echo '4320'
    ls -lhtd /media/4320/4320
    echo

    If I do not get a reply from 250 and 4320 then they have switched. The fix is
    another short script.

    umount /dev/sdc1 &&
    umount /dev/sdd1 &&
    mount -t ext3 /dev/sdc1 /media/4320 &&
    mount -t ext3 /dev/sdd1 /media/250

    This has to be done as root.

    Other considerations

    External USB drives are much slower than an internal drive mounted on thebus.
    Also when moving data from one external to another the speed is only halfof
    that already slow speed. This speed is more than bearable to me as I onlyuse
    them for storage. If you have an application needing a lot of disk accesses do
    not put that data on an external drive. Of course this doubly slow speed is also
    true of the internal bus but that is so fast you barely notice it. USB isso
    slow that when you read and write to the USB the difference is very obvious.

    Also because the access is slow if you are running other applications which also
    have reads and writes to an external drive and there is clearly interference one
    of them has to be run with nice.

    --
    The decline began a century ago when Arabs with their religious prohibition
    against making graven images of humans and animals took over western art.
    -- The Iron Webmaster, 3776
    nizkor http://www.giwersworld.org/nizkook/nizkook.phtml
    Larry Shiff http://www.giwersworld.org/computers/newsagent.phtml a8



  2. Re: Installing multiple external USB hard drives

    On Sat, 02 Jun 2007 04:31:02 -0400, Matt Giwer wrote:

    > Be the first! Start a tradition!
    >
    > Spoken the Pseudelus the slave to his owner Hero.
    >
    > Following the linux threads since I started using external storage I have not
    > seen a thing on this subject. If anyone knows better please let me know so I can
    > improve the article.
    >
    > Overall, I guess I should add I am happy with the USB storage drives as they do
    > as advertised and as expected. And now that I start to explain what I wrote I
    > realize I should point out I have several fixed USB ports rather than a hub,
    > that I have not tried it with a hub and a couple other things.
    >
    > But if anyone is interested in trying it, get one. You will not be disappointed.
    >
    > http://www.giwersworld.org/computers...b-drives.phtml
    >
    > Installing multiple external USB hard drives
    > by Matt Giwer, 2007 [May]
    >
    > Storage increases to fill the space available. It is a fundamental rule of
    > nature. Don't try to fight it.
    >
    > Why would you want to do a stupid thing like this?
    > You are a home user. You only have four slots for drives. At least two are your
    > main hard drive and your CD or DVD drive. So you have two empty slots. Unless
    > you are richer than the average user you can afford maybe two 500MB internal drives.
    >
    > You do not want to do this if the speed of any application depends upon disk
    > access time.


    >

    USB 2.0 is fairly fast. The rated burst speed is 480Mb/s. Are you using
    USB 1.1?

    When doing an rsync between two USB drives which are both using device
    mapper encryption the sustained transfer rate is on the order of >5MB/s
    (>40Mb/s). While slower than the burst rate, large amounts of data can be
    transferred reasonably quickly.


    --
    Douglas Mayne



  3. Re: Installing multiple external USB hard drives

    Douglas Mayne wrote:
    > On Sat, 02 Jun 2007 04:31:02 -0400, Matt Giwer wrote:
    >
    >> Be the first! Start a tradition!
    >>
    >> Spoken the Pseudelus the slave to his owner Hero.
    >>
    >> Following the linux threads since I started using external storage I have not
    >> seen a thing on this subject. If anyone knows better please let me know so I can
    >> improve the article.
    >>
    >> Overall, I guess I should add I am happy with the USB storage drives as they do
    >> as advertised and as expected. And now that I start to explain what I wrote I
    >> realize I should point out I have several fixed USB ports rather than a hub,
    >> that I have not tried it with a hub and a couple other things.
    >>
    >> But if anyone is interested in trying it, get one. You will not be disappointed.
    >>
    >> http://www.giwersworld.org/computers...b-drives.phtml
    >>
    >> Installing multiple external USB hard drives
    >> by Matt Giwer, © 2007 [May]
    >>
    >> Storage increases to fill the space available. It is a fundamental rule of
    >> nature. Don't try to fight it.
    >>
    >> Why would you want to do a stupid thing like this?
    >> You are a home user. You only have four slots for drives. At least twoare your
    >> main hard drive and your CD or DVD drive. So you have two empty slots.Unless
    >> you are richer than the average user you can afford maybe two 500MB internal drives.
    >>
    >> You do not want to do this if the speed of any application depends upon disk
    >> access time.

    >
    > USB 2.0 is fairly fast. The rated burst speed is 480Mb/s. Are you using
    > USB 1.1?


    That comment was in anticipation of the kind of question that is common here.
    "I just installed a USB 2.0 external and it appears slow. Is this normal?" My
    pre-answer is yes. It is not that slow. It is simply noticeable where internal
    disk i/o usually is not. I should add I am talking mostly file sizes and
    collections of files for animation of 175MB and up.

    I am going to get into movie editing soon and for that I have a 320MB internal.
    With the present SATA internal, 500MHz FSB and 3GB cpu putting together atwo
    minute animation can take a half hour.

    BTW: This is just a hobby. http://www.giwersworld.org/artv/ artiv and artiii
    have examples. The above numbers are for those size animations. I am going for
    at 640 to 1012 pixel width animations. At 30 frames per second you can see where
    the storage space is going.



+ Reply to Thread