Installing multiple external USB hard drives - Setup
This is a discussion on Installing multiple external USB hard drives - Setup ; Be the first! Start a tradition!
Spoken the Pseudelus the slave to his owner Hero.
Following the linux threads since I started using external storage I have not
seen a thing on this subject. If anyone knows better please let ...
-
Installing multiple external USB hard drives
Be the first! Start a tradition!
Spoken the Pseudelus the slave to his owner Hero.
Following the linux threads since I started using external storage I have not
seen a thing on this subject. If anyone knows better please let me know so I can
improve the article.
Overall, I guess I should add I am happy with the USB storage drives as they do
as advertised and as expected. And now that I start to explain what I wrote I
realize I should point out I have several fixed USB ports rather than a hub,
that I have not tried it with a hub and a couple other things.
But if anyone is interested in trying it, get one. You will not be disappointed.
http://www.giwersworld.org/computers...b-drives.phtml
Installing multiple external USB hard drives
by Matt Giwer, © 2007 [May]
Storage increases to fill the space available. It is a fundamental rule of
nature. Don't try to fight it.
Why would you want to do a stupid thing like this?
You are a home user. You only have four slots for drives. At least two are your
main hard drive and your CD or DVD drive. So you have two empty slots. Unless
you are richer than the average user you can afford maybe two 500MB internal drives.
You do not want to do this if the speed of any application depends upon disk
access time. The boring part
For the first time since PCs came out there have not gone to machines speed
increasing as fast as my budget for computer hardware. The speed increasehas
maxed for two years while Intel and AMD developed multi-core and 64bit CPUs.
There is no point to switching to those until your most used applicationscan
make use of them. In the mean time my budget has gone into mass storage.
It started innocently enough with some unused drives from old machines and
finding an add-on for an external enclosure that would hold a drive and plug
into a USB port. These were 40 GB drives and my computer had a 160 GB drive. I
had the idea that for the first time I could afford to back up all data of any
serious importance without the hassle of putting it on floppies or CD/DVDs. That
is a nuisance and I was constantly having to decide what was really, really
important and keeping track of all the disks and update intervals. You know the
drill.
At first I had to switch the two 40s back and forth but it worked. Then Igot a
sale notice from Tigerdirect on a 250 GB USB drive for something like $110 or
so. Whatever the price it was the first affordable drive that got me to look
further. I bought and in one night backed up everything in my home directory and
had an update scheme in formulated for the future.
The problem was backup was only 80 GB out of 230 GB available after formatting.
All that free space ate at me. I found other things to collect from the web. In
a few weeks I started running out of disk space and bought another and another
and found a 120GB I had forgotten about to put in the original purchase holder.
If only I had started with that drive I might never have learned the fundamental
rule of nature.
Soon I had hundreds of GB of data and no way to back up all of it. Thingsto be
stored expand to fill the space available.
The price of hard drives is dropping rapidly. I assume this is to empty
inventory while switching over to the denser vertical polarization technology.
The bottom line is I have a terabyte of external storage for about $US400. The
original machine came with a 160 GB drive to which I have added a 320 GB drive
for nearly 1.5 TB total storage. The added internal 320 GB drive was alsoabout
$US100. So for about $US1000 I have a 3 GHz machine with 1.5 TB of storage.
End of boring part. Beginning of useful part.
I have always bought by best price per gigabyte. (I see no reason to consider
the size of the buffer because of the low data rate through the USB port.) They
are all different different vendors. They all work with nothing but plugging
them in. I use Redhat linux FC6 and KDE.
To satisfy the Microsoft "plug n play" advertising they come with VFAT file
format. Yes, most Linuxes will read and write that format but it is very
unreliable and can be wiped out by a simple power glitch. Put a real filesystem
on it.
I format them to ext3. Were I to use ext2 the available space would be larger.
How much larger I have no idea. However, a power glitch can force a filecheck
with ext2 format. Since a filecheck takes about an hour per 100 GB I'll take the
hit. With ext3 the advertised size reduces as follows.
120 GB => 113 GB
250 GB => 230 GB
320 GB => 294 GB
For formatting I use this as root.
/sbin/mke2fs -c -j -L 320 /dev/sdX1
The -c does a read-only check for bad sectors. This takes about an hour per 100
GB on a 3 GHz machine. I have read this isn't really necessary these days. I
don't know. I am not in a hurry.
The -j creates an ext3 filesystem. The j is for journaling. The -L is creates a
label on the disk in this case the label is 320. I have not found a use for this
yet but if I ever do, there it is. Of course make all the names different.
There are many other options for mke2fs detailed on the man page. I cannot find
any that are worth getting a handle on to see if there is significantly more
usable space. But if you are going to use them for more than offline storage you
likely do want the defaults without special configuration. But if you do make a
special configuration, write it down on paper and put that in a safe place. I
guarantee you a year from now you will not remember what you did.
The first problem, formatting
Specifically finding the dev number to use. This is the /dev/sdX1 in the format
command above. Using KDE when you plug it in a browser window will open and show
its contents. There will be no contents at this point but the browser window
will give you how it is mounted showing the sdX1 device name. (I assume
something similar happens in Gnome.) Write that down, close the browser and go root.
Just in case as root do
umount /dev/sdX1
/sbin/mke2fs -c -j -L 320 /dev/sdX1
with the right letter for the X. Get a cup of coffee, have lunch, make love to
your spouse. Whatever you do this will take an hour or three. As this does not
interfere with anything else you can just get on with whatever it is you do on
the computer. If you are in a hurry and are confident the bad sector check is
not needed these days leave out the -c and it will only take a minute or two.
Instead of using KDE you can read the file /etc/mtab to see what device number
was assigned to it. It will be the last line of the file.
The second problem, mounting
Unless you use it only for occasional backup you want these drives to mount
themselves and behave just like an internal drive. The problem is when they are
automatically mounted at boot by fstab they are not given the same drive number
as when they are plugged into running computer. I am certain there is a way to
do with without rebooting but I have no idea what it might be.
KDE to the rescue again. On the desktop there will be an icon for each drive
after boot. It you gave it a name, that will be the label under the icon
otherwise it will be the amount of storage on the device. Now you click on the
icon and that browser will appear again. The navigation window will show you the
device number it was assigned at boot.
Here are the relevant entries from my /etc/fstab file
/dev/hda /media/cdrecorder iso9660 defaults 0 0
/dev/hdb /media/cdrom iso9660 defaults 0 0
/dev/sde /media/120 ext3 defaults 0 0
/dev/sdc1 /media/250 ext3 defaults 0 0
#/dev/sdd1 /media/250 ext3 defaults 0 0
/dev/sdb /media/320 ext3 defaults 0 0
/dev/sdd1 /media/4320 ext3 defaults 0 0
#/dev/sdc1 /media/4320 ext3 defaults 0 0
As you can I labeled my drives by the size and also created mountpoints of the
same name in /media. At first none of the /dev/sdX existed. Once finding their
device number assigned at boot time I add a line to put it at the right
mountpoint. (The second 320 is named 4320 simply to make use of tab completion.
To create the mountpoints you have to be root. But after creating them do
chown user:user /media/120
so you can read and write them without being root. There is probably a better
owner designation. I am the only one using this computer. This works for me.
Known problem
Notice from my FSTAB two drives are devices with an appended 1 and two are not.
When I had only three, only one had an appended 1 and there was no problem. With
the fourth two have the appended 1 and they occasionally switch at boot time.
That is not good.
Here is what happens. If you try to format say /dev/sdc1 as /dev/sdc you get a
warning that reads dire. I have ignored it and no problems. I assume adding a
partition to those drives now is impossible without a complete reformat of the
disk. If that is the worst penalty so be it.
At my first opportunity meaning when I can offload one of the appended 1 drives
I will reformat it without the 1 and see if the switching stops.
In the mean time I have created a file at the top of each drive with its label
as the file name and then a simple script
echo
echo '120'
ls -lhtd /media/120/120
echo
echo '250'
ls -lhtd /media/250/250
echo
echo '320-TV'
ls -lhtd /media/320/320
echo
echo '4320'
ls -lhtd /media/4320/4320
echo
If I do not get a reply from 250 and 4320 then they have switched. The fix is
another short script.
umount /dev/sdc1 &&
umount /dev/sdd1 &&
mount -t ext3 /dev/sdc1 /media/4320 &&
mount -t ext3 /dev/sdd1 /media/250
This has to be done as root.
Other considerations
External USB drives are much slower than an internal drive mounted on thebus.
Also when moving data from one external to another the speed is only halfof
that already slow speed. This speed is more than bearable to me as I onlyuse
them for storage. If you have an application needing a lot of disk accesses do
not put that data on an external drive. Of course this doubly slow speed is also
true of the internal bus but that is so fast you barely notice it. USB isso
slow that when you read and write to the USB the difference is very obvious.
Also because the access is slow if you are running other applications which also
have reads and writes to an external drive and there is clearly interference one
of them has to be run with nice.
--
The decline began a century ago when Arabs with their religious prohibition
against making graven images of humans and animals took over western art.
-- The Iron Webmaster, 3776
nizkor http://www.giwersworld.org/nizkook/nizkook.phtml
Larry Shiff http://www.giwersworld.org/computers/newsagent.phtml a8
-
Re: Installing multiple external USB hard drives
On Sat, 02 Jun 2007 04:31:02 -0400, Matt Giwer wrote:
> Be the first! Start a tradition!
>
> Spoken the Pseudelus the slave to his owner Hero.
>
> Following the linux threads since I started using external storage I have not
> seen a thing on this subject. If anyone knows better please let me know so I can
> improve the article.
>
> Overall, I guess I should add I am happy with the USB storage drives as they do
> as advertised and as expected. And now that I start to explain what I wrote I
> realize I should point out I have several fixed USB ports rather than a hub,
> that I have not tried it with a hub and a couple other things.
>
> But if anyone is interested in trying it, get one. You will not be disappointed.
>
> http://www.giwersworld.org/computers...b-drives.phtml
>
> Installing multiple external USB hard drives
> by Matt Giwer, © 2007 [May]
>
> Storage increases to fill the space available. It is a fundamental rule of
> nature. Don't try to fight it.
>
> Why would you want to do a stupid thing like this?
> You are a home user. You only have four slots for drives. At least two are your
> main hard drive and your CD or DVD drive. So you have two empty slots. Unless
> you are richer than the average user you can afford maybe two 500MB internal drives.
>
> You do not want to do this if the speed of any application depends upon disk
> access time.
>
USB 2.0 is fairly fast. The rated burst speed is 480Mb/s. Are you using
USB 1.1?
When doing an rsync between two USB drives which are both using device
mapper encryption the sustained transfer rate is on the order of >5MB/s
(>40Mb/s). While slower than the burst rate, large amounts of data can be
transferred reasonably quickly.
--
Douglas Mayne
-
Re: Installing multiple external USB hard drives
Douglas Mayne wrote:
> On Sat, 02 Jun 2007 04:31:02 -0400, Matt Giwer wrote:
>
>> Be the first! Start a tradition!
>>
>> Spoken the Pseudelus the slave to his owner Hero.
>>
>> Following the linux threads since I started using external storage I have not
>> seen a thing on this subject. If anyone knows better please let me know so I can
>> improve the article.
>>
>> Overall, I guess I should add I am happy with the USB storage drives as they do
>> as advertised and as expected. And now that I start to explain what I wrote I
>> realize I should point out I have several fixed USB ports rather than a hub,
>> that I have not tried it with a hub and a couple other things.
>>
>> But if anyone is interested in trying it, get one. You will not be disappointed.
>>
>> http://www.giwersworld.org/computers...b-drives.phtml
>>
>> Installing multiple external USB hard drives
>> by Matt Giwer, © 2007 [May]
>>
>> Storage increases to fill the space available. It is a fundamental rule of
>> nature. Don't try to fight it.
>>
>> Why would you want to do a stupid thing like this?
>> You are a home user. You only have four slots for drives. At least twoare your
>> main hard drive and your CD or DVD drive. So you have two empty slots.Unless
>> you are richer than the average user you can afford maybe two 500MB internal drives.
>>
>> You do not want to do this if the speed of any application depends upon disk
>> access time.
>
> USB 2.0 is fairly fast. The rated burst speed is 480Mb/s. Are you using
> USB 1.1?
That comment was in anticipation of the kind of question that is common here.
"I just installed a USB 2.0 external and it appears slow. Is this normal?" My
pre-answer is yes. It is not that slow. It is simply noticeable where internal
disk i/o usually is not. I should add I am talking mostly file sizes and
collections of files for animation of 175MB and up.
I am going to get into movie editing soon and for that I have a 320MB internal.
With the present SATA internal, 500MHz FSB and 3GB cpu putting together atwo
minute animation can take a half hour.
BTW: This is just a hobby. http://www.giwersworld.org/artv/ artiv and artiii
have examples. The above numbers are for those size animations. I am going for
at 640 to 1012 pixel width animations. At 30 frames per second you can see where
the storage space is going.