Disk setup options (s/w RAID 1, encryption, LVM) for new server - Ubuntu

Hi all

Am just setting up a new amd64 dual opteron server for high-performance
number crunching and data storage. Key requirements are:

- floating point performance on multi-threaded codes with significant
but not huge memory footprint
- large amounts of long-term safe data storage
- high electrical efficiency & low noise/heat output
- tower case of modest physical size
- reasonable cost

Disk speed performance is not a big issue. Have assembled components
that I believe will satisfy all this:

- 2x Opteron 2354 (good compromise between power & SPEC fp_rate2006/OMP2001)
- tyan thunder 2927-E mobo (only ATX board I know of for dual opty)
- enermax 82+ psu (apparently both very quiet & very high efficiency)
- 2x WD RE-2 greenpower 750G hard disks (ditto + long warranty)
- 4x 2G ddr2 667 ECC (kingston 3.2w per stick)
- large passive heatpipe CPU heatsinks & Noctua case fan
- Antec solo ATX case (nice build & noise reduction features)

Probably more detail than needed but better more than less I guess.

I will be installing either Ubuntu 8.04 LTS or debian Lenny.

Although I am not a Linux novice - have been developing on and
admin’ing an Ubuntu amd64 box for about 5 years - I’m not an expert
either. My Linux skills are mainly self-taught and focus on floating
point maths code development and simple admin tasks such as apt-get,
rsync backup, setting up firewall/NFS/ssh.

The main issue that I am considering now is the exact setup of the
disks and file structure. I definitely need redundancy & no nasty
surprises but not speed, so software RAID 1 using mdadm would seem like
the best option from that point of view. I would like to have a
strongly encrypted /home directory if at all possible. I would also
like to have at least / & /home on separate partitions so that system
and data are separated to make changes easier in the future. Possibly
also for /var & /tmp as I have been advised that if I ever expose this
machine to the net and run ftp or a website on it the logs and other
temporary files can get large - I have no experience of this but am
trying to plan wisely in advance.

Having googled the net I have seen many articles on this but quite a
few show different ways of doing the same thing or even contradictory
info. From what I have gathered so far it seems that one of the
preferred options is:

- form a RAID 1 setup with mdadm on the two HD’s
- encrypt this whole array? dm-crypt?
- on top of the encrypted array use LVM to create the various partitions

...but I am unsure (because I have not done any of these before):

- if encrypting the _whole_ array is necessary or even a good idea
from the point of
view of safety/redundancy of data & is dm-crypt the best route?
- if this is the right order? i.e. could I use LVM on top of the RAID
array and then
just encrypt /home
- does disk encryption and software RAID 1 interact in a potentially
troublesome way?
- does LVM allow a new system to be installed without touching /home?
- what tools are available to make the whole setup process relatively
easy and error-
free

Another thing that I would like to try is to put /boot separately on a
little IDE->CF adaptor so that if a HD in the array fails I can always
guarantee a re-boot, as I believe there can be issues booting from the
‘other’ disk in a software RAID 1 array in that circumstance. Also
planning to use an APC UPS with the apcupsd daemon but I believe that
is all fairly straightforward to set up.

Any thoughts - particularly on the reliability, flexibility and
feasibility of the various disk/file setup options - would be much
appreciated.

TIA

Michael

______________________________________________

Hopkins Research Touch the Future
______________________________________________

Can you do hardware RAID?

i

On 2008-09-17 12:56:48 +0100, Ignoramus19342
said:

> Can you do hardware RAID?
>
> i

Would prefer not unless I have to, and have heard good things about mdadm.

Do you recommend any quality cards (3ware/Adaptec/LSI) that run well on
open-source drivers and don't cost too much?

M

--
______________________________________________

Hopkins Research Touch the Future
______________________________________________

On 2008-09-17, Michael Hopkins wrote:
> On 2008-09-17 12:56:48 +0100, Ignoramus19342
> said:
>
>> Can you do hardware RAID?
>>
>> i
>
> Would prefer not unless I have to, and have heard good things about mdadm.
>
> Do you recommend any quality cards (3ware/Adaptec/LSI) that run well on
> open-source drivers and don't cost too much?
>

I use 3ware on servers, and had good results.
--
Due to extreme spam originating from Google Groups, and their inattention
to spammers, I and many others block all articles originating
from Google Groups. If you want your postings to be seen by
more readers you will need to find a different means of
posting on Usenet.
http://improve-usenet.org/

On 2008-09-17, Michael Hopkins wrote:
> On 2008-09-17 12:56:48 +0100, Ignoramus19342
> said:
>
>> Can you do hardware RAID?
>>
>> i
>
> Would prefer not unless I have to, and have heard good things about mdadm.

Why not? Hardware Raid will be faster, more reliable, and easier to
recover than software Raid in every case. Software raid is fine if no
other option exists, but it isn't ideal...

>
> Do you recommend any quality cards (3ware/Adaptec/LSI) that run well on
> open-source drivers and don't cost too much?

Most system boards these days, especially those with SATA controllers,
have hardware RAID built-in. Once you create the array, just build
the system with LVM, and use encryption where desired.


--
Joe - Linux User #449481/Ubuntu User #19733
joe at hits - buffalo dot com
"Hate is baggage, life is too short to go around pissed off all the
time..." - Danny, American History X