single core high performance computing/max memory?

Hi,
I'm looking for a high performance computing environment for genetic analysis.
As far as I understood, as well Opterons as Xeons can only address 4GB of main
memory per core. Is this right?
Looking for alternate platfoms, I came across the IBM Power 520 Express
([url]http://www-03.ibm.com/systems/power/hardware/520/[/url]), which is capable of
addressing up to 16GB as a single core system.
Furthermore, it runs linux, and should therefore be able to compile and run
common gentic analysis software such as plink
([url]http://pngu.mgh.harvard.edu/~purcell/plink/[/url]) or birdsuite
([url]http://www.broad.mit.edu/mpg/birdsuite/[/url]).
Did anyone ever try this before?
Recommedations, please!
Thx,
Chris

Mon, 30 Jun 2008 10:35:59 +0000, Chris:
[color=blue]
> Hi,
> I'm looking for a high performance computing environment for genetic
> analysis. As far as I understood, as well Opterons as Xeons can only
> address 4GB of main memory per core. Is this right?[/color]
Deadly wrong.
address sizes : 36 bits physical, 48 bits virtual

Do your math and discover you could address 64G of ram per processor.
(and 256TB of total memory)

Of course you have to use a 64bit os if you're not asking for troubles.

Mon, 30 Jun 2008 10:35:59 +0000, Chris:
[color=blue]
> Hi,
> I'm looking for a high performance computing environment for genetic
> analysis. As far as I understood, as well Opterons as Xeons can only
> address 4GB of main memory per core. Is this right?[/color]
Deadly wrong
address sizes : 36 bits physical, 48 bits virtual


i.e. 64GB ram (256TB of total memory).

Mon, 30 Jun 2008 10:35:59 +0000, Chris:
[color=blue]
> Hi,
> I'm looking for a high performance computing environment for genetic
> analysis. As far as I understood, as well Opterons as Xeons can only
> address 4GB of main memory per core. Is this right?[/color]
Deadly wrong
address sizes : 36 bits physical, 48 bits virtual


i.e. 64GB ram (256TB of total memory).

On Mon, 30 Jun 2008 10:35:59 UTC in comp.os.linux.hardware, Chris
<me@privacy.net> wrote:
[color=blue]
> As far as I understood, as well Opterons as Xeons can only address 4GB of main
> memory per core. Is this right?[/color]

No.

--
Trevor Hemsley, Brighton, UK
Trevor dot Hemsley at ntlworld dot com

Jim <Jim@j.eu> wrote:[color=blue]
> Mon, 30 Jun 2008 10:35:59 +0000, Chris:
>[color=green]
>> Hi,
>> I'm looking for a high performance computing environment for genetic
>> analysis. As far as I understood, as well Opterons as Xeons can only
>> address 4GB of main memory per core. Is this right?[/color]
> Deadly wrong
> address sizes : 36 bits physical, 48 bits virtual
>
>
> i.e. 64GB ram (256TB of total memory).[/color]
Thanks.
So, to get a single-core Xeon 32GB Ram system, which hardware is needed?
Most mainboards applicable do not support that much Ram.
Is it reasonable at all?
How is the performance compared to an Opteron for single core apps?
Thx,
Chris

In article <g4acuf$291d$1@gwdu112.gwdg.de>, Chris <me@privacy.net> wrote:[color=blue]
> I'm looking for a high performance computing environment for genetic analysis.
> As far as I understood, as well Opterons as Xeons can only address 4GB of main
> memory per core. Is this right?[/color]

Hm, even the 64-bit chips? I would've thought they'd be able to address
2^64 B (16 EiB) per core. Not so?

--
"Never go off on tangents, which are lines that intersect a curve at
only one point and were discovered by Euclid, who lived in the 6th
century, which was an era dominated by the Goths, who lived in what we
now know as Poland." - from Nov. 1998 issue of Infosystems Executive.

Chris <me@privacy.net> writes:[color=blue]
>Jim <Jim@j.eu> wrote:[color=green]
>> address sizes : 36 bits physical, 48 bits virtual
>>
>>
>> i.e. 64GB ram (256TB of total memory).[/color][/color]

I am pretty sure today's Xeons support more than 64GB. Certainly
there are motherboards that support 128GB of physical RAM.
[color=blue]
>Thanks.
>So, to get a single-core Xeon 32GB Ram system, which hardware is needed?
>Most mainboards applicable do not support that much Ram.[/color]

There are quite a number of Xeon motherboards that support 32GB and
more. They are typically dual-socket boards, but you can run it with
only one socket equipped. I think nowadays you can but only dual- and
quad-core Xeons, but maybe you can get a single-core Xeon that fits on
the second-hand market.
[color=blue]
>Is it reasonable at all?[/color]

Doing it with a single-core Xeon is not reasonable (they are based on
the Pentium 4, if you can get them at all). I suggest getting a
dual-core Xeon 51xx or 52xx (i.e., Core 2 Duo-based), which tend to be
faster (even for single-threaded applications) and consume less power.
[color=blue]
>How is the performance compared to an Opteron for single core apps?[/color]

The fastest of the Xeons mentioned above are faster than the fastest
Opterons on most single-threaded applications. Also, if you want
32GB, it may be cheaper to equip both sockets of a dual-core board to
get two additional memory channels than to try to do it with one
Opteron (and forget about single-socket Opteron boards, they don't
have enough memory slots).

- anton
--
M. Anton Ertl Some things have to be seen to be believed
[email]anton@mips.complang.tuwien.ac.at[/email] Most things have to be believed to be seen
[url]http://www.complang.tuwien.ac.at/anton/home.html[/url]

Chris wrote:[color=blue]
> Hi,
> I'm looking for a high performance computing environment for genetic analysis.
> As far as I understood, as well Opterons as Xeons can only address 4GB of main
> memory per core. Is this right?[/color]

No, Opterons are able to address upto 1TB of memory, no matter how many
cores, and something like 256TB of memory spread across multiple sockets
(rather than cores). Xeons are dependent on their chipset memory
controller, and their limitations. Depending on the chipset, Xeons
should also be able to address upto 1TB of memory, and more spread
across independent chipsets (rather than shared chipsets).

This is also dependent on current RAM manufacturing technology. If the
RAM itself can't be made dense enough, then you're probably limited in
the amount of memory just based on that.

Yousuf Khan

Mon, 30 Jun 2008 16:15:13 +0000, Anton Ertl:
[color=blue]
> Chris <me@privacy.net> writes:[color=green]
>>Jim <Jim@j.eu> wrote:[color=darkred]
>>> address sizes : 36 bits physical, 48 bits virtual
>>>
>>>
>>> i.e. 64GB ram (256TB of total memory).[/color][/color]
>
> I am pretty sure today's Xeons support more than 64GB. Certainly there
> are motherboards that support 128GB of physical RAM.[/color]

Show me address sizes and I'll tell you.

Mon, 30 Jun 2008 16:15:13 +0000, Anton Ertl:
[color=blue]
> Chris <me@privacy.net> writes:[color=green]
>>Jim <Jim@j.eu> wrote:[color=darkred]
>>> address sizes : 36 bits physical, 48 bits virtual
>>>
>>>
>>> i.e. 64GB ram (256TB of total memory).[/color][/color]
>
> I am pretty sure today's Xeons support more than 64GB. Certainly there
> are motherboards that support 128GB of physical RAM.[/color]

Show me address sizes and I'll tell you.

On Mon, 30 Jun 2008 14:11:05 +0000, Chris wrote:
[color=blue]
> Jim <Jim@j.eu> wrote:[color=green]
>> Mon, 30 Jun 2008 10:35:59 +0000, Chris:
>>[color=darkred]
>>> Hi,
>>> I'm looking for a high performance computing environment for genetic
>>> analysis. As far as I understood, as well Opterons as Xeons can only
>>> address 4GB of main memory per core. Is this right?[/color]
>> Deadly wrong
>> address sizes : 36 bits physical, 48 bits virtual
>>
>>
>> i.e. 64GB ram (256TB of total memory).[/color]
> Thanks.
> So, to get a single-core Xeon 32GB Ram system, which hardware is needed?
> Most mainboards applicable do not support that much Ram. Is it
> reasonable at all?
> How is the performance compared to an Opteron for single core apps? Thx,
> Chris[/color]

Forget about single core chips, all of the modern processors are at least
dual core. Also if you are looking at a Xeon or Opteron system then there
will be at least 2 processor chips which brings you up to at least 4 cores
on the board. If a server board only has one processor chip on it then
it's basically a desktop system and it will have desktop system memory
limits.

There are plenty of Xeon boards that will support 32G or even more, check
out

[url]http://www.supermicro.com/[/url]

The current generation of Core2 boards using the P45 chipset can support
16G if you use 4G DIMMs. 4G DIMMs are pricey as compared to 2G DIMMs but
in absolute terms it's not bad, about $1000 for 16G vs $160 for 8G using
2G DIMMs.

On Monday 30 June 2008 12:35, someone who identifies as *Chris* wrote
in /comp.os.linux.hardware:/
[color=blue]
> Hi,
> I'm looking for a high performance computing environment for genetic
> analysis. As far as I understood, as well Opterons as Xeons can only
> address 4GB of main memory per core. Is this right?[/color]

Opterons are 64-bit processors with an on-chip memory controller, common to
all cores on the chip. It's got 40 bits for RAM, so you do the math
yourself. ;-) When multiple Opterons are installed on a motherboard, then
each Opteron has its dedicated memory controller and dedicated memory. All
processors can see all memory in the machine, but will tend to use the
memory that's "closer" to the processor. It's called ccNUMA
(cache-coherent Non-Uniform Memory Addressing). ;-)

I have a dual-Opteron machine with two dualcore Opterons. It's got 32 GB of
ECC registered DDR-2 memory installed, and all is recognized.

Xeons come in two variants: 32-bit and 64-bit. Both use a memory controller
on the motherboard for all cores and all processor sockets in the machine,
so it's actually an SMP machine and not a NUMA. 32-bit machines are
limited to 4 GB - or 64 GB in paged mode if PAE is enabled in the kernel,
which does however require that the kernel be compiled for i686 at the
least, because i586 (i.e. Intel Pentium and Pentium MMX, AMD
K5/K6/K6-II/K6-III) does not have PAE yet - but the 64-bit Intels have the
same RAM and virtual memory capacity as their AMD counterparts.


--
*Aragorn*
(registered GNU/Linux user #223157)

On Monday 30 June 2008 18:15, someone who identifies as *Anton Ertl* wrote
in /comp.os.linux.hardware:/
[color=blue]
> [...] Also, if you want 32GB, it may be cheaper to equip both sockets of a
> dual-core board to get two additional memory channels than to try to do it
> with one Opteron (and forget about single-socket Opteron boards, they
> don't have enough memory slots).[/color]

Sorry Anton, but the current batch[1] of Intel processors still uses a
Northbridge and a single on-motherboard memory controller. It'll support
dual-channel memory access, yes, but it won't add another memory channel if
you have two or more sockets.

Opterons on the other hand have an on-chip memory controller and use ccNUMA
memory access[2] if you have more than one Opteron on the motherboard.


[1] Future Intels will do away with the Northbridge and will have an on-chip
memory controller as has already been the case with AMD since the first
AMD64 processor years ago.

[2] Unless disabled through node interleaving or a non-NUMA aware SMP
kernel, of course.

--
*Aragorn*
(registered GNU/Linux user #223157)

General Schvantzkopf <schvantzkopf@yahoo.com> wrote:[color=blue]
> On Mon, 30 Jun 2008 14:11:05 +0000, Chris wrote:
>
> Forget about single core chips, all of the modern processors are at least
> dual core. Also if you are looking at a Xeon or Opteron system then there
> will be at least 2 processor chips which brings you up to at least 4 cores
> on the board. If a server board only has one processor chip on it then
> it's basically a desktop system and it will have desktop system memory
> limits.
>[/color]
Yeah, I guess the Xeon dual cores are a pretty good choice!
[color=blue]
> There are plenty of Xeon boards that will support 32G or even more, check
> out
>
> [url]http://www.supermicro.com/[/url]
>
> The current generation of Core2 boards using the P45 chipset can support
> 16G if you use 4G DIMMs. 4G DIMMs are pricey as compared to 2G DIMMs but
> in absolute terms it's not bad, about $1000 for 16G vs $160 for 8G using
> 2G DIMMs.[/color]
I found the
Asus DSBF-D16,
which could take up to two Dual/Quad-Core Xeons and supports up to 16 Dimms.
I'm just not sure if the all 16 Dimm slot can be addressed if there's only one
CPU on the Mainboard.
Regards,
Chris

Aragorn <aragorn@chatfactory.invalid> writes:[color=blue]
>On Monday 30 June 2008 18:15, someone who identifies as *Anton Ertl* wrote
>in /comp.os.linux.hardware:/[/color]
[About Opterons:][color=blue][color=green]
>> [...] Also, if you want 32GB, it may be cheaper to equip both sockets of a
>> dual-core board to get two additional memory channels than to try to do it
>> with one Opteron (and forget about single-socket Opteron boards, they
>> don't have enough memory slots).[/color]
>
>Sorry Anton, but the current batch[1] of Intel processors still uses a
>Northbridge and a single on-motherboard memory controller.[/color]

Sorry if it was not clear that I meant Opterons with the statement above.
[color=blue]
> It'll support
>dual-channel memory access, yes, but it won't add another memory channel if
>you have two or more sockets.[/color]

Actually, you can buy dual-socket Xeon boards with four (FB-DIMM)
channels; I don't think anybody builds single-socket boards with these
memory controllers. But of course with Xeons you don't get additional
memory controllers by equiping both sockets, so if you need only one
core, there is no reason to equip both sockets.

- anton
--
M. Anton Ertl Some things have to be seen to be believed
[email]anton@mips.complang.tuwien.ac.at[/email] Most things have to be believed to be seen
[url]http://www.complang.tuwien.ac.at/anton/home.html[/url]

Chris <me@privacy.net> wrote:
: Hi,
: I'm looking for a high performance computing environment for
: genetic analysis. As far as I understood, as well Opterons as
: Xeons can only address 4GB of main memory per core. Is this
: right?
: Looking for alternate platfoms, I came across the IBM Power
: 520 Express
: ([url]http://www-03.ibm.com/systems/power/hardware/520/[/url]), which is
: capable of addressing up to 16GB as a single core system.
: Furthermore, it runs linux, and should therefore be able to
: compile and run common gentic analysis software such as plink
: ([url]http://pngu.mgh.harvard.edu/~purcell/plink/[/url]) or birdsuite
: ([url]http://www.broad.mit.edu/mpg/birdsuite/[/url]).
: Did anyone ever try this before?
: Recommedations, please!
: Thx,
: Chris

This would be a great question to post over at "comp.arch".

G7

In comp.sys.ibm.pc.hardware.chips Chris <me@privacy.net> wrote in part:[color=blue]
> I'm looking for a high performance computing environment
> for genetic analysis. As far as I understood, as
> well Opterons as Xeons can only address 4GB of main
> memory per core. Is this right?[/color]

No, it is very dated. For about 10 years, x86 CPUs have
had the capability of addressing far more than 4 GB. 64 GB
at least with PAE and 1 TB with x86_64. The problem is most
motherboards have much smaller capabilities.

The 4 GB limit applies per _process_ on a 32-bit OS.
A good 32 bit OS can handle multiple 4 GB processes.

But for your app (presumably some sort of pattern search),
you are probably best going with x86_64 where you get 48
bit virtual addresses and ~40 bit physical as well as the
64 bit ops. Linux is very available for x86_64, and it is
getting difficult to find any PC without x86_64 capabilities.


-- Robert