Windows in High Performance Computing - Linux

This is a discussion on Windows in High Performance Computing - Linux ; I am 'one of the few' people who has actually "used" Windows Compute Cluster Server 2003 in a work environment (for running Computation Fluid Dynamics programs). Here are my experiences: Firstly, basically it is just Windows 2003 server for AMD64. ...

+ Reply to Thread
Results 1 to 11 of 11

Thread: Windows in High Performance Computing

  1. Windows in High Performance Computing


    I am 'one of the few' people who has actually "used" Windows Compute
    Cluster Server 2003 in a work environment (for running Computation
    Fluid Dynamics programs).
    Here are my experiences:
    Firstly, basically it is just Windows 2003 server for AMD64. There is
    an additional CD with the 'Compute Cluster' pack that must be
    installed.
    There is a graphical queue manager MMC snapin etc.

    There are some current problems with Computer Cluster Server:
    * The kernel is old, and as usual, there are troubles finding the
    correct 64-bit driver for the on-board SCSI in our brand new Dell
    nodes. Windows requires a floppy disk. None of the nodes have a floppy
    drive, so we have to hook up a USB floppy, and locate a floppy disk!!
    * Default update behaviour. You must be careful to make sure that
    Windows update is disabled, because it will reboot that machines in
    the middle of the night, killing hundreds of hours worth of jobs, and
    driving users into a searing rage. Updates that require reboots are
    far too frequent for production use systems. Updates at present
    require additional IT management to manually install.
    * Jobs randomly crash - avoid virus checkers. I think because our IT
    department's amazing virus scanner decides to eat up all the non-paged
    pool (i'm still investigating this one). Installing virus scanners on
    compute nodes, where intensive disk access is the norm borders on
    insanity in my opinion.
    * Infiniband drivers - We have beta drivers from a major provider of
    infiniband cards (OpenIB stack). They are, in their current state, not
    usable. The compute cluster node management software faults repeatedly
    in the user mode DLL of the OpenIB device drivers. We have never been
    able to run a job with it over more than 2 nodes.
    Generally, we use HP MPI, which is pretty solid over ethernet, but
    because of bad drivers, can't currently be used over Infiniband. We'll
    have to get updated drivers when I get time to look at this.
    Saying this, we've also had issues in the past with Linux Infiniband
    drivers, but these are usually resolved quickly by vendor, or by a
    kernel update.
    * Lack of a decent scripting language that is common on all platforms
    requires that cygwin/bash/python be deployed to all nodes to allow
    platform independent scripts to execute.
    * Management of shared drives on Windows has always been a problem.
    Some Windows programs just don't like UNC paths. Drive letter mapping
    gets mixed up with domain policy, update scripts, user's personal
    mappings, and our source control system's mappings. What a mess.
    Windows NEEDS an automounter. Consequently, we tend to copy files
    about. This is better for the performance of the programs that we are
    running in many cases anyway.
    * There are two Terminal Server licenses per node for administration,
    so if your users need to be able to login to compute nodes (as they
    frequently do with some programs), you will have to fork out for
    expensive terminal server licenses.
    * Few HPC applications actually support Windows compute nodes.
    Save yourself some effort, and just deploy Linux on HPC nodes.


    http://blogs.computerworld.com/where..._is_2_to_linux

    Ballmer recently said that High Performance Computing is one area
    Microsoft aims to have a large presence in.

  2. Re: Windows in High Performance Computing

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    ____/ nessuno@wigner.berkeley.edu on Tuesday 30 September 2008 23:05 : \____

    >
    > I am 'one of the few' people who has actually "used" Windows Compute
    > Cluster Server 2003 in a work environment (for running Computation
    > Fluid Dynamics programs).
    > Here are my experiences:
    > Firstly, basically it is just Windows 2003 server for AMD64. There is
    > an additional CD with the 'Compute Cluster' pack that must be
    > installed.
    > There is a graphical queue manager MMC snapin etc.
    >
    > There are some current problems with Computer Cluster Server:
    > * The kernel is old, and as usual, there are troubles finding the
    > correct 64-bit driver for the on-board SCSI in our brand new Dell
    > nodes. Windows requires a floppy disk. None of the nodes have a floppy
    > drive, so we have to hook up a USB floppy, and locate a floppy disk!!
    > * Default update behaviour. You must be careful to make sure that
    > Windows update is disabled, because it will reboot that machines in
    > the middle of the night, killing hundreds of hours worth of jobs, and
    > driving users into a searing rage. Updates that require reboots are
    > far too frequent for production use systems. Updates at present
    > require additional IT management to manually install.
    > * Jobs randomly crash - avoid virus checkers. I think because our IT
    > department's amazing virus scanner decides to eat up all the non-paged
    > pool (i'm still investigating this one). Installing virus scanners on
    > compute nodes, where intensive disk access is the norm borders on
    > insanity in my opinion.
    > * Infiniband drivers - We have beta drivers from a major provider of
    > infiniband cards (OpenIB stack). They are, in their current state, not
    > usable. The compute cluster node management software faults repeatedly
    > in the user mode DLL of the OpenIB device drivers. We have never been
    > able to run a job with it over more than 2 nodes.
    > Generally, we use HP MPI, which is pretty solid over ethernet, but
    > because of bad drivers, can't currently be used over Infiniband. We'll
    > have to get updated drivers when I get time to look at this.
    > Saying this, we've also had issues in the past with Linux Infiniband
    > drivers, but these are usually resolved quickly by vendor, or by a
    > kernel update.
    > * Lack of a decent scripting language that is common on all platforms
    > requires that cygwin/bash/python be deployed to all nodes to allow
    > platform independent scripts to execute.
    > * Management of shared drives on Windows has always been a problem.
    > Some Windows programs just don't like UNC paths. Drive letter mapping
    > gets mixed up with domain policy, update scripts, user's personal
    > mappings, and our source control system's mappings. What a mess.
    > Windows NEEDS an automounter. Consequently, we tend to copy files
    > about. This is better for the performance of the programs that we are
    > running in many cases anyway.
    > * There are two Terminal Server licenses per node for administration,
    > so if your users need to be able to login to compute nodes (as they
    > frequently do with some programs), you will have to fork out for
    > expensive terminal server licenses.
    > * Few HPC applications actually support Windows compute nodes.
    > Save yourself some effort, and just deploy Linux on HPC nodes.
    >

    >
    > http://blogs.computerworld.com/where..._is_2_to_linux
    >
    > Ballmer recently said that High Performance Computing is one area
    > Microsoft aims to have a large presence in.


    He also boasted about search ambitions for like half a decade. And where are
    they? Still declining, despite /paying/ people to use the thing.

    My colleagues tried Windows high perfor~1 and it sucked so badly that it was
    embarrassing. Like many other such clusters, there was an escape from it,
    despite Microsoft 'freebies'.

    http://groups.google.com/group/comp....6892356?lnk=st

    - --
    ~~ Best of wishes

    "Linux is a very complete and sophisticated operating system. And there is a
    lot of work being done to improve it in and of itself, particularly to make it
    easier to use and easier for people to set up on their personal computers."

    --Paul Maritz, senior vice-president, Microsoft
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (GNU/Linux)

    iEYEARECAAYFAkjivs4ACgkQU4xAY3RXLo6dLQCeNuX1qMebbI gZTg9NMw1bMQ18
    yE4An04Nn+BVP+fn3Ssu0ffE+fA3bgRu
    =g1W3
    -----END PGP SIGNATURE-----

  3. Re: Windows in High Performance Computing

    On Sep 30, 5:05*pm, Roy Schestowitz
    wrote:
    > -----BEGIN PGP SIGNED MESSAGE-----
    > Hash: SHA1
    >
    > ____/ ness...@wigner.berkeley.edu on Tuesday 30 September 2008 23:05 : \____
    >
    >
    >
    > >
    > > I am 'one of the few' people who has actually "used" Windows Compute
    > > Cluster Server 2003 in a work environment (for running Computation
    > > Fluid Dynamics programs).
    > > Here are my experiences:
    > > Firstly, basically it is just Windows 2003 server for AMD64. There is
    > > an additional CD with the 'Compute Cluster' pack that must be
    > > installed.
    > > There is a graphical queue manager MMC snapin etc.

    >
    > > There are some current problems with Computer Cluster Server:
    > > * The kernel is old, and as usual, there are troubles finding the
    > > correct 64-bit driver for the on-board SCSI in our brand new Dell
    > > nodes. Windows requires a floppy disk. None of the nodes have a floppy
    > > drive, so we have to hook up a USB floppy, and locate a floppy disk!!
    > > * Default update behaviour. You must be careful to make sure that
    > > Windows update is disabled, because it will reboot that machines in
    > > the middle of the night, killing hundreds of hours worth of jobs, and
    > > driving users into a searing rage. Updates that require reboots are
    > > far too frequent for production use systems. Updates at present
    > > require additional IT management to manually install.
    > > * Jobs randomly crash - avoid virus checkers. I think because our IT
    > > department's amazing virus scanner decides to eat up all the non-paged
    > > pool (i'm still investigating this one). Installing virus scanners on
    > > compute nodes, where intensive disk access is the norm borders on
    > > insanity in my opinion.
    > > * Infiniband drivers - We have beta drivers from a major provider of
    > > infiniband cards (OpenIB stack). They are, in their current state, not
    > > usable. The compute cluster node management software faults repeatedly
    > > in the user mode DLL of the OpenIB device drivers. We have never been
    > > able to run a job with it over more than 2 nodes.
    > > Generally, we use HP MPI, which is pretty solid over ethernet, but
    > > because of bad drivers, can't currently be used over Infiniband. We'll
    > > have to get updated drivers when I get time to look at this.
    > > Saying this, we've also had issues in the past with Linux Infiniband
    > > drivers, but these are usually resolved quickly by vendor, or by a
    > > kernel update.
    > > * Lack of a decent scripting language that is common on all platforms
    > > requires that cygwin/bash/python be deployed to all nodes to allow
    > > platform independent scripts to execute.
    > > * Management of shared drives on Windows has always been a problem.
    > > Some Windows programs just don't like UNC paths. Drive letter mapping
    > > gets mixed up with domain policy, update scripts, user's personal
    > > mappings, and our source control system's mappings. What a mess.
    > > Windows NEEDS an automounter. Consequently, we tend to copy files
    > > about. This is better for the performance of the programs that we are
    > > running in many cases anyway.
    > > * There are two Terminal Server licenses per node for administration,
    > > so if your users need to be able to login to compute nodes (as they
    > > frequently do with some programs), you will have to fork out for
    > > expensive terminal server licenses.
    > > * Few HPC applications actually support Windows compute nodes.
    > > Save yourself some effort, and just deploy Linux on HPC nodes.
    > >

    >
    > >http://blogs.computerworld.com/where..._is_2_to_linux

    >
    > > Ballmer recently said that High Performance Computing is one area
    > > Microsoft aims to have a large presence in.

    >
    > He also boasted about search ambitions for like half a decade. And where are
    > they? Still declining, despite /paying/ people to use the thing.
    >
    > My colleagues tried Windows high perfor~1 and it sucked so badly that it was
    > embarrassing. Like many other such clusters, there was an escape from it,
    > despite Microsoft 'freebies'.
    >
    > http://groups.google.com/group/comp....rowse_thread/t...
    >
    > - --
    > * * * * * * * * ~~ Best of wishes
    >
    > "Linux is a very complete and sophisticated operating system. And there is a
    > lot of work being done to improve it in and of itself, particularly to make it
    > easier to use and easier for people to set up on their personal computers.."
    >
    > * * * * * * * * --Paul Maritz, senior vice-president, Microsoft
    > -----BEGIN PGP SIGNATURE-----
    > Version: GnuPG v1.4.9 (GNU/Linux)
    >
    > iEYEARECAAYFAkjivs4ACgkQU4xAY3RXLo6dLQCeNuX1qMebbI gZTg9NMw1bMQ18
    > yE4An04Nn+BVP+fn3Ssu0ffE+fA3bgRu
    > =g1W3
    > -----END PGP SIGNATURE-----


    Ballmer has actually made quite a few specific predictions lately----
    someone should keep track of them, as most of them will no doubt turn
    out to have the same quality as Gates' promise to eliminate spam
    within 2 years (was that 2004?). Among others, he plans to take a
    major presence in search---in spite of the fact that Microsoft is
    descending ever lower into the single digits in search. And to kick
    Apple in the smart phone market---good luck. What's interesting is
    that Ballmer is clearly very concerned about competition from Apple,
    that is, real competition. He made some statement about expecting the
    old tried and true Microsoft methods to work, ie extend, extinguish
    etc. It's like he doesn't get it, that when he doesn't control a
    monopoly, he's got to compete on the basis of quality. It seems to be
    an alien thought around Microsoft.

  4. Re: Windows in High Performance Computing


    wrote in message
    news:dafb2d5f-4b3a-4308-b945-b6f0d10ce7f5@b1g2000hsg.googlegroups.com...
    >
    > Ballmer recently said that High Performance Computing is one area
    > Microsoft aims to have a large presence in.


    Down the road, eh? Do you think that they are incapable of advancing beyond
    Win2K3 server? That is kind of arrogant.


  5. Re: Windows in High Performance Computing

    On Tue, 30 Sep 2008 16:05:35 -0700 (PDT), nessuno@wigner.berkeley.edu
    wrote:

    >
    > I am 'one of the few' people who has actually "used" Windows Compute
    > Cluster Server 2003 in a work environment (for running Computation
    > Fluid Dynamics programs).
    > Here are my experiences:
    > Firstly, basically it is just Windows 2003 server for AMD64. There is
    > an additional CD with the 'Compute Cluster' pack that must be
    > installed.



    It's amazing that Linux can do all that stuff but yet can't seem to make a
    dent in the desktop system area.
    Linux has been free for 15+ years and still is ignored by most people.

    Figure out why, fix it and you might have a winner on your hands.
    Keep churning out new distributions by the hour and you will remain doomed.


    --
    Moshe Goldfarb
    Collector of soaps from around the globe.
    Please visit The Hall of Linux Idiots:
    http://linuxidiots.blogspot.com/
    Please Visit www.linsux.org

  6. Re: Windows in High Performance Computing

    The Lying Idiot Roy wrote:

    >
    > He also boasted about search ambitions for like half a decade. And
    > where are they? Still declining, despite /paying/ people to use the
    > thing.
    >
    > My colleagues tried W




    What about your "Semantic Knowledge Engine" Iuron?

  7. Re: Windows in High Performance Computing

    Sean Keese writes:

    > The Lying Idiot Roy wrote:
    >
    >>
    >> He also boasted about search ambitions for like half a decade. And
    >> where are they? Still declining, despite /paying/ people to use the
    >> thing.
    >>
    >> My colleagues tried W

    >
    >
    >
    > What about your "Semantic Knowledge Engine" Iuron?


    Have the bearded masses not been summoned by Roy's magic horn?

    http://www.ucforums.com/image.php?u=...ine=1130925429

    He has had a vision, now all that is required is some socially inept geek with
    a huge brain to turn it into a reality!

    I'm amazed that that big head Kelsey has not "knocked it up" on one his
    386 cluster computers in ISO C and had it meeting his needs in less than
    half an hour.

    Steady there folks - all this talk of "horn" and "needs" will get a few
    here dribbling....

    --
    "You're a condescending, arrogant asshole, Quack."
    -- Tattoo Vampire in alt.os.linux.ubuntu, comp.os.linux.advocacy

  8. Re: Windows in High Performance Computing

    Roy Schestowitz writes:

    >> * Lack of a decent scripting language that is common on all platforms
    >> requires that cygwin/bash/python be deployed to all nodes to allow
    >> platform independent scripts to execute.


    The cluelessness or rabid stupidity of that statement is simply
    breathtaking.

  9. Re: Windows in High Performance Computing

    Hadron wrote:

    > Roy Schestowitz writes:
    >
    >>> * Lack of a decent scripting language that is common on all platforms
    >>> requires that cygwin/bash/python be deployed to all nodes to allow
    >>> platform independent scripts to execute.

    >
    > The cluelessness or rabid stupidity of that statement is simply
    > breathtaking.


    What "common scripting language" would you take, Hadron Quark?

    --
    Microsoft: The company that made email dangerous
    And web browsing. And viewing pictures. And...


  10. Re: Windows in High Performance Computing

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA1

    ____/ Sean Keese on Wednesday 01 October 2008 14:29 : \____

    > The Lying Idiot Roy wrote:


    ^^^^^^^^^

    No posting history, using the gateway all the Microsoft Munchkins use. Gee, you
    certainly are a nymshifting munchkin.

    Go back where you came from. Sweaty Ballmer needs a massage.

    ..-------------------------------------------------.
    |____________________________________________O_-_X|
    | File Options Tools Help |
    `_-_____-________-______-_________________________'
    | Alert |____________
    | *Plonk* ' Subject |______________
    `----------. *Sender* ' Troll |
    `----------. Idiot |
    | Moron |
    | Twit |
    | Imbecile |
    | Changeling |
    | Cretin |
    | Asshole |
    | Weird |
    | Dunce |
    | Kook |
    | Eerie |
    | *Nym* |
    | Morph |
    `------------'


    > >
    > > He also boasted about search ambitions for like half a decade. And
    > > where are they? Still declining, despite /paying/ people to use the
    > > thing.
    > >
    > > My colleagues tried W

    >
    >
    >
    > What about your "Semantic Knowledge Engine" Iuron?


    - --
    ~~ Best of wishes

    Roy S. Schestowitz | Switch to GNU/Linux. < http://www.getgnulinux.org/ >
    http://Schestowitz.com | GNU is Not UNIX | PGP-Key: 0x74572E8E
    http://iuron.com - proposing a non-profit search engine
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.9 (GNU/Linux)

    iEYEARECAAYFAkjjp5AACgkQU4xAY3RXLo6RygCgtK5sAQgJ/mPcwqVYHUC+iz7x
    hZkAn3F0C6bRlB9yGvqN7ttmekkVwO2W
    =ZMEd
    -----END PGP SIGNATURE-----

  11. Re: Windows in High Performance Computing

    [snips]

    On Tue, 30 Sep 2008 16:05:35 -0700, nessuno@wigner.berkeley.edu wrote:

    > There are some current problems with Computer Cluster Server: * The
    > kernel is old, and as usual, there are troubles finding the correct
    > 64-bit driver for the on-board SCSI in our brand new Dell nodes.


    No, that must be wrong; one of the never-ending cries of the Wintrolls is
    that for new hardware, Windows has the support, Linux doesn't.

    > Windows
    > requires a floppy disk. None of the nodes have a floppy drive, so we
    > have to hook up a USB floppy, and locate a floppy disk!!


    This in an age where floppies are all but extinct.

    > * Default
    > update behaviour. You must be careful to make sure that Windows update
    > is disabled, because it will reboot that machines in the middle of the
    > night, killing hundreds of hours worth of jobs, and driving users into a
    > searing rage.


    Ain't it great? Supposedly reliable OSen which reboot themselves even
    when active tasks are running. To think, someone actually got _paid_ to
    implement that particular bit of idiocy.

    > Updates that require reboots are far too frequent for
    > production use systems.


    They must use a _very_ actively developed kernel. Oh, wait, I forgot;
    that OS can require reboots for upgrading a *web browser*.

    > management to manually install. * Jobs randomly crash - avoid virus
    > checkers.


    What's a virus checker do? Aside from consume resources, I mean? Oh,
    wait - that other OS has problems with such things, doesn't it?

    > I think because our IT department's amazing virus scanner
    > decides to eat up all the non-paged pool (i'm still investigating this
    > one). Installing virus scanners on compute nodes, where intensive disk
    > access is the norm borders on insanity in my opinion.


    Needing them _at all_ borders on insanity.

    > drivers when I get time to look at this. Saying this, we've also had
    > issues in the past with Linux Infiniband drivers, but these are usually
    > resolved quickly by vendor, or by a kernel update.


    There ain't nothing perfect, but some things are a little _less_ perfect
    than others.

    > * Lack of a decent scripting language that is common on all platforms
    > requires that cygwin/bash/python be deployed to all nodes to allow
    > platform independent scripts to execute.


    So, basically, you're bringing the usability of Linux into Windows - but
    wasn't Windows supposed to be the ultimate in usability? Isn't that the
    battle cry of the Windroids?

    > * Management of shared drives
    > on Windows has always been a problem. Some Windows programs just don't
    > like UNC paths. Drive letter mapping gets mixed up with domain policy,
    > update scripts, user's personal mappings, and our source control
    > system's mappings. What a mess. Windows NEEDS an automounter.


    Windows needs to abolish its mid-70's-era drive letter nonsense. *Then*
    it needs an automounter.

    > * There are two Terminal Server licenses per node for administration, so
    > if your users need to be able to login to compute nodes (as they
    > frequently do with some programs), you will have to fork out for
    > expensive terminal server licenses.


    Funny; I can have 2, 20, or 2,000 people log into any of my Linux boxen
    without worrying about such things; the only limiting factor are physical
    resources - memory, bandwidth, etc.

    > Ballmer recently said that High Performance Computing is one area
    > Microsoft aims to have a large presence in.


    And in 2025, they might actually get somewhere.


+ Reply to Thread