After the recently concluded HPC/InternationalSupercomputing conference, there is quite a bit of talk about Exa-scalecomputing. Theidea here is to push Supercomputing into the realm of sustainableExaflop computation (by 2018). [Lets ignore for a moment that itspretty hard to sustain even the current Petaflop levels, a problem thatwill no doubt be solved in next few years ... ]

Jack Dongarra, a leader in the area of Supercomputing (and co-creatorof leading mathematical packages such as Linpack, EISpack, etc),recently gave an interview on this topic which I think makes for an interestingread (you can read the full interview here). The salient points areinteresting, and I'm listing here a few that I found most worthpondering over:
  • Going from Petascale to Exascale will mean going from hundreds ofthousands of threads to billions ofthreads
    • This shift is similar to the shift from vector programs toparallel programming
    • The strategy used to achieve petascale will no longer scale tothe exascale level, so programs will need to be redesigned
    • Programs will need to have asynchronous handling built in.
  • Exascale programs/machines willessentially be hybrids and purely MPI or loop-based programswill no longer be viable for this scale. Thus a fork-join model will nolonger work
  • Memory is going to play at least as big a factor as CPU. Forcosts, for heat considerations and for latency/computational issues.
  • Programs will have to build infault-tolerance. At that scale, something is bound to fail. Andyou can restart using checkpointing
  • Machines will be both lightweight parallel (Blue-gene style oflots of simple threads) or commodity processors with GPU accelerators.
  • International cooperation is amust. Government (and international) involvement of bodies likeG-8 will be critical drivers.
  • Community will drive developmentinto vendors.
This is an ambitious and complex goal and the journey will beinteresting to follow as much for the human pursuit as it is for thetechnical pursuit. As a major HPC vendor, Sun systems group (insideOracle) is watching and following these developments very closely.Compilers and tools are an integral part of such a pursuit; they havealways been and will continue to be critical.

Read More about [(HPC) Challenges to Exascale Super-computing...