A colleague found out something interesting about the rperf of IBM. rPerf stands for "relative Performance" and is a construction of IBM to compare the performance of their systems. My colleague got to this points while studying the recent TPC-H result for the p6 595.

When you look at the rPerf value of a p5 595 with the 1,9 GHz in document as current of May 2009, it yields a value of 306.21 times the performance of the rPerf baseline (a pSeries 640 system). When you look at the rPerf value of the p6 p595 yo will find a value that this system delivers 553.01 times the performance of the baseline. That's a difference of 81% percent.

Why did i talked about especially those values: These systems were used in the TPC-H benchmarks by IBM. In those benchmarks the p5 p595 yielded 100,512 QphH@3000GB, the p6 p595 yielded 156,537 QphH@3000GB. When you calculate the difference you will get just 56 percent difference. There are 25% missing percent.

I know that the p5 p595 used vastly more disks, but i want to point you on certain facts:
  • The new system is using AIX 6.1 instead of AIX 5.3. 6.1 is regarded as an important step in scaling on large systems.
  • The p6 configuration is using twice the memory of the p5 configuration. TPC-H is known to be a very memory intensive benchmark. And usual ... only a cached I/O is a good I/O)
  • The p6 provides PCIe instead of PCI-X, thus providing a much faster I/O
  • The p5 config used a general purpose database, while the p6 config used a specialized DWH database
  • The p6 config used 4 GB SAN instead of 2 GB FC.
  • The p6 config used 667 MHz DRAM instead of 533 MHz

Let's just assume for a moment, that those changes would just outweight the number of disks.

You could draw a conclusion out of this: rPerf is vastly overscaling the performance for I/O intensive tasks. The difference is 25% less performance between estimated performance by rperf and real performance measured in the benchmark.

I think IBM is pretty aware of the fact, that rPerf isn't really usable for system sizings, at least when you look at the rPerf webpage you can get to the conclusion that they want to say "Nice numbers. But don't use them ...". Furthermore it doesn't test important parts of commercial computing:
The rPerf model is not intended to represent any specific public benchmark results and should not be reasonably used in that way. The model simulates some of the system operations such as CPU, cache and memory. However, the model does not simulate disk or network I/O operations.
However i'm aware of several situations where an competing offer was based on an rPerf calculation. independent of the point if its really appropriate to sthe situation. So when you get an offer of IBM using rPerf used in the sizing you should be cautious (in the sense of very cautious) if it's really matching your needs.

Read More about [Caveat emptor 2...