What sort of throughput should one expect across an encrypted IPSec
tunnel? I know fancy dedicated hardware can do multiple gigabits per
sec using 3DES, but I have no idea what I should be getting on plain
computer hardware.

Here's my setup:
private network 1 <--> gateway1 <--> internet <--> gateway2 <-->
private network 2

The gateways are a 2.5GHz P4 and a newish Opteron both running openbsd
4.1 with broadcom and intel gigabit nics and 'gigabit' uplinks to the
net. They have no dedicated encryption hardware. Everything seems to
work fine, ie computers on the two private networks can access each
other just fine for nfs/ssh/etc

Testing via both coping a large file using NFS and scp:
With the default encryption (AES) I get exactly 2.0MB/s across the
tunnel
Switching to 3DES I get exactly 2.3MB/s
Bypassing the tunnel (by binat'ing a public ip to a private one) gets
about 50MB/s, give or take 10MB/s depending on how busy the upstream
networks are.

Looking at top, I get about 15% CPU usage on the slower gateway when
using the tunnel, and about 30% when bypassing the tunnel (no
encryption). I also get about 10-20% interrupt time, but it bounces
around a lot.

Is ~2MB/s all I can expect, or should I be trying to troubleshoot? In
my naivety I would expect the cpu to be at 100% if the encryption was
the bottleneck, but I have no idea how network stacks/NICs work, so
maybe there's some other bottleneck introduced when using IPSec.