> Its probably worth playing with vfs.zfs.cache_flush_disable when using
> the hardware RAID.
>
> By default, ZFS will flush the entire hardware cache just to make sure
> the ZFS Intent Log (ZIL) has been written.
>
> This isn't so bad on a group of hard disks with small caches, but bad if
> you have 256mb of controller write cache.


Flushing the cache to constituent drives also has a direct impact on
latency, even without any dirty data (save what you just written) in
the cache. If you're doing anything that does frequent fsync():s,
you're likely to not want to wait for actual persistence to disk (with
battery backed cache).

In any case, why would the actual RAID controller cache be flushed,
unless someone expliclitly configured it such? I would expect a
regular BIO_FLUSH (that's all ZFS is going right?) to be satisfied by
the data being contained in the controller cache, under the assumption
that it is battery backed, and that the storage volume/controller has
not been explicitly configured otherwise to not rely on the battery
for persistence.

Please correct me if I'm wrong, but if synchronous writing to your
RAID device results in actually waiting for underlying disks to commit
the data to platters, that sounds like a driver/controller
problem/policy issue rather than anything that should be fixed by
tweaking ZFS.

Or is it the case that ZFS does both a "regular" request to commit
data (which I thought was the purpose of BIO_FLUSH, even though the
"FLUSH" sounds more specific) and separately does a "flush any actual
caches no matter what" type of request that ends up bypassing
controller policy (because it is needed on stupid SATA drives or
such)?

--
/ Peter Schuller

PGP userID: 0xE9758B7D or 'Peter Schuller '
Key retrieval: Send an E-Mail to getpgpkey@scode.org
E-Mail: peter.schuller@infidyne.com Web: http://www.scode.org


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (FreeBSD)

iEYEARECAAYFAkkNwqQACgkQDNor2+l1i30+tQCfYKIGEHETqN g1gfwi9XwBuFuq
dMAAoK7OczyR6rVe6gigw+EG797v0L2I
=5oRU
-----END PGP SIGNATURE-----