Some light intro first: OpenSSL has a concept of plugins/add-ons called 'engines' which can supply alternative implementation of crypto operations (digests, symmetric and asymmetric ciphers and random data generation). The main reason for the existence of the engines is the ability to offload crypto ops to hardware. (Open)Solaris ships with an engine called PKCS#11 engine which provides access to Solaris Cryptographic Framework which in turn can provide access to HW crypto.

I spent some time fixing bugs in OpenSSL PKCS#11 engine in Solaris so I got quite intimate with its internals. Recently while discussing an upcoming feature with Jan he asked me why one particular detail in the engine is done one way and not the other (it's the fork() detection not done via atfork handlers; for the curious). It took me some thinking to find the answer (I focused on the other changes at that time) which made us realize that it would be good to summarize the design choices behind the engine and also to document the internals so that others can quickly see what's going on inside and also be able to do changes in the engine without reverse engineer the thoughts behind it. The outcome is a set of slides which I hope succinctly describe both the overall picture and the gritty details.

The presentation can be downloaded here.