Browser error on Intel Mac but not on PPC Mac or Windows

Hello,
I'm updating a library which provides IP/SSL functionality in an
application development environment that most people have never heard
of, so I'll spare you the gory details. I'm updating it so it builds
as a universal binary, and I'm running into problems on the Mactel
side. I'm looking for ideas on how to debug this. If anyone has any
suggestions, I'm all ears, because this SSL stuff is very new to me.
My library statically links against openssl 0.9.8a.

Basically, I have a very simple test application that uses my library
to create an SSL context and listen on port 6112, then it kicks off a
server process when a valid connection has been made. The server
process just continuously reads (via SSL_read()) until it gets an HTTP
request, then it just sends some text back. So, my test is just
opening a browser and connecting to "https://<hostname>:6112".

I've built this library on Windows and tested it with IE7, and it
works fine. Just so you know, the certificate I'm creating the SSL
context with is self-signed and so I get the warnings in the browser
about how I shouldn't trust it, but if I tell it to continue, it does.
The same is true when I test it on a G5/PPC Mac, using the same
certificate with Firefox and Safari...works as expected. However, when
I test it on my Intel Mac using either Firefox, I get the error "Could
not establish an encrypted connection because the certificate
presented by <hostname> has an invalid signature." And with Safari, I
get the error "Safari can't open the page 'https://<hostname>:6112/'
because it couldn't establish a secure connection to the server
'<hostname>'." The connection is made because I see the server process
get spawned. However, the browser never sends the HTTP request,
presumably because it doesn't like the certificate.

I was able use the old version of the library on my Mactel and the
test behaves as expected with the old version using the same version
of the browsers and the same certificate. So, it seems like it has to
be something with the way the library is built/works on Mactel, not
the certificate itself.


Does anyone have any thoughts on what could be going wrong, or what I
should try to debug this?


Thanks in advance,
Robbie

I was able to resolve it. In case anyone runs into something similar,
it ended up being a problem with the Configure script in openssl
0.9.8a (and possibly other versions too). It was a byte ordering issue
that was happening in the SSL handshake. This was due to the -
DB_ENDIAN flag getting thrown in their when calling openssl's
Configure with the darwin-i386-cc configuration. It should be -
DL_ENDIAN because x86 is little endian.

So, if you're running into this problem (really, any SSL handshaking
problem or invalid certificate/signature problem) on Mactel, but not
PPC, the way to fix it is to edit the Configure script, search for
"darwin-i386-cc" and on that line, change DB_ENDIAN to DL_ENDIAN.

Best regards,
Robbie