When we installed openssl version 0.9.8a, SSH linked against it started to
crash, sometimes before connection was made and sometimes well into the
connection. One of our users provided the following backtrace:

Core was generated by `ssh kekkonen.cs.hut.fi'.
Program terminated with signal 11, Segmentation fault.
#0 0xff2084f4 in bn_sub_words () from /usr/lib/libcrypto.so.0.9.8
(gdb) bt
#0 0xff2084f4 in bn_sub_words () from /usr/lib/libcrypto.so.0.9.8
#1 0xff20038c in bn_sub_part_words () from /usr/lib/libcrypto.so.0.9.8
#2 0xff200984 in bn_mul_recursive () from /usr/lib/libcrypto.so.0.9.8
#3 0xff200bfc in bn_mul_recursive () from /usr/lib/libcrypto.so.0.9.8
#4 0xff20208c in BN_mul () from /usr/lib/libcrypto.so.0.9.8
#5 0xff209894 in BN_mod_mul_montgomery () from /usr/lib/libcrypto.so.0.9.8
#6 0xff1fdca4 in BN_mod_exp_mont_consttime () from /usr/lib/libcrypto.so.0.9.8
#7 0xff228f10 in generate_key () from /usr/lib/libcrypto.so.0.9.8
#8 0x0003b7c0 in dh_gen_key ()
#9 0x00039aa4 in kexdh_client ()
#10 0x0003727c in kex_input_kexinit ()
#11 0x000368a8 in dispatch_run ()
#12 0x0001e41c in client_loop ()
#13 0x0001950c in main ()


SSH is version OpenSSH_4.2p1. This problem does not occur when ssh is
linked against openssl version 0.9.7i. Here is the self test report:



OpenSSL self-test report:

OpenSSL version: 0.9.8a
Last change: Remove the functionality of SSL_OP_MSIE_SSLV2_RSA_PADDI...
Options: --prefix=/c/openssl enable-shared no-gmp no-krb5 no-mdc2 no-rc5 no-zlib no-zlib-dynamic
OS (uname): SunOS camelot 5.8 Generic_117350-25 sun4u sparc SUNW,Ultra-4
OS (config): sun4u-whatever-solaris2
Target (default): solaris-sparcv9-cc
Target: solaris-sparcv9-cc
Compiler: cc: Sun C 5.6 2004/07/15

Failure!



>From the end of the file 'testlog':


testing ECDSA_sign() and ECDSA_verify() with some internal curves:
secp160k1: ....... ok
secp160r1: ....... ok
secp160r2: ....... ok
secp192k1: ....... ok
secp224k1: ....... ok
secp224r1: ....... ok
secp256k1: ....... ok
secp384r1: ....... ok
secp521r1: ... failed

ECDSA test failed




--
Juha Aatrokoski, System Administrator, CS lab / HUT

__________________________________________________ ____________________
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager majordomo@openssl.org