Re: BUG #17760: SCRAM authentication fails with "modern" (rsassaPss signature) server certificate

On Sat, Feb 11, 2023 at 12:58:02PM +0200, Heikki Linnakangas wrote:
> X509_get_signature_info() calls X509_check_purpose(), which calls internal
> function ossl_x509v3_cache_extensions(), which extracts and caches quite a
> lot of information from the certificate. It calculates and caches its SHA1
> hash, for example. That seems acceptable, the overhead is negligible and I
> don't see any scenario where X509_get_signature_nid() would succeed but
> X509_get_signature_info() would fail.

Excellent find!  On 1.1.1, that's x509v3_cache_extensions, it seems.
The extra overhead at authentication does not worry me, FWIW.  I am
wondering why the OpenSSL folks have done nothing for
X509_get_signature_nid() in 1.1.1~ in this case, but whatever.

From my notes, I was also wondering whether we should improve the
situation with the two following things:
- Addition of a new GUC called, say, ssl_check_cert_hash to check at
TLS init time in the backend if a certificate's hash can be found or
not.
- Improvement of error messages in this area when a hash function
cannot be found.

However, this proves to be unnecessary once we use
X509_get_signature_info() as loading an RSA-PSS cert with OpenSSL
1.0.2 or 1.1.0 leads to this error, simply:
FATAL:  could not load server certificate file "server.crt": unsupported algorithm

Would more facility make sense for other cert types where OpenSSL
cannot map with these yet, though?  I am not really convinced that we
need to burn more resources until we get a complaint and adapt again,
as it may be possible that OpenSSL also improves in-between for such
cases.

The SSL tests need to be patched so as they adapt on-the-fly depending
on if RSA-PSS is supported, of course, and we cannot do a
switch_server_cert() for versions older than 1.1.1.

> +1 on your patch. I think the only thing it's missing is changes in
> meson.build and Solution.pm to match the configure.ac changes.

Done.

> Agreed, if we have an easy solution for RSA-PSS, that's good enough
> for now.

Indeed.

With all that in mind, I am finishing with the attached with the
tests, the meson tweaks and the MSVC tweaks.  I have tested it on
HEAD, with OpenSSL down to 1.0.1 which is the minimum version of this
branch.  Making sure that this mostly works with 1.0.0 and 0.9.8 on
older branches would not be an issue here.
--
Michael