Re: Failing SSL connection due to weird interaction with openssl

On Tue, Oct 23, 2012 at 4:09 AM, Lars Kanis <lars@greiz-reinsdorf.de> wrote:
> While investigating a ruby-pg issue [1], we noticed that a libpq SSL
> connection can fail, if the running application uses OpenSSL for other work,
> too. Root cause is the thread local error queue of OpenSSL, that is used to
> transmit textual error messages to the application after a failed crypto
> operation. In case that the application leaves errors on the queue, the
> communication to the PostgreSQL server can fail with a message left from the
> previous failed OpenSSL operation, in particular when using non-blocking
> operations on the socket. This issue with openssl is quite old now - see
> [3].
>
> For [1] it turned out that the issue is subdivided into these three parts:
> 1. the ruby-openssl binding does not clear the thread local error queue of
> OpenSSL after a certificate verify
> 2. OpenSSL makes use of a shared error queue for different crypto contexts.
> 3. libpq does not ensure a cleared error queue when doing SSL_* calls
>
> To 1: Remaining messages on the error queue can generally lead to failing
> operations, later on. I'd talk to the ruby-openssl developers, to discuss
> how we can avoid any remaining messages on the queue.
>
> To 2: SSL_get_error() inspects the shared error queue under some conditions.
> It's maybe poor API design, but it's documented behaviour [2]. So we
> certainly have to get along with it.
>
> To 3: To make libpq independent to a previous error state, the error queue
> might be cleared with a call to ERR_clear_error() prior
> SSL_connect/read/write as in the attached trivial patch. This would make
> libpq robust against other uses of openssl within the application.
>
> What do you think about clearing the OpenSSL error queue in libpq in that
> way?

Shouldn't it be the job of whatever code is consuming the error to
clear the error queue afterwards?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company