BUG #4869: No proper initialization of OpenSSL-Engine in libpq

The following bug has been logged online:

Bug reference:      4869
Logged by:          Lars Kanis
Email address:      kanis@comcard.de
PostgreSQL version: 8.4rc1
Operating system:   Linux c1170lx 2.6.24-23-generic #1 SMP Wed Apr 1
21:47:28 UTC 2009 i686 GNU/Linux
Description:        No proper initialization of OpenSSL-Engine in libpq
Details:

When using OpenSSL-engine pkcs11 with PGSSLKEY=pkcs11:id_45 the
authentication to the PG-server fails with "engine not initialized".

According to the OpenSSL-docs
(http://www.openssl.org/docs/crypto/engine.html) the structural reference
returned by ENGINE_by_id needs to be initialized first before use. The
buildin engine doesn't need this, but most of external engines don't work
otherwise.

Moreover the structural and functional references should be freed in any
case.


The following patch solves the problem:

diff -ur postgresql-8.4rc1.orig/src/interfaces/libpq/fe-secure.c
postgresql-8.4rc1/src/interfaces/libpq/fe-secure.c
--- postgresql-8.4rc1.orig/src/interfaces/libpq/fe-secure.c     2009-06-11
16:49:14.000000000 +0200
+++ postgresql-8.4rc1/src/interfaces/libpq/fe-secure.c  2009-06-22
10:56:38.000000000 +0200
@@ -689,6 +689,20 @@
                                ERR_pop_to_mark();
                                return 0;
                        }
+
+                       if (ENGINE_init(engine_ptr) == 0)
+                       {
+                               char     *err = SSLerrmessage();
+
+                               printfPQExpBuffer(&conn->errorMessage,
+
libpq_gettext("could not initialize SSL engine \"%s\": %s\n"),
+
engine_str, err);
+                               SSLerrfree(err);
+                               ENGINE_free(engine_ptr);
+                               free(engine_str);
+                               ERR_pop_to_mark();
+                               return 0;
+                       }

                        *pkey = ENGINE_load_private_key(engine_ptr,
engine_colon,

  NULL, NULL);
@@ -700,6 +714,8 @@

libpq_gettext("could not read private SSL key \"%s\" from engine \"%s\":
%s\n"),

engine_colon, engine_str, err);
                                SSLerrfree(err);
+                               ENGINE_finish(engine_ptr);
+                               ENGINE_free(engine_ptr);
                                free(engine_str);
                                ERR_pop_to_mark();
                                return 0;