Обсуждение: thread-safe libpq and DBD::Pg

When experimenting with    threaded perl
I noticed I had to lock access to the database
a one user at the time in order get reliable (no SEGV).

Unfortunely it is 3 layer between my server and the wire,
DBI, DBD::Pg och libpq.
Thinking I would start from the bottom:
Has anyone any experience of the thread-safeness of libpq?

    have a good weekend,

        Göran

Goran Thyni <goran@bildbasen.se> writes:
> Has anyone any experience of the thread-safeness of libpq?

PQconnectdb() is not thread-safe, because it scribbles temporary
data in the static PQconninfoOptions array.  You can get around
that easily enough by not using PQconnectdb (or PQconndefaults)
--- use PQsetdbLogin() instead.  In any case the problem exists
only if different threads try to open database connections
at the same time.

As far as I can think at the moment, ordinary operations while
the database connection is open should be thread safe.  That's
assuming that your libc (esp. malloc/free) is thread safe of course.
It also assumes that only one thread is using any one PGconn
object (at a time, anyway).  Multiple threads using the same
PGconn to issue concurrent requests definitely will not work.

There is also some thread risk during connection shutdown because
libpq momentarily commandeers the SIGPIPE handler.  I am thinking
it'd be nice to get rid of that code if possible --- see discussion
from a couple weeks ago on the hackers (or was it interfaces?) list.


I had thought a little bit about ripping out PQconnectdb's use of
changeable fields in PQconninfoOptions, but it looks like this
could break applications that access PQconninfoOptions.
Does anyone have any feelings about that, pro or con?

            regards, tom lane

I wrote:
> Goran Thyni <goran@bildbasen.se> writes:
>> Has anyone any experience of the thread-safeness of libpq?

> There is also some thread risk during connection shutdown because
> libpq momentarily commandeers the SIGPIPE handler.  I am thinking
> it'd be nice to get rid of that code if possible --- see discussion
> from a couple weeks ago on the hackers (or was it interfaces?) list.

I have removed that SIGPIPE hacking from closePGconn() in a patch
just submitted to pgsql-patches; that's one less thread safety violation
in libpq.  However, whilst I was poking about in the libpq innards I
noticed a couple of other ones :-(.

PQprint() also manipulates the SIGPIPE handler in order to prevent
application termination if the pager subprocess it starts up decides
to quit early.  This is a good thing, I think, but it does create
a problem if the app is multi-threaded and other threads would prefer
a different SIGPIPE setting.  (I suppose if signal handlers are
thread-local in your environment then it's a non-issue anyway.)

I also noticed that PQoidStatus() returns a pointer to a static
char array, which is a classic thread-safety booboo.  I am thinking
that the cleanest solution is for it to copy the OID number into
conn->errorMessage and return a pointer to that.  This would mean
that the value would not be good after the next operation on the
PGconn object, but I wouldn't imagine that any apps hold onto the
pointer very long anyway --- they probably always feed it to atoi
immediately.  (Actually, why the dickens doesn't this routine
return an Oid value directly?  It could return InvalidOid in
case of trouble, just like PQftype does... but I suppose we don't
want to break application code just to make the API a tad cleaner.)

            regards, tom lane

> I also noticed that PQoidStatus() returns a pointer to a static
> char array, which is a classic thread-safety booboo.  I am thinking
> that the cleanest solution is for it to copy the OID number into
> conn->errorMessage and return a pointer to that.  This would mean
> that the value would not be good after the next operation on the
> PGconn object, but I wouldn't imagine that any apps hold onto the
> pointer very long anyway --- they probably always feed it to atoi
> immediately.  (Actually, why the dickens doesn't this routine
> return an Oid value directly?  It could return InvalidOid in
> case of trouble, just like PQftype does... but I suppose we don't
> want to break application code just to make the API a tad cleaner.)

Yes, it is very silly that PQoidStatus() returns a char*.  I believe it
was only done that way because it was pulling the oid out of the
cmdStatus field.  We really should return an int.  Perhaps we don't
because we would be hardcoding the oid type on the client side.  By
default, we return all types as char *.  I don't think we do this
anywhere else in the protocol.

This is a complete guess.  I believe it was added by Vadim in 6.2 or
6.3.

Perhaps we could double-use the cmdStatus field, perhaps when they ask
for the oid, we put a null after the oid value, and return a pointer to
the inside of the cmdStatus field.  If the call for PQcmdStatus(), we
remove the null and return the string.  Strange, I admit.

--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Tom Lane wrote:
>
> Goran Thyni <goran@bildbasen.se> writes:
> > Has anyone any experience of the thread-safeness of libpq?
>
> PQconnectdb() is not thread-safe, because it scribbles temporary
> data in the static PQconninfoOptions array.  You can get around
> that easily enough by not using PQconnectdb (or PQconndefaults)
> --- use PQsetdbLogin() instead.  In any case the problem exists
> only if different threads try to open database connections
> at the same time.
> [...}

Tom,
thanks for the reply.
DBD::Pg dos indeed use PQconnectdb(),
but at least in my light test I get SEGV in $sth->fetch.
The problems are probably somewhere in DBDI or DBD::Pg or
somewhere in the interaction of the parts.

I am looking into how much work a "pure Perl" interface
should amount to. Using native perl IO and pgsql-fe/be-protocol
should make "pgsql-clienting" even more portable over platforms.

    regards,
        Göran

Goran Thyni <goran@bildbasen.se> writes:
> DBD::Pg dos indeed use PQconnectdb(),
> but at least in my light test I get SEGV in $sth->fetch.
> The problems are probably somewhere in DBDI or DBD::Pg or
> somewhere in the interaction of the parts.

FWIW, I finally got around to installing Perl5 (5.004 pl4 to
be specific), and so far the perl5 interface supplied with pgsql
seems to work fine.  I only just now realized that what you
are talking about is something different ... the code included
in the distribution calls itself Pg not DBD::Pg.  What versions
of everything are you using?

> I am looking into how much work a "pure Perl" interface
> should amount to. Using native perl IO and pgsql-fe/be-protocol
> should make "pgsql-clienting" even more portable over platforms.

Of course, it will also mean that you have to maintain that code
every time we change the fe/be protocol ... there is a change
upcoming for 6.4.

            regards, tom lane

Bruce Momjian <maillist@candle.pha.pa.us> writes:
> Yes, it is very silly that PQoidStatus() returns a char*.  I believe it
> was only done that way because it was pulling the oid out of the
> cmdStatus field.  We really should return an int.  Perhaps we don't
> because we would be hardcoding the oid type on the client side.

Type Oid is already visible to libpq clients: it's returned by PQftype()
as the type code for attributes.  I think encouraging clients to store
oids as type Oid rather than some-int-type-chosen-at-random would be a
good thing, not a bad thing.

However, improving the cleanliness of the API might not be worth the
cost of breaking application code, even though this routine is probably
not very widely used.  Does anyone else have an opinion pro or con?

> Perhaps we could double-use the cmdStatus field, perhaps when they ask
> for the oid, we put a null after the oid value, and return a pointer to
> the inside of the cmdStatus field.  If the call for PQcmdStatus(), we
> remove the null and return the string.  Strange, I admit.

I had a variant of that idea: what's stored in cmdStatus looks like
        INSERT oidvalue count
but there's a fair amount of room left over (cmdStatus is 40 bytes ...
and we could make it bigger if we wanted).  We can make PQoidStatus
copy the oidvalue into the free space:
        cmdStatus = "INSERT oidvalue count\0oidvalue\0"
and return a pointer to here ...............^
That way PQcmdStatus and PQoidStatus don't interfere with each other.
The result of PQoidStatus will still get zapped by the next interaction
with the backend, but at least it's not as transient as it would be in
the errorMessage field.

But, still, converting to an integer return value would be a *lot*
cleaner.

If we were going to do that, I'd be strongly inclined to change
PQcmdTuples to return an int (or more likely a long) as well.
I can't envision any situation where people aren't writing
atoi(PQcmdTuples(conn)) so why not save them the trouble...

            regards, tom lane

Tom Lane wrote:
>
> Goran Thyni <goran@bildbasen.se> writes:
> > DBD::Pg dos indeed use PQconnectdb(),
> > but at least in my light test I get SEGV in $sth->fetch.
> > The problems are probably somewhere in DBDI or DBD::Pg or
> > somewhere in the interaction of the parts.
>
> FWIW, I finally got around to installing Perl5 (5.004 pl4 to
> be specific), and so far the perl5 interface supplied with pgsql
> seems to work fine.  I only just now realized that what you
> are talking about is something different ... the code included
> in the distribution calls itself Pg not DBD::Pg.  What versions
> of everything are you using?

I am using perl 5.005_01 with threads support and DBI/DBD::Pg.
DBI/DBD is the "standard" RDBMS-interface for perl available
from CPAN.

> > I am looking into how much work a "pure Perl" interface
> > should amount to. Using native perl IO and pgsql-fe/be-protocol
> > should make "pgsql-clienting" even more portable over platforms.
>
> Of course, it will also mean that you have to maintain that code
> every time we change the fe/be protocol ... there is a change
> upcoming for 6.4.

I have done some experimenting:
The easy parts are done (if not cleanly), connect/exec/close.
The hairy parts is to extract row descriptors and tuples
from select/fetch statements.

I am working against the current CVS tree, are there more
protocol changes in the pipeline?

    regards,
        Göran.

> Bruce Momjian <maillist@candle.pha.pa.us> writes:
> > Yes, it is very silly that PQoidStatus() returns a char*.  I believe it
> > was only done that way because it was pulling the oid out of the
> > cmdStatus field.  We really should return an int.  Perhaps we don't
> > because we would be hardcoding the oid type on the client side.
>
> Type Oid is already visible to libpq clients: it's returned by PQftype()
> as the type code for attributes.  I think encouraging clients to store
> oids as type Oid rather than some-int-type-chosen-at-random would be a
> good thing, not a bad thing.
>
> However, improving the cleanliness of the API might not be worth the
> cost of breaking application code, even though this routine is probably
> not very widely used.  Does anyone else have an opinion pro or con?
>
> > Perhaps we could double-use the cmdStatus field, perhaps when they ask
> > for the oid, we put a null after the oid value, and return a pointer to
> > the inside of the cmdStatus field.  If the call for PQcmdStatus(), we
> > remove the null and return the string.  Strange, I admit.
>
> I had a variant of that idea: what's stored in cmdStatus looks like
>         INSERT oidvalue count
> but there's a fair amount of room left over (cmdStatus is 40 bytes ...
> and we could make it bigger if we wanted).  We can make PQoidStatus
> copy the oidvalue into the free space:
>         cmdStatus = "INSERT oidvalue count\0oidvalue\0"
> and return a pointer to here ...............^
> That way PQcmdStatus and PQoidStatus don't interfere with each other.
> The result of PQoidStatus will still get zapped by the next interaction
> with the backend, but at least it's not as transient as it would be in
> the errorMessage field.
>
> But, still, converting to an integer return value would be a *lot*
> cleaner.
>
> If we were going to do that, I'd be strongly inclined to change
> PQcmdTuples to return an int (or more likely a long) as well.
> I can't envision any situation where people aren't writing
> atoi(PQcmdTuples(conn)) so why not save them the trouble...

Another idea I had was to a char* field to the result structure.
Initialize it to NULL, then when they want oidStatus, we malloc() some
memory, assign it to our char*, and copy the string into there, and
return that to the user.  If we get called later, we can re-use the
memory, and free() it when we free the PQresult.

It was added in 6.2.

If we change it, we will have to require people using those calls to
modify their apps, rather than just relink with the new libpq.  However,
old binaries will run fine because they would be linked to the old
libpq.

php3 uses PQoidStatus, so I can imagine great problems if we change the
libpq interface.  I am afraid we are stuck with char*.


--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Goran Thyni <goran@bildbasen.se> writes:
> I am using perl 5.005_01 with threads support and DBI/DBD::Pg.
> DBI/DBD is the "standard" RDBMS-interface for perl available
> from CPAN.

Hmm.  Well, I'm afraid I have too much on my plate already to go chasing
an incompatibility with someone else's software.  If you or someone else
can track down what the problem is, I'll gladly help fix libpq (if
that's what needs fixed) ... but I can't spare the time right now to
chase a problem that doesn't show up in my setup.

>> Of course, it will also mean that you have to maintain that code
>> every time we change the fe/be protocol ... there is a change
>> upcoming for 6.4.

> I am working against the current CVS tree, are there more
> protocol changes in the pipeline?

You're in good shape then; we agreed several weeks ago that there would
be no more FE-to-BE link protocol changes for 6.4.  I was afraid you
might be working from 6.3.2 sources, that's all.

            regards, tom lane

Bruce Momjian <maillist@candle.pha.pa.us> writes:
> php3 uses PQoidStatus, so I can imagine great problems if we change the
> libpq interface.  I am afraid we are stuck with char*.

Yeah, I can't really see breaking applications just for a marginal
improvement in cleanliness.

There is a possible compromise however: we could leave PQcmdTuples and
PQoidStatus defined as-is (but do something to get rid of PQoidStatus'
use of a static return area), and add two more functions that have more
reasonable return conventions.  The documentation could describe the
older functions as deprecated.

Perhaps the int-returning forms could be named "PQCmdTuples" and
"PQOidStatus" (note difference in capitalization) ... unless someone
has a better idea.

Does anyone think this is worth the trouble, or shall we leave bad
enough alone?

I do intend to get rid of the static return area for PQoidStatus in any
case.  I'd also like to fix the problem with PQconninfoOptions not being
treated as a constant (specifically, the "val" fields are being used as
working storage).  Is anyone aware of any applications that would be
broken by removing "val" from the PQconninfoOption struct?

            regards, tom lane

> Bruce Momjian <maillist@candle.pha.pa.us> writes:
> > php3 uses PQoidStatus, so I can imagine great problems if we change the
> > libpq interface.  I am afraid we are stuck with char*.
>
> Yeah, I can't really see breaking applications just for a marginal
> improvement in cleanliness.
>
> There is a possible compromise however: we could leave PQcmdTuples and
> PQoidStatus defined as-is (but do something to get rid of PQoidStatus'
> use of a static return area), and add two more functions that have more
> reasonable return conventions.  The documentation could describe the
> older functions as deprecated.
>
> Perhaps the int-returning forms could be named "PQCmdTuples" and
> "PQOidStatus" (note difference in capitalization) ... unless someone
> has a better idea.
>
> Does anyone think this is worth the trouble, or shall we leave bad
> enough alone?

Perhaps we can leave the change for a time when we want to change the
libpq interface in a more significant way.  Having two functions just
seems like a waste for such a rarely-called fuction.


>
> I do intend to get rid of the static return area for PQoidStatus in any
> case.  I'd also like to fix the problem with PQconninfoOptions not being
> treated as a constant (specifically, the "val" fields are being used as
> working storage).  Is anyone aware of any applications that would be
> broken by removing "val" from the PQconninfoOption struct?
>
>             regards, tom lane
>


--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Tom,

Tom Lane wrote:
>
> Goran Thyni <goran@bildbasen.se> writes:
> > I am using perl 5.005_01 with threads support and DBI/DBD::Pg.
> > DBI/DBD is the "standard" RDBMS-interface for perl available
> > from CPAN.
>
> Hmm.  Well, I'm afraid I have too much on my plate already to go chasing
> an incompatibility with someone else's software.  If you or someone else
> can track down what the problem is, I'll gladly help fix libpq (if
> that's what needs fixed) ... but I can't spare the time right now to
> chase a problem that doesn't show up in my setup.

Since there are 3 pcs of s/w involved (all 3 with many lines of C-code)
and non of them are really well tested in a threaded environment
it is a real kludge.
I understand anyone who do not want to stick his head into that,
I pulled my own back and started experimenting with my "pure perl"
interface instead.
Your work cleaning up libpq will be appriciated by lots of programmers
out there.

As a status report on my module, it now fetch records OK too.
On the ToDo-list:
    * INET-socket support (now it is Unix domain sockets only)
    * explicit cursor support
    * BLOB-support
    * clean-up of error handling/reporting
    * test suite
    * DBI-DBD layer on top of standard module
I will release a snap-shot as soon as it is cleaned up a bit and
somewhat more tested.

    regards,
--
---------------------------------------------
Göran Thyni, sysadm, JMS Bildbasen, Kiruna

Goran Thyni wrote:
>
> Tom,
>
> Tom Lane wrote:
> >
> > Goran Thyni <goran@bildbasen.se> writes:
> > > I am using perl 5.005_01 with threads support and DBI/DBD::Pg.
> > > DBI/DBD is the "standard" RDBMS-interface for perl available
> > > from CPAN.
> >
> > Hmm.  Well, I'm afraid I have too much on my plate already to go chasing
> > an incompatibility with someone else's software.  If you or someone else
> > can track down what the problem is, I'll gladly help fix libpq (if
> > that's what needs fixed) ... but I can't spare the time right now to
> > chase a problem that doesn't show up in my setup.
>
> Since there are 3 pcs of s/w involved (all 3 with many lines of C-code)
> and non of them are really well tested in a threaded environment
> it is a real kludge.
> I understand anyone who do not want to stick his head into that,
> I pulled my own back and started experimenting with my "pure perl"
> interface instead.
> Your work cleaning up libpq will be appriciated by lots of programmers
> out there.
>
> As a status report on my module, it now fetch records OK too.
> On the ToDo-list:
>         * INET-socket support (now it is Unix domain sockets only)
>         * explicit cursor support
>         * BLOB-support
>         * clean-up of error handling/reporting
>         * test suite
>         * DBI-DBD layer on top of standard module
> I will release a snap-shot as soon as it is cleaned up a bit and
> somewhat more tested.
>
>         regards,
> --
> ---------------------------------------------
> Göran Thyni, sysadm, JMS Bildbasen, Kiruna


I suggest to spend the effort on making libpq thread safe.
Otherwise we end up with two DBD modules for postgres.
If you prepare a module, which does not depend on libpq,
you will have much more effort on keeping it up to date.
The main goal of an interface like libpq is to avoid such
dependencies.

Edmund
--
Edmund Mergl          mailto:E.Mergl@bawue.de
Im Haldenhau 9        http://www.bawue.de/~mergl
70565 Stuttgart       fon: +49 711 747503
Germany

At 19:44 +0300 on 10/8/98, Edmund Mergl wrote:


> I suggest to spend the effort on making libpq thread safe.
> Otherwise we end up with two DBD modules for postgres.
> If you prepare a module, which does not depend on libpq,
> you will have much more effort on keeping it up to date.
> The main goal of an interface like libpq is to avoid such
> dependencies.

On the other hand, making a DBD module which is not dependent on libpq will
make it possible to use the perl interface on platforms other than UNIX
(e.g. Win, Mac).

So, if potential users have any say on the matter, I'd ask that interfaces
for various cross-platform languages be kept as platform-independent (that
is, as free of native code) as possible.

Herouth

--
Herouth Maoz, Internet developer.
Open University of Israel - Telem project
http://telem.openu.ac.il/~herutma

Edmund,

I certainly respect your opinions and take note. We have used your
DBD::Pg-module
in several projects here with great success.

This is the conclusions I have reach:

* A "pure perl"-interface is useful for
    - systems without decent C-compiler
    - systems not supported by libpq
          (there is already a pure TCL interface, isn't it?)
    - in other special applications where "pure perl" is preferable.
* I will release my module, probably called PgSQL since,
    - I need a "pure perl" interface for a couple of project
    - The basic module is mostly done already
    - Why not share?
* I will *NOT* make a DBD interface, because
    - I don't need it
    - the C-code in DBI make the arguments for "pure perl" mute
    - It would confuse J. Random Looser and friends :-)

As I see it there is a niche for all three perl interfaces:
* DBI/DBD::Pg:
    excellent portability across RDBMSs and nice standard API
    makes it the natural choise for most project.
* Pg:
    the choice where low-level access and maximum performance is needed
* PgSQL:
    "pure perl" with OO-design makes it easily customizable for
    PostgreSQL-only applications.

Maybe I will look at thread-safety for DBI/DBD::Pg/libpq but
debugging C-based perl-modules is pretty time consuming
(I wrote the DBD::Informix4 module a couple of yeasr ago).
Combine that with threads/debugging headache and isn't
so funny anymore. :-)

    regards,
--
---------------------------------------------
Göran Thyni, sysadm, JMS Bildbasen, Kiruna


---------------------------------------------------------------

Edmund Mergl wrote:
> I suggest to spend the effort on making libpq thread safe.
> Otherwise we end up with two DBD modules for postgres.
> If you prepare a module, which does not depend on libpq,
> you will have much more effort on keeping it up to date.
> The main goal of an interface like libpq is to avoid such
> dependencies.

Goran Thyni <goran@bildbasen.se> writes:
> * A "pure perl"-interface is useful for
>     - systems without decent C-compiler
>     - systems not supported by libpq
>           (there is already a pure TCL interface, isn't it?)

I agree that a "pure Perl" or "pure Tcl" front end would be more
easily portable than what we have.  It'd probably be slower, but
maybe not so much slower as to be a problem.

The present libpgtcl is NOT pure Tcl --- it is a C-coded Tcl extension,
besides which it is layered on top of libpq.

I think I remember someone speculating about writing a pure Tcl
front end by programming directly to the socket-level FE/BE interface,
but I haven't heard about any results.

            regards, tom lane

Tom Lane wrote:
>
> Goran Thyni <goran@bildbasen.se> writes:
> > * A "pure perl"-interface is useful for
> >       - systems without decent C-compiler
> >       - systems not supported by libpq
> >           (there is already a pure TCL interface, isn't it?)
>
> I agree that a "pure Perl" or "pure Tcl" front end would be more
> easily portable than what we have.  It'd probably be slower, but
> maybe not so much slower as to be a problem.

I once did an incomplete "pure Python" PgSQL (v6.2) interface, which ran
on both Linux and Win32. It was not that much slower.

> The present libpgtcl is NOT pure Tcl --- it is a C-coded Tcl extension,
> besides which it is layered on top of libpq.
>
> I think I remember someone speculating about writing a pure Tcl
> front end by programming directly to the socket-level FE/BE interface,
> but I haven't heard about any results.

I've got the impression that there is currently a pure Tcl interface
as well that some use with pgAccess on win32.

Hannu

> > I think I remember someone speculating about writing a pure Tcl
> > front end by programming directly to the socket-level FE/BE interface,
> > but I haven't heard about any results.
>
> I've got the impression that there is currently a pure Tcl interface
> as well that some use with pgAccess on win32.
>
> Hannu

I am not sure what "pure" means , but there is a tcl script equivalent
of libpgtcl.so which is called pgsql.tcl. It is available from the same
site that you get pgaccess

I needed it to enable running pgaccess on a MacOS client. Works fine.

--
-----------------------------------------------------------------
|John Dzilvelis                                                 |
-----------------------------------------------------------------

Goran Thyni wrote:
>
> Edmund,
>
> I certainly respect your opinions and take note. We have used your
> DBD::Pg-module
> in several projects here with great success.
>
> This is the conclusions I have reach:
>
> * A "pure perl"-interface is useful for
>         - systems without decent C-compiler
>         - systems not supported by libpq
>           (there is already a pure TCL interface, isn't it?)
>         - in other special applications where "pure perl" is preferable.
> * I will release my module, probably called PgSQL since,
>         - I need a "pure perl" interface for a couple of project
>         - The basic module is mostly done already
>         - Why not share?
> * I will *NOT* make a DBD interface, because
>         - I don't need it
>         - the C-code in DBI make the arguments for "pure perl" mute
>         - It would confuse J. Random Looser and friends :-)
>

ok, I agree. I thought you are just working on a second DBD module
with the additional feature to be thread safe. A pure perl module
is certainly interesting for many people.

Edmund
--
Edmund Mergl          mailto:E.Mergl@bawue.de
Im Haldenhau 9        http://www.bawue.de/~mergl
70565 Stuttgart       fon: +49 711 747503
Germany

>>>>> "TL" == Tom Lane <tgl@sss.pgh.pa.us> writes:

  TL> I think I remember someone speculating about writing a pure Tcl
  TL> front end by programming directly to the socket-level FE/BE interface,
  TL> but I haven't heard about any results.

Can the FE/BE interface be expected to remain stable over time? As
much so as libpq?

--
Eric Marsden
emarsden @ mail.dotcom.fr
It's elephants all the way down

>>>>> "HK" == Hannu Krosing <hannu@trust.ee> writes:
>>>>> "TL" == Tom Lane <tgl@sss.pgh.pa.us> writes:

  TL> The present libpgtcl is NOT pure Tcl --- it is a C-coded Tcl
  TL> extension, besides which it is layered on top of libpq.

This is my impression too: in postgresql-6.3.2/src/interfaces/libpgtcl/
there are C files which reference functions like PQconnectdb. However,
the documentation for libpgtcl states (this is from the top of
postgresql-6.3.2/doc/src/sgml/libpgtcl.sgml)

   pgtcl is a tcl package for front-end programs to interface with
   <ProductName>Postgres</ProductName> backends. pgtcl does not use
   the libpq library but communicates to the backend directly via the
   frontend-backend protocol. Thus, it is more efficient than previous
   postgres->tcl bindings which are layered on top of libpq. In
   addition, pgtcl can handle multiple backend connections from a
   single frontend application.

The documentation looks wrong, or perhaps it is talking about a
development version of libpgtcl? Should I report a documentation bug?


  HK> I've got the impression that there is currently a pure Tcl interface
  HK> as well that some use with pgAccess on win32.

This is ftp://ftp.flex.ro/pub/pgaccess/pgsql.tcl (Tcl interface at the
socket level).

--
Eric Marsden
emarsden @ mail.dotcom.fr
It's elephants all the way down

Eric Marsden <emarsden@mail.dotcom.fr> writes:
>>>>>> "TL" == Tom Lane <tgl@sss.pgh.pa.us> writes:
TL> I think I remember someone speculating about writing a pure Tcl
TL> front end by programming directly to the socket-level FE/BE interface,
TL> but I haven't heard about any results.

> Can the FE/BE interface be expected to remain stable over time? As
> much so as libpq?

Well, we changed it significantly for 6.4, so I wouldn't care to promise
it'll never change again.  BUT: thanks to the fact that the client sends
a protocol version number at the start of a session, the server can and
does support old-protocol clients.  So piecemeal updates of client
libraries seem feasible.  I'd be a lot more worried about proliferating
clients that know the wire-level protocol if this were not so.

It occurs to me that there is a potential versioning bug in the way
libpq deals with this issue: it sends out the version number with

    sp.protoVersion = (ProtocolVersion) htonl(PG_PROTOCOL_LATEST);

But PG_PROTOCOL_LATEST is defined (in some backend include file or
other) as the latest protocol version supported by the *server*.
Thus, merely recompiling libpq against source files of an updated
server will cause it to claim that it speaks the latest protocol
version, whether or not you've actually touched libpq.

This is bogus.

I think libpq needs to have its *own* constant defining what protocol
version number it speaks, which you'd have to update manually whenever
you change libpq to handle a different version.

Of course, the same goes for any other client library that deals with
the wire protocol directly.

            regards, tom lane

Eric Marsden <emarsden@mail.dotcom.fr> writes:
>>>>>> "TL" == Tom Lane <tgl@sss.pgh.pa.us> writes:
TL> The present libpgtcl is NOT pure Tcl --- it is a C-coded Tcl
TL> extension, besides which it is layered on top of libpq.

> the documentation for libpgtcl states (this is from the top of
> postgresql-6.3.2/doc/src/sgml/libpgtcl.sgml)

>    pgtcl is a tcl package for front-end programs to interface with
>    <ProductName>Postgres</ProductName> backends. pgtcl does not use
>    the libpq library but communicates to the backend directly via the
>    frontend-backend protocol. Thus, it is more efficient than previous
>    postgres->tcl bindings which are layered on top of libpq. In
>    addition, pgtcl can handle multiple backend connections from a
>    single frontend application.

Yeah, that paragraph is all wet --- perhaps there once was a libpgtcl
written that way, but the current distribution certainly is not it.
I took that para out of the CVS sources some time ago.

            regards, tom lane