Re: [HACKERS] Pipelining executions to postgresql server

On 2014-11-02 12:11:49 +0000, Mikko Tiihonen wrote:
> > From: Andres Freund <andres@2ndquadrant.com>
> > On 2014-11-01 14:04:05 +0000, Mikko Tiihonen wrote:
> > > I created a proof of concecpt patch for postgresql JDBC driver that
> > > allows the caller to do pipelining of requests within a
> > > transaction. The pipelining here means same as for HTTP: the client
> > > can send the next execution already before waiting for the response of
> > > the previous request to be fully processed.
> >
> > Slightly confused here. To my knowledge the jdbc driver already employs
> > some pipelining? There's some conditions where it's disabled (IIRC
> > RETURNING for DML is one of them), but otherwise it's available.
> >
> > I'm very far from a pgjdbc expert, but that's what I gathered from the
> > code when investigating issues a while back and from my colleague Craig.
>
> Most DB interfaces make the server operations look synchronous.

You IIRC can use jdbc's batch interface.

> I should have searched earlier a better reference to libpg. I am planning on adding support for something similar to
> http://www.postgresql.org/docs/9.3/static/libpq-async.html
> More specifically operations like:
>   int PQsendQuery(PGconn *conn, const char *command);
>   PGresult *PQgetResult(PGconn *conn);

That doesn't really support pipelining though - you can still only send
one query. It's nonblocking, but that's a different thing.

> In libpg terms: Is it safe to do multiple PQsendQuery operations
> before invoking PQgetResult as many times?

The network protocol allows for pipelining, yes. But libpq unfortunately
doesn't.

You should read the protocol definition.

> It should be, if the server startd to process (read in) the next query
> only after it has sent the previous response out.

There's complexities around error handling and such making it slightly
more complex.

Greetings,

Andres Freund

--
 Andres Freund                       http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services