Re: libpq pipelining

On 12/04/2014 03:11 AM, Matt Newell wrote:
> The recent discussion about pipelining in the jodbc driver prompted me to look
> at what it would take for libpq.

Great!

> I have a proof of concept patch working.  The results are even more promising
> than I expected.
>
> While it's true that many applications and frameworks won't easily benefit, it
> amazes me that this hasn't been explored before.
>
> I developed a simple test application that creates a table with a single auto
> increment primary key column, then runs a 4 simple queries x times each:
> ...
>
> I plan to write documentation, add regression testing, and do general cleanup
> before asking for feedback on the patch itself.  Any suggestions about
> performance testing or api design would be nice.  I haven't played with
> changing the sync logic yet, but I'm guessing that an api to allow manual sync
> instead of a sync per PQsendQuery will be needed.  That could make things
> tricky though with multi-statement queries, because currently the only way to
> detect when results change from one query  to the next are a ReadyForQuery
> message.

A good API is crucial for this. It should make it easy to write an 
application that does pipelining, and to handle all the error conditions 
in a predictable way. I'd suggest that you write the documentation 
first, before writing any code, so that we can discuss the API. It 
doesn't have to be in SGML format yet, a plain-text description of the 
API will do.

- Heikki