Re: speeding up inserts by pipelining

On 04.03.2013 12:01, Todd Owen wrote:
> Is there any chance of psqlODBC one day supporting query pipelining inside
> the scope of calls to SQLExecute? This doesn't seem like too much effort to
> implement, although the tricky part is probably the semantics in the case
> where some rows succeed and some fail (constaint violations, etc). I'm
> tempted to give it a try myself, if I manage to get the build working under
> VS2010...

Yeah, seems reasonable. I think you'll want to send all the Bind and
Execute messages, followed by a single Sync message. That means that
all the queries will run as a single transaction, and if any of them
fail, all will be rolled back. That seems reasonable, although I don't
know what the ODBC spec has to say about that.

One tricky aspect is that if you just naively send all the queries
first, and then start to read the results, you can get a deadlock caused
by full network buffers. If the client doesn't read the responses before
it's sent all the queries, the client's receive buffer can fill up with
the responses from the backend, so that the backend blocks, trying to
send more responses. And at the same time, the client is blocked while
trying to send more queries, because the backend's receive buffer is
already full of incoming queries. I think you'll need to put the socket
into non-blocking mode, and use select()/poll().

- Heikki