Re: libpq COPY handling

On Thu, Apr 25, 2013 at 1:56 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> However, the documentation in libpq.sgml is a bit bogus too, because it
> counsels trying the PQputCopyEnd() call again, which will not work
> (since we already changed the asyncStatus).  We could make that say "a
> zero result is informational, you might want to try PQflush() later".
> The trouble with this, though, is that any existing callers that were
> coded to the old spec would now be broken.
>
> It might be better to consider that the code is correct and fix the
> documentation.  I notice that the other places in fe-exec.c that call
> pqFlush() generally say "In nonblock mode, don't complain if we're unable
> to send it all", which is pretty much the spirit of what this is doing
> though it lacks that comment.

Changing the meaning of a 0 return code seems like a bad idea.
However, not ever returning 0 isn't great either: someone could be
forgiven for writing code that calls PQputCopyData/End() until they
get a 0 result, then waits for the socket to become write-OK before
continuing.  They might be surprised to find that this results in
libpq's internal buffer expanding until malloc fails.

I'm gathering that what I should probably do is something like this.
Send some data using PQputCopyData() or an end-of-copy using
PQputCopyEnd().  Then, call PQflush().  If the latter returns 1, then
wait for the socket to become write-OK and try again.  Once it returns
0, send the next hunk of copy-data.  This is a bit of an exasperating
interface because when the socket buffer fills up, PQputCopyData()
will do a write() but we won't know what happened; we'll have to call
PQflush() and try a second write of the same data without an
intervening wait to find out that the buffer is full.  If this worked
as currently documented, that wouldn't be necessary.  But it seems
tolerable.

>> Also, I noticed that there are a few places in fe-protocol3.c that
>> seem not to know about COPY-BOTH mode.  I'm not sure that any of these
>> are actually bugs right now given the current very limited use of
>> COPY-BOTH mode, but I'm wondering whether it wouldn't be better to
>> minimize the chance of future surprises.  Patch attached.
>
> +1 for these changes, anyway.

OK, committed, thanks.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company