Re: Non-blocking synchronization in libpq using pipeline mode

On Mon, 25 Mar 2024 15:47:00 +0100
Jan Behrens <jbe-mlist@magnetkern.de> wrote:

> On Tue, 19 Mar 2024 12:49:23 +0100
> Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> 
> > On 2024-Mar-06, PG Doc comments form wrote:
> > 
> > > [...]
> > 
> > [...]
> 
> [...]
> 
> [...] the documentation [of PQsendFlushRequest] specifically states
> that:
> 
> "Note that the request is not itself flushed to the server
> automatically; use PQflush if necessary."
> 
> However, that sentence only applies to "PQsendFlushRequest", not to
> "PQpipelineSync". On the contrary, section "34.5.1.1. Issuing Queries"
> reads:
> 
> "[Flushing to the server] occurs when PQpipelineSync is used to
> establish a synchronization point in the pipeline, or when PQflush is
> called."
> 
> As I understand it, this means that PQpipelineSync will flush the
> client-side's output, while PQsendFlushRequest won't flush the
> client-side. So both commands act differently with regard to flushing,
> is that right?
> 
> If that is the case, then the question is what happens if you call
> "PQpipelineSync" in non-blocking mode. That isn't clear to me (and I
> believe it is not explicitly made clear in the documentation).

I had a look into the source code of PostgreSQL 16.2 to answer that
question, and I found:

int
PQpipelineSync(PGconn *conn)
{
    /* ... */

    /*
     * Give the data a push (in pipeline mode, only if we're past the size
     * threshold).  In nonblock mode, don't complain if we're unable to send
     * it all; PQgetResult() will do any additional flushing needed.
     */
    if (pqPipelineFlush(conn) < 0)
        return 0;

    return 1;
}

Moreover:

/*
 * pqPipelineFlush
 *
 * In pipeline mode, data will be flushed only when the out buffer reaches the
 * threshold value.  In non-pipeline mode, it behaves as stock pqFlush.
 *
 * Returns 0 on success.
 */
static int
pqPipelineFlush(PGconn *conn)
{
    if ((conn->pipelineStatus != PQ_PIPELINE_ON) ||
        (conn->outCount >= OUTBUFFER_THRESHOLD))
        return pqFlush(conn);    
    return 0;
} 

It is not entirely clear to me what this means for the user of libpq.

It seems like invoking PQpipelineSync with pipeline mode switched on,
will *NOT* always flush the synchronization request out to the server.

While the comment on PQpipelineSync suggests that "PQgetResult() will
do any additional flushing needed", I believe that an event loop that
only waits for incoming data (e.g. POLLIN on the file descriptor) may
block indefinitely because a synchronization request hasn't been
flushed out yet.

If that is true, the documentation should clarify that PQflush might
need to be invoked manually by the user of libpq. Do I understand this
right?

> 
> In section "34.4. Asynchronous Command Processing", we find:
> 
> "In the nonblocking state, successful calls to PQsendQuery, PQputline,
> PQputnbytes, PQputCopyData, and PQendcopy will not block; their changes
> are stored in the local output buffer until they are flushed.
> Unsuccessful calls will return an error and must be retried."
> 
> There is no reference to "PQpipelineSync" there. Does that mean that
> "PQsetnonblocking" has no effect on "PQpipelineSync"'s blocking
> behavior? If so, is it correct that "PQpipelineSync" always blocks when
> the socket is not ready for writing (since, as stated in the
> documentation, it does flush)?

Following the source code, PQpipelineSync will not block (but also
sometimes not flush, which is not documented, except in the source
code).

> 
> I think section 34.5.1.4 should clarifiy how exactly PQpipelineSync
> acts in regard to flushing and blocking in that matter. It doesn't seem
> very clear at the moment.
> 
> > 
> > > 2. the synchronization or flush request need to be flushed manually with
> > > successive PQflush calls
> > 
> > Yes.
> 
> Regarding "PQpipelineSync", the documentation implies otherwise. See
> cite above:
> 
> "[Flushing to the server] occurs when PQpipelineSync is used to
> establish a synchronization point in the pipeline, or when PQflush is
> called."
> 
> That sentence doesn't make sense if PQpipelineSync wouldn't flush.

See above: PQpipelineSync seems to "sometimes" flush.

> [...]

Regards
Jan Behrens