Re: Introducing coarse grain parallelism by postgres_fdw.

Hello,

> In order to minimize the impact, what can be done is to execute
> fetch_more_data() in asynchronous mode every time, when there only few rows
> left to be consumed. So in current code below
> 1019     /*
> 1020      * Get some more tuples, if we've run out.
> 1021      */
> 1022     if (fsstate->next_tuple >= fsstate->num_tuples)
> 1023     {
> 1024         /* No point in another fetch if we already detected EOF,
> though. */
> 1025         if (!fsstate->eof_reached)
> 1026             fetch_more_data(node, false);
> 1027         /* If we didn't get any tuples, must be end of data. */
> 1028         if (fsstate->next_tuple >= fsstate->num_tuples)
> 1029             return ExecClearTuple(slot);
> 1030     }
> 
> replace line 1022 with if (fsstate->next_tuple >= fsstate->num_tuples)
> with if (fsstate->next_tuple >= fsstate->num_tuples -
> SOME_BUFFER_NUMBER_ROWS)
> Other possibility is to call PQsendQuery(conn, sql), after line 2100 and if
> eof_reached is false.
> 
> 2096             /* Must be EOF if we didn't get as many tuples as we asked
> for. */
> 2097             fsstate->eof_reached = (numrows < fetch_size);
> 2098
> 2099             PQclear(res);
> 2100             res = NULL;

I see, I'll consider it. If late (lazy) error detection is
allowed, single row mode seems available, too.

regards,

-- 
Kyotaro Horiguchi
NTT Open Source Software Center