Re: Parallel Append can break run-time partition pruning

On Thu, 23 Apr 2020 at 11:39, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> David Rowley <dgrowleyml@gmail.com> writes:
> > On Thu, 23 Apr 2020 at 11:11, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> Well, anytime the parallel startup cost is significant, for starters.
> >> But maybe we account for that at some other point, like when building
> >> the Gather?
>
> > Yeah. There's no mention of parallel_setup_cost or parallel_tuple_cost
> > in any of the Append costing code. Those are only applied when we cost
> > Gather / GatherMerge At the point Amit and I are talking about, we're
> > only comparing two Append paths. No Gather/GatherMerge in sight yet,
> > so any additional costs from those is not applicable.
>
> Right, so really the costs of partial and non-partial paths are not
> commensurable, and comparing them directly is just misleading.
> I trust we're not throwing away non-partial paths on that basis?

There is a case in both master and in the patch where we compare the
cost of the cheapest path in partial_pathlist. However, in this case,
the pathlist path will be used in an Append or Parallel Append with a
Gather below it, so those parallel_(setup|tuple)_costs will be applied
regardless.  The non-parallel Append in this case still requires a
Gather since it still is using multiple workers to execute the
subpaths. e.g the plan I posted in [1].

The code comparing the path costs is:

else if (nppath == NULL ||
(cheapest_partial_path != NULL &&
  cheapest_partial_path->total_cost < nppath->total_cost))

[1] https://www.postgresql.org/message-id/CAApHDvqcOD3ObPgPAeU+3qyFL_wzE5kmczw70qMAh7qJ-3wuzw@mail.gmail.com