Re: [BUGS] BUG #14505: explain verbose for postgresql_fdw

Jeff Janes <jeff.janes@gmail.com> writes:
> On Wed, Jan 18, 2017 at 5:27 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> jeff.janes@gmail.com writes:
>>> The "Output:" line of explain verbose lists columns of foreign table that
>>> are not actually output, or at least ought not be output.

>> That's an optimization to avoid an unnecessary projection step.  It has
>> nothing to do with what's retrieved from the remote, and for that matter
>> it's not specific to foreign tables.

> Is all of that data actually being transferred from the foreign to the
> local?

Well, that would depend on the particular FDW, but postgres_fdw doesn't
transfer unneeded columns, as you can tell by looking at the "Remote SQL"
command.

The key point here is that the output of the ForeignScan plan node has to
match the declared rowtype of the foreign table.  So what postgres_fdw is
going to do, after retrieving the subset of columns that it detected were
used in the query, is construct a dummy tuple that has all the columns but
fills in nulls for the unused ones.  The point of what the planner is
doing with the node output is that we don't need to run ExecProject just
to throw away those nulls again; we can pass the dummy tuple up to the
join node as-is.  (In some cases, like if we had to pass the tuples
through a sort, it'd be worth projecting to remove unused columns ... but
the planner knows that, and generates a minimal output tlist in those
cases.  This isn't one of those cases.)

> If so, I don't think the optimization is a good choice in this case.

I think you're mistaken.

The alternative that we could have used is to require FDWs to emit tuples
containing only used columns --- but that would *require* all FDWs to
contain the logic about identifying used columns, and for at least some
kinds of FDW it would add processing not subtract it.  It'd also
complicate matters elsewhere, because ForeignScan would act quite a bit
different from scans on regular tables.  In those scans, the raw tuples
are necessarily going to contain all columns.

> Do you know where can I find the relevant source code which decides
> on this?

See use_physical_tlist in createplan.c.

            regards, tom lane


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs