Re: Custom Scan APIs (Re: Custom Plan node)

According to the suggestion, I dropped the enhancement of postgres_fdw on
top of custom-scan interface from my submission, and also move the supplemental
functions into the part-1 portion (implementation of custom-scan interface).
Even though the ctidscan module is under discussion, I still include this module
because of its usefulness as demonstration / example of custom-scan node.

Thanks,
--
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai@ak.jp.nec.com>


> -----Original Message-----
> From: Kaigai Kouhei(海外 浩平)
> Sent: Tuesday, March 04, 2014 8:26 AM
> To: 'Stephen Frost'; Kohei KaiGai
> Cc: Shigeru Hanada; Jim Mlodgenski; Robert Haas; Tom Lane; PgHacker; Peter
> Eisentraut
> Subject: RE: Custom Scan APIs (Re: [HACKERS] Custom Plan node)
>
> > > As I mentioned
> > > up-thread, I'd really like to see FDW join push-down, FDW aggregate
> > > push-down, parallel query execution, and parallel remote-FDW
> > > execution and I don't see this CustomScan approach as the right
> > > answer to any of those.
> >
> > In accordance with the above, what I'd like to see with this patch is
> > removal of the postgres_fdw changes and any changes which were for that
> support.
> >
> I don't argue this approach. It might be useful to *demonstrate* how custom-
> scan node works as replacement of join, however,
>
> > In addition, I'd like to understand why 'ctidscan' makes any sense to
> > have as an example of what to use this for- if that's valuable, why
> > wouldn't we simply implement that in core?  I do want an example in
> > contrib of how to properly use this capability, but I don't think that's
> it.
> >
> Do you think it makes sense if my submission was only interface portion
> without working example? The purpose of ctidscan module is, similar to
> postgres_fdw, to demonstrate the usage of custom-scan interface with enough
> small code scale. If tons of code example were attached, nobody will want
> to review the patch.
> The "cache_scan" module that I and Haribabu are discussing in another thread
> also might be a good demonstration for custom-scan interface, however, its
> code scale is a bit larger than ctidscan.
>
> > For one thing, an example where you could have this CustomScan node
> > calling other nodes underneath would be interesting.  I realize the
> > CTID scan can't do that directly but I would think your GPU-based
> > system could; after all, if you're running a join or an aggregate with
> > the GPU, the rows could come from nearly anything.  Have you
> > considered that, or is the expectation that users will just go off and
> > access the heap and/or whatever indexes directly, like ctidscan does?
> How would such a requirement be handled?
> >
> In case when custom-scan node has underlying nodes, it shall be invoked
> using ExecProcNode as built-in node doing, then it will be able to fetch
> tuples come from underlying nodes. Of course, custom-scan provider can
> perform the tuples come from somewhere as if it came from underlying relation.
> It is responsibility of extension module. In some cases, it shall be required
> to return junk system attribute, like ctid, for row-level locks or table
> updating.
> It is also responsibility of the extension module (or, should not add custom-
> path if this custom-scan provider cannot perform as required).
>
> Thanks,
> --
> NEC OSS Promotion Center / PG-Strom Project KaiGai Kohei
> <kaigai@ak.jp.nec.com>