Re: WIP: Collecting statistics on CSV file data

(2011/11/07 20:26), Shigeru Hanada wrote:
> (2011/10/20 18:56), Etsuro Fujita wrote:
>> I revised the patch according to Hanada-san's comments. Attached is the
>> updated version of the patch.
>>
>> Changes:
>>
>>     * pull up of logging "analyzing foo.bar"
>>     * new vac_update_relstats always called
>>     * tab-completion in psql
>>     * add "foreign tables are not analyzed automatically..." to 23.1.3
>> Updating Planner Statistics
>>     * some other modifications
>
> Submission review
> =================
>
> - Patch can be applied, and all regression tests passed. :)

Thank you for your testing.  I updated the patch according to your
comments.  Attached is the updated version of the patch.

> - file_fdw_do_analyze_rel is almost copy of do_analyze_rel.  IIUC,
> difference against do_analyze_rel are:
>      * don't logging analyze target
>      * don't switch userid to the owner of target table
>      * don't measure elapsed time for autoanalyze deamon
>      * don't handle index
>      * some comments are removed.
>      * sample rows are acquired by file_fdw's routine
>
> I don't see any problem here, but would you confirm that all of them are
> intentional?

Yes.  But in the updated version, I've refactored analyze.c a little bit
to allow FDW authors to simply call do_analyze_rel().

> - In your design, each FDW have to copy most of do_analyze_rel to their
> own source.  It means that FDW authors must know much details of ANALYZE
> to implement ANALYZE handler.  Actually, your patch exports some static
> functions from analyze.c.  Have you considered hooking
> acquire_sample_rows()?  Such handler should be more simple, and
> FDW-specific.  As you say, such design requires FDWs to skip some
> records, but it would be difficult for some FDW (e.g. twitter_fdw) which
> can't pick sample data up easily.  IMHO such problem *must* be solved by
> FDW itself.

The updated version enables FDW authors to just write their own
acquire_sample_rows().  On the other hand, by retaining to hook
AnalyzeForeignTable routine at analyze_rel(), higher level than
acquire_sample_rows() as before, it allows FDW authors to write
AnalyzeForeignTable routine for foreign tables on a remote server to ask
the server for its current stats instead, as pointed out earlier by Tom
Lane.

Best regards,
Etsuro Fujita