Re: textarray option for file FDW

On 01/15/2011 12:29 PM, Andrew Dunstan wrote:
>
> I've been waiting for the latest FDW patches as patiently as I can,
> and I've been reviewing them this morning, in particular the file_fdw
> patch and how it interacts with the newly exposed COPY API. Overall it
> seems to be a whole lot cleaner, and the wholesale duplication of the
> copy code is gone, so it's much nicer and cleaner. So now I'd like to
> add a new option to it: "textarray". This option would require that
> the foreign table have exactly one field, of type text[], and would
> compose all the field strings read from the file for each record into
> the array (however many there are). This would require a few changes
> to contrib/file_fdw/file_fdw.c and a few changes to
> src/backend/commands/copy.c, which I can probably have done in fairly
> short order, Deo Volente. This will allow something like:
>
>    CREATE FOREIGN TABLE arr_text (
>         t text[]
>    )  SERVER file_server
>    OPTIONS (format 'csv', filename '/path/to/ragged.csv', textarray
>    'true');
>    SELECT t[3]::int as a, t[1]::timestamptz as b, t[99] as not_there
>    FROM arr_text;
>
>

A WIP patch is attached. It's against Shigeru Hanada's latest FDW
patches. It's surprisingly tiny. Right now it probably leaks memory like
a sieve, and that's the next thing I'm going to chase down.

cheers

andrew