Re: Ragged CSV import

Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
> >> I have received a requirement for the ability to import ragged CSV 
> >> files, i.e. files that contain variable numbers of columns per row.
> 
> BTW, one other thought about this: I think the historical reason for
> COPY being strict about the number of incoming columns was that it
> provided a useful cross-check that the parsing hadn't gone off into
> the weeds.  We have certainly seen enough examples where the reported
> manifestation of, say, an escaping mistake was that COPY saw the row
> as having too many or too few columns.  So being permissive about it
> would lose some error detection capability.  I am not clear about
> whether CSV format is sufficiently more robust than the traditional
> COPY format to render this an acceptable loss.  Comments?

I think accepting less columns and filling with nulls should be
protected enough for this not to be a problem; if the parser goes nuts,
it will die eventually.  Silently dropping excessive trailing columns
does not seem acceptable though; you could lose entire rows and not
notice.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.