Re: Ragged CSV import

Robert Haas wrote:
> I agree that ignoring extra columns is a bad idea, but I don't even
> like the idea of ignoring missing columns.  It doesn't seem like a
> good idea to take a spreadsheet and feed it into COPY without doing
> any validation anyway, and this is the kind of thing that is trivial
> to clean up with a thin layer of Perl or your scripting language of
> choice.
>
>   

If it's an optional feature then I don't see why there is a problem. 
What skin is it off anyone else's nose but those whose choose this 
behaviour?

I am perfectly familiar with Perl and so is the client that requested 
this feature. They are quite technically savvy. They are using a 
scripting solution now but they find it cumbersome.

As for general validation, the requestor's application in fact loads the 
spreadsheet into a temp table of text columns and then subjects it to a 
large variety of complex business rule checking before adding the data 
to the main tables. It is a whole lot faster and cleaner to do it that 
way than before the data is loaded at all. That's why they aren't 
concerned about missing columns.

cheers

andrew