Re: Bulkloading using COPY - ignore duplicates?

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I think we can allow something like:
> COPY FROM '/tmp/x' WITH ERRORS 2

> Yes, I realize we need subtransactions or something, but we should add
> it to the TODO list if it is a valid request, right?

Well, I don't like that particular API in any case.  Why would I think
that 2 errors are okay and 3 are not, if I'm loading a
many-thousand-line COPY file?  Wouldn't it matter *what* the errors
are, at least as much as how many there are?  "Discard duplicate rows"
is one thing, but "ignore bogus data" (eg, unrecognizable timestamps)
is not the same animal at all.

As someone already remarked, the correct, useful form of such a feature
is to echo the rejected lines to some sort of output file that I can
look at afterwards.  How many errors there are is not the issue.
        regards, tom lane