On Thu, 2009-10-08 at 11:01 -0400, Tom Lane wrote:
> So as far as I can see, the only form of COPY error handling that
> wouldn't be a cruel joke is to run a separate subtransaction for each
> row, and roll back the subtransaction on error. Of course the
> problems
> with that are (a) speed, (b) the 2^32 limit on command counter IDs
> would mean a max of 2^32 rows per COPY, which is uncomfortably small
> these days. Previous discussions of the problem have mentioned trying
> to batch multiple rows per subtransaction to alleviate both issues.
> Not easy of course, but that's why it's not been done yet. With a
> patch like this you'd also have (c) how to avoid rolling back the
> insertions into the logging table.
(d) using too many xids will force the system to begin immediate
wraparound-avoidance vacuuming to freeze rows.
Dimitri's pgloader is looking even more attractive, not least because it
exists and it works. (And is the reason I personally stopped considering
the COPY-error-logging feature as important).
-- Simon Riggs www.2ndQuadrant.com