Re: COPY enhancements

Hi all,

I think there is a misunderstanding about what the current patch is about.
The patch includes 2 things:
- error logging in a table for bad tuples in a COPY operation (see 
http://wiki.postgresql.org/wiki/Error_logging_in_COPY for an example; 
the error message, command and so on are automatically logged)
- auto-partitioning in a hierarchy of child table if the COPY targets a 
parent table.

The patch does NOT include:
- logging errors into a file (a feature we can add later on (next commit 
fest?))

I can put the auto-partitioning in a separate patch if that helps but 
this patch relies on error logging to log possible errors in the routing 
of tuples.

I think that the way to move forward is first to have a basic error 
logging facility that logs into a database table (like the current patch 
does) and then add enhancements. I don't think we should try to do the 
file logging at the same time. If you prefer to postpone the 
auto-partitioning to the next commit fest, I can strip it from the 
current patch and re-submit it for the next fest (but it's just 2 
isolated methods really easy to review).

Emmanuel


Robert Haas wrote:
> On Wed, Oct 7, 2009 at 3:17 AM, Greg Smith <gsmith@gregsmith.com> wrote:
>   
>> I know this patch is attracting more reviewers lately, is anyone tracking
>> the general architecture of the code yet?  Emmanuel's work is tough to
>> review just because there's so many things mixed together, and there's other
>> inputs I think should be considered at the same time while we're all testing
>> in there (such as the COPY patch Andrew Dunstan put together).
>>     
>
> I hadn't realized this was an issue, but I think it's a good point: a
> patch that does one thing well is much more likely to get accepted
> than a patch that does two things well, let alone two things poorly.
> It's just much easier to review and verify.  Or maybe the name of the
> patch maybe should have tipped me off: "COPY enhancements" vs. "make
> COPY have feature X".
>
>   
>> What I'd like to see is for everything to get broken more into component
>> chunks that can get commited and provide something useful one at a time,
>> because I doubt taskmaster Robert is going to let this one linger around
>> with scope creep for too long before being pushed out to the next
>> CommitFest.
>>     
>
> I'm can't decide whether to feel good or bad about that appelation, so
> I'm going with both.  But in all seriousness if this patch needs
> substantial reworking (which it sounds like it does) we should
> postpone it to the next CF; we are quickly running out of days, and
> it's not fair to reviewers or committers to ask for new reviews of
> substantially revised code with a only a week to go.
>
> ...Robert
>   


-- 
Emmanuel Cecchet
Aster Data Systems
Web: http://www.asterdata.com