Re: COPY enhancements

On Wed, 7 Oct 2009, Emmanuel Cecchet wrote:

> I think there is a misunderstanding about what the current patch is 
> about...the patch does NOT include logging errors into a file (a feature 
> we can add later on (next commit fest?))

I understand that (as one of the few people who has read the patch it 
seems), but as you can might guess from the feedback here that's not the 
way I expect your patch is actually going to be handled.  Something that 
logs only to a table without the interface for the file output at least 
specified isn't the sort of thing that this group tends to go along with 
very well.  Since as it is we haven't even nailed down how the file output 
is even going to look or work yet, the table logging isn't very likely to 
go in unless it's at least clear that the future plans there will cleanly 
stack on top.  That's what people are alluding to mentioning the whole 
roadmap concept for all this.

I get the impression you were hoping to get another chunk of this commited 
on this round.  I would guess that realistically it's going to take at 
least a few more weeks for the rest of this to get nailed down, and that a 
really solid feature should be ready by the next CF instead.  You should 
actually be proud of the progress you spurred on the COPY options stuff 
that's in there now, that idea has been floating around for a while now 
but until your patch there wasn't any momentum behind doing something 
about it.  The problem you're facing now is that so many people have been 
thinking about this particular area for so long without any code to talk 
about that you're now stuck with all that pent up uncoded design to clear.

> I can put the auto-partitioning in a separate patch if that helps but this 
> patch relies on error logging to log possible errors in the routing of 
> tuples.

Understood.  I know you've gotten some other coding feedback here, and 
you've been very good about taking that and producing updated versions 
which is appreciated.  I would recommend that the next time you resubmit 
this, if you could break the logging and partitioning patches into a pair 
that depend on one another, that would make life easier for everyone else 
(albeit probably a harder for you).  At that point we should be set to 
have some others take over some of that tweaking, with myself, Selena, and 
Jeff all interested and capable of helping out here.

> If you prefer to postpone the auto-partitioning to the next commit fest, 
> I can strip it from the current patch and re-submit it for the next fest 
> (but it's just 2 isolated methods really easy to review).

That was one of points I was trying to make--that would be an easier to 
review by itself, but as it is people interested in it can't consider it 
without also staring at the logging stuff.  And people who are focusing on 
the logging bits find it distracting, so nobody is really happy with the 
current combined patch.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD