Re: Quality and Performance

Andrew Dunstan <andrew@dunslane.net> writes:
> Josh Berkus wrote:
>> ... DW operations aren't 
>> really testable without 18 hours to generate data ... but we could test a 
>> lot of things.

> Performance isn't just about humungous DW apps.

Indeed.  I think the real take-home lesson from these past few days'
discussion is that *any* particular view of performance is going to
miss things that don't affect that case, but do affect somebody else.

What I find most worrisome about the notion of setting up a
performance-farm is that it will encourage us to optimize with blinkers
on --- that is, that we will consider only the specific cases measured
by whatever tests are included in the farm, and will happily pessimize
other cases.  We can ameliorate that a bit if we can get a sufficiently
wide variety of test cases, but it will always be a concern.  And
dogmatic positions like "only cases involving terabytes of data are
worth testing" are definitely not going to help.
        regards, tom lane