Robert Haas <robertmhaas@gmail.com> writes:
> On Wed, Aug 12, 2015 at 2:04 PM, Peter Geoghegan <pg@heroku.com> wrote:
>> This resistance to adding tests seems quite short sighted to me,
>> especially when the concern is about queries that will each typically
>> take less than 1ms to execute. Like Noah, I think that it would be
>> very helpful to simply be more inclusive of additional tests that
>> don't increase test coverage by as much as each query in a minimal
>> subset. I am not at all convinced by arguments about the cost of
>> maintaining tests when a simple behavioral change occurs.
> I've removed tests from patches that in my opinion were unlikely to
> fail either (a) for any reason or (b) for any reason other than an
> intentional change, and I think that's a reasonable thing to do.
FWIW, I've objected in the past to tests that would significantly
increase the runtime of "make check", unless I thought they were
especially valuable (which enumerating every minor behavior of a
feature patch generally isn't IMO). I still think that that's an
important consideration: every second you add to "make check" is
multiplied many times over when you consider how many developers
run that how many times a day.
We've talked about having some sort of second rank of tests that
people wouldn't necessarily run before committing, and that would
be allowed to eat more time than the core regression tests would.
I think that might be a valuable direction to pursue if people start
submitting very bulky tests.
regards, tom lane