Andrew Dunstan <andrew@dunslane.net> writes:
> On 08/09/2015 08:58 PM, Michael Paquier wrote:
>> Sure. Attached is what I have in mind. Contrary to your version we
>> keep around temp paths should a run succeed after one that has failed
>> when running make check multiple times in a row. Perhaps it does not
>> matter much in practice as log files get removed at each new run but I
>> think that this allows a finer control in case of failure. Feel free
>> to have a look.
> Actually, I don't think this is a very good idea. You could end up with
> a whole series of opaquely named directories from a series of failing
> runs. If you want to keep the directory after a failure, and want to do
> another run, then rename the directory. That's what you have to do with
> the main regression tests, too. My version also has the benefit of
> changing exactly 3 lines in the source :-)
FWIW, I think we should keep the behavior of the TAP tests as close as
possible to the established behavior of the main regression tests.
It's certainly possible that there's room for improvement in that
benchmark behavior. But let's first converge the test behaviors,
and then have a discussion about whether we want changes, and if
so change all the tests at the same time.
regards, tom lane