Re: Buildfarm feature request: some way to track/classify failures

Andrew Dunstan wrote:
> Martijn van Oosterhout wrote:
>> On Tue, Mar 20, 2007 at 02:57:13AM -0400, Tom Lane wrote:
>>  
>>> Maybe we should think about filtering the noise.  Like, say, discarding
>>> every report from mongoose that involves an icc core dump ...
>>> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=mongoose&dt=2007-03-20%2006:30:01 
>>>
>>>     
>>
>> Maybe a simple compromise would be being able to setup a set of regexes
>> that search the output and set a flag it that string is found. If you
>> find the string, it gets marked with a flag, which means that when you
>> look at mongoose, any failures that don't have the flag become easier
>> to spot.
>>
>> It also means that once you've found a common failure, you can create
>> the regex and then any other failures with the same string get tagged
>> also, making unexplained ones easier to spot.
>>
>>
>>   
> 
> You need to show first that this is an adequate tagging mechanism, both 
> in tagging things adequately and in not picking up false positives, 
> which would make things worse, not better. And even then you need 
> someone to do the analysis to create the regex.
> 
> The buildfarm works because it leverages our strength, namely automating 
> things. But all the tagging suggestions I've seen will involve regular, 
> repetitive and possibly boring work, precisely the thing we are not good 
> at as a group.

this is probably true - however as a buildfarm admin I occasionally 
wished i had a way to invalidate reports generated from my boxes to 
prevent someone wasting time to investigate them (like errors caused by 
system upgrades,configuration problems or other local issues).

But I agree that it might be difficult to make that "manual tagging" 
process scalable and reliable enough so that it really is an improvment 
over what we have now.

Stefan