On Tue, Mar 21, 2023 at 3:15 AM Greg Stark <stark@mit.edu> wrote:
> The next level of this would be something like notifying the committer
> with a list of patches in the CF that a commit broke. I don't
> immediately see how to integrate that with our workflow but I have
> seen something like this work well in a previous job. When committing
> code you often went and updated other unrelated projects to adapt to
> the new API (or could adjust the code you were committing to cause
> less breakage).
I've been hesitant to make it send email. The most obvious message to
send would be "hello, you posted a patch, but it fails on CI" to the
submitter. Cfbot has been running for about 5 years now, and I'd say
the rate of spurious/bogus failures has come down a lot over that time
as we've chased down the flappers in master, but it's still enough
that you would quickly become desensitised/annoyed by the emails, I
guess (one of the reasons I recently started keeping history is to be
able to do some analysis of that so we can direct attention to chasing
down the rest, or have some smart detection of known but not yet
resolved flappers, I dunno, something like that). Speaking as someone
who occasionally sends patches to other projects, it's confusing and
unsettling when you get automated emails from github PRs telling you
that this broke, that broke and the other broke, but only the project
insiders know which things are newsworthy and which things are "oh
yeah that test is a bit noisy, ignore that one".