Re: Commitfest overflow

On Thu, Aug 05, 2021 at 03:06:54PM +0200, Tomas Vondra wrote:
> On 8/5/21 8:39 AM, Andrey Borodin wrote:
> >>...
> >>
> >>Early commitfests recognized a rule that patch authors owed one review per
> >>patch registered in the commitfest.  If authors were holding to that, then
> >>both submissions and reviews would slow during vacations, but the neglected
> >>fraction of the commitfest would be about the same.  I think it would help to
> >>track each author's balance (reviews_done - reviews_owed).
> >
> >+1 for tracking this.
> 
> Yeah, I agree we should be stricter about this rule, but I'm somewhat
> skeptical about tracking it in the CF app - judging patch and review
> complexity seems quite subjective, etc.

The CF app presently lacks the data to track this, but with relevant feature
additions, I suspect it would be the best place to manage tracking.  Something
like this: when a CF app user changes a patch status from Needs Review or
Ready for Committer to anything else, invite that user to name a recipient of
review credit.

> >BTW when review is done? When first revision is published? Or when patch is committed\rollbacked?
> >When the review is owed? At the moment when patch is submitted? Or when it is committed?

Those are important questions.  Here are answers that feel reasonable to me,
but they may need refinement.  Anyone can increase their reviews_done by
sending a review that causes a patch to leave Needs Review status in the CF
app for the first time in a given commitfest.  Trivial defect reports,
e.g. that the patch doesn't compile, are not reviews; the person setting
Waiting on Author shall not assign review credit.  Committers can increase
their reviews_done by sending a review that causes a patch to leave Ready for
Committer status for the first time in a given commitfest; however, nobody
receives two credits for the same patch, in the same commitfest.  Whenever a
CF entry is increasing someone's reviews_done, it also increases reviews_owed
for the entry's author.  What do you think?

(Consequences: each CF entry can increment reviews_done of up to two users per
commitfest, but it increments reviews_owed just once.  If the patch bounces
between Needs Review and Waiting on Author several times in a single CF, that
counts as a total of one review.)

> I think the rule is roughly that when you submit a patch to a CF, you're
> expected to review a patch of comparable complexity in the same CF. It's not
> tied to whether the patch is committed, etc.

Yep.  Would we get reasonable incentives without tracking "comparable
complexity" explicitly?  That bit is harder to judge.