Re: More efficient build farm animal wakeup?

On 2022-11-20 Su 17:32, Thomas Munro wrote:
> On Sun, Nov 20, 2022 at 2:44 AM Andrew Dunstan <andrew@dunslane.net> wrote:
>> It might not suit your use case, but one of the things I do to reduce
>> fetch load is to run a local mirror which runs
>>
>>    git fetch -q --prune
>>
>> every 5 minutes. It also runs a git daemon, and several of my animals
>> point at that.
> Thanks.  I understand now that my configuration without a local mirror
> is super inefficient (it spends the first ~25s of each minute running
> git commands).  Still, even though that can be improved by me setting
> up more stuff, I'd like something event-driven rather than short
> polling-based for lower latency.
>
>> If there's a better git API I'll be happy to try to use it.
> Cool.  Seems like we just have to invent something first...
>
> FWIW I'm also trying to chase the short polling out of cfbot.  It
> regularly harasses the git servers at one end (could be fixed with
> this approach), and wastes a percentage of our allotted CPU slots on
> the other end by scheduling periodically (could be fixed with webhooks
> from Cirrus).



I think I have solved most of the actual issues without getting too complex.

Here's how:

The buildfarm server now creates a companion to branches_of_interest.txt
called branches_of_interest.json which looks like this:

[
   {
      "REL_11_STABLE" : "140c803723"
   },
   {
      "REL_12_STABLE" : "4cbcb7ed85"
   },
   {
      "REL_13_STABLE" : "c13667b518"
   },
   {
      "REL_14_STABLE" : "5cda142bb9"
   },
   {
      "REL_15_STABLE" : "ff9d27ee2b"
   },
   {
      "HEAD" : "51b5834cd5"
   }
]