Re: There's something rotten in the PG infrastructure

<p dir="ltr"><br /> On May 17, 2015 08:30, "Stephen Frost" <<a
href="mailto:sfrost@snowman.net">sfrost@snowman.net</a>>wrote:<br /> ><br /> > * Tom Lane (<a
href="mailto:tgl@sss.pgh.pa.us">tgl@sss.pgh.pa.us</a>)wrote:<br /> > > I wrote:<br /> > > > * However,
neitherof my buildfarm critters noticed anything had happened<br /> > > > for about an hour and a half. 
prairiedoglit off with a run around 0:55<br /> > > > Sunday, dromedary around 1:10.  (Both of them check every
20minutes, not<br /> > > > on the same schedule.)  dromedary's run finished around 1:25.<br /> > ><br />
>> > * As of right now, 1:56 AM, the buildfarm status page is not showing an<br /> > > > update from
dromedary,or indeed any other machine for nearly four hours.<br /> > > > There should have been a lot of
updatesby now.<br /> > ><br /> > > Hmm ... actually, it looks like both prairiedog and dromedary are<br />
>> rebuilding repeatedly, which is odd because certainly nothing is happening<br /> > > on gitmaster.  But
therun start times quoted above probably only<br /> > > represent the runs that were active when I got annoyed
enoughto go look<br /> > > at what was happening.<br /> ><br /> > I've been poking around and I don't see
anyobvious issues on either<br /> > gitmaster or git.p.o.  The cronjobs appear to be running.  Not saying<br /> >
thereisn't an issue but it's at least less than obvious if there is.<br /> > Further, these systems are still on
wheezy(many other boxes have been<br /> > upgraded to jessie at this point), so nothing much has changed on them<br
/>> in quite some time.  It's possible that there is an issue due to jessie<br /> > being on the host server, but
hopefullynot.<br /> ><p dir="ltr">We've seen some emails from  our buildfarm client as well, where the buildfarm
complainsabout database errors. I'm guessing some intermittent errors there, since the site itself works. That's also
thereason some snapshots on the ftp site are currently slightly out of date - they are uploaded at the end of
successfulbuildfarm runs.. <br /><p dir="ltr">So Afaict, the problem seems to be in the buildfarm, not in the pg
infrastructure.<p dir="ltr">The quick check for the replication between the git servers is to just look at the gitweb
interface.That one reads directly from the same got repository as the buildfarn clients would. <p dir="ltr">/Magnus