Обсуждение: buildfarm does not test "make check"
So I added the brin isolation test back. Because it needs the pageinspect module, it can only work under "make check", not installcheck. The problem is that this means buildfarm will not run it, because it only runs installcheck :-( I realized that the important modules that run under "make check" already have custom buildfarm modules, namely pg_upgrade and test_decoding. I wonder if it would make more sense to have this be customizable from the buildfarm server side, without having to resort to writing a buildfarm module for each new thing. I envision a file hosted in the buildfarm server, perhaps a JSON blob, that lists modules that need "make check" treatment. Within the JSON object for each such module there could live the details such as what files need to be saved. That way, we could add more things to test without requiring the buildfarm client to be upgraded each time. This approach seems to have worked very well to customize the branches that each animal builds, which is why I suggest that it be used here too. -- Álvaro Herrera 33.5S 70.5W "The Gord often wonders why people threaten never to come back after they've been told never to return" (www.actsofgord.com)
On 08/13/2015 02:11 PM, Alvaro Herrera wrote: > So I added the brin isolation test back. Because it needs the > pageinspect module, it can only work under "make check", not > installcheck. The problem is that this means buildfarm will not run it, > because it only runs installcheck :-( > > I realized that the important modules that run under "make check" > already have custom buildfarm modules, namely pg_upgrade and > test_decoding. I wonder if it would make more sense to have this be > customizable from the buildfarm server side, without having to resort to > writing a buildfarm module for each new thing. I envision a file hosted > in the buildfarm server, perhaps a JSON blob, that lists modules that > need "make check" treatment. Within the JSON object for each such > module there could live the details such as what files need to be > saved. That way, we could add more things to test without requiring the > buildfarm client to be upgraded each time. > > This approach seems to have worked very well to customize the branches > that each animal builds, which is why I suggest that it be used here > too. > The buildfarm server doesn't control anything the clients do except it provides them with a list of branches we are interested in. Even that is invisible to the main program, and only used by the wrapper run_branches.pl. The main program can run entirely offline and I'm going to resist moves to make it otherwise. (Yes it needs access to git, but even that can be finessed.) If you want some way of specifying this, put it in the source, e.g. invent a directory src/test/buildfarm_support and put it there. That way committers have automatic access to it, which they certainly don't to the buildfarm server. I'd rather just make it a small piece of perl instead of JSON,though. You're also going to have to handle the msvc side of things. That won't be trivial. See discussion elsewhere today about how we've got that wrong recently. cheers andrew
Andrew Dunstan wrote: > The buildfarm server doesn't control anything the clients do except it > provides them with a list of branches we are interested in. Even that is > invisible to the main program, and only used by the wrapper run_branches.pl. > The main program can run entirely offline and I'm going to resist moves to > make it otherwise. (Yes it needs access to git, but even that can be > finessed.) Understood. > If you want some way of specifying this, put it in the source, e.g. invent a > directory src/test/buildfarm_support and put it there. That way committers > have automatic access to it, which they certainly don't to the buildfarm > server. I'd rather just make it a small piece of perl instead of > JSON,though. Makes plenty of sense, thanks. > You're also going to have to handle the msvc side of things. That won't be > trivial. See discussion elsewhere today about how we've got that wrong > recently. Oh my. The pg_upgrade code in src/tools/msvc/vcregress.pl looks rather unhealthy, and I don't see anything there to handle test_decoding, so I assume that's done differently somehow. (For pg_upgrade surely we should not be using a shell script ...) I don't have time to go through this right now, but it's on my list of things to think about. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 08/13/2015 02:52 PM, Alvaro Herrera wrote: >> You're also going to have to handle the msvc side of things. That won't be >> trivial. See discussion elsewhere today about how we've got that wrong >> recently. > Oh my. The pg_upgrade code in src/tools/msvc/vcregress.pl looks rather > unhealthy, and I don't see anything there to handle test_decoding, so I > assume that's done differently somehow. (For pg_upgrade surely we should > not be using a shell script ...) > > I don't have time to go through this right now, but it's on my list of > things to think about. > Don't worry about the existing cases. Just create something that handles new test cases in a generic way. cheers andrew
On Thu, Aug 13, 2015 at 2:11 PM, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote: > So I added the brin isolation test back. Because it needs the > pageinspect module, it can only work under "make check", not > installcheck. The problem is that this means buildfarm will not run it, > because it only runs installcheck :-( Aren't "make installcheck" and "make check" supposed to test the same set of things? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 08/14/2015 09:27 AM, Robert Haas wrote: > On Thu, Aug 13, 2015 at 2:11 PM, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote: >> So I added the brin isolation test back. Because it needs the >> pageinspect module, it can only work under "make check", not >> installcheck. The problem is that this means buildfarm will not run it, >> because it only runs installcheck :-( > Aren't "make installcheck" and "make check" supposed to test the same > set of things? > There are some cases that don't provide installcheck targets. cheers andrew