Обсуждение: faster testing with symlink installs
I'm proposing a way to make test runs a bit faster. A fair amount of time is taken by creating the test installation. Not huge compared to the whole test run, but still long enough to get boring. The idea (that I stole from somewhere else) is that we make the installation as symlinks pointing back to the source tree. So after the first run, the installation is automatically up to date after each build. Then we can skip the whole temp install run if we find that we already have one (unless you change the set of installed files in a major way, which is very rate). This also helps if you use make installcheck: You just need to rebuild and restart the server, skipping the make install run. So that's what the patch does: make install INSTALL_SYMLINKS=1 installs files by making relative symlinks. make check TEMP_INSTALL_SYMLINKS=1 creates the temporary installation using symlinks and skips the install step if a tmp_install already exists. It requires the "ln" program from GNU coreutils. So it would be optional. Except ... this doesn't actually work. find_my_exec() resolves symlinks to find the actual program installation, and so for example the installed initdb will look for postgres in src/bin/initdb/. I would like to revert this behavior, because it seems to do more harm than good. The original commit message indicates that this would allow symlinking a program to somewhere outside of the installation tree and still allow it to work and find its friends. But that could just as well be done with a shell script. Reverting this behavior would also allow Homebrew-like installations to work better. The actual installation is in a versioned directory like /usr/local/Cellar/postgresql/10.1/... but symlinked to /usr/local/opt/postgresql/. But because of the symlink resolution, calling /usr/local/opt/postgresql/bin/pg_config returns paths under Cellar, which then causes breakage of software built against that path on postgresql upgrades the change the version number. Thoughts? -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Вложения
On Wed, Feb 28, 2018 at 9:34 PM, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote: > Except ... this doesn't actually work. find_my_exec() resolves symlinks > to find the actual program installation, and so for example the > installed initdb will look for postgres in src/bin/initdb/. I would > like to revert this behavior, because it seems to do more harm than > good. The original commit message indicates that this would allow > symlinking a program to somewhere outside of the installation tree and > still allow it to work and find its friends. But that could just as > well be done with a shell script. > > Reverting this behavior would also allow Homebrew-like installations to > work better. The actual installation is in a versioned directory like > /usr/local/Cellar/postgresql/10.1/... but symlinked to > /usr/local/opt/postgresql/. But because of the symlink resolution, > calling /usr/local/opt/postgresql/bin/pg_config returns paths under > Cellar, which then causes breakage of software built against that path > on postgresql upgrades the change the version number. My initial gut feeling is that changing this would hurt more people than it would help. It seems entirely reasonable to install PostgreSQL in, say, /opt/local/postgresql, and then symlink psql and pg_ctl into /opt/local/bin or /usr/bin or wherever you like to find your binaries. I don't think we want to break that. It's true that it will work if you symlink everything, as in your Homebrew example, but you might not. TBH I find that Homebrew example pretty odd. I would understand installing each major release in a version directory, but putting every point release in a different versioned directory seems like a bad plan. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes: > On Wed, Feb 28, 2018 at 9:34 PM, Peter Eisentraut > <peter.eisentraut@2ndquadrant.com> wrote: >> Except ... this doesn't actually work. find_my_exec() resolves symlinks >> to find the actual program installation, and so for example the >> installed initdb will look for postgres in src/bin/initdb/. I would >> like to revert this behavior, because it seems to do more harm than >> good. The original commit message indicates that this would allow >> symlinking a program to somewhere outside of the installation tree and >> still allow it to work and find its friends. But that could just as >> well be done with a shell script. > My initial gut feeling is that changing this would hurt more people > than it would help. I agree. My recollection is that we expended substantial sweat to make that type of setup work, and I do not think it was for idle reasons. The fact that the behavior is very old doesn't mean it was a bad choice. (Also, the fact that the commit message didn't explain the reasoning in detail is not much of an argument; we didn't go in for that back then.) regards, tom lane
On Wed, Mar 07, 2018 at 05:06:59PM -0500, Robert Haas wrote: > TBH I find that Homebrew example pretty odd. I would understand > installing each major release in a version directory, but putting > every point release in a different versioned directory seems like a > bad plan. That's a project policy on their side. The version of all components is kept around this way to give users the easier possibility to link or use back a previous version if a component update does wrong. There are advantages to such models as well for projects which care a lot about some precise compatibility that some upstream maintainer overlooked, while not having to patch an existing instance to filter out only a set of components. -- Michael
Вложения
On Wed, Mar 07, 2018 at 06:17:15PM -0500, Tom Lane wrote: > I agree. My recollection is that we expended substantial sweat to make > that type of setup work, and I do not think it was for idle reasons. > The fact that the behavior is very old doesn't mean it was a bad choice. > (Also, the fact that the commit message didn't explain the reasoning in > detail is not much of an argument; we didn't go in for that back > then.) Yes, the current behavior is here for some time: commit: 336969e490d71c316a42fabeccda87f798e562dd author: Tom Lane <tgl@sss.pgh.pa.us> date: Sat, 6 Nov 2004 23:06:29 +0000 Add code to find_my_exec() to resolve a symbolic link down to the actual executable location. This allows people to continue to use setups where, eg, postmaster is symlinked from a convenient place. Per gripe from Josh Berkus. And the original thread is here: https://www.postgresql.org/message-id/4973.1099605411%40sss.pgh.pa.us The last complain on the matter I can find actually involves the same people as this thread :) https://www.postgresql.org/message-id/54DE457F.2090206%40gmx.net So the patch should be marked as rejected or at least returned with feedback? -- Michael
Вложения
Michael Paquier <michael@paquier.xyz> writes: > The last complain on the matter I can find actually involves the same > people as this thread :) > https://www.postgresql.org/message-id/54DE457F.2090206%40gmx.net > So the patch should be marked as rejected or at least returned with > feedback? IMO, changing this behavior is more likely to draw complaints than praise. regards, tom lane
On Wed, Mar 21, 2018 at 11:43 PM, Michael Paquier <michael@paquier.xyz> wrote: > On Wed, Mar 07, 2018 at 05:06:59PM -0500, Robert Haas wrote: >> TBH I find that Homebrew example pretty odd. I would understand >> installing each major release in a version directory, but putting >> every point release in a different versioned directory seems like a >> bad plan. > > That's a project policy on their side. The version of all components is > kept around this way to give users the easier possibility to link or use > back a previous version if a component update does wrong. There are > advantages to such models as well for projects which care a lot about > some precise compatibility that some upstream maintainer overlooked, > while not having to patch an existing instance to filter out only a set > of components. Well, under that definition, the behavior Peter is complaining about upthread is a feature, not a bug. If you want a separate install of PostgreSQL for every minor release, you should have a separate version of each other piece of software you build against it. Or so it seems to me, anyway. I suppose we could provide a build-time option to change this behavior. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Thu, Mar 22, 2018 at 11:07:48AM -0400, Robert Haas wrote: > I suppose we could provide a build-time option to change this behavior. it seems to me that it is going to be hard to reach a consensus by the end of the commit fest, so I am marking this patch as returned with feedback. Not sure if that's worth it either.. -- Michael