Обсуждение: Makefile.global will override configure parameters if "pgsql" and "postgres" appear anywhere in the source path name
Hi All, Recently I was debugging an issue in which builds of postgresql succeeded on a developer's workstation, but failed in our company's build system. After some investigation, I found that the value of pkglibdir (as used by src/Makefile.global and various other places) was different even though the ./configure invocations were identical. Eventually, I found that Makefile.global was the culprit, with some sections like: pkglibdir = $(libdir) ifeq "$(findstring pgsql, $(pkglibdir))" "" ifeq "$(findstring postgres, $(pkglibdir))" "" override pkglibdir := $(pkglibdir)/postgresql endif endif On the developer's workstation, the sources had been unpacked in a directory, e.g. (the structure is a quirk convention of an internal tool) $HOME/src/postgresql13-13.7/source On the production build system, the path used was: /build/source The --prefix used is a subdirectory of the source directory (in this context we're only building the source to get the static libpq and headers, which we use to build another piece of software). On the developer's workstation, the string "postgres" is part of the path, and thus pkglibdir does not get "postgresql" appended via an override (nor any of the other variables). But on the build system, both tests pass, and the override with appended "postgresql" do occur. I'm assuming that the intention of findstring postgres was to tidy up values like pkglibdir when libdir is e.g. /usr/lib/postgres/, which makes sense, but not when "postgres" appears in libdir for some other reason. We're able to patch around this, but I thought it might be worth reporting. I considered devising a patch but haven't yet thought of a good strategy, and I suspect it might be trick to avoid breaking other users who may (even unwittingly) rely on this quirk. I would be delighted to accept any suggestions however. Thanks! .andy
On 2022-Jul-15, Andy Bailey wrote: > After some investigation, I found that the value of pkglibdir (as used > by src/Makefile.global and various other places) was different even > though the ./configure invocations were identical. Eventually, I found > that Makefile.global was the culprit, with some sections like: > > pkglibdir = $(libdir) > ifeq "$(findstring pgsql, $(pkglibdir))" "" > ifeq "$(findstring postgres, $(pkglibdir))" "" > override pkglibdir := $(pkglibdir)/postgresql > endif > endif Right. See commit 984b0b4df83f and its discussion at [1]. The working theory (as I remember it) is that if the installation is already postgres-specific (because the user has made it so by having "pgsql" or "postgres" in the argument to --prefix), then we don't need to add another level of postgres-specificity to it; but otherwise we add "/postgresql" to the path so that the resulting files don't end up mixed with files installed by unrelated packages. [1] https://www.postgresql.org/message-id/flat/Pine.LNX.4.21.0010042208350.934-100000%40peter.localdomain -- Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
Hi, On 2022-07-15 20:38:17 +0200, Alvaro Herrera wrote: > On 2022-Jul-15, Andy Bailey wrote: > > > After some investigation, I found that the value of pkglibdir (as used > > by src/Makefile.global and various other places) was different even > > though the ./configure invocations were identical. Eventually, I found > > that Makefile.global was the culprit, with some sections like: > > > > pkglibdir = $(libdir) > > ifeq "$(findstring pgsql, $(pkglibdir))" "" > > ifeq "$(findstring postgres, $(pkglibdir))" "" > > override pkglibdir := $(pkglibdir)/postgresql > > endif > > endif > > Right. See commit 984b0b4df83f and its discussion at [1]. The working > theory (as I remember it) is that if the installation is already > postgres-specific (because the user has made it so by having "pgsql" or > "postgres" in the argument to --prefix), then we don't need to add > another level of postgres-specificity to it; but otherwise we add > "/postgresql" to the path so that the resulting files don't end up mixed > with files installed by unrelated packages. I think the current approach isn't great. E.g., putting extension .so's into the same directories as "normal shared libraries", which typically is either in the library search path via system configuration, or in the library search path due to rpath, strikes me as quite wrongheaded. They're not libraries that should be linked to, so they shouldn't be in the library dir directly. Nor do we really allow the user to opt in/out of us changing the user-specified install locations. Greetings, Andres Freund
Andres Freund <andres@anarazel.de> writes:
> On 2022-07-15 20:38:17 +0200, Alvaro Herrera wrote:
>> Right. See commit 984b0b4df83f and its discussion at [1]. The working
>> theory (as I remember it) is that if the installation is already
>> postgres-specific (because the user has made it so by having "pgsql" or
>> "postgres" in the argument to --prefix), then we don't need to add
>> another level of postgres-specificity to it; but otherwise we add
>> "/postgresql" to the path so that the resulting files don't end up mixed
>> with files installed by unrelated packages.
> I think the current approach isn't great.
> E.g., putting extension .so's into the same directories as "normal shared
> libraries", which typically is either in the library search path via system
> configuration, or in the library search path due to rpath, strikes me as quite
> wrongheaded.
Well, we don't, with a normal not-PG-specific install path.
What I get with --prefix=/home/tgl/testversion is
$ ls ~/testversion/lib
libecpg.a libecpg_compat.so@ libpgtypes.so@ libpq.so.5@
libecpg.so@ libecpg_compat.so.3@ libpgtypes.so.3@ libpq.so.5.5*
libecpg.so.6@ libecpg_compat.so.3.4* libpgtypes.so.3.3* postgresql/
libecpg.so.6.4* libpgport.a libpq.a
libecpg_compat.a libpgtypes.a libpq.so@
$ ls ~/testversion/lib/postgresql
ascii_and_mic.so* pgxs/ utf8_and_gb18030.so*
cyrillic_and_mic.so* plperl.so* utf8_and_gbk.so*
dict_snowball.so* plpgsql.so* utf8_and_iso8859.so*
euc2004_sjis2004.so* utf8_and_ascii.so* utf8_and_iso8859_1.so*
euc_cn_and_mic.so* utf8_and_big5.so* utf8_and_johab.so*
euc_jp_and_sjis.so* utf8_and_cyrillic.so* utf8_and_sjis.so*
euc_kr_and_mic.so* utf8_and_euc2004.so* utf8_and_sjis2004.so*
euc_tw_and_big5.so* utf8_and_euc_cn.so* utf8_and_uhc.so*
latin2_and_win1250.so* utf8_and_euc_jp.so* utf8_and_win.so*
latin_and_mic.so* utf8_and_euc_kr.so*
libpqwalreceiver.so* utf8_and_euc_tw.so*
which seems perfectly sane from here. The issue is that if the
install directory path already has "postgres" somewhere in it,
you get a different layout where the PG-specific subdir is smashed
together with the user libraries.
I tend to agree with the OP that that case isn't great, but we've
had it like this for so long that I'm not sure we can change it.
OTOH, the issue probably only affects PG developers, so maybe
we could?
> Nor do we really allow the user to opt in/out of us changing the
> user-specified install locations.
Yeah, you'd have to resort to editing Makefile.global after
configuring, which is annoying.
regards, tom lane
Hi, On 2022-07-15 19:22:12 -0400, Tom Lane wrote: > Andres Freund <andres@anarazel.de> writes: > > On 2022-07-15 20:38:17 +0200, Alvaro Herrera wrote: > >> Right. See commit 984b0b4df83f and its discussion at [1]. The working > >> theory (as I remember it) is that if the installation is already > >> postgres-specific (because the user has made it so by having "pgsql" or > >> "postgres" in the argument to --prefix), then we don't need to add > >> another level of postgres-specificity to it; but otherwise we add > >> "/postgresql" to the path so that the resulting files don't end up mixed > >> with files installed by unrelated packages. > > > I think the current approach isn't great. > > > E.g., putting extension .so's into the same directories as "normal shared > > libraries", which typically is either in the library search path via system > > configuration, or in the library search path due to rpath, strikes me as quite > > wrongheaded. > > Well, we don't, with a normal not-PG-specific install path. Right - I was commenting on the "postgres"/"pgsql" specific logic. > which seems perfectly sane from here. The issue is that if the > install directory path already has "postgres" somewhere in it, > you get a different layout where the PG-specific subdir is smashed > together with the user libraries. > > I tend to agree with the OP that that case isn't great, but we've > had it like this for so long that I'm not sure we can change it. > OTOH, the issue probably only affects PG developers, so maybe > we could? I'm not sure it just devs - I think several distribution install things into per-major-version paths containing postgres or pgsql. But I think they also tend to patch postgres to avoid some of the Makefile.global magic. So I suspect it'd not be too hard for them to adjust. Greetings, Andres Freund