Обсуждение: Lower priority of the configure option --with-libraries causes wrong lib resolution

Поиск
Список
Период
Сортировка

Lower priority of the configure option --with-libraries causes wrong lib resolution

От
Charles Samborski
Дата:
Hello,
I am having issues building Postgres due to the `./configure` script. I 
found a workaround but I would appreciate a more reliable fix.

I use Arch Linux 64bit and want to build Postgres with the latest libicu 
version which is version 77.1. Unfortunately the Arch package is still 
at version 76 currently [0]. To solve this, I built libicu 77.1 locally. 
I also built the latest libxml2 with the latest libicu to ensure compat. 
This means that I have a local directory with shared objects and include 
files for libicu 77. Let's say that these dependencies are in 
`/postgres-deps/lib` and `/postgres-deps/include`. I also still have the 
system-level ICU version 76 present in `/usr/lib`.

My goal is to build Postgres with the libicu version from `libicuuc.so` 
(77) instead of the one from `/usr/lib` (76). Both directories contain a 
file named `libicuuc.so`.


To achieve it, I called the `configure` script with the arguments 
`--with-libraries` and `--with-includes`, as such: `./configure 
--with-libraries=/postgres-deps/lib 
--with-includes=/postgres-deps/include ...` (+ some extra `--with` flags 
to enable features, irrelevant to this issue I think).

I then initiate the build with `LD_LIBRARY_PATH="/postgres-deps/lib " 
make all`. The C compilation into object code succeeds using the libicu 
77 includes, but then it fails to link with many errors such as:

```
/usr/bin/ld: commands/collationcmds.o: in function `get_icu_locale_comment':
collationcmds.c:(.text+0x1a20): undefined reference to 
`uloc_getDisplayName_77'
```

The failing command is fairly long so I shortened it to focus on the 
main part:

```

gcc [...COMPILER_FLAGS] [...OBJECT_FILES] -L../../src/port 
-L../../src/common  -L/usr/lib -L/postgres-deps/lib -Wl,--as-needed  
-Wl,--export-dynamic -lzstd -llz4 -lxslt -lxml2 -lpam -lssl -lcrypto 
-lgssapi_krb5 -lz -lm -lldap -licui18n -licuuc -lsystemd -o postgres
```

In particular, notice that the lib locations give priority to the system 
directory instead of the `--with-libraries` directory that I passed to 
`./configure`: `-L/usr/lib  -L/postgres-deps/lib`. Since the command 
requests `-licuuc`, the file `/usr/lib/libicuuc.so` (version 76) is 
matched first and the file `/usr/lib/libicuuc.so` (77) is ignored.


Swapping the two flags so the order is `-L/postgres-deps/lib -L/usr/lib` 
fixes the build. I consider it very surprising that that libraries 
passed with `--with-libraries` have lower priority, I would expect 
explicitly requested libraries to have the highest priority.


I've searched a bit to find where both locations are inserted.

First, `/usr/lib` is added in the clang configuration [1]. It is 
retrieved from `/usr/bin/llvm-config --ldflags`.

Second, the `--with-libraries` directories are collected into 
`$LIBDIRS`, with a check to verify that the dirs exists, this adds 
`/postgres-deps/lib` [2].

Finally, $LIBDIRS is _appended at the end of LDFLAGS_.using 
`LDFLAGS="$LDFLAGS $LIBDIRS"` [3].


My workaround is to update the configure script to instead prepend 
`LIBDIRS` at the start of `LDFLAGS` using `LDFLAGS="$LIBDIRS $LDFLAGS"`. 
With this change, the lib dir order is `-L/postgres-deps/lib -L/usr/lib` 
and the version 77 of libicuuc.so is picked by the linker. The build 
completes and I get a fully functional Postgres (at least it passes the 
test suite of my application depending on it).

My expectation that moving the lib directories passed using 
`--with-libraries` before any other linker flags makes sense, however 
I'm not an expert in this area and I guess that there may be situations 
where having the CLI libs first may cause issues. I can send a patch 
swapping the application order or `LIBDIRS` as described in my 
workaround if it makes sense. If there is a better solution to give 
higher priority to the libs passed through the configure CLI, I would 
gladly use it.


[0]: https://archlinux.org/packages/core/x86_64/icu/

[1]: 
https://github.com/postgres/postgres/blob/6d6480066c1a96c7130b97b1139fdada9d484f80/configure#L5197

[2]: 
https://github.com/postgres/postgres/blob/6d6480066c1a96c7130b97b1139fdada9d484f80/configure#L8104

[3]: 
https://github.com/postgres/postgres/blob/6d6480066c1a96c7130b97b1139fdada9d484f80/configure#L9824


Thank you in advance,
Charles "demurgos" Samborski




Charles Samborski <demurgos@demurgos.net> writes:
> Swapping the two flags so the order is `-L/postgres-deps/lib -L/usr/lib` 
> fixes the build. I consider it very surprising that that libraries 
> passed with `--with-libraries` have lower priority, I would expect 
> explicitly requested libraries to have the highest priority.

Ugh.

> First, `/usr/lib` is added in the clang configuration [1]. It is 
> retrieved from `/usr/bin/llvm-config --ldflags`.

I would pin the blame here.  This code should not be messing with
the global LDFLAGS.  -L switches from llvm-config should probably
go into LLVM_LIBS instead, so that they're only applied while
linking llvmjit.so.  (Compare the handling of -L switches from
python or perl: those go into python_libspec or perl_embed_ldflags,
they're not applied globally.)

Can you check whether a change along that line fixes the problem
in your environment?

> My workaround is to update the configure script to instead prepend 
> `LIBDIRS` at the start of `LDFLAGS` using `LDFLAGS="$LIBDIRS $LDFLAGS"`. 

I don't think we'd accept that: it seems about as likely to break
builds as fix them.  Notably, users might've themselves modified
LDFLAGS to contain -L switches.  While that's not good practice
IMO, we shouldn't make changes that are more sweeping than
necessary.

            regards, tom lane



I wrote:
> I would pin the blame here.  This code should not be messing with
> the global LDFLAGS.

Looking closer, we've made the same mistake elsewhere.
I think we need something like the attached to ensure
that -L switches coming from libraries' configure helpers
don't override user-specified directories.

            regards, tom lane

diff --git a/config/llvm.m4 b/config/llvm.m4
index fa4bedd9370..9d6fe8199e3 100644
--- a/config/llvm.m4
+++ b/config/llvm.m4
@@ -4,7 +4,7 @@
 # -----------------
 #
 # Look for the LLVM installation, check that it's new enough, set the
-# corresponding LLVM_{CFLAGS,CXXFLAGS,BINPATH} and LDFLAGS
+# corresponding LLVM_{CFLAGS,CXXFLAGS,BINPATH,LIBS}
 # variables. Also verify that CLANG is available, to transform C
 # into bitcode.
 #
@@ -55,7 +55,7 @@ AC_DEFUN([PGAC_LLVM_SUPPORT],
 
   for pgac_option in `$LLVM_CONFIG --ldflags`; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LLVM_LIBS="$LLVM_LIBS $pgac_option";;
     esac
   done
 
diff --git a/configure b/configure
index 4f15347cc95..09890286e2a 100755
--- a/configure
+++ b/configure
@@ -5194,7 +5194,7 @@ fi
 
   for pgac_option in `$LLVM_CONFIG --ldflags`; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LLVM_LIBS="$LLVM_LIBS $pgac_option";;
     esac
   done
 
@@ -9441,7 +9441,7 @@ fi
   done
   for pgac_option in $XML2_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi
@@ -9671,7 +9671,7 @@ fi
   done
   for pgac_option in $LZ4_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi
@@ -9812,7 +9812,7 @@ fi
   done
   for pgac_option in $ZSTD_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi
diff --git a/configure.ac b/configure.ac
index 4b8335dc613..99bb2fb5698 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1108,7 +1108,7 @@ if test "$with_libxml" = yes ; then
   done
   for pgac_option in $XML2_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi
@@ -1157,7 +1157,7 @@ if test "$with_lz4" = yes; then
   done
   for pgac_option in $LZ4_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi
@@ -1182,7 +1182,7 @@ if test "$with_zstd" = yes; then
   done
   for pgac_option in $ZSTD_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi

I wrote:
> I think we need something like the attached to ensure
> that -L switches coming from libraries' configure helpers
> don't override user-specified directories.

I looked this over again, and was about ready to commit it when
I realized that we have essentially the same problem for -I
switches.  If there's a --with-includes switch, then the -I
switches from that should override anything supplied by external
configuration helpers, but we didn't reliably get that right.
(Some of those bugs are new in v18, but others are older.)

However, -I switches pointing to in-tree directories should
override all of those.  We fixed one instance of that problem
a few months ago in cb36f8ec2, but I was dismayed to find that
there are more.  This could result in build failures thanks to
pulling in the wrong version of some Postgres header.

Hence, 0001 attached is the same as before (but now with
a commit message), and then 0002 tackles the problems with -I
switches.

            regards, tom lane

From c8cf76dc696e28c541ccfa3416c2ca2cd859b6c8 Mon Sep 17 00:00:00 2001
From: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun, 27 Jul 2025 17:21:50 -0400
Subject: [PATCH v2 1/2] Avoid putting library-supplied -L switches before
 user-supplied ones.

For many optional libraries, we extract the -L and -l switches needed
to link the library from a helper program such as llvm-config.  In
some cases we put the resulting -L switches into LDFLAGS ahead of
-L switches specified via --with-libraries.  That risks breaking
the user's intention for --with-libraries.

It's not such a problem if the library's -L switch points to a
directory containing only that library, but on some platforms a
library helper may "helpfully" offer a switch such as -L/usr/lib
that points to a directory holding all standard libraries.  If the
user specified --with-libraries in hopes of overriding the standard
build of some library, the -L/usr/lib switch prevents that from
happening since it will come before the user-specified directory.

To fix, avoid inserting these switches directly into LDFLAGS during
configure, instead adding them to LIBDIRS or SHLIB_LINK.  They will
still eventually get added to LDFLAGS, but only after the switches
coming from --with-libraries.

The Meson build scripts may or may not have any comparable problem,
but I'll leave it to someone else to investigate that.

Reported-by: Charles Samborski <demurgos@demurgos.net>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/70f2155f-27ca-4534-b33d-7750e20633d7@demurgos.net
---
 config/llvm.m4 | 4 ++--
 configure      | 8 ++++----
 configure.ac   | 6 +++---
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/config/llvm.m4 b/config/llvm.m4
index fa4bedd9370..9d6fe8199e3 100644
--- a/config/llvm.m4
+++ b/config/llvm.m4
@@ -4,7 +4,7 @@
 # -----------------
 #
 # Look for the LLVM installation, check that it's new enough, set the
-# corresponding LLVM_{CFLAGS,CXXFLAGS,BINPATH} and LDFLAGS
+# corresponding LLVM_{CFLAGS,CXXFLAGS,BINPATH,LIBS}
 # variables. Also verify that CLANG is available, to transform C
 # into bitcode.
 #
@@ -55,7 +55,7 @@ AC_DEFUN([PGAC_LLVM_SUPPORT],

   for pgac_option in `$LLVM_CONFIG --ldflags`; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LLVM_LIBS="$LLVM_LIBS $pgac_option";;
     esac
   done

diff --git a/configure b/configure
index 6d7c22e153f..a2a15813c85 100755
--- a/configure
+++ b/configure
@@ -5194,7 +5194,7 @@ fi

   for pgac_option in `$LLVM_CONFIG --ldflags`; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LLVM_LIBS="$LLVM_LIBS $pgac_option";;
     esac
   done

@@ -9441,7 +9441,7 @@ fi
   done
   for pgac_option in $XML2_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi
@@ -9671,7 +9671,7 @@ fi
   done
   for pgac_option in $LZ4_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi
@@ -9812,7 +9812,7 @@ fi
   done
   for pgac_option in $ZSTD_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi
diff --git a/configure.ac b/configure.ac
index c2877e36935..4972fb600ad 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1108,7 +1108,7 @@ if test "$with_libxml" = yes ; then
   done
   for pgac_option in $XML2_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi
@@ -1157,7 +1157,7 @@ if test "$with_lz4" = yes; then
   done
   for pgac_option in $LZ4_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi
@@ -1182,7 +1182,7 @@ if test "$with_zstd" = yes; then
   done
   for pgac_option in $ZSTD_LIBS; do
     case $pgac_option in
-      -L*) LDFLAGS="$LDFLAGS $pgac_option";;
+      -L*) LIBDIRS="$LIBDIRS $pgac_option";;
     esac
   done
 fi
--
2.43.7

From 3c56c2869380a3664b701c56c611c76d17796622 Mon Sep 17 00:00:00 2001
From: Tom Lane <tgl@sss.pgh.pa.us>
Date: Sun, 27 Jul 2025 18:14:06 -0400
Subject: [PATCH v2 2/2] Avoid putting library-supplied -I switches before
 user-supplied ones.

This patch fixes the same problem as the previous one, but with
respect to -I switches: those coming from --with-includes should
appear before any coming from outside sources such as llvm-config.
We have not heard field complaints about this case, but it seems
certain that a user attempting to override a standard library
could have issues.

The changes for this go well beyond configure itself, however,
because many Makefiles have occasion to manipulate CPPFLAGS to
insert locally-desirable -I switches, and some of them got it wrong.
The correct ordering is any -I switches pointing at within-the-
source-tree-or-build-tree directories, then those from the tree-wide
CPPFLAGS, then those from outside sources such as llvm-config.
There were several places that risked pulling in a system-supplied
copy of libpq headers, for example, instead of the in-tree files.

The Meson build scripts may or may not have any comparable problem,
but I'll leave it to someone else to investigate that.

Reported-by: Charles Samborski <demurgos@demurgos.net>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/70f2155f-27ca-4534-b33d-7750e20633d7@demurgos.net
---
 config/programs.m4                  |  4 ++--
 configure                           | 16 ++++++++--------
 configure.ac                        | 12 ++++++------
 src/Makefile.global.in              |  2 +-
 src/backend/jit/llvm/Makefile       |  2 +-
 src/bin/initdb/Makefile             |  2 +-
 src/common/Makefile                 |  2 +-
 src/interfaces/libpq-oauth/Makefile |  2 +-
 src/interfaces/libpq/Makefile       |  2 +-
 src/pl/plpython/Makefile            |  2 +-
 src/pl/tcl/Makefile                 |  2 +-
 11 files changed, 24 insertions(+), 24 deletions(-)

diff --git a/config/programs.m4 b/config/programs.m4
index c73d9307ea8..e57fe4907b8 100644
--- a/config/programs.m4
+++ b/config/programs.m4
@@ -290,8 +290,8 @@ AC_DEFUN([PGAC_CHECK_LIBCURL],
   pgac_save_LDFLAGS=$LDFLAGS
   pgac_save_LIBS=$LIBS

-  CPPFLAGS="$LIBCURL_CPPFLAGS $CPPFLAGS"
-  LDFLAGS="$LIBCURL_LDFLAGS $LDFLAGS"
+  CPPFLAGS="$CPPFLAGS $LIBCURL_CPPFLAGS"
+  LDFLAGS="$LDFLAGS $LIBCURL_LDFLAGS"

   AC_CHECK_HEADER(curl/curl.h, [],
                   [AC_MSG_ERROR([header file <curl/curl.h> is required for --with-libcurl])])
diff --git a/configure b/configure
index a2a15813c85..8a535da6b7a 100755
--- a/configure
+++ b/configure
@@ -9436,7 +9436,7 @@ fi
   # Note the user could also set XML2_CFLAGS/XML2_LIBS directly
   for pgac_option in $XML2_CFLAGS; do
     case $pgac_option in
-      -I*|-D*) CPPFLAGS="$CPPFLAGS $pgac_option";;
+      -I*|-D*) INCLUDES="$INCLUDES $pgac_option";;
     esac
   done
   for pgac_option in $XML2_LIBS; do
@@ -9666,7 +9666,7 @@ fi
   # note that -llz4 will be added by AC_CHECK_LIB below.
   for pgac_option in $LZ4_CFLAGS; do
     case $pgac_option in
-      -I*|-D*) CPPFLAGS="$CPPFLAGS $pgac_option";;
+      -I*|-D*) INCLUDES="$INCLUDES $pgac_option";;
     esac
   done
   for pgac_option in $LZ4_LIBS; do
@@ -9807,7 +9807,7 @@ fi
   # note that -lzstd will be added by AC_CHECK_LIB below.
   for pgac_option in $ZSTD_CFLAGS; do
     case $pgac_option in
-      -I*|-D*) CPPFLAGS="$CPPFLAGS $pgac_option";;
+      -I*|-D*) INCLUDES="$INCLUDES $pgac_option";;
     esac
   done
   for pgac_option in $ZSTD_LIBS; do
@@ -12723,8 +12723,8 @@ if test "$with_libcurl" = yes ; then
   pgac_save_LDFLAGS=$LDFLAGS
   pgac_save_LIBS=$LIBS

-  CPPFLAGS="$LIBCURL_CPPFLAGS $CPPFLAGS"
-  LDFLAGS="$LIBCURL_LDFLAGS $LDFLAGS"
+  CPPFLAGS="$CPPFLAGS $LIBCURL_CPPFLAGS"
+  LDFLAGS="$LDFLAGS $LIBCURL_LDFLAGS"

   ac_fn_c_check_header_mongrel "$LINENO" "curl/curl.h" "ac_cv_header_curl_curl_h" "$ac_includes_default"
 if test "x$ac_cv_header_curl_curl_h" = xyes; then :
@@ -16658,7 +16658,7 @@ fi

 if test "$with_icu" = yes; then
   ac_save_CPPFLAGS=$CPPFLAGS
-  CPPFLAGS="$ICU_CFLAGS $CPPFLAGS"
+  CPPFLAGS="$CPPFLAGS $ICU_CFLAGS"

   # Verify we have ICU's header files
   ac_fn_c_check_header_mongrel "$LINENO" "unicode/ucol.h" "ac_cv_header_unicode_ucol_h" "$ac_includes_default"
@@ -18876,7 +18876,7 @@ Use --without-tcl to disable building PL/Tcl." "$LINENO" 5
     fi
     # now that we have TCL_INCLUDE_SPEC, we can check for <tcl.h>
     ac_save_CPPFLAGS=$CPPFLAGS
-    CPPFLAGS="$TCL_INCLUDE_SPEC $CPPFLAGS"
+    CPPFLAGS="$CPPFLAGS $TCL_INCLUDE_SPEC"
     ac_fn_c_check_header_mongrel "$LINENO" "tcl.h" "ac_cv_header_tcl_h" "$ac_includes_default"
 if test "x$ac_cv_header_tcl_h" = xyes; then :

@@ -18945,7 +18945,7 @@ fi
 # check for <Python.h>
 if test "$with_python" = yes; then
   ac_save_CPPFLAGS=$CPPFLAGS
-  CPPFLAGS="$python_includespec $CPPFLAGS"
+  CPPFLAGS="$CPPFLAGS $python_includespec"
   ac_fn_c_check_header_mongrel "$LINENO" "Python.h" "ac_cv_header_Python_h" "$ac_includes_default"
 if test "x$ac_cv_header_Python_h" = xyes; then :

diff --git a/configure.ac b/configure.ac
index 4972fb600ad..e72201e679b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1103,7 +1103,7 @@ if test "$with_libxml" = yes ; then
   # Note the user could also set XML2_CFLAGS/XML2_LIBS directly
   for pgac_option in $XML2_CFLAGS; do
     case $pgac_option in
-      -I*|-D*) CPPFLAGS="$CPPFLAGS $pgac_option";;
+      -I*|-D*) INCLUDES="$INCLUDES $pgac_option";;
     esac
   done
   for pgac_option in $XML2_LIBS; do
@@ -1152,7 +1152,7 @@ if test "$with_lz4" = yes; then
   # note that -llz4 will be added by AC_CHECK_LIB below.
   for pgac_option in $LZ4_CFLAGS; do
     case $pgac_option in
-      -I*|-D*) CPPFLAGS="$CPPFLAGS $pgac_option";;
+      -I*|-D*) INCLUDES="$INCLUDES $pgac_option";;
     esac
   done
   for pgac_option in $LZ4_LIBS; do
@@ -1177,7 +1177,7 @@ if test "$with_zstd" = yes; then
   # note that -lzstd will be added by AC_CHECK_LIB below.
   for pgac_option in $ZSTD_CFLAGS; do
     case $pgac_option in
-      -I*|-D*) CPPFLAGS="$CPPFLAGS $pgac_option";;
+      -I*|-D*) INCLUDES="$INCLUDES $pgac_option";;
     esac
   done
   for pgac_option in $ZSTD_LIBS; do
@@ -1944,7 +1944,7 @@ fi

 if test "$with_icu" = yes; then
   ac_save_CPPFLAGS=$CPPFLAGS
-  CPPFLAGS="$ICU_CFLAGS $CPPFLAGS"
+  CPPFLAGS="$CPPFLAGS $ICU_CFLAGS"

   # Verify we have ICU's header files
   AC_CHECK_HEADER(unicode/ucol.h, [],
@@ -2344,7 +2344,7 @@ Use --without-tcl to disable building PL/Tcl.])
     fi
     # now that we have TCL_INCLUDE_SPEC, we can check for <tcl.h>
     ac_save_CPPFLAGS=$CPPFLAGS
-    CPPFLAGS="$TCL_INCLUDE_SPEC $CPPFLAGS"
+    CPPFLAGS="$CPPFLAGS $TCL_INCLUDE_SPEC"
     AC_CHECK_HEADER(tcl.h, [], [AC_MSG_ERROR([header file <tcl.h> is required for Tcl])])
     CPPFLAGS=$ac_save_CPPFLAGS
 fi
@@ -2381,7 +2381,7 @@ fi
 # check for <Python.h>
 if test "$with_python" = yes; then
   ac_save_CPPFLAGS=$CPPFLAGS
-  CPPFLAGS="$python_includespec $CPPFLAGS"
+  CPPFLAGS="$CPPFLAGS $python_includespec"
   AC_CHECK_HEADER(Python.h, [], [AC_MSG_ERROR([header file <Python.h> is required for Python])])
   CPPFLAGS=$ac_save_CPPFLAGS
 fi
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 04952b533de..8b1b357beaa 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -254,7 +254,7 @@ CPP = @CPP@
 CPPFLAGS = @CPPFLAGS@
 PG_SYSROOT = @PG_SYSROOT@

-override CPPFLAGS := $(ICU_CFLAGS) $(LIBNUMA_CFLAGS) $(LIBURING_CFLAGS) $(CPPFLAGS)
+override CPPFLAGS += $(ICU_CFLAGS) $(LIBNUMA_CFLAGS) $(LIBURING_CFLAGS)

 ifdef PGXS
 override CPPFLAGS := -I$(includedir_server) -I$(includedir_internal) $(CPPFLAGS)
diff --git a/src/backend/jit/llvm/Makefile b/src/backend/jit/llvm/Makefile
index e8c12060b93..68677ba42e1 100644
--- a/src/backend/jit/llvm/Makefile
+++ b/src/backend/jit/llvm/Makefile
@@ -31,7 +31,7 @@ endif
 # All files in this directory use LLVM.
 CFLAGS += $(LLVM_CFLAGS)
 CXXFLAGS += $(LLVM_CXXFLAGS)
-override CPPFLAGS := $(LLVM_CPPFLAGS) $(CPPFLAGS)
+override CPPFLAGS += $(LLVM_CPPFLAGS)
 SHLIB_LINK += $(LLVM_LIBS)

 # Because this module includes C++ files, we need to use a C++
diff --git a/src/bin/initdb/Makefile b/src/bin/initdb/Makefile
index 997e0a013e9..c0470efda92 100644
--- a/src/bin/initdb/Makefile
+++ b/src/bin/initdb/Makefile
@@ -20,7 +20,7 @@ include $(top_builddir)/src/Makefile.global
 # from libpq, else we have risks of version skew if we run with a libpq
 # shared library from a different PG version.  Define
 # USE_PRIVATE_ENCODING_FUNCS to ensure that that happens.
-override CPPFLAGS := -DUSE_PRIVATE_ENCODING_FUNCS -I$(libpq_srcdir) -I$(top_srcdir)/src/timezone $(ICU_CFLAGS)
$(CPPFLAGS)
+override CPPFLAGS := -DUSE_PRIVATE_ENCODING_FUNCS -I$(libpq_srcdir) -I$(top_srcdir)/src/timezone $(CPPFLAGS)
$(ICU_CFLAGS)

 # We need libpq only because fe_utils does.
 LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) $(ICU_LIBS)
diff --git a/src/common/Makefile b/src/common/Makefile
index 1e2b91c83c4..2c720caa509 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -163,7 +163,7 @@ libpgcommon_shlib.a: $(OBJS_SHLIB)
 # The JSON API normally exits on out-of-memory; disable that behavior for shared
 # library builds. This requires libpq's pqexpbuffer.h.
 jsonapi_shlib.o: override CPPFLAGS += -DJSONAPI_USE_PQEXPBUFFER
-jsonapi_shlib.o: override CPPFLAGS += -I$(libpq_srcdir)
+jsonapi_shlib.o: override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)

 # Because this uses its own compilation rule, it doesn't use the
 # dependency tracking logic from Makefile.global.  To make sure that
diff --git a/src/interfaces/libpq-oauth/Makefile b/src/interfaces/libpq-oauth/Makefile
index 270fc0cf2d9..682f17413b3 100644
--- a/src/interfaces/libpq-oauth/Makefile
+++ b/src/interfaces/libpq-oauth/Makefile
@@ -24,7 +24,7 @@ NAME = pq-oauth-$(MAJORVERSION)
 override shlib := lib$(NAME)$(DLSUFFIX)
 override stlib := libpq-oauth.a

-override CPPFLAGS := -I$(libpq_srcdir) -I$(top_builddir)/src/port $(LIBCURL_CPPFLAGS) $(CPPFLAGS)
+override CPPFLAGS := -I$(libpq_srcdir) -I$(top_builddir)/src/port $(CPPFLAGS) $(LIBCURL_CPPFLAGS)

 OBJS = \
     $(WIN32RES)
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 47d67811509..da6650066d4 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -24,7 +24,7 @@ NAME= pq
 SO_MAJOR_VERSION= 5
 SO_MINOR_VERSION= $(MAJORVERSION)

-override CPPFLAGS :=  -I$(srcdir) $(CPPFLAGS) -I$(top_builddir)/src/port -I$(top_srcdir)/src/port
+override CPPFLAGS := -I$(srcdir) -I$(top_builddir)/src/port -I$(top_srcdir)/src/port $(CPPFLAGS)
 ifneq ($(PORTNAME), win32)
 override CFLAGS += $(PTHREAD_CFLAGS)
 endif
diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile
index f959083a0bd..25f295c3709 100644
--- a/src/pl/plpython/Makefile
+++ b/src/pl/plpython/Makefile
@@ -11,7 +11,7 @@ ifeq ($(PORTNAME), win32)
 override python_libspec =
 endif

-override CPPFLAGS := -I. -I$(srcdir) $(python_includespec) $(CPPFLAGS)
+override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS) $(python_includespec)

 rpathdir = $(python_libdir)

diff --git a/src/pl/tcl/Makefile b/src/pl/tcl/Makefile
index ea52a2efc22..dd57f7d694c 100644
--- a/src/pl/tcl/Makefile
+++ b/src/pl/tcl/Makefile
@@ -11,7 +11,7 @@ top_builddir = ../../..
 include $(top_builddir)/src/Makefile.global


-override CPPFLAGS := -I. -I$(srcdir) $(TCL_INCLUDE_SPEC) $(CPPFLAGS)
+override CPPFLAGS := -I. -I$(srcdir) $(CPPFLAGS) $(TCL_INCLUDE_SPEC)

 # On Windows, we don't link directly with the Tcl library; see below
 ifneq ($(PORTNAME), win32)
--
2.43.7