Обсуждение: meson: Add _static and _shared suffixes to the library names
Hi, Windows CI images are updated ~15 hours ago [1] and Windows CI task started to fail with [2]: ``` FAILED: [code=2] src/interfaces/libpq/libpq.a.p/meson_pch-c.obj src/interfaces/libpq/libpq.a.p/postgres_fe_pch.pch "cl" "-Isrc/interfaces/libpq\libpq.a.p" "-Isrc/interfaces/libpq" "-I..\src\interfaces\libpq" "-Isrc\port" "-I..\src\port" "-Isrc\include" "-I..\src\include" "-Ic:\openssl\1.1\include" "-I..\src\include\port\win32" "-I..\src\include\port\win32_msvc" "-IC:/cirrus/src/include/pch" "/nologo" "/showIncludes" "/utf-8" "/W2" "/Od" "/Zi" "/Zc:preprocessor" "/DWIN32" "/DWINDOWS" "/D__WINDOWS__" "/D__WIN32__" "/D_CRT_SECURE_NO_DEPRECATE" "/D_CRT_NONSTDC_NO_DEPRECATE" "/wd4018" "/wd4244" "/wd4273" "/wd4101" "/wd4102" "/wd4090" "/wd4267" "/Ycpostgres_fe_pch.h" "/Fpsrc/interfaces/libpq\libpq.a.p\postgres_fe_pch.pch" "/Fosrc/interfaces/libpq\libpq.a.p\meson_pch-c.obj" "/MDd" "/nologo" "/showIncludes" "/utf-8" "/W2" "/Od" "/Zi" "/Zc:preprocessor" "/DWIN32" "/DWINDOWS" "/D__WINDOWS__" "/D__WIN32__" "/D_CRT_SECURE_NO_DEPRECATE" "/D_CRT_NONSTDC_NO_DEPRECATE" "/wd4018" "/wd4244" "/wd4273" "/wd4101" "/wd4102" "/wd4090" "/wd4267" "-DSO_MAJOR_VERSION=5" "/FS" "/FdC:\cirrus\build\src/interfaces/libpq\libpq.pdb" "/c" src/interfaces/libpq/libpq.a.p/meson_pch-c.c src/interfaces/libpq/libpq.a.p/meson_pch-c.c: fatal error C1052: program database file, 'C:\cirrus\build\src\interfaces\libpq\libpq.pdb', was generated by the linker with /DEBUG:fastlink; compiler cannot update such PDB files; please delete it or use /Fd to specify a different PDB filename ``` After talking with Andres, we realized that the problem is caused by static and shared libraries having the same name, so they overwrite each other's debug file. This was not seen before, our guess is that the ninja version is upgraded on the Windows CI image from 1.11.1 to 1.13.0 and new ninja optimized the build. Older ninja was building two libraries concurrently and that hid the problem but the new ninja optimized/changed the build, so these two libraries weren't built concurrently and caused the failure. The solution is changing the libraries' names [3]. Changing conflicting libraries' names is actually enough but I wanted to add _static and _shared suffixes to all of the libraries' names as I think this is more future proof. Any feedback would be appreciated. Note: Since there is no new commit after the Windows CI image update [1], this problem is not visible on the upstream Postgres for now. The first commit will trigger this problem. [1] https://cirrus-ci.com/task/5683237017616384 [2] https://cirrus-ci.com/task/5906793164963840 [3] https://cirrus-ci.com/task/5204554066690048 -- Regards, Nazir Bilal Yavuz Microsoft
Вложения
On Tue, Aug 12, 2025 at 8:46 AM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
> The solution is changing the libraries' names [3]. Changing
> conflicting libraries' names is actually enough but I wanted to add
> _static and _shared suffixes to all of the libraries' names as I think
> this is more future proof.
>
> Any feedback would be appreciated.
Just as a kneejerk reaction, I think it'd be really strange if the
Meson version of libpq-oauth.a was named libpq-oauth_static.a on disk,
and clients had to figure out which build system was used in order to
link. (Especially since the original name doesn't actually conflict
with libpq-oauth-19.so.)
> -libpq_so = shared_library('libpq',
> +libpq_so = shared_library('libpq_shared',
This, in particular, can't happen; it's akin to an SONAME bump.
Is there a way to work around this problem in a way that affects Windows only?
--Jacob
Jacob Champion <jacob.champion@enterprisedb.com> writes:
> On Tue, Aug 12, 2025 at 8:46 AM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
>> The solution is changing the libraries' names [3]. Changing
>> conflicting libraries' names is actually enough but I wanted to add
>> _static and _shared suffixes to all of the libraries' names as I think
>> this is more future proof.
>>
>> Any feedback would be appreciated.
> Just as a kneejerk reaction, I think it'd be really strange if the
> Meson version of libpq-oauth.a was named libpq-oauth_static.a on disk,
> and clients had to figure out which build system was used in order to
> link.
This proposal seems utterly unacceptable from a client compatibility
standpoint. Sure, we could get away with renaming purely internal
things like boot_parser, but libpq? It won't do for the ecpg
libraries either.
> Is there a way to work around this problem in a way that affects Windows only?
Even on Windows, the proposal is unacceptable. The build products
have to remain the same as what they've been for decades. If meson
is unable to build them, we can't use meson. But I imagine there is
something you can do to persuade the newer version to do what it was
doing before.
regards, tom lane
On Tue, Aug 12, 2025 at 9:27 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > > Is there a way to work around this problem in a way that affects Windows only? > > Even on Windows, the proposal is unacceptable. Sure, but I'm hoping that there's some Windows-specific Meson twiddling that can be done to disambiguate the debug files on disk. --Jacob
On 12.08.25 18:37, Jacob Champion wrote: > On Tue, Aug 12, 2025 at 9:27 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> Is there a way to work around this problem in a way that affects Windows only? >> >> Even on Windows, the proposal is unacceptable. > > Sure, but I'm hoping that there's some Windows-specific Meson > twiddling that can be done to disambiguate the debug files on disk. Here is an older discussion that also involved having concurrent shared_library() and static_library() on Windows, and the general sentiment there appeared to be that this should (be made to) work: https://github.com/mesonbuild/meson/issues/459 So I don't know what changed now, but I think we should think about in terms of what changed rather than fixing our code.
Hi, Thank you all for sharing your thoughts and I am sorry for the first proposal. It is a mistake on my part, I should have thought more. On Tue, 12 Aug 2025 at 21:47, Peter Eisentraut <peter@eisentraut.org> wrote: > > On 12.08.25 18:37, Jacob Champion wrote: > > On Tue, Aug 12, 2025 at 9:27 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > >>> Is there a way to work around this problem in a way that affects Windows only? > >> > >> Even on Windows, the proposal is unacceptable. > > > > Sure, but I'm hoping that there's some Windows-specific Meson > > twiddling that can be done to disambiguate the debug files on disk. > > Here is an older discussion that also involved having concurrent > shared_library() and static_library() on Windows, and the general > sentiment there appeared to be that this should (be made to) work: > https://github.com/mesonbuild/meson/issues/459 I think the problem in the link you shared is different, we have problems with Windows debug files (*.pdb files) but they have problems with the library files themselves. Our problem is that the latest built library tries to overwrite the early built library's .pdb file and the build fails. So, my guess is that the libraries are in different paths but their .pdb files are still in the same path. I did a quick check and found that we have only one .pdb file after the build, which I think confirms my guess. After the #1 fix applied, we have two .pdb files; one .pdb file for each library type. > So I don't know what changed now, but I think we should think about in > terms of what changed rather than fixing our code. I think we had this problem for a long time but the new ninja version showed that problem. I found two ways to fix that problem, both approaches fix the problem by themselves: 1- Setting different names for .pdb files for shared libraries only on the Windows OS. I think that is the correct fix, it just adds _shared suffix to .pdb files of shared libraries on the Windows OS. 2- Using '/DEBUG:FULL' instead of '/DEBUG:FASTLINK' in the Windows CI task but this causes more memory to be used. It seems that the error appears only when the '/DEBUG:FASTLINK' is set. '/DEBUG:FULL' is a default option, so we may decide to not add it at all. I explicitly added it as I found this easier to understand. This approach fixes the build but I think it is not the correct fix, we will have one .pdb file after this fix; not one for each library type. Here is how the directories look like after the fixes are applied: $ dir C:\cirrus\build\src\interfaces\libpq -> master with old ninja (master with new ninja fails): Directory of C:\cirrus\build\src\interfaces\libpq 08/13/2025 05:47 AM <DIR> . 08/13/2025 05:47 AM <DIR> .. 08/13/2025 05:47 AM 4,483 exports.def 08/13/2025 05:47 AM 973,700 libpq.a 08/13/2025 05:47 AM <DIR> libpq.a.p 08/13/2025 05:47 AM 450,560 libpq.dll 08/13/2025 05:47 AM <DIR> libpq.dll.p 08/13/2025 05:47 AM 41,952 libpq.lib 08/13/2025 05:47 AM 1,413,120 libpq.pdb 08/13/2025 05:47 AM 700,416 lnk{6BBDFFB2-4393-4C5B-AE04-F33F1E9BBA12}.tmp 08/13/2025 05:47 AM <DIR> test 6 File(s) 3,584,231 bytes 5 Dir(s) 22,671,282,176 bytes free -------------------------------------------------- #1 fix with new ninja: Directory of C:\cirrus\build\src\interfaces\libpq 08/13/2025 05:55 AM <DIR> . 08/13/2025 05:53 AM <DIR> .. 08/13/2025 05:53 AM 4,483 exports.def 08/13/2025 05:55 AM 973,700 libpq.a 08/13/2025 05:54 AM <DIR> libpq.a.p 08/13/2025 05:53 AM 450,560 libpq.dll 08/13/2025 05:53 AM <DIR> libpq.dll.p 08/13/2025 05:53 AM 41,952 libpq.lib 08/13/2025 05:54 AM 700,416 libpq.pdb 08/13/2025 05:53 AM 1,413,120 libpq_shared.pdb 08/13/2025 05:55 AM <DIR> test 6 File(s) 3,584,231 bytes 5 Dir(s) 22,891,589,632 bytes free ---------------------------------------- #1 fix with old ninja: Directory of C:\cirrus\build\src\interfaces\libpq 08/13/2025 06:19 AM <DIR> . 08/13/2025 06:19 AM <DIR> .. 08/13/2025 06:19 AM 4,483 exports.def 08/13/2025 06:19 AM 973,700 libpq.a 08/13/2025 06:19 AM <DIR> libpq.a.p 08/13/2025 06:19 AM 450,560 libpq.dll 08/13/2025 06:19 AM <DIR> libpq.dll.p 08/13/2025 06:19 AM 41,952 libpq.lib 08/13/2025 06:19 AM 700,416 libpq.pdb 08/13/2025 06:19 AM 1,413,120 libpq_shared.pdb 08/13/2025 06:19 AM <DIR> test 6 File(s) 3,584,231 bytes 5 Dir(s) 22,671,474,688 bytes free -------------------------------------------------- #2 fix with new ninja: Directory of C:\cirrus\build\src\interfaces\libpq 08/13/2025 06:04 AM <DIR> . 08/13/2025 06:02 AM <DIR> .. 08/13/2025 06:02 AM 4,483 exports.def 08/13/2025 06:04 AM 973,700 libpq.a 08/13/2025 06:03 AM <DIR> libpq.a.p 08/13/2025 06:02 AM 450,560 libpq.dll 08/13/2025 06:02 AM <DIR> libpq.dll.p 08/13/2025 06:02 AM 41,952 libpq.lib 08/13/2025 06:03 AM 1,961,984 libpq.pdb 08/13/2025 06:04 AM <DIR> test 5 File(s) 3,432,679 bytes 5 Dir(s) 22,859,337,728 bytes free ---------------------------------------- #2 fix with old ninja: Directory of C:\cirrus\build\src\interfaces\libpq 08/13/2025 06:07 AM <DIR> . 08/13/2025 06:07 AM <DIR> .. 08/13/2025 06:07 AM 4,483 exports.def 08/13/2025 06:07 AM 973,700 libpq.a 08/13/2025 06:07 AM <DIR> libpq.a.p 08/13/2025 06:07 AM 450,560 libpq.dll 08/13/2025 06:07 AM <DIR> libpq.dll.p 08/13/2025 06:07 AM 41,952 libpq.lib 08/13/2025 06:07 AM 1,961,984 libpq.pdb 08/13/2025 06:07 AM <DIR> test 5 File(s) 3,432,679 bytes 5 Dir(s) 22,639,058,944 bytes free - We have one .pdb file and one .tmp file on the master. I think this .tmp file is a corrupted .pdb file - #1 fix has correct .pdb files on both ninja versions. - We have only one .pdb file with the #2 fix on both ninja versions, there should be one for each library type. -- Regards, Nazir Bilal Yavuz Microsoft
Вложения
Nazir Bilal Yavuz: > 1- Setting different names for .pdb files for shared libraries only on > the Windows OS. I think that is the correct fix, it just adds _shared > suffix to .pdb files of shared libraries on the Windows OS. How about renaming the .pdb files, but instead of adding _shared, rename both .pdb files to libpq.a.pdb and libpq.dll.pdb? I think that would be much clearer than libpq.pdb and libpq_shared.pdb. Best, Wolfgang
Hi,
Thank you for looking into this!
On Wed, 13 Aug 2025 at 10:50, Wolfgang Walther <walther@technowledgy.de> wrote:
>
> Nazir Bilal Yavuz:
> > 1- Setting different names for .pdb files for shared libraries only on
> > the Windows OS. I think that is the correct fix, it just adds _shared
> > suffix to .pdb files of shared libraries on the Windows OS.
>
> How about renaming the .pdb files, but instead of adding _shared, rename
> both .pdb files to libpq.a.pdb and libpq.dll.pdb?
>
> I think that would be much clearer than libpq.pdb and libpq_shared.pdb.
That works for me. I think the only downside is that there are 22
static_library() and 4 shared_library() calls inside the meson build.
So we need to copy this logic [1] 22 times more if we want to rename
static libraries' .pdb files.
[1]
${name}_pdb_args = []
if host_system == 'windows'
${name}_pdb_args = ['/PDB:' + meson.current_build_dir() / '${name}.pdb']
endif
--
Regards,
Nazir Bilal Yavuz
Microsoft
Hi, On 2025-08-13 09:36:23 +0300, Nazir Bilal Yavuz wrote: > Thank you all for sharing your thoughts and I am sorry for the first > proposal. It is a mistake on my part, I should have thought more. That's just iterative development, don't worry about it. > On Tue, 12 Aug 2025 at 21:47, Peter Eisentraut <peter@eisentraut.org> wrote: > > On 12.08.25 18:37, Jacob Champion wrote: > > > On Tue, Aug 12, 2025 at 9:27 AM Tom Lane <tgl@sss.pgh.pa.us> wrote: > > >>> Is there a way to work around this problem in a way that affects Windows only? > > >> > > >> Even on Windows, the proposal is unacceptable. > > > > > > Sure, but I'm hoping that there's some Windows-specific Meson > > > twiddling that can be done to disambiguate the debug files on disk. > > > > Here is an older discussion that also involved having concurrent > > shared_library() and static_library() on Windows, and the general > > sentiment there appeared to be that this should (be made to) work: > > https://github.com/mesonbuild/meson/issues/459 > > I think the problem in the link you shared is different, we have > problems with Windows debug files (*.pdb files) but they have problems > with the library files themselves. Our problem is that the latest > built library tries to overwrite the early built library's .pdb file > and the build fails. So, my guess is that the libraries are in > different paths but their .pdb files are still in the same path. I did > a quick check and found that we have only one .pdb file after the > build, which I think confirms my guess. After the #1 fix applied, we > have two .pdb files; one .pdb file for each library type. Yea. > > So I don't know what changed now, but I think we should think about in > > terms of what changed rather than fixing our code. > > I think we had this problem for a long time but the new ninja version > showed that problem. Right - I'm fairly sure that if you built sequentially with an older ninja you'd also see the issue. > I found two ways to fix that problem, both approaches fix the problem > by themselves: > > 1- Setting different names for .pdb files for shared libraries only on > the Windows OS. I think that is the correct fix, it just adds _shared > suffix to .pdb files of shared libraries on the Windows OS. I don't think it is right to do this "below" meson - presumably this means that meson install won't work correctly, as it doesn't know about the changed pdb file names. > 2- Using '/DEBUG:FULL' instead of '/DEBUG:FASTLINK' in the Windows CI > task but this causes more memory to be used. It seems that the error > appears only when the '/DEBUG:FASTLINK' is set. '/DEBUG:FULL' is a > default option, so we may decide to not add it at all. I explicitly > added it as I found this easier to understand. This approach fixes the > build but I think it is not the correct fix, we will have one .pdb > file after this fix; not one for each library type. I think it's an acceptable fix for now. I added /DEBUG:FASTLINK to the CI task when it was using windows containers, as we'd run out of memory occasionally. But since we aren't using those anymore, I think the best way to make CI work again is to simply stop using /DEBUG:FASTLINK. Separately I think we should report this as a bug to meson. Could you perhaps create a minimal reproducer of the issue and report it? Greetings, Andres Freund
Hi, On 2025-08-13 10:24:07 -0400, Andres Freund wrote: > > 2- Using '/DEBUG:FULL' instead of '/DEBUG:FASTLINK' in the Windows CI > > task but this causes more memory to be used. It seems that the error > > appears only when the '/DEBUG:FASTLINK' is set. '/DEBUG:FULL' is a > > default option, so we may decide to not add it at all. I explicitly > > added it as I found this easier to understand. This approach fixes the > > build but I think it is not the correct fix, we will have one .pdb > > file after this fix; not one for each library type. > > I think it's an acceptable fix for now. I added /DEBUG:FASTLINK to the CI > task when it was using windows containers, as we'd run out of memory > occasionally. But since we aren't using those anymore, I think the best way to > make CI work again is to simply stop using /DEBUG:FASTLINK. I'll push it this way once the test completes that way in my repo. I want to get cfbot green again. If we want to do it differently than in Bilal's v2-0002-ci-windows-Use-DEBUG-FULL-instead-of-DEBUG-FASTLI.patch I however removed DEBUG:FULL, I don't see a point of explicitly specifying the default value. Greetings, Andres Freund
Hi, On Wed, 13 Aug 2025 at 23:03, Andres Freund <andres@anarazel.de> wrote: > > Hi, > > On 2025-08-13 10:24:07 -0400, Andres Freund wrote: > > > 2- Using '/DEBUG:FULL' instead of '/DEBUG:FASTLINK' in the Windows CI > > > task but this causes more memory to be used. It seems that the error > > > appears only when the '/DEBUG:FASTLINK' is set. '/DEBUG:FULL' is a > > > default option, so we may decide to not add it at all. I explicitly > > > added it as I found this easier to understand. This approach fixes the > > > build but I think it is not the correct fix, we will have one .pdb > > > file after this fix; not one for each library type. > > > > I think it's an acceptable fix for now. I added /DEBUG:FASTLINK to the CI > > task when it was using windows containers, as we'd run out of memory > > occasionally. But since we aren't using those anymore, I think the best way to > > make CI work again is to simply stop using /DEBUG:FASTLINK. > > I'll push it this way once the test completes that way in my repo. I want to > get cfbot green again. If we want to do it differently than in Bilal's > v2-0002-ci-windows-Use-DEBUG-FULL-instead-of-DEBUG-FASTLI.patch Thank you! > Separately I think we should report this as a bug to meson. Could you perhaps > create a minimal reproducer of the issue and report it? Yes, I will share the issue link for tracking when I report it. -- Regards, Nazir Bilal Yavuz Microsoft
Hi, On Thu, 14 Aug 2025 at 10:37, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote: > > On Wed, 13 Aug 2025 at 23:03, Andres Freund <andres@anarazel.de> wrote: > > > > v2-0002-ci-windows-Use-DEBUG-FULL-instead-of-DEBUG-FASTLI.patch > > > Separately I think we should report this as a bug to meson. Could you perhaps > > create a minimal reproducer of the issue and report it? > > Yes, I will share the issue link for tracking when I report it. Here is the link: https://github.com/mesonbuild/meson/issues/14925 -- Regards, Nazir Bilal Yavuz Microsoft