Обсуждение: [PATCH] jit: fix build with LLVM-21
(please cc: me on replies) Hello! I tried building against LLVM-21 and noticed that a function for symbol lookup was renamed (without semantic changes), breaking the LLVM JIT. The following patch fixes this by adding a version guard. It applies equally to both master and 17.6. Passes the test suite and verified on 17.6 with the jit example from the documentation. cheers, Holger
Вложения
On 08.09.25 15:20, Holger Hoffstätte wrote: > I tried building against LLVM-21 and noticed that a function for symbol > lookup was renamed (without semantic changes), breaking the LLVM JIT. > The following patch fixes this by adding a version guard. It applies > equally > to both master and 17.6. Passes the test suite and verified on 17.6 with > the > jit example from the documentation. I can confirm that this change seems correct. See [0] for reference. [0]: https://github.com/llvm/llvm-project/commit/d3d856ad84698fa4ec66177d00558b2f5b438d3b As a small style request, I would flip the conditional around so that the new code appears first. I see that we don't do this very consistently in the existing code, but maybe we can start a new trend. ;-) In my testing with LLVM 21, I'm getting an additional error: ../src/backend/jit/llvm/llvmjit_wrap.cpp:56:18: error: no matching constructor for initialization of 'llvm::orc::RTDyldObjectLinkingLayer' 56 | return wrap(new llvm::orc::RTDyldObjectLinkingLayer( | ^ 57 | *unwrap(ES), [] { return std::make_unique<llvm::backport::SectionMemoryManager>(nullptr, true); })); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:58:3: note: candidate constructor not viable: no known conversion from '(lambda at ../src/backend/jit/llvm/llvmjit_wrap.cpp:57:16)' to 'GetMemoryManagerFunction' (aka 'unique_function<std::unique_ptr<RuntimeDyld::MemoryManager> (const MemoryBuffer &)>') for 2nd argument 58 | RTDyldObjectLinkingLayer(ExecutionSession &ES, | ^ 59 | GetMemoryManagerFunction GetMemoryManager); | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:37:16: note: candidate constructor (the implicit copy constructor) not viable: requires 1 argument, but 2 were provided 37 | class LLVM_ABI RTDyldObjectLinkingLayer | ^~~~~~~~~~~~~~~~~~~~~~~~ I gather you're not seeing that?
On 2025-09-12 08:36, Peter Eisentraut wrote: > On 08.09.25 15:20, Holger Hoffstätte wrote: >> I tried building against LLVM-21 and noticed that a function for symbol >> lookup was renamed (without semantic changes), breaking the LLVM JIT. >> The following patch fixes this by adding a version guard. It applies equally >> to both master and 17.6. Passes the test suite and verified on 17.6 with the >> jit example from the documentation. > > I can confirm that this change seems correct. See [0] for reference. Excellent! Thanks for taking a look. This was my first post to pgsql-hackers and I wasn't sure if I had done something wrong. > [0]: > https://github.com/llvm/llvm-project/commit/d3d856ad84698fa4ec66177d00558b2f5b438d3b > > As a small style request, I would flip the conditional around so > that the new code appears first. I see that we don't do this very > consistently in the existing code, but maybe we can start a new > trend. ;-) I knew this would come up since I pondered the same thing :D As you said, the existing code is not consistent, but I can switch this to let the new code appear first, if you prefer. A new patch is attached. > In my testing with LLVM 21, I'm getting an additional error: > > ../src/backend/jit/llvm/llvmjit_wrap.cpp:56:18: error: no matching constructor for initialization of 'llvm::orc::RTDyldObjectLinkingLayer' > 56 | return wrap(new llvm::orc::RTDyldObjectLinkingLayer( > | ^ > 57 | *unwrap(ES), [] { return std::make_unique<llvm::backport::SectionMemoryManager>(nullptr, true);})); > | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > /opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:58:3: note: candidate constructornot viable: no known conversion from '(lambda at ../src/backend/jit/llvm/llvmjit_wrap.cpp:57:16)' to 'GetMemoryManagerFunction'(aka 'unique_function<std::unique_ptr<RuntimeDyld::MemoryManager> (const MemoryBuffer &)>') for2nd argument > 58 | RTDyldObjectLinkingLayer(ExecutionSession &ES, > | ^ > 59 | GetMemoryManagerFunction GetMemoryManager); > | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > /opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:37:16: note: candidate constructor(the implicit copy constructor) not viable: requires 1 argument, but 2 were provided > 37 | class LLVM_ABI RTDyldObjectLinkingLayer > | ^~~~~~~~~~~~~~~~~~~~~~~~ > > I gather you're not seeing that? I do not - I dropped the patch into my Gentoo package build for 17.6 and it has been working fine when building with gcc-15.2 and the jit part with clang-21.1.0. I also just rebuilt everything with clang-21.1.1 (no gcc) and also do not see the problem. This is on amd64 Linux. The following commit seems to be involved: https://github.com/postgres/postgres/commit/9044fc1d45a0212fd123bd8f364eac058f60fed7 which is also in my 17.6 tree. I verified that the new SectionMemoryManager.cpp is compiled without error. Since you're using homebrew I guess this is triggered by the __aarch64__ guard in src/include/jit/llvmjit_backport.h. Unfortunately this is as far as I can help with this. Please let me know if there is anything else I can do. cheers Holger
Вложения
On 2025-09-12 09:47, Holger Hoffstätte wrote:
> On 2025-09-12 08:36, Peter Eisentraut wrote:
>> In my testing with LLVM 21, I'm getting an additional error:
>>
>> ../src/backend/jit/llvm/llvmjit_wrap.cpp:56:18: error: no matching constructor for initialization of
'llvm::orc::RTDyldObjectLinkingLayer'
>> 56 | return wrap(new llvm::orc::RTDyldObjectLinkingLayer(
>> | ^
>> 57 | *unwrap(ES), [] { return std::make_unique<llvm::backport::SectionMemoryManager>(nullptr,
true);}));
>> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> /opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:58:3: note: candidate
constructornot viable: no known conversion from '(lambda at ../src/backend/jit/llvm/llvmjit_wrap.cpp:57:16)' to
'GetMemoryManagerFunction'(aka 'unique_function<std::unique_ptr<RuntimeDyld::MemoryManager> (const MemoryBuffer &)>')
for2nd argument
>> 58 | RTDyldObjectLinkingLayer(ExecutionSession &ES,
>> | ^
>> 59 | GetMemoryManagerFunction GetMemoryManager);
>> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> /opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:37:16: note: candidate
constructor(the implicit copy constructor) not viable: requires 1 argument, but 2 were provided
>> 37 | class LLVM_ABI RTDyldObjectLinkingLayer
>> | ^~~~~~~~~~~~~~~~~~~~~~~~
>>
>> I gather you're not seeing that?
I temporarily removed the __arch64__ guard and can reproduce the above error,
using either gcc-15 or clang-21. It seems this llvm monkey patch backport thing
is affected by the following commit in llvm-21:
https://github.com/llvm/llvm-project/commit/cd585864c0bbbd74ed2a2b1ccc191eed4d1c8f90
Trying to figure out how to adapt the code..
chers
Holger
On 2025-09-12 14:47, Holger Hoffstätte wrote: <snip> > I temporarily removed the __arch64__ guard and can reproduce the above error, > using either gcc-15 or clang-21. It seems this llvm monkey patch backport thing > is affected by the following commit in llvm-21: > https://github.com/llvm/llvm-project/commit/cd585864c0bbbd74ed2a2b1ccc191eed4d1c8f90 > > Trying to figure out how to adapt the code.. Try the attached patch on your homebrew setup. This compiles and passes "make check", though I do not think it runs any jit tests. So intead I dropped this into my 17.6, disabled the __arch64__ guard to make sure I get the backport class, rebuilt and ran the jit on/off example from the docs. This showed the expected performance difference with jit=on vs. off. cheers Holger
Вложения
On 12.09.25 09:47, Holger Hoffstätte wrote: >> As a small style request, I would flip the conditional around so >> that the new code appears first. I see that we don't do this very >> consistently in the existing code, but maybe we can start a new >> trend. ;-) > I knew this would come up since I pondered the same thing :D > As you said, the existing code is not consistent, but I can switch this > to let the new code appear first, if you prefer. > A new patch is attached. I have committed this one. For the remaining issue on arm, let's wait for some additional reviews.
Hi,
I've tested the patch: it fixes the compilation on arm and I'm
correctly linked to llvm21:
ldd /var/lib/postgresql/.local/lib/llvmjit.so | grep libLLVM
libLLVM.so.21.1 => /lib/aarch64-linux-gnu/libLLVM.so.21.1
(0x0000ec34112d0000)
Testing the query that used to trigger the jit segfault, the llvm
backport is working as expected.
pgbench -i --partitions=256
psql options=-cjit_above_cost=0 -c 'SELECT count(bid) from pgbench_accounts;'
count
--------
100000
As a comparison, if I forcefully disable
USE_LLVM_BACKPORT_SECTION_MEMORY_MANAGER, I have the expected crash as
the issue is still present in LLVM (the fix would be to switch to
JITLink).
psql options=-cjit_above_cost=0 -c 'SELECT count(bid) from pgbench_accounts;'
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
connection to server was lost