Re: Oom on temp (un-analyzed table caused by JIT) V16.1 [ NOT Fixed ]

Поиск
Список
Период
Сортировка
От Kirk Wolak
Тема Re: Oom on temp (un-analyzed table caused by JIT) V16.1 [ NOT Fixed ]
Дата
Msg-id CACLU5mSN=VGmkAWyhBPCokZiBG3sdXz-yXZ18CY8iMBiLw-W8g@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Oom on temp (un-analyzed table caused by JIT) V16.1 [ NOT Fixed ]  (Michael Banck <mbanck@gmx.net>)
Список pgsql-hackers
On Thu, Feb 22, 2024 at 4:49 PM Michael Banck <mbanck@gmx.net> wrote:
Hi,

On Wed, Jan 24, 2024 at 02:50:52PM -0500, Kirk Wolak wrote:
> On Mon, Jan 22, 2024 at 1:30 AM Kirk Wolak <wolakk@gmail.com> wrote:
> > On Fri, Jan 19, 2024 at 7:03 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> >> > On 19 Jan 2024, at 23:09, Kirk Wolak <wolakk@gmail.com> wrote:
> > Thank you, that made it possible to build and run...
> > UNFORTUNATELY this has a CLEAR memory leak (visible in htop)
> > I am watching it already consuming 6% of my system memory.
...
I had a look at this (and blogged about it here[1]) and was also
wondering what was going on with 17devel and the recent back-branch
releases, cause I could also reproduce those continuing memory leaks.

Adding some debug logging when llvm_inline_reset_caches() is called
solves the mystery: as you are calling a function, the fix that is in
17devel and the back-branch releases is not applicable and only after
the function returns llvm_inline_reset_caches() is being called (as
llvm_jit_context_in_use_count is greater than zero, presumably, so it
never reaches the call-site of llvm_inline_reset_caches()).

If you instead run your SQL in a DO-loop (as in the blog post) and not
as a PL/PgSQL function, you should see that it no longer leaks. This
might be obvious to some (and Andres mentioned it in
https://www.postgresql.org/message-id/20210421002056.gjd6rpe6toumiqd6%40alap3.anarazel.de)
but it took me a while to figure out/find.

Thanks for confirming.  Inside a routine is required.  But disabling JIT was good enough for us.

Curious if there was another way to end up calling the cleanup?  But it had that "feel" that the context of the cleanup was
being lost between iterations of the loop? 

В списке pgsql-hackers по дате отправления:

Предыдущее
От: "Shankaran, Akash"
Дата:
Сообщение: RE: Popcount optimization using AVX512
Следующее
От: Peter Eisentraut
Дата:
Сообщение: Re: Idea Feedback: psql \h misses -> Offers Links?