Re: [PoC] Improve dead tuple storage for lazy vacuum
От | John Naylor |
---|---|
Тема | Re: [PoC] Improve dead tuple storage for lazy vacuum |
Дата | |
Msg-id | CAFBsxsGv9GF6Vvzi-c_xwoFi1mbxY_qgy_Gx39M5KHHCfWCMKA@mail.gmail.com обсуждение исходный текст |
Ответ на | Re: [PoC] Improve dead tuple storage for lazy vacuum (Masahiko Sawada <sawada.mshk@gmail.com>) |
Ответы |
Re: [PoC] Improve dead tuple storage for lazy vacuum
|
Список | pgsql-hackers |
On Mon, Dec 19, 2022 at 2:14 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Tue, Dec 13, 2022 at 1:04 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > Looking at other code using DSA such as tidbitmap.c and nodeHash.c, it
> > seems that they look at only memory that are actually dsa_allocate'd.
> > To be exact, we estimate the number of hash buckets based on work_mem
> > (and hash_mem_multiplier) and use it as the upper limit. So I've
> > confirmed that the result of dsa_get_total_size() could exceed the
> > limit. I'm not sure it's a known and legitimate usage. If we can
> > follow such usage, we can probably track how much dsa_allocate'd
> > memory is used in the radix tree.
>
> I've experimented with this idea. The newly added 0008 patch changes
> the radix tree so that it counts the memory usage for both local and
> shared cases. As shown below, there is an overhead for that:
>
> w/o 0008 patch
> 298453544 | 282
> w/0 0008 patch
> 293603184 | 297
> > seems that they look at only memory that are actually dsa_allocate'd.
> > To be exact, we estimate the number of hash buckets based on work_mem
> > (and hash_mem_multiplier) and use it as the upper limit. So I've
> > confirmed that the result of dsa_get_total_size() could exceed the
> > limit. I'm not sure it's a known and legitimate usage. If we can
> > follow such usage, we can probably track how much dsa_allocate'd
> > memory is used in the radix tree.
>
> I've experimented with this idea. The newly added 0008 patch changes
> the radix tree so that it counts the memory usage for both local and
> shared cases. As shown below, there is an overhead for that:
>
> w/o 0008 patch
> 298453544 | 282
> w/0 0008 patch
> 293603184 | 297
This adds about as much overhead as the improvement I measured in the v4 slab allocator patch. That's not acceptable, and is exactly what Andres warned about in
I'm guessing the hash join case can afford to be precise about memory because it must spill to disk when exceeding workmem. We don't have that design constraint.
--
John Naylor
EDB: http://www.enterprisedb.com
--
John Naylor
EDB: http://www.enterprisedb.com
В списке pgsql-hackers по дате отправления: