Memory Accounting
От | Jeff Davis |
---|---|
Тема | Memory Accounting |
Дата | |
Msg-id | 027a129b8525601c6a680d27ce3a7172dab61aab.camel@j-davis.com обсуждение исходный текст |
Ответы |
Re: Memory Accounting
Re: Memory Accounting |
Список | pgsql-hackers |
Previous discussion: https://postgr.es/m/1407012053.15301.53.camel%40jeff-desktop This patch introduces a way to ask a memory context how much memory it currently has allocated. Each time a new block (not an individual chunk, but a new malloc'd block) is allocated, it increments a struct member; and each time a block is free'd, it decrements the struct member. So it tracks blocks allocated by malloc, not what is actually used for chunks allocated by palloc. The purpose is for Memory Bounded Hash Aggregation, but it may be useful in more cases as we try to better adapt to and control memory usage at execution time. I ran the same tests as Robert did before[1] on my laptop[2]. The only difference is that I also set max_parallel_workers[_per_gather]=0 to be sure. I did 5 runs, alternating between memory-accounting and master, and I got the following results for "elapsed" (as reported by trace_sort): regression=# select version, min(s), max(s), percentile_disc(0.5) within group (order by s) as median, avg(s)::numeric(10,2) from tmp group by version; version | min | max | median | avg -------------------+-------+-------+--------+------- master | 13.92 | 14.40 | 14.06 | 14.12 memory accounting | 13.43 | 14.46 | 14.11 | 14.09 (2 rows) So, I don't see any significant performance impact for this patch in this test. That may be because: * It was never really significant except on PPC64. * I changed it to only update mem_allocated for the current context, not recursively for all parent contexts. It's now up to the function that reports memory usage to recurse or not (as needed). * A lot of changes to sort have happened since that time, so perhaps it's not a good test of memory contexts any more. pgbench didn't show any slowdown either. I also did another test with hash aggregation that uses significant memory (t10m is a table with 10M distinct values and work_mem is 1GB): postgres=# select (select (i, count(*)) from t10m group by i having count(*) > n) from (values(1),(2),(3),(4),(5)) as s(n); I didn't see any noticable difference there, either. Regards, Jeff Davis [1] https://postgr.es/m/CA%2BTgmobnu7XEn1gRdXnFo37P79bF%3DqLt46%3D37ajP3Cro9dBRaA%40mail.gmail.com [2] Linux jdavis 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Вложения
В списке pgsql-hackers по дате отправления: