Re: VACUUM memory management

On 2019-Dec-09, Ibrar Ahmed wrote:

> Hi,
>
> The memory consumption of VACUUM has some issues and could be improved.
> Some of its limitations are recorded in the comments of the “vacuumlazy.c”
> file. The current design of VACUUM memory usage is that it stores the TID
> in a fixed-size array which is allocated at the start, based upon
> maintenance_work_mem. There are three problems with that design

Did you see this thread?
https://postgr.es/m/CAGTBQpbDCaR6vv9=scXzuT8fSbckf=a3NgZdWFWZbdVugVht6Q@mail.gmail.com

Robert: "What I think we need to do is make some provision to initially
allocate only a small amount of memory and then grow the allocation
later if needed. For example, instead of having
vacrelstats->dead_tuples be declared as ItemPointer, declare it as
ItemPointer * and allocate the array progressively in segments. I'd
actually argue that the segment size should be substantially smaller
than 1 GB, like say 64MB; there are still some people running systems
which are small enough that allocating 1 GB when we may need only 6
bytes can drive the system into OOM."

I change vacrelstats->dead_tuples to ItemPointer * and allocate small memory and added more when needed. What I did new is divide maintenance_work_mem in fixed-size chunks.