Re: pgsql: Generational memory allocator

Simon Riggs <simon@2ndquadrant.com> writes:
> On 26 November 2017 at 08:46, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I've confirmed that the attached is sufficient to stop the valgrind crash
>> on my machine.  But as I said, I think we should be more aggressive at
>> resizing the buffer, to reduce resize cycles.  I'm inclined to start out
>> with a buffer size of 128 or 256 or so bytes and double it when needed.
>> Anybody have a feeling for a typical size for the "main data" part
>> of a WAL record?

> We reuse the buffer and only pfree/palloc when we need to enlarge the
> buffer, so not sure we need to do the doubling thing and it probably
> doesn't matter what the typical size is.

Well, I'm concerned about the possibility of a lot of palloc thrashing
if the first bunch of records it reads happen to have steadily increasing
sizes.  However, rather than doubling, it might be sufficient to set a
robust minimum on the first allocation, ie use something along the lines
of Max(1024, MAXALIGN(state->main_data_len)).
        regards, tom lane