Re: copy.c allocation constant

On Wed, Jan 24, 2018 at 1:43 PM, Andres Freund <andres@anarazel.de> wrote:
> Indeed. Don't think RAW_BUF_SIZE is quite big enough for that on most
> platforms though. From man mallopt:
>  Balancing  these  factors  leads  to a default setting of 128*1024 for the M_MMAP_THRESHOLD parameter.
> Additionally, even when malloc() chooses to use mmap() to back an
> allocation, it'll still needs a header to know the size of the
> allocation and such. So exactly using a size of a multiple of 4KB will
> still leave you with wasted space.  Due to the latter I can't see it
> mattering whether or not we add +1 to a power-of-two size.

Well, it depends on how it works.  dsa_allocate, for example, never
adds a header to the size of the allocation.  Allocations < 8kB are
bucketed by size class and stored in superblocks carved up into
equal-sized chunks.  Allocations > 8kB are rounded to a multiple of
the 4kB page size and we grab that many consecutive free pages.  I
didn't make those behaviors up; I copied them from elsewhere.  Some
other allocator I read about did small-medium-large allocations: large
with mmap(), medium with multiples of the page size, small with
closely-spaced size classes.

It doesn't seem like a particularly good idea to take a 64kB+1 byte
allocation, stick a header on it, and pack it tightly up against other
allocations on both sides.  Seems like that could lead to
fragmentation problems.  Is that really what it does?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company