Re: GiST memory allocation

Neil Conway <neilc@samurai.com> writes:
> AFAICS, GiST doesn't take any advantage of the palloc() infrastructure 
> beyond treating palloc() as a better malloc().

This is pretty much true of all the index AMs, I think.  I looked
briefly at using a short-term memory context in the btree code, but
gave it up after seeing that many of the allocations have to survive
from one index_getnext call to the next.  It didn't seem that there
would be much net gain in notational simplicity.

> (the observation here is that 99% of the allocations done by 
> gist.c are for internal use only -- we rarely allocate anything that 
> needs to live longer than the current GiST API call).

You sure about that?  In btree quite a lot of the allocs need to survive
across the current scan.

> One relatively easy fix would be to create a per-backend "GiST context" 
> as a static variable in gist.c.
> ...
> One downside is that the GiST API won't be reentrant, but I don't think 
> that's a major problem.

I think this is not acceptable.  It certainly wouldn't be acceptable for
btree --- you couldn't even use a PL-language function as an index
operator, because the PL itself would need to do system catalog accesses
that could result in re-entrant btree scans.  If you got away with it
for GiST it would only be because GiST is a stepchild that the system
doesn't depend on.  That doesn't sound like the direction to go in.

You could look at creating per-scan and/or per-tuple-cycle memory
contexts during gistbeginscan, keeping pointers to them in the indexscan
structure.  However this only works for scans --- not sure if there are
any data structures that are common to scans and the non-scan operations.
        regards, tom lane