On Sat, Feb 17, 2024 at 1:54 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> A variant of this approach:
>
> 5. Re-map the shared_buffers when needed.
>
> Between transactions, a backend should not hold any buffer pins. When
> there are no pins, you can munmap() the shared_buffers and mmap() it at
> a different address.
I really like this idea, but I think Andres has latched onto the key
issue, which is that it supposes that the underlying shared memory
object upon which shared_buffers is based can be made bigger and
smaller, and that doesn't work for anonymous mappings AFAIK.
Maybe that's not really a problem any more, though. If we don't depend
on the address of shared_buffers anywhere, we could move it into a
DSM. Now that the stats collector uses DSM, it's surely already a
requirement that DSM works on every machine that runs PostgreSQL.
We'd still need to do something about the buffer mapping table,
though, and I bet dshash is not a reasonable answer on performance
grounds.
Also, it would be nice if the granularity of resizing could be
something less than a whole transaction, because transactions can run
for a long time. We don't really need to wait for a transaction
boundary, probably -- a time when we hold zero buffer pins will
probably happen a lot sooner, and at least some of those should be
safe points at which to remap.
Then again, somebody can open a cursor, read from it until it holds a
pin, and then either idle the connection or make it do arbitrary
amounts of unrelated work, forcing the remapping to be postponed for
an arbitrarily long time. But some version of this problem will exist
in any approach to this problem, and long-running pins are a nuisance
for other reasons, too. We probably just have to accept this sort of
issue as a limitation of our implementation.
--
Robert Haas
EDB: http://www.enterprisedb.com