Re: refactoring relation extension and BufferAlloc(), faster COPY

Hi,

Attached is v6. Changes:

- Try to address Melanie and Horiguchi-san's review. I think there's one or
  two further things that need to be done

- Avoided inserting newly extended pages into the FSM while holding a buffer
  lock. If we need to do so, we now drop the buffer lock and recheck if there
  still is space (very very likely). See also [1]. I use the infrastructure
  introduced over in that in this patchset.

- Lots of comment and commit message polishing. More needed, particularly for
  the latter, but ...

- Added a patch to fix the pre-existing undefined behaviour in localbuf.c that
  Melanie pointed out. Plan to commit that soon.

- Added a patch to fix some pre-existing DO_DB() format code issues. Plan to
  commit that soon.


I did some benchmarking on "bufmgr: Acquire and clean victim buffer
separately" in isolation. For workloads that do a *lot* of reads, that proves
to be a substantial benefit on its own.  For the, obviously unrealistically
extreme, workload of N backends doing
  SELECT pg_prewarm('pgbench_accounts', 'buffer');
in a scale 100 database (with a 1281MB pgbench_accounts) and shared_buffers of
128MB, I see > 2x gains at 128, 512 clients.  Of course realistic workloads
will have much smaller gains, but it's still good to see.


Looking at the patchset, I am mostly happy with the breakdown into individual
commits. However "bufmgr: Move relation extension handling into
ExtendBufferedRel{By,To,}" is quite large. But I don't quite see how to break
it into smaller pieces without making things awkward (e.g. due to static
functions being unused, or temporarily duplicating the code doing relation
extensions).


Greetings,

Andres Freund

[1] https://www.postgresql.org/message-id/20230325025740.wzvchp2kromw4zqz%40awork3.anarazel.de