BUG #18259: Assertion in ExtendBufferedRelLocal() fails after no-space-left condition

The following bug has been logged on the website:

Bug reference:      18259
Logged by:          Alexander Lakhin
Email address:      exclusion@gmail.com
PostgreSQL version: 16.1
Operating system:   Ubuntu 22.04
Description:

The following script:
mkdir /tmp/100m
sudo mount -t tmpfs -o size=100M tmpfs /tmp/100m
export PGDATA=/tmp/100m/tmpdb

initdb
pg_ctl -l server.log start

cat << 'EOF' | psql
CREATE UNLOGGED TABLE filler(a int, b text STORAGE plain);
INSERT INTO filler SELECT g, repeat('x', 1000) FROM generate_series(1,
50000) g;
CREATE TEMP TABLE tbl(a int);
INSERT INTO tbl SELECT g FROM generate_series(1, 200000) g;
INSERT INTO tbl SELECT g FROM generate_series(1, 200000) g;
DROP TABLE filler;
INSERT INTO tbl SELECT g from generate_series(1, 200000) g;
EOF

triggers an assertion failure following "no space left" errors:
...
CREATE TABLE
ERROR:  could not extend file "base/5/t3_16391": No space left on device
HINT:  Check free disk space.
ERROR:  could not extend file "base/5/t3_16391": No space left on device
HINT:  Check free disk space.
DROP TABLE
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
connection to server was lost
TRAP: failed Assert("buf_state & BM_TAG_VALID"), File: "localbuf.c", Line:
390, PID: 25978

The call stack of the failure is:
ExtendBufferedRelLocal at localbuf.c:391:4
ExtendBufferedRelCommon at bufmgr.c:1801:17
ExtendBufferedRelBy at bufmgr.c:862:9
RelationAddBlocks at hio.c:342:16
RelationGetBufferForTuple at hio.c:768:11
heap_insert at heapam.c:1862:11
heapam_tuple_insert at heapam_handler.c:253:2
table_tuple_insert at tableam.h:1402:1
ExecInsert at nodeModifyTable.c:1138:21
ExecModifyTable at nodeModifyTable.c:3810:12
ExecProcNodeFirst at execProcnode.c:465:1
ExecProcNode at executor.h:274:1
ExecutePlan at execMain.c:1670:10
standard_ExecutorRun at execMain.c:365:3
ExecutorRun at execMain.c:310:1
ProcessQuery at pquery.c:165:5
PortalRunMulti at pquery.c:1277:5
PortalRun at pquery.c:795:5
exec_simple_query at postgres.c:1274:10
PostgresMain at postgres.c:4641:27
ExitPostmaster at postmaster.c:5047:1
BackendStartup at postmaster.c:4196:5
ServerLoop at postmaster.c:1788:6
PostmasterMain at postmaster.c:1466:11

The first bad commit for this anomaly is 31966b15 (and exactly that commit
added the Assert).

With debug logging added in this code within ExtendBufferedRelLocal():
        if (found)
        {
            BufferDesc *existing_hdr =
GetLocalBufferDescriptor(hresult->id);
            uint32      buf_state;

            UnpinLocalBuffer(BufferDescriptorGetBuffer(victim_buf_hdr));

            existing_hdr = GetLocalBufferDescriptor(hresult->id);
            PinLocalBuffer(existing_hdr, false);
            buffers[i] = BufferDescriptorGetBuffer(existing_hdr);

            buf_state = pg_atomic_read_u32(&existing_hdr->state);
            Assert(buf_state & BM_TAG_VALID);
            Assert(!(buf_state & BM_DIRTY));
            buf_state &= BM_VALID;
            pg_atomic_unlocked_write_u32(&existing_hdr->state, buf_state);
...
I see that it reached for the second INSERT (and NOSPC error) with
existing_hdr->state == 0x2040000, but for the third INSERT I observe
state == 0x0.