Re: Relation extension scalability

I think we need to start testing these patches not only in terms of
how *fast* they are but how *large* the relation ends up being when
we're done.  A patch that inserts the rows slower but the final
relation is smaller may be better overall.  Can you retest v13, v14,
and master, and post not only the timings but the relation size
afterwards?  And maybe post the exact script you are using?

Maybe something like this would help:

    if (needLock)
    {
        if (!use_fsm)
            LockRelationForExtension(relation, ExclusiveLock);
        else if (!ConditionLockRelationForExtension(relation, ExclusiveLock))
        {
            BlockNumber     last_blkno = RelationGetNumberOfBlocks(relation);

            targetBlock = GetPageWithFreeSpaceExtended(relation,
last_blkno, len + saveFreeSpace);
            if (targetBlock != InvalidBlockNumber)
                goto loop;

            LockRelationForExtension(relation, ExclusiveLock);
            targetBlock = GetPageWithFreeSpace(relation, len + saveFreeSpace);
            if (targetBlock != InvalidBlockNumber)
            {
                UnlockRelationForExtension(relation, ExclusiveLock);
                goto loop;
            }
            RelationAddExtraBlocks(relation, bistate);
        }
    }

I think this is better than what you had before with lastValidBlock,
because we're actually interested in searching the free space map at
the *very end* of the relation, not wherever the last target block
happens to have been.

We could go further still and have GetPageWithFreeSpace() always
search the last, say, two pages of the FSM in all cases.  But that
might be expensive. The extra call to RelationGetNumberOfBlocks seems
cheap enough here because the alternative is to wait for a contended
heavyweight lock.