Re: GIN improvements part 1: additional information

Aside from the pg_upgrade discussion, here's an updated version of the
patch, rebased over master. It also contains some other misc refactoring
I've done while reading through the patch. I haven't tested this much, I
may well have also broken something, but I wanted to post an update
before the weekend.

Thinking about the page format, I think we should start using the
pd_lower/upper pointers in the data page format. For a non-leaf page,
pd_upper would always point to the beginning of the special area, and
pd_lower would indicate the end of PostingItems. For a leaf page,
pd_lower would indicate the end of the compressed posting list, and
pd_upper would point to the "leaf-index" at the end of the page. That
matches the standard page layout in the sense that the space between
pd_lower and pd_upper is free, although the data stored in the non-free
areas would be quite different. That would allow us to mark full-page
images with buffer_std, allowing the "gap" to be left out. I think that
would be a more natural way to keep track of the used/unused space on
the page, anyway, compared to the current maxoff/endoffset field in the
special area.

In the attached patch, I in fact already did that for data leaf pages,
but didn't change the format of non-leaf pages yet. If we want to
support pg_upgrade, we might want to refrain from changing the non-leaf
format.

- Heikki