Re: RFI: Extending the TOAST Pointer

On Tue, 23 May 2023 at 18:34, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Thu, May 18, 2023 at 8:06 AM Matthias van de Meent
> <boekewurm+postgres@gmail.com> wrote:
> > This enum still has many options to go before it exceeds the maximum
> > of the uint8 va_tag field. Therefore, I don't think we have no disk
> > representations left, nor do I think we'll need to add another option
> > to the ToastCompressionId enum.
> > As an example, we can add another VARTAG option for dictionary-enabled
> > external toast; like what the pluggable toast patch worked on. I think
> > we can salvage some ideas from that patch, even if the main idea got
> > stuck.
>
> Adding another VARTAG option is somewhat different from adding another
> ToastCompressionId. I think that right now we have embedded in various
> places the idea that VARTAG_EXTERNAL is the only thing that shows up
> on disk, and we'd need to track down all such places and adjust them
> if we add other VARTAG types in the future. Depending on how it is to
> be used, adding a new ToastCompressionId might be less work. However,
> I don't think we can use the possibility of adding a new VARTAG value
> as a reason why it's OK to use up the last possible ToastCompressionId
> value for something non-extensible.

I think you might not have picked up on what I was arguing for, but I
agree with what you just said.

My comment on not needing to invent a new ToastCompressionId was on
the topic of adding capabilities^ to toast pointers that do things
differently than the current TOAST and need more info than just sizes,
2x 32-bit ID and a compression algorithm.

^ capabilities such as compression dictionaries (which would need to
store a dictionary ID in the pointer), TOAST IDs that are larger than
32 bits, and other such advances.

Kind regards,

Matthias van de Meent
Neon, Inc.