Re: GIST and TOAST

>> Does every data type define a compress/decompress method? Even if it's not a
>> data type that normally gets very large?
>
> Yes, any GiST opclass should have such methods. In trivial case it just returns
> input value. As I remember, only R-Tree emulation over boxes, contrib/seg and
> contrib/cube have simple compress method.

Hm, if they just return the original datum without detoasting it then it could
be an issue. I'll check.

>> Well we cheated a bit and had heap/index_form_tuple convert the data to packed
>> format. This saves having to push small tuples through the toaster. So now
>> tuples can magically become toasted as soon as they go into a tuple even if
>> they never get pushed through the toaster. 
>
> Ok, it should be safe for GiST except some possible memory management issue.
> index_form_tuple in GiST works in GiST's memory context which is short-live. Is
> it possible issue for your patch? BTW, that's connected to GIN too.

index_form_tuple doesn't leak memory. packed varlena format just has a shorter
header so it can store the header and then copy the data to the new location.
It doesn't have to create a copy of the data (except in the tuple, obviously).

But it means index_getattr can return a toasted tuple. I see several calls to
index_getattr that immediately put the datum into a GISTENTRY and call support
functions like the union function. For example gistMakeUnionItVec does this.

>> So it's perfectly safe to just use DatumGetType and PG_GETARG_TYPE instead of
>> using DatumGetPointer and PG_GETARG_POINTER and having to manually cast
>> everywhere, no? It seems like there's a lot of extra pain to maintain the code
>> in the present style with all the manual casts.
>
> Of course, I agree. Just PG_FREE_IF_COPY is extra call in support methods.

Well if you're doing everything in short-lived memory contexts then we don't
even need this. btree procedures do it because the btree code expects to be
able to do comparisons without having to set up short-lived memory contexts.

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com