Обсуждение: Detoasting and memory usage

Hi,
    I discovered a slightly misleading (or at least non-intuitive to
me) behaviour of the pg_detoast_datum call while processing variable length
objects.   I was reading a large number of text fields on fetched
results and noticed some strange memory behavior.   I was seeing that in
some situations, memory usage would shoot up dramatically, while in
others it would not.  Puzzled, I tracked it down my usage of
DatumGetTextP(), which eventually reduces to pg_detoast_datum.  As it turns
out, most of the text fields I was processing were small enough not to
be toasted and everything was OK.   For the larger ones, pg_detoast_datum
would give me a palloced copy, and since I was making all these calls in
the same memory context and not pfreeing the results, I had a memory
leak.   Just pfreeing the result would obviously not work since
pg_detoast_datum doesn't return a palloced result in most cases (i.e.
smaller varlength objects).  I ended up using DatumGetTextPCopy (to
ensure that the result was palloced) with a pfree and all problems were
solved.

I guess my situation is an odd fringe case that probably doesn't happen
too much, but I thought that knowledge of it might be useful to someone
somewhere.

    -Aaron

apb18@cornell.edu wrote:
> I guess my situation is an odd fringe case that probably doesn't happen
> too much, but I thought that knowledge of it might be useful to someone
> somewhere.

See PG_FREE_IF_COPY in fmgr.h

/*
  * Support for cleaning up detoasted copies of inputs.  This must only
  * be used for pass-by-ref datatypes, and normally would only be used
  * for toastable types.  If the given pointer is different from the
  * original argument, assume it's a palloc'd detoasted copy, and pfree
  * it.
  *
  * NOTE: most functions on toastable types do not have to worry about
  * this, but we currently require that support functions for indexes
  * not leak memory.
  */
#define PG_FREE_IF_COPY(ptr,n) \
     do { \
         if ((Pointer) (ptr) != PG_GETARG_POINTER(n)) \
             pfree(ptr); \
     } while (0)

Maybe you can use that or do something similar?

HTH,

Joe

Hmm.. I can't believe I missed that.  While that exact macro wouldn't
apply in my situation (because I fetch the value from a field in a tuple and
not the argument of a function), the underlying concept is the same as
what I would need.   Now I do not need to feel the wrath of unnecessary
memory allocation/deallocation for the non-toasted case.  Excellent.
Thanks very much!

    -Aaron

> #define PG_FREE_IF_COPY(ptr,n) \
>      do { \
>          if ((Pointer) (ptr) != PG_GETARG_POINTER(n)) \
>              pfree(ptr); \
>      } while (0)
>
> Maybe you can use that or do something similar?
>
> HTH,
>
> Joe