Re: [postgis-users] ERROR: array size exceeds themaximumallowed(134217727)

On Mon, Mar 22, 2010 at 4:45 PM, Paul Ramsey <pramsey@cleverelephant.ca> wrote:
> Did you already try replacing your postgis functions with array_agg
> calls to see if we can push the problem back over the fence to pgsql
> land?

On Sat, Mar 20, 2010 at 9:17 AM, Mike Leahy <mgleahy@alumni.uwaterloo.ca> wrote:
> Running this query on various data will produce one of two results.  One is
> the error mentioned in the subject (ERROR:  array size exceeds the maximum
> allowed (134217727)).  I can find very little information on this error.  The
> other outcome is that it often causes the PostgreSQL backend to segfault (see
> gdboutput.txt).

gdboutput.txt:

Program terminated with signal 11, Segmentation fault.
#0  0x00007fa4be23615b in pfree () from
/usr/lib/postgresql/8.4/bin/postgres
#1  0x00007fa4be18091b in makeMdArrayResult () from
/usr/lib/postgresql/8.4/bin/postgres
#2  0x00007fa4b7f038bc in pgis_accum_finalfn () from
/usr/lib/postgresql/8.4/lib/postgis-1.5.so
#3  0x00007fa4b7f039ee in pgis_geometry_union_finalfn () from
/usr/lib/postgresql/8.4/lib/postgis-1.5.so

Any chance you can generate one of these seg faults from a build with
symbols and with assertions enabled? It might catch the problem
earlier and provide more info.

IIRC there were some memory management changes which required changes
to array_agg() and which had some risk of causing problems for other
sites with similar coding. Is it possible your'e missing some of these
changes? I'm having trouble tracking them all down but at least
there's these:


commit 3d332de2eab8a01c0ef3f58ea69de2010fe8a1e1
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Thu Jul 23 20:45:27 2009 +0000
   In a non-hashed Agg node, reset the "aggcontext" at group
boundaries, instead   of individually pfree'ing pass-by-reference transition values.  This should   be at least as fast
asthe prior coding, and it has the major advantage of   clearing out any working data an aggregate function may have
storedin or   underneath the aggcontext.  This avoids memory leakage when an aggregate   such as array_agg() is used in
GROUPBY mode.  Per report from Chris Spotts. 
   Back-patch to 8.4.  In principle the problem could arise in prior versions,   but since they didn't have array_agg
theissue seems not critical. 

commit 7f83b61cc26eeac0c5a09add49f6cf899f87fc0b
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Sat Jun 20 18:45:28 2009 +0000
   Fix things so that array_agg_finalfn does not modify or free its input   ArrayBuildState, per trouble report from
MerlinMoncure.  By adopting   this fix, we are essentially deciding that aggregate final-functions   should not modify
theirinputs ever.  Adjust documentation and comments   to match that conclusion. 

--
greg