Re: failed assertion in toasting code

"Sergey E. Koposov" <math@sai.msu.ru> writes:

> Hello -hackers,
>
> In the process of converting a multi-Tb datadabe from 8.2 to 8.3, Postgres 8.3
> died at the failed assertion:
>
> TRAP: FailedAssertion("!(((toast_pointer).va_extsize < (toast_pointer).va_rawsize - ((int32) sizeof(int32))))", File:
"tuptoaster.c",Line: 1134)
 
> LOG:  server process (PID 8874) was terminated by signal 6: Aborted
> LOG:  terminating any other active server processes
> LOG:  all server processes terminated; reinitializing
> LOG:  database system was interrupted; last known up at 2008-02-20 07:43:00 MSK
> LOG:  database system was not properly shut down; automatic recovery in progress
> LOG:  redo starts at 78/BA00E060
> LOG:  record with zero length at 78/BC7A5FA8
> LOG:  redo done at 78/BC7A5F78
> LOG:  last completed transaction was at log time 2008-02-20 07:43:03.292665+03
> LOG:  autovacuum launcher started
> LOG:  database system is ready to accept connections
>
> Unfortunately I cannot tell much more right now (I don't have the exact name of
> the table even), although I suspect which column is that (because I have only
> one column which is subject to toasting).
>
> I was doing basically pg_dumpall_8.2 | psql_8.3 I can say that the tables are
> large ~ 100-500 Gb in size and contain the column with the image in a special
> type "image", which is toasted. cas=# \d sdssdr5.frame
>          Table "sdssdr5.frame"
>  Column  |       Type       | Modifiers
> ---------+------------------+-----------
>  fieldid | bigint           | not null
> ..................
>  htmid   | bigint           | not null
>  img     | image            | not null
>
> CREATE TYPE image (
>         INPUT = image_in,
>         OUTPUT = image_out,
>         INTERNALLENGTH = -1,
>         STORAGE = external
> );

You aren't doing anything funny in the image_in function to generate
compressed varlenas manually are you?

Assuming not then it must be a case where we're saving less than 4 bytes and
that's appearing as a saving in one place but then not somewhere else once you
take into account the headers. Except I've just gone through the code looking
for that kind of error and didn't spot it.

I'll keep looking (or someone else will probably spot it before I do anyways)
but if these images are mostly incompressible data you would probably be
better off marking the columns as storage "external" so Postgres just toasts
them as-is instead of trying to compress them first with:
ALTER column SET STORAGE EXTERNAL

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com Ask me about EnterpriseDB's On-Demand Production
Tuning