Re: [GENERAL] Invalid field size

    Tom Lane wrote:

> Moreno Andreo <moreno.andreo@evolu-s.it> writes:
> > So the hint is to abandon manual COPY and let pg_dump do the hard work?
>
> If it is a newline-conversion problem, compressed pg_dump archives would
> be just as subject to corruption as your binary COPY file is.

It's mentioned in [1] that the signature at the beginning of these files
embed a CRLF to detect this newline-conversion problem early on,
so I would expect COPY IN to stumble on a corrupted signature
and abort earlier in the process, if that conversion occurred.
Instead the report says it fails after a number of tuples:

> ERROR:  invalid field size
> CONTEXT:  COPY tab, line 619, column thumbnail

[1] https://www.postgresql.org/docs/current/static/sql-copy.htm

The file header consists of 15 bytes of fixed fields, followed by a
variable-length header extension area. The fixed fields are:

Signature
  11-byte sequence PGCOPY\n\377\r\n\0 — note that the zero byte is a
  required part of the signature. (The signature is designed to allow
  easy identification of files that have been munged by a
  non-8-bit-clean transfer. This signature will be changed by
  end-of-line-translation filters, dropped zero bytes, dropped high
  bits, or parity changes.)
  ...


Best regards,
--
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite