Re: [GENERAL] Invalid field size

Il 04/07/2017 16:51, Tom Lane ha scritto:
> Moreno Andreo <moreno.andreo@evolu-s.it> writes:
>> I've implemented a backup procedure in C# with Npgsql (using COPY TO I
>> dump all tables in a compressed file) that's been working well in the
>> last 5 years (and it's still working, since this is a single, isolated
>> case).
>> OS: Windows 7
>> PG: 9.1.6 (I know, it's EOL, but I think it's not matter here)
>> [ got corrupted data with: ]
>> 2017-07-04 12:55:27 CEST STATEMENT:  COPY
>> tab(cod,guid,data,blob,thumbnail,descr,type,url,user,home,codrec,table,op,dagg,last)
>> FROM STDIN WITH BINARY
> Pushing binary data around on Windows is always a hazardous proposition.
> I'd bet something somewhere did a newline format conversion on your
> data, either adding or removing CR characters.  There might not have
> been any CR or LF bytes in the data fields proper, but it'd be quite
> plausible for some of the length words used in binary-COPY format to
> contain such bytes.
>
> You might be better off using plain text COPY format; it can withstand
> this sort of thing much better.
>
>             regards, tom lane
>
When we wrote this function, we first used plain COPY format, but we
were not satisfied by the file size we got (too big compared to data
size), so we switched to BINARY (I don't remember if there was also some
performance matter involved).
So what you are saying is "in the last 5 years you've been extremely
lucky?" :-)

Thanks
Moreno.