Re: [SPAM] Re: [GENERAL] Invalid field size

Il 04/07/2017 17:39, Adrian Klaver ha scritto:
> On 07/04/2017 08:19 AM, Moreno Andreo wrote:
>> Il 04/07/2017 16:51, Tom Lane ha scritto:
>>> Moreno Andreo <moreno.andreo@evolu-s.it> writes:
>>>> I've implemented a backup procedure in C# with Npgsql (using COPY TO I
>>>> dump all tables in a compressed file) that's been working well in the
>>>> last 5 years (and it's still working, since this is a single, isolated
>>>> case).
>>>> OS: Windows 7
>>>> PG: 9.1.6 (I know, it's EOL, but I think it's not matter here)
>>>> [ got corrupted data with: ]
>>>> 2017-07-04 12:55:27 CEST STATEMENT:  COPY
>>>> tab(cod,guid,data,blob,thumbnail,descr,type,url,user,home,codrec,table,op,dagg,last)
>>>>
>>>> FROM STDIN WITH BINARY
>>> Pushing binary data around on Windows is always a hazardous
>>> proposition.
>>> I'd bet something somewhere did a newline format conversion on your
>>> data, either adding or removing CR characters.  There might not have
>>> been any CR or LF bytes in the data fields proper, but it'd be quite
>>> plausible for some of the length words used in binary-COPY format to
>>> contain such bytes.
>>>
>>> You might be better off using plain text COPY format; it can withstand
>>> this sort of thing much better.
>>>
>>>             regards, tom lane
>>>
>> When we wrote this function, we first used plain COPY format, but we
>> were not satisfied by the file size we got (too big compared to data
>> size), so we switched to BINARY (I don't remember if there was also
>> some performance matter involved).
>> So what you are saying is "in the last 5 years you've been extremely
>> lucky?" :-)
>
> Your original post went back and forth on whether you where lucky in
> the past:
>
> "... that's been working well in the last 5 years (and it's still
> working, since this is a single, isolated case)"
>
> "As for many error I got in the past I assume we are trying to COPY
> FROM corrupted data (when using cheap pendrives we get often this
> error)."
The bunch of errors I mention here is related to file management (issues
with file copying or unzipping), sometines I had errors like
"unrecognized Unicode character: 0xFF", and making a new backup always
resolved the error.
This is the very first time we have this kind of error.
If I had the source machine I'd try to make a new backup...