Re: pg_read_file() and non-ascii input file

Robert Haas <robertmhaas@gmail.com> wrote:

> > If we want to keep backward compatibility, the issue can be fixed
> > by adding pg_verifymbstr() to the function.
>
> I don't feel good about changing the return type of an existing
> function, so I guess +1 from me on the approach quoted above.

Ok, I just added pg_verifymbstr() instead of changing the result type.

I didn't add any additinal file reading functions in the patch, but
I'm willing to add them if someone want them:
  - pg_read_file_with_encoding()
  - pg_read_binary_file() RETURNS bytea
  - pg_read_text_file() RETURNS SETOF text -- returns set of lines


One thing bothering me is the HINT message on error is just pointless;
The encoding is controlled by "server_encoding" here. We will have the
same error message in server-side COPY commands. We'd better improving
the message, though it should be done by another patch.

=# SELECT pg_read_file('invalid.txt', 0, (pg_stat_file('invalid.txt')).size);
ERROR:  invalid byte sequence for encoding "UTF8": 0x93
HINT:  This error can also happen if the byte sequence does not match the
       encoding expected by the server, which is controlled by "client_encoding".

=# COPY tbl FROM 'invalid.txt'; -- server-side copy from a local file.
(the same message -- but the encoding should match with "server_encoding")

Regards,
---
Takahiro Itagaki
NTT Open Source Software Center