Re: Using psql -f to load a UTF8 file

On 09/20/2012 11:44 PM, Leif Biberg Kristensen wrote:
>   Torsdag 20. september 2012 16.56.16 skrev Alan Millington :
>> psql". But how am I supposed to remove the byte order mark from a UTF8
>> file? I thought that the whole point of the byte order mark was to tell
>> programs what the file encoding is. Other programs, such as Python, rely
>> on this.
>
> http://en.wikipedia.org/wiki/Byte_order_mark
>
> While the Byte Order Mark is important for UTF-16, it's totally irrelevant to
> the UTF-8 encoding.

I strongly disagree. The BOM provides a useful and standard way to
differentiate UTF-8 encoded text files from the random pile of encodings
that any given file could be.

On many platforms (including all Windows versions) the default system
text encoding for 8-bit text is not UTF-8. On such systems, a BOM in a
UTF-8 file allows a program/editor to reliably work out that it's UTF-8
and treat it as such, rather than mangling it by interpreting it as the
local system encoding.

psql should accept UTF-8 with BOM.

--
Craig Ringer