Re: EOL characters and multibyte encodings

"Joe Conway" <mail@joeconway.com>
> Tom Lane wrote:
>> Joe Conway <mail@joeconway.com> writes:
>>> My first thought on fixing this issue was to simply replace all 
>>> instances of '\r' in pg_proc.prosrc with '\n' prior to sending it to the 
>>> R parser. As far as I know, any instances of '\r' embedded in a 
>>> syntactically valid R statement must be escaped (i.e. literally the 
>>> characters "\" and "r"), so that should not be a problem. But I am 
>>> concerned about how this potentially plays against multibyte characters. 
>>> Is it safe to do this, or do I need to use a mb-aware replace algorithm?
>>
>> It's safe, because you'll be dealing with prosrc inside the backend,
>> therefore using a backend-legal encoding, and those don't have any ASCII
>> aliasing problems (all bytes of an MB character must have high bit set).

The lower byte of some characters in BIG5, GBK, GB18030 may be less than
0x7F and don't have the high bit set. Fortunately, they don't use 0x0D and
0x0A (CR and LF).

Regards,
William ZHANG

> Great -- I wasn't sure about that.
>