Re: Bug in UTF8-Validation Code?

On 2007-04-05, Tatsuo Ishii <ishii@postgresql.org> wrote:
>> Andrew - Supernews <andrew+nonews@supernews.com> writes:
>> > Thinking about this made me realize that there's another, ahem, elephant
>> > in the room here: convert().
>> > By definition convert() returns text strings which are not valid in the
>> > server encoding. How can this be addressed?
>> 
>> Remove convert().  Or at least redefine it as dealing in bytea not text.
>
> That would break some important use cases. 
>
> 1) A user have UTF-8 database which contains various language
>    data. Each language has its own table. He wants to sort a SELECT
>    result by using ORDER BY. Since locale cannot handle multiple
>    languages, he uses C locale and do the SELECT something like this:
>
>    SELECT * FROM french_table ORDER BY convert(t, 'LATIN1');
>    SELECT * FROM japanese_table ORDER BY convert(t, 'EUC_JP');

That works without change if convert(text,text) returns bytea.
>
> 2) A user has a UTF-8 database but unfortunately his OS's UTF-8 locale
>    is broken. He decided to use C locale and want to sort the result
>    from SELECT like this.
>
>    SELECT * FROM japanese_table ORDER BY convert(t, 'EUC_JP');

That also works without change if convert(text,text) returns bytea.

-- 
Andrew, Supernews
http://www.supernews.com - individual and corporate NNTP services