Re: libpq Unicode support?

Are we not going to lose some characters if we are putting a UTF-16 to UTF-8
translation in front of libpq?

Ale.

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Friday, April 22, 2005 12:14 PM
To: Bruce Momjian
Cc: Ale Raza; pgsql-general@postgresql.org
Subject: Re: [GENERAL] libpq Unicode support?


Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Tom Lane wrote:
>> Oh?  Who's working on it, or even interested?  Was there discussion
>> of adding it to TODO?

> TODO has:

>         o Add support for Unicode

>           To fix this, the data needs to be converted to/from UTF16/UTF8
>           so the Win32 wcscoll() can be used, and perhaps other functions
>           like towupper().  However, UTF8 already works with normal
>           locales but provides no ordering or character set classes.

That's completely unrelated --- it's talking about making correct use of
Windows' locale support in one small bit inside the server.

To make libpq UTF-16 capable, we'd have to change its API for all
strings; either make the strings counted rather than null-terminated,
or make the string elements wchar instead of char.  After that we'd
have to hack the FE/BE protocol too (or more likely, require libpq
to translate UTF-16 to UTF-8 before sending to the server).  I don't
foresee anyone doing any of this, at least not in the near term.

Putting a UTF-16 to UTF-8 translation in front of libpq seems a lot
more practical.

            regards, tom lane