Re: full text search, utf8

I can answer in english if you like.

This error happening also when i'm trying to CREATE TEXT SEARCH DICTIONARY:

ports=# CREATE TEXT SEARCH DICTIONARY ruispell (
ports(#     TEMPLATE = ispell,
ports(#     DictFile = russian,
ports(#     AffFile = russian,
ports(#     StopWords = russian
ports(# );
ERROR:  неверная последовательность байт имя кодировки "UTF8": 0xd1
ПОДСКАЗКА:  This error can also happen if the byte sequence does not
match the encoding expected by the server, which is controlled by
"client_encoding".
ports=#

All data in table populated with perl script that read text file in UTF8
and make INSERTs, and i think if there was illegal character, error
would appear after INSERT.


Andrew Boag wrote:
> sorry for English response (I don't have Russian keyboard here)
>
> 0xd1 may be an illegal UTF8 chaacter that was mistakenly allowed into
> the table. Not all libraries (or all versions of postgres) prevent
> illegal UTF8 characters from getting into DB.
>
> We saw similar issues with a 7.4 -> 8.1 postgres data migration.
>
> However, I don't fully understand your select query so there may be
> another cause.
>
> alexander lunyov wrote:
>> Здравствуйте.
>>
>> Имеется freebsd 6.2, postgresql-8.3.1
>>
>> В env:
>>
>> % env | grep UTF
>> LANG=ru_RU.UTF-8
>> MM_CHARSET=UTF-8
>>
>> % psql ports -U pgsql
>> Welcome to psql 8.3.1, the PostgreSQL interactive terminal.
>>
>> Type:  \copyright for distribution terms
>>        \h for help with SQL commands
>>        \? for help with psql commands
>>        \g or terminate with semicolon to execute query
>>        \q to quit
>>
>> ports=# \encoding
>> UTF8
>> ports=# \l
>>         Список баз данных
>>     Имя    | Владелец | Кодировка
>> -----------+----------+-----------
>>  ports     | pgsql    | UTF8
>>  postgres  | pgsql    | UTF8
>>  template0 | pgsql    | UTF8
>>  template1 | pgsql    | UTF8
>> (4 rows)
>>
>> Пробую поискать в таблице, и вот результат:
>>
>> ports=# select name from abonents where to_tsvector(name) @@
>> to_tsquery('s');
>> ERROR:  неверная последовательность байт имя кодировки "UTF8": 0xd1
>> ПОДСКАЗКА:  This error can also happen if the byte sequence does not
>> match the encoding expected by the server, which is controlled by
>> "client_encoding".
>>
>> при этом в конфигурации english работает нормально.
>>
>> # select count(name) from abonents where to_tsvector('english',name)
>> @@ to_tsquery('some');
>>  count
>> -------
>>      6
>> (1 запись)
>>
>> Почему?
>>
>
>


--
С уважением
Александр Лунев
ОАО РТК