Re: Another tsearch bug...

On Fri, 23 Aug 2002, Christopher Kings-Lynne wrote:

> Hi guys,
>
> Hate to keep coming up with these bugs without patches - but I really don't
> have time to look into the source code atm :(
>
> OK, attached is an example of the problem.  Notice how trademarks and
> copyright symbols are being indexed along with the word.  This means that if
> someone searches for 'balance' in the above data set, they won't find
> anything.
>
> I'm not sure how this would be handled.  In the English language, it'd
> probably be safe to say that high ascii characters would be stripped from
> the index?  But you'd want to leave accents and stuff in I guess.  Tricky.

Rather tricky. The problem is that we don't know how to get flex to works
with locale. Parser recognizes latin words ([a-zA-Z]), nonLatin ([\0200-\0377])
and mixed words ([a-zA-Z\0200-\0377]). Your case (balanceR) is the mixed word.
The right way is to have locale aware parser to properly recognize words.
We incline to refuse a flex.

>
> Anyway, just bringing it to your attention...
>
> Chris
>
Regards,    Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83