tsearch parser overhaul

re:
http://archives.postgresql.org/pgsql-hackers/2009-11/msg00754.php

Alvaro Herrera <alvherre@commandprompt.com> wrote:
> Kevin Grittner wrote:
> 
>> (Note: I personally would much rather see the performance
>> penalty addressed this way, and a TODO added for the more
>> invasive work, than to leave this alone for the next release if
>> there's nobody willing to tackle the problem at a more
>> fundamental level.)
> 
> +1
I haven't added a TODO yet because I'm not sure how to frame it. 
I'm inclined that it would be no more work to replace the current
recursively called state engine with something easier to read and
understand than to try to fix the current oddities.  Perhaps
something along the lines of this?:
http://vo.astronet.ru/arxiv/dict_regex.html
I suspect we'd need to get it to use the same regexp code used
elsewhere in PostgreSQL.
Thoughts?
-Kevin