Re: multi terabyte fulltext searching

On Wed, 21 Mar 2007, Benjamin Arai wrote:

> Can't you implement something similar to google by aggregating results for
> TSearch2 over many machines?

tsearch2 doesn't use any global statistics, so, in principle, you
should be able to run fts on several machines and combine them using
dblink (contrib/dblink).

>
> Benjamin
> On Mar 21, 2007, at 8:59 AM, Teodor Sigaev wrote:
>
>> I'm afraid that fulltext search on multiterabytes set of documents can not
>> be implemented on any RDBMS, at least on single box. Specialized fulltext
>> search engines (with exact matching and time to search about one second)
>> has practical limit near 20 millions of docs, cluster - near 100 millions.
>> Bigger collections require engines like a google.
>>
>>
>> --
>> Teodor Sigaev                                   E-mail: teodor@sigaev.ru
>>                                                   WWW:
>> http://www.sigaev.ru/
>>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will ignore your desire to
>     choose an index scan if your joining column's datatypes do not
>     match

     Regards,
         Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83