Re: string_to_array eats too much memory?

> > > I'm playing with GIN to make a full text search system. GIN comes
> > > with built-in TEXT[] support and I use string_to_array()
> to make a
> > > TEXT[]. Problem is, if there's large number of array elemets,
> > > string_to_array() consumes too much memory. For example, to make
> > > ~70k array elements, string_to_array seems to eat several
> Gig bytes
> > > of memory. ~70k array elements means there are same
> number of words
> > > in a document which is not too big in a large text IMO.
> >
> > Do you mean 70k unique lexemes? Ugh.
>
> I'm testing how GIN scales.
>
> > Why do not you use tsearch framework?
>
> ? I thought GIN is superior than tsearch2.
>
> From your GIN proposal posted to pgsql-hackers:
>
> "The primary goal of the Gin index is a scalable full text
> search in PostgreSQL"

tsearch2 *uses* GIN in 8.2. Just CREATE INDEX foo ON bar USING
gin(mytsvector).

And tsearch2 in 8.2 with GIN can be a *lot* faster than with GIST. I've
been running experiments on the website search with tsearch2/GIN and
i've been seeing fantastic performance compared top revious versions.



//Magnus