Re: IN list processing performance (yet again)

Bruno Wolff III wrote:

On Wed, May 28, 2003 at 13:58:14 -0400, Dave Tenny <tenny@attbi.com> wrote: 

A join isn't an option, these elements come a a selection of entity ID's 
that are specific to some client context.
Some other people suggested joins too.    

You can union the values together and then join (or use where exists) with the
result. This may not be faster and you may not be able to union several
thousand selects together in a single statement. But it shouldn't be too
much work to test it out. 

I assume you mean something like:

test=# select million.id, million.val from million, (select 10000 as a union select 20000 as a) t2 where million.id = t2.a;
  id   |  val
-------+-------
 10000 |     0
 20000 | 10000
(2 rows)

Ouch!  That's deviant.   Haven't tried it yet and I cringe at the thought of it, but I might take a run at it.  However that's going to
run up the buffer space quickly.  That was one of my as yet unsnaswered questions, what is the pragmatic buffer size limit
for queries?

I'm really hoping we'll come up with something better, like an understanding of why IN lists are non-linear in the first
place when the column is indexed, and whether it's fixable through some other means or whether it's a bug that should be fixed.

After all, I'm trying to support multiple databases, and other databases kick butt on this.  It's just postgresql that's
having difficulty.

(Btw, I've also tried statement batching, but that's a lose for now, at least with the current JDBC drivers and 7.3.2).