Re: [HACKERS] Hashjoin status report

On Thu, 6 May 1999, Tom Lane wrote:

> I've committed fixes that deal with all of the coredump problems
> I could find in nodeHash.c (there were several :-().
> 
> But the code still has a fundamental design flaw: it uses a fixed-size
> overflow area to hold tuples that don't fit into the hashbuckets they
> are assigned to.  This means you get "hashtable out of memory" errors
> if the distribution of tuples is skewed enough, or if the number of
> hashbuckets is too small because the system underestimated the number
> of tuples in the relation.  Better than a coredump I suppose, but still
> very bad, especially since the optimizer likes to use hashjoins more
> than it used to.
> 
> What I would like to do to fix this is to store the tuples in a Portal
> instead of in a fixed-size palloc block.  While at it, I'd rip out the
> "relative vs. absolute address" cruft that is in the code now.
> (Apparently there was once some thought of using a shared memory block
> so that multiple processes could share the work of a hashjoin.  All that
> remains is ugly, bug-prone code ...)
> 
> The reason I bring all this up is that it'd be a nontrivial revision
> to nodeHash.c, and I'm uncomfortable with the notion of making such a
> change this late in a beta cycle.  On the other hand it *is* a bug fix,
> and a fairly important one IMHO.
> 
> Opinions?  Should I plow ahead, or leave this to fix after 6.5 release?

Estimate of time involved to fix this?  vs likelihood of someone
triggering the bug in production?

Marc G. Fournier                   ICQ#7615664               IRC Nick: Scrappy
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org