Re: Solving hash table overrun problems

Bruno Wolff III <bruno@wolff.to> writes:
>   Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> * Estimate the number of batches N using the planner's estimate.
>> We will always choose N a power of 2.  A tuple's batch number is
>> ((H div K) mod N).

> If K is way low this could be very slow.

How so?  You're not concerned about the time to do the division itself
are you?

>> * Now begin scanning the outer join input.  Tuples of batch number
>> zero (according to the current calculation) can be matched to the
>> current hashtable contents.  Tuples of higher batch numbers are dropped
>> into holding files for the outer input, one per batch.

> For new keys at this step do you know their final disposition so that making
> new hash entries won't be necessary?

Well, we probably have a pretty fair idea of N at this point, so it'll
usually be right --- but we reserve the right to increase N again later
in case we have to do so because one of the later inner batches is much
bigger than the zero batch we are currently processing.
        regards, tom lane