Re: Use my (date) index, darn it!

After a long battle with technology, siracusa@mindspring.com (John Siracusa), an earthling, wrote:
> On 1/5/04 1:55 AM, Tom Lane wrote:
>> John Siracusa <siracusa@mindspring.com> writes:
>>> Obviously the planner is making some bad choices here.
>>
>> A fair conclusion ...
>>
>>> I know that it is trying to avoid random seeks or other scary things
>>> implied by a "correlation" statistic that is not close to 1 or -1, but
>>> it is massively overestimating the hit caused by those seeks and
>>> seemingly not taking into account the size of the table!
>>
>> You haven't given any evidence to support these conclusions, though.
>
> Well here's what I was basing that theory on: before clustering, the
> correlation for the date column was around 0.3.  After clustering, it was 1,
> and the index was always used.  Does clustering change any other statistics
> other that correlation?  I ran analyze immediately before and after the
> cluster operation.
>
>> Could we see some table schemas, EXPLAIN ANALYZE output, and relevant
>> pg_stats entries for the various cases?
>
> Well, the table is clustered now, so I can't reproduce the situation.  Is
> there any way to "uncluster" a table?  Should I just cluster it on a
> different column?

That would presumably work...

It sounds to me as though the statistics that are being collected
aren't "good enough."  That tends to be a sign that the quantity of
statistics (e.g. - bins in the histogram) are insufficient.

This would be resolved by changing the number of bins (default of 10)
via "ALTER TABLE FOO ALTER COLUMN BAR SET STATISTICS 100" (or some
other value higher than 10).

Clustering would rearrange the contents of the table, and perhaps make
the histogram 'more representative.'  Increasing the "SET STATISTICS"
value will quite likely be even more helpful, and is a lot less
expensive than clustering the table...
--
If this was helpful, <http://svcs.affero.net/rm.php?r=cbbrowne> rate me
http://www.ntlug.org/~cbbrowne/nonrdbms.html
Rules of  the Evil Overlord  #158. "I will  exchange the labels  on my
folder of  top-secret plans and  my folder of family  recipes. Imagine
the  hero's  surprise when  he  decodes  the  stolen plans  and  finds
instructions for Grandma's Potato Salad."
<http://www.eviloverlord.com/>