Re: Postgres 8.2 memory weirdness

On Thu, 24 Jan 2008, Tory M Blue wrote:

> Postg: 8.2.1fc6

8.2.1 has a nasty bug related to statistics collection that causes
performance issues exactly in the kind of heavy update situation you're
in.  That's actually why i asked for the exact 8.2 version.  You should
plan an upgrade as soon as feasible to the current release just to
eliminate this as a possible influence on your problems.  No need to dump
the database or do anything fancy, just get the new version going and
point it at the existing database.

To do a quick check on whether this is impacting things, run top, press
"c" to show the full process lines, and note whether the statistics
collector process is taking up a significant amount of CPU time.  If it
is, you're being nailed by the bug, and you really need that ugprade.

> 8 Gigs of Ram
> shared_buffers = 75000   <--- Believe these need tuning (based on the
> reading last night)

Probably, but if you're having checkpoint problems now making
shared_buffers bigger will likely make them worse.  Some people with
update-heavy workloads end up reducing this to a very small value (<250MB)
even with large amounts of RAM because that makes less information to dump
at checkpoint time.

> checkpoint_segments = 50
> checkpoint_timeout = 300
> checkpoint_warning = 3600s              <--- set this last night and
> already see instances of
>
> "2008-01-24 03:54:39 PST    LOG:  checkpoints are occurring too
> frequently (89 seconds apart)
> 2008-01-24 03:54:39 PST    HINT:  Consider increasing the
> configuration parameter "checkpoint_segments"."

If you're getting checkpoints every 89 seconds it's no wonder your system
is dying.  You may need to consider a large increase to
checkpoint_segments to get the interval between checkpoints to increase.
It should at least be a few minutes between them if you want any
reasonable performance level.

> effective_cache_size = 330000  <-- This appears totally wrong and
> something I noticed last night.  left over from previous versions of
> postgres on different hardware. (thinking to set this to 6-7G)

Right, that's where it should be.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD