Re: tuning questions

Jack,

> latest changes:
> shared_buffers = 35642

This is fine, it's about 14% of available RAM.  Though the way you calculated
it still confuses me.   It's not complicated; it should be between 6% and 15%
of available RAM; since you're doing a data-transformation DB, yours should
be toward the high end.

> max_fsm_relations = 1000
> max_fsm_pages = 10000

You want to raise this a whole lot if your data transformations involve large
delete or update batches.    I'd suggest running "vacuum analyze verbose"
between steps to see how many dead pages you're accumulating.

> wal_buffers = 64
> sort_mem = 32768
> vacuum_mem = 32768
> effective_cache_size = 10000

This is way the heck too low.  it's supposed to be the size of all available
RAM; I'd set it to 2GB*65% as a start.

> IO is active, but hardly saturated. CPU load is hefty though, load
> average is at 4 now.

Unless you're doing huge statistical aggregates (like radar charts), or heavy
numerical calculations-by-query, high CPU and idle I/O usually indicates a
really bad query, like badly mismatched data types on a join or unconstrained
joins or  overblown formatting-by-query.

--
-Josh Berkus
 Aglio Database Solutions
 San Francisco