Re: High CPU Utilization

On Mon, 16 Mar 2009, Joe Uhl wrote:

> Here is vmstat 1 30.  We are under peak load right now so I can gather
> information from the real deal

Quite helpful, reformatting a bit and picking an informative section:

procs -----------memory----------    ---swap- ----io--- -system-- ----cpu----
r  b   swpd     free   buff  cache   si   so   bi    bo   in   cs us sy id wa
0 34  95048 11025880  56988 15020168 0    0  3852   160 3616 8614 11  1  6 82
3 25  95048 10996356  57044 15044796 0    0  7892   456 3126 7115  4  3  8 85
1 26  95048 10991692  57052 15050100 0    0  5188   176 2566 5976  3  2 12 83

This says that your server is spending all its time waiting for I/O,
actual CPU utilization is minimal.  You're only achieving around 3-8MB/s
of random I/O.  That's the reality of what your disk I/O subsystem is
capable of, regardless of what its sequential performance with dd looks
like.  If you were to run a more complicated benchmark like bonnie++
instead, I'd bet that your "seeks/second" results are very low, even
though sequential read/write is fine.

The Perc5 controllers have a pretty bad reputation for performance on this
list, even in RAID10.  Not much you can do about that beyond scrapping the
controller and getting a better one.

What you might do in order to reduce the total number of writes needed is
some standard postgresql.conf tuning; see
http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server

What you could do here is increase shared_buffers, checkpoint_segments,
and checkpoint_completion_target as described there.  Having more buffers
dedicated to the database and having less checkpoints can result in less
random I/O showing up, as popular data pages will stay in RAM for longer
without getting written out so much.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD