Re: Linux: more cores = less concurrency.

On Mon, Apr 11, 2011 at 5:06 PM, Kevin Grittner
<Kevin.Grittner@wicourts.gov> wrote:
> Glyn Astill <glynastill@yahoo.co.uk> wrote:
>
>> The issue I'm seeing is that 8 real cores outperform 16 real
>> cores, which outperform 32 real cores under high concurrency.
>
> With every benchmark I've done of PostgreSQL, the "knee" in the
> performance graph comes right around ((2 * cores) +
> effective_spindle_count).  With the database fully cached (as I
> believe you mentioned), effective_spindle_count is zero.  If you
> don't use a connection pool to limit active transactions to the
> number from that formula, performance drops off.  The more CPUs you
> have, the sharper the drop after the knee.

I was about to say something similar with some canned advice to use a
connection pooler to control this.  However, OP scaling is more or
less topping out at cores / 4...yikes!.  Here are my suspicions in
rough order:

1. There is scaling problem in client/network/etc.  Trivially
disproved, convert the test to pgbench -f and post results
2. The test is in fact i/o bound. Scaling is going to be
hardware/kernel determined.  Can we see iostat/vmstat/top snipped
during test run?  Maybe no-op is burning you?
3. Locking/concurrency issue in heavy_seat_function() (source for
that?)  how much writing does it do?

Can we see some iobound and cpubound pgbench runs on both servers?

merlin