Re: Linux: more cores = less concurrency.

On Tue, Apr 12, 2011 at 12:00 PM, Greg Smith <greg@2ndquadrant.com> wrote:
> Kevin Grittner wrote:
>>
>> Glyn Astill <glynastill@yahoo.co.uk> wrote:
>>
>>>
>>> Results from Greg Smiths stream_scaling test are here:
>>>
>>> http://www.privatepaste.com/4338aa1196
>>>
>>
>>  Well, that pretty much clinches it.  Your RAM access tops out at 16
>> processors.  It appears that your processors are spending most of
>> their time waiting for and contending for the RAM bus.
>>
>
> I've pulled Glyn's results into https://github.com/gregs1104/stream-scaling
> so they're easy to compare against similar processors, his system is the one
> labled 4 X X7550.  I'm hearing this same story from multiple people lately:
>  these 32+ core servers bottleneck on aggregate memory speed with running
> PostgreSQL long before the CPUs are fully utilized.  This server is close to
> maximum memory utilization at 8 cores, and the small increase in gross
> throughput above that doesn't seem to be making up for the loss in L1 and L2
> thrashing from trying to run more.  These systems with many cores can only
> be used fully if you have a program that can work efficiency some of the
> time with just local CPU resources.  That's very rarely the case for a
> database that's moving 8K pages, tuple caches, and other forms of working
> memory around all the time.
>
>
>> I have gotten machines in where moving a jumper, flipping a DIP
>> switch, or changing BIOS options from the default made a big
>> difference.  I'd be looking at the manuals for my motherboard and
>> BIOS right now to see what options there might be to improve that
>
> I already forwarded Glyn a good article about tuning these Dell BIOSs in
> particular from an interesting blog series others here might like too:
>
> http://bleything.net/articles/postgresql-benchmarking-memory.html
>
> Ben Bleything is doing a very thorough walk-through of server hardware
> validation, and as is often the case he's already found one major problem
> with the vendor config he had to fix to get expected results.

For posterity, since it looks like you guys have nailed this one, I
took a look at some of the code off list and I can confirm there is no
obvious bottleneck coming from locking type issues.  The functions are
'stable' as implemented with no fancy tricks.

merlin