Re: Linux kernel impact on PostgreSQL performance

On 1/13/14, 3:04 PM, Jeff Janes wrote:
>
> I think the above is pretty simple for both interaction (allow us to inject a clean page into the file page cache)
andpolicy (forget it after you hand it to us, then remember it again when we hand it back to you clean).  And I think
itwould pretty likely be an improvement over what we currently do.  But I think it is probably the wrong way to get the
improvement. I think the real problem is that we don't trust ourselves to manage more of the memory ourselves.
 
>
> As far as I know, we still don't have a publicly disclosable and readily reproducible test case for the reports of
performancedegradation when we have more than 8GB in shared_buffers. If we had one of those, we could likely reduce the
doublebuffering problem by fixing our own scalability issues and therefore taking responsibility for more of the data
ourselves.

While I agree we need to fix the 8GB limit, we're always going to have a problem here unless we put A LOT of new
abilitiesinto our memory capabilities. Like, for example, stealing memory from shared buffers to support a sort. Or
implementinga system-wide limit on WORK_MEM. Or both.
 

I would much rather teach the OS and Postgres to work together on memory management than for us to try and re-implement
everythingthe OS has already done for us.
 
-- 
Jim C. Nasby, Data Architect                       jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net