Re: Clock sweep not caching enough B-Tree leaf pages?

On 2014-04-16 11:28:04 -0400, Robert Haas wrote:
> On Wed, Apr 16, 2014 at 10:49 AM, Andres Freund  wrote:
> >> 1. Improving the rate at which we can evict buffers, which is what
> >> you're talking about here.
> >>
> >> 2. Improving the choice of which buffers we evict, which is what
> >> Peter's talking about, or at least what I think he's talking about.
> >>
> >> Those things are both important, but they're different, and I'm not
> >> sure that working on one precludes working on the other.  There's
> >> certainly the potential for overlap, but not necessarily.
> >
> > I don't think that that they neccessarily preclude each other
> > either. But my gut feeling tells me that it'll be hard to have
> > interesting algorithmic improvements on the buffer eviction choice
> > because any additional complexity around that will have prohibitively
> > high scalability impacts due to the coarse locking.
> 
> Doesn't that amount to giving up?  I mean, I'm not optimistic about
> the particular approach Peter's chosen here being practical for the
> reasons that you and I already articulated.  But I don't think that
> means there *isn't* a viable approach; and I think Peter's test
> results demonstrate that the additional complexity of a better
> algorithm can more than pay for itself.  That's a pretty important
> point to keep in mind.

Well, I think it could be a very good idea to invest more resources
(cpu, bus, memory) in buffer management - but doing so right *now* where
it's all done under one monolithic lock will have noticeable
consequences for many workloads. Spending more cycles per buffer won't
be very noticeable if it's not done under a gigantic lock - right now it
will be.

> [ reasonable proposal ].  In such a world, it's still not
> permissible for reclaim calculations to be super-complex, but you hope
> that most of the activity is happening in the background process, so
> cycle-shaving becomes less critical.

Yes, agreed.

Greetings,

Andres Freund

-- Andres Freund	                   http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services