Re: WAL prefetch

On 2018-06-18 16:44:09 -0400, Robert Haas wrote:
> On Sat, Jun 16, 2018 at 3:41 PM, Andres Freund <andres@anarazel.de> wrote:
> >> The posix_fadvise approach is not perfect, no doubt about that. But it
> >> works pretty well for bitmap heap scans, and it's about 13249x better
> >> (rough estimate) than the current solution (no prefetching).
> >
> > Sure, but investing in an architecture we know might not live long also
> > has it's cost. Especially if it's not that complicated to do better.
> 
> My guesses are:
> 
> - Using OS prefetching is a very small patch.
> - Prefetching into shared buffers is a much bigger patch.

Why?  The majority of the work is standing up a bgworker that does
prefetching (i.e. reads WAL, figures out reads not in s_b, does
prefetch). Allowing a configurable number + some synchronization between
them isn't that much more work.


> - It'll be five years before we have direct I/O.

I think we'll have lost a significant market share by then if that's the
case. Deservedly so.

Greetings,

Andres Freund