Re: WIP: WAL prefetch (another approach)

On 4/8/21 1:46 PM, Thomas Munro wrote:
> On Thu, Apr 8, 2021 at 3:27 AM Tomas Vondra
> <tomas.vondra@enterprisedb.com> wrote:
>> On 4/7/21 1:24 PM, Thomas Munro wrote:
>>> I made one other simplifying change: previously, the prefetch module
>>> would read the WAL up to the "written" LSN (so, allowing itself to
>>> read data that had been written but not yet flushed to disk by the
>>> walreceiver), though it still waited until a record's LSN was
>>> "flushed" before replaying.  That allowed prefetching to happen
>>> concurrently with the WAL flush, which was nice, but it felt a little
>>> too "special".  I decided to remove that part for now, and I plan to
>>> look into making standbys work more like primary servers, using WAL
>>> buffers, the WAL writer and optionally the standard log-before-data
>>> rule.
>>
>> Not sure, but the removal seems unnecessary. I'm worried that this will
>> significantly reduce the amount of data that we'll be able to prefetch.
>> How likely it is that we have data that is written but not flushed?
>> Let's assume the replica is lagging and network bandwidth is not the
>> bottleneck - how likely is this "has to be flushed" a limit for the
>> prefetching?
> 
> Yeah, it would have been nice to include that but it'll have to be for
> v15 due to lack of time to convince myself that it was correct.  I do
> intend to look into more concurrency of that kind for v15.  I have
> pushed these patches, updated to be disabled by default.  I will look
> into how I can run a BF animal that has it enabled during the recovery
> tests for coverage.  Thanks very much to everyone on this thread for
> all the discussion and testing so far.
> 

OK, understood. I'll rerun the benchmarks on this version, and if
there's a significant negative impact we can look into that during the
stabilization phase.

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company