Re: Slow standby snapshot

Hi,

> 3 авг. 2021 г., в 03:01, Andres Freund <andres@anarazel.de> написал(а):
>
> Hi,
>
> On 2021-08-03 00:07:23 +0300, Michail Nikolaev wrote:
>> The main idea is simple optimistic optimization - store offset to next
>> valid entry. So, in most cases, we could just skip all the gaps.
>> Of course, it adds some additional impact for workloads without long
>> (few seconds) transactions but it is almost not detectable (because of
>> CPU caches).
>
> I'm doubtful that that's really the right direction. For workloads that
> are replay heavy we already often can see the cost of maintaining the
> known xids datastructures show up significantly - not surprising, given
> the datastructure. And for standby workloads with active primaries the
> cost of searching through the array in all backends is noticeable as
> well.  I think this needs a bigger data structure redesign.

KnownAssignedXids implements simple membership test idea. What kind of redesign would you suggest? Proposed
optimisationmakes it close to optimal, but needs eventual compression. 

Maybe use a hashtable of running transactions? It will be slightly faster when adding\removing single transactions. But
muchworse when doing KnownAssignedXidsRemove(). 

Maybe use a tree? (AVL\RB or something like that) It will be slightly better, because it does not need eventual
compressionlike exiting array. 

Best regards, Andrey Borodin.