Обсуждение: Optimize walsender handling invalid messages of 'drop publication'

Hi all,

Before PG 14, walsender process has to handle invalid message in one
XLOG (PG 14 provide a particular XLOG type: XLOG_XACT_INVALIDATIONS).
This may bring some problems which has been discussed in previous
mail:
https://www.postgresql.org/message-id/flat/CAM_vCufO3eeRZ_O04z9reiE%2BB644%2BRgczbAVo9C5%2BoHV9S7%2B-g%40mail.gmail.com#981e65567784e0aefa4474cc3fd840f6

This patch can solve the problem. It has three parts:
1. pgoutput do not do useless invalid cache anymore;
2. Add a relid->relfilenode hash map to invoid hash seq search;
3. test case: It needs two or three minutes to finish.

The patch is based on the HEAD of branch REL_13_STABLE. It also works
for PG 10~12.

Thanks.
Bowenshi

Dears,

This issue has been pending for several months without any response.
And this problem still exists in the latest minor versions of PG 12
and PG 13.

I believe that the fix in this patch is helpful.

The patch has been submitted
https://commitfest.postgresql.org/43/4393/ .  Anyone who is interested
in this issue can help with the review.

Regards!
BowenShi

On Mon, 27 Feb 2023 at 16:12, Bowen Shi <zxwsbg12138@gmail.com> wrote:
>
> Hi all,
>
> Before PG 14, walsender process has to handle invalid message in one
> XLOG (PG 14 provide a particular XLOG type: XLOG_XACT_INVALIDATIONS).
> This may bring some problems which has been discussed in previous
> mail:
https://www.postgresql.org/message-id/flat/CAM_vCufO3eeRZ_O04z9reiE%2BB644%2BRgczbAVo9C5%2BoHV9S7%2B-g%40mail.gmail.com#981e65567784e0aefa4474cc3fd840f6
>
> This patch can solve the problem. It has three parts:
> 1. pgoutput do not do useless invalid cache anymore;
> 2. Add a relid->relfilenode hash map to invoid hash seq search;
> 3. test case: It needs two or three minutes to finish.
>
> The patch is based on the HEAD of branch REL_13_STABLE. It also works
> for PG 10~12.
>
> Thanks.
> Bowenshi

Hi,

On 2023-06-26 15:01:22 +0800, Bowen Shi wrote:
> This issue has been pending for several months without any response.
> And this problem still exists in the latest minor versions of PG 12
> and PG 13.
>
> I believe that the fix in this patch is helpful.
>
> The patch has been submitted
> https://commitfest.postgresql.org/43/4393/ .  Anyone who is interested
> in this issue can help with the review.

ISTM that the path for people encountering this issue is to upgrade.

It's not unheard of that we'd backpatch a performance improvements to the
backbranches, but it's pretty rare.  It's one thing to decide to backpatch an
optimization if it had time to "mature" in the development branch, but from
what I undestand you're proposing to apply this just to the back branches.

Greetings,

Andres Freund