Обсуждение: Use WALReadFromBuffers in more places
Hi, Commit 91f2cae7a4e that introduced WALReadFromBuffers only used it for physical walsenders. It can also be used in more places benefitting logical walsenders, backends running pg_walinspect and logical decoding functions if the WAL is available in WAL buffers. I'm attaching a 0001 patch for this. While at it, I've also added a test module in 0002 patch to demonstrate 2 things: 1) how the caller can ensure the requested WAL is fully copied to WAL buffers using WaitXLogInsertionsToFinish before reading from WAL buffers. 2) how one can implement an xlogreader page_read callback to read unflushed/not-yet-flushed WAL directly from WAL buffers. FWIW, a separate test module to explicitly test the new function is suggested here - https://www.postgresql.org/message-id/CAFiTN-sE7CJn-ZFj%2B-0Wv6TNytv_fp4n%2BeCszspxJ3mt77t5ig%40mail.gmail.com. Please have a look at the attached patches. I will register this for the next commit fest. -- Bharath Rupireddy PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com
Вложения
Hi, Bharath. I've been testing this. It's cool. Is there any way we could
monitor the hit rate about directly reading from WAL buffers by exporting
to some views?
---
Regards, Jingtang
On Wed, May 8, 2024 at 9:51 AM Jingtang Zhang <mrdrivingduck@gmail.com> wrote: > > Hi, Bharath. I've been testing this. It's cool. Is there any way we could > monitor the hit rate about directly reading from WAL buffers by exporting > to some views? Thanks for looking into this. I used purpose-built patches for verifying the WAL buffers hit ratio, please check USE-ON-HEAD-Collect-WAL-read-from-file-stats.txt and USE-ON-PATCH-Collect-WAL-read-from-buffers-and-file-stats.txt at https://www.postgresql.org/message-id/CALj2ACU9cfAcfVsGwUqXMace_7rfSBJ7%2BhXVJfVV1jnspTDGHQ%40mail.gmail.com. In the long run, it's better to extend what's proposed in the thread https://www.postgresql.org/message-id/CALj2ACU_f5_c8F%2BxyNR4HURjG%3DJziiz07wCpQc%3DAqAJUFh7%2B8w%40mail.gmail.com. I haven't had a chance to dive deep into that thread yet, but a quick glance over v8 patch tells me that it hasn't yet implemented the idea of collecting WAL read stats for both walsenders and the backends reading WAL. If that's done, we can extend it for WAL read from WAL buffers. -- Bharath Rupireddy PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com