Re: Batching in executor

On Fri, Sep 26, 2025 at 10:28:33PM +0900, Amit Langote wrote:
> At PGConf.dev this year we had an unconference session [1] on whether
> the community can support an additional batch executor. The discussion
> there led me to start hacking on $subject. I have also had off-list
> discussions on this topic in recent months with Andres and David, who
> have offered useful thoughts.
> 
> This patch series is an early attempt to make executor nodes pass
> around batches of tuples instead of tuple-at-a-time slots. The main
> motivation is to enable expression evaluation in batch form, which can
> substantially reduce per-tuple overhead (mainly from function calls)
> and open the door to further optimizations such as SIMD usage in
> aggregate transition functions. We could even change algorithms of
> some plan nodes to operate on batches when, for example, a child node
> can return batches.

For background, people might want to watch these two videos from POSETTE
2025.  The first video explains how data warehouse query needs are
different from OLTP needs:

    Building a PostgreSQL data warehouse
    https://www.youtube.com/watch?v=tpq4nfEoioE

and the second one explains the executor optimizations done in PG 18:

    Hacking Postgres Executor For Performance
    https://www.youtube.com/watch?v=D3Ye9UlcR5Y

I learned from these two videos that to handle new workloads, I need to
think of the query demands differently, and of course can this be
accomplished without hampering OLTP workloads?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.