Re: Enforce work_mem per worker

On Sat, Nov 27, 2021 at 04:33:07PM +0000, Arne Roland wrote:
> Hello!
> 
> Since I used a lot of my time chasing short lived processes eating away big chunks of memory in recent weeks, I am
wonderingabout a decent way to go about this.
 
> The problem I am facing essentially relates to the fact that work_mem settings, while they are enforced per hash and
sortnode, aren't enforced globally.
 
> One common case, that causes this problem more frequently than a few years ago, is the partitionwise_join. If there
area lot of partitions hash joined, we get a lot of hash nodes, each one potentially consuming work_mem.
 

> While avoiding oom seems a big deal to me, my search didn't turn up previous hackers discussions about this. There is
agood chance I am missing something here, so I'd appreciate any pointers.
 

Here's some pointers ;)

https://www.postgresql.org/message-id/flat/20190708164401.GA22387%40telsasoft.com

https://www.postgresql.org/message-id/flat/20191216215314.qvrtgxaz4m755geq%40development#75e9930ac2cd353a8036dc71e8f5e6f7
https://www.postgresql.org/message-id/flat/CAH2-WzmNwV%3DLfDRXPsmCqgmm91mp%3D2b4FvXNF%3DcCvMrb8YFLfQ%40mail.gmail.com
 - I don't recall reading all of this last one before, and it's got interesting
   historic value, so I'm reading it myself now...

-- 
Justin