Re: BGWorkers, shared memory pointers, and postmaster restart

On 2014-04-16 19:11:37 +0800, Craig Ringer wrote:
> On 04/16/2014 02:37 PM, Craig Ringer wrote:
> > Hi all
> > 
> > I've been using the dynamic BGWorker support for some recent work, and I
> > think I've found an issue with how postmaster restarts are handled.
> > 
> > TL;DR: I don't think there's a safe way to use a BGWorker (static or
> > dynamic) with bgw_restart_time != BGW_NEVER_RESTART and a bgw_main_arg
> > Datum that points into shared memory, and think we might need a API
> > change to fix that.
> 
> Andres sensibly points out that part of this is easily solved by passing
> the bgworker an index into an array in a named shmem block. A simple
> pass-by-value Datum that can be turned into a pointer to a shmem struct.
> 
> This still doesn't solve the other half of the issue, which is how to
> handle dynamic bgworkers after a postmaster restart. They're effectively
> lost/leaked: there's no way to retain a bgworker handle across restart,
> and no way to list bgworkers, nor is there any idempotent way to say
> "Start a worker to do <x> only if it doesn't already exist" (unique
> names, magic cookie hashes, whatever).
> 
> With the current API the only solution to the second half that I see is
> to have bgworkers run in non-restart mode and manage them yourself. Not
> ideal.
> 
> Instead we need one of:
> 
> - A flag like BGW_UNREGISTER_ON_RESTART;
> 
> - To always unregister dynamic bgws on postmaster shm clear + restart;
> 
> - A way to list bgws, inspect their BackgroundWorker structs and obtain
> their handles; or
> 
> - A way to idempotently register a bgw only if it doesn't already exist

I think we should go for always unregistering dynamic bgws. There's
really little justification for keeping them around after a crash cycle.

While not the nicest place architecturally, it seems easy enough to do
in BackgroundWorkerShmemInit() which happens to be called conveniently
in a crash/restart cycle...

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services