Re: BGWorkers, shared memory pointers, and postmaster restart

On Wed, Apr 16, 2014 at 7:11 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
>> TL;DR: I don't think there's a safe way to use a BGWorker (static or
>> dynamic) with bgw_restart_time != BGW_NEVER_RESTART and a bgw_main_arg
>> Datum that points into shared memory, and think we might need a API
>> change to fix that.
>
> Andres sensibly points out that part of this is easily solved by passing
> the bgworker an index into an array in a named shmem block. A simple
> pass-by-value Datum that can be turned into a pointer to a shmem struct.

Yes.  I think the answer to your original complaint is that we don't
currently support that, and adding support for that is material for a
future release.

> This still doesn't solve the other half of the issue, which is how to
> handle dynamic bgworkers after a postmaster restart. They're effectively
> lost/leaked: there's no way to retain a bgworker handle across restart,
> and no way to list bgworkers, nor is there any idempotent way to say
> "Start a worker to do <x> only if it doesn't already exist" (unique
> names, magic cookie hashes, whatever).

Fair point.

> With the current API the only solution to the second half that I see is
> to have bgworkers run in non-restart mode and manage them yourself. Not
> ideal.
>
> Instead we need one of:
>
> - A flag like BGW_UNREGISTER_ON_RESTART;

I would be OK with this, maybe modulo the name.

> - To always unregister dynamic bgws on postmaster shm clear + restart;

I don't particularly care for this.  Let's suppose the background
worker is a long-running daemon, like a PG equivalent of cron.  In
static-background worker land, the admin has to restart the cluster to
get this going.  In dynamic-background worker land, he can load it on
the fly.  But once he gets it up and running, he wants it to stay up
and running, surviving crashes and everything.  That's a big part of
the value of having a background worker interface in the first place.

> - A way to list bgws, inspect their BackgroundWorker structs and obtain
> their handles; or

This is certainly a good idea.

> - A way to idempotently register a bgw only if it doesn't already exist

This is probably a good idea, too.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company