Re: parallelizing the archiver

On 9/29/21, 9:49 PM, "Bossart, Nathan" <bossartn@amazon.com> wrote:
> I'm sure there are other ways to approach this, but I thought I'd give
> it a try to see what was possible and to get the conversation started.

BTW I am also considering the background worker approach that was
mentioned upthread.  My current thinking is that the backup extension
would define a special background worker that communicates with the
archiver via shared memory.  As noted upthread, this would enable
extension authors to do whatever batching, parallelism, etc. that they
want, and it should also prevent failures from taking down the
archiver process.  However, this approach might not make sense for
things like recovery_end_command that are only executed once.  Maybe
it's okay to leave that one alone for now.

Nathan