Re: How to estimate the shared memory size required for parallel scan?

On Thu, Aug 16, 2018 at 11:36 PM, Masayuki Takahashi
<masayuki038@gmail.com> wrote:
> I am trying to change cstore_fdw to scan in parallel. FDW interface provide
> 'EstimateDSMForeignScan' for required shared memory size to scan in parallel.
>
> How to estimate the shared memory size required for parallel scan?

It's a slightly strange use of the word "estimate".  It means "tell me
how much shared memory you need".  Later, your
InitializeDSMForeignScan() callback will receive a pointer to exactly
that much shared memory to initialise.  Then
InitializeWorkerForeignScan will also receive a pointer to that
memory, inside every worker process.  Note that it may be mapped at a
different address in each process, so be careful not to use raw
pointers.  It's up to you to design a struct to hold whatever data,
spinlocks, LWLocks, atomics etc you might need to orchestrate your
parallel scan.

It works much the same way for built-in executor nodes that are
parallel-aware by the way.  For example, ExecHashJoinEstimate()
reserves sizeof(ParallelHashJoinState), and then in
ExecHashJoinInitializeDSM() it allocates and initialises it, and
ExecHashJoinIntializeWorker() tells the workers about it.  The
built-in executor nodes have to do a little bit more work than FDWs,
using the plan node ID to allocate and look things up in a "TOC"
(table of contents), but nodeForeignScan.c does that work for you in
your case: it just asks you how much you want, and then gives it to
you.

-- 
Thomas Munro
http://www.enterprisedb.com