Re: concurrent snapshots

On Thu, Sep 8, 2011 at 11:33 AM, Ants Aasma <ants.aasma@eesti.ee> wrote:
> On Thu, Sep 8, 2011 at 5:28 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>> On Thu, Sep 8, 2011 at 9:26 AM, Ants Aasma <ants.aasma@eesti.ee> wrote:
>>> When go try to find the new csnmin
>>> and discover that a backend has a csnmin that is too old, we go through
>>> the snapshots of that backend and convert every snapshot under the
>>> desired csnmin to a traditional snapshot.
>>
>> I thought about something along these lines (though I didn't flesh out
>> the details as much as you have here), but rejected it because the
>> step described above would require all snapshots to be stored in
>> shared memory.  That's problematic for several reasons:
>>
>> 1. A backend can have lots of snapshots, potentially requiring an
>> unbounded amount of shared memory.  We can't accommodate that.
>
> If PostgreSQL gets POSIX shared memory capability at some point in the
> future, would it be enough to accommodate snapshots in shared memory?

By itself, that particular change would not help with this problem.

>> 2. You'd need to protect all of those snapshots with spinlocks or
>> something, which would be wicked expensive, because the owning process
>> would need to take and release that spinlock every time it touched the
>> snapshot.
>
> It seems to me that converting a transactions type can be done
> lock-free. The process that does the converting needs to ensure that
> the new transaction type flag is visible before releasing any xids.
> For visibility checking, the additional cost is a read barrier, two
> volatile reads (recheck snapshot type and dense horizon) and occasional
> retry after getting a visibility result.
>
> Maybe I'm missing something. I'll take a deeper look at the snapshot
> handling code tonight to see if anything else might have any
> implications.

I'm not convinced it's anywhere near that easy.  For one thing, on at
least one big server I'm playing with, memory latency on shared memory
is vastly higher (like >10x!) than on backend-local memory due to NUMA
effects.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company