Re: cheaper snapshots

On Thu, Jul 28, 2011 at 3:46 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
> Sounds like the right set of thoughts to be having.

Thanks.

> If you do this, you must cover subtransactions and Hot Standby. Work
> in this area takes longer than you think when you take the
> complexities into account, as you must.

Right.  This would replace the KnownAssignedXids stuff (a non-trivial
project, I am sure).

> I think you should take the premise of making snapshots O(1) and look
> at all the ways of doing that. If you grab too early at a solution you
> may grab the wrong one.

Yeah, I'm just brainstorming at this point.  This is, I think, the
best idea of what I've come up with so far, but it's definitely not
the only approach.

> For example, another approach would be to use a shared hash table.
> Snapshots are O(1), committing is O(k), using the snapshot is O(logN).
> N can be kept small by regularly pruning the hash table. If we crash
> we lose the hash table - no matter. (I'm not suggesting this is
> better, just a different approach that should be judged across
> others).

Sorry, I'm having a hard time understanding what you are describing
here.  What would the keys and values in this hash table be, and what
do k and N refer to here?

> What I'm not sure in any of these ideas is how to derive a snapshot xmin.

That is a problem.  If we have to scan the ProcArray every time we
take a snapshot just to derive an xmin, we are kind of hosed.  One
thought I had is that we might be able to use a sort of sloppy xmin.
In other words, we keep a cached xmin, and have some heuristic where
we occasionally try to update it.  A snapshot with a too-old xmin
isn't wrong, just possibly slower.  But if xmin is only slightly stale
and xids can be tested relatively quickly, it might not matter very
much.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company