Re: Avoiding repeated snapshot computation
От | Robert Haas |
---|---|
Тема | Re: Avoiding repeated snapshot computation |
Дата | |
Msg-id | CA+TgmoYOcJAqtdLyTkFEeLSmgbNGEbcMnrG26gm7NyqMaTDrFg@mail.gmail.com обсуждение исходный текст |
Ответ на | Avoiding repeated snapshot computation (Pavan Deolasee <pavan.deolasee@gmail.com>) |
Ответы |
Re: Avoiding repeated snapshot computation
(Pavan Deolasee <pavan.deolasee@gmail.com>)
|
Список | pgsql-hackers |
On Sat, Nov 26, 2011 at 10:52 AM, Pavan Deolasee <pavan.deolasee@gmail.com> wrote: > What we can do is when a transaction comes to compute its snapshot, it > checks if some other transaction is already computing a snapshot for > itself. If so, it just sleeps on the lock. When the other process > finishes computing the snapshot, it saves the snapshot is a shared > area and wakes up all processes waiting for the snapshot. All those > processes then just copy the snapshot from the shared area and they > are done. This will not only reduce the total CPU consumption by > avoiding repetitive work, but would also reduce the total time for > which ProcArrayLock is held in SHARED mode by avoiding pipeline of > GetSnapshotData calls. I am currently trying the shared work queue > mechanism to implement this, but I am sure we can do it this in some > other way too. I don't quite understand how this is going to work. Transaction A ends, invaliding the shared snapshot. Now transactions B, C, and D come along wishing to take snapshots. B begins taking a snapshot, so C and D wait (blech!) for that to be complete. Then, they start copying the snapshot. Meanwhile, transaction E ends, invalidating the shared snapshot, and then transaction F comes along, wanting to take a snapshot. If F constructs a new shared snapshot, then doesn't that overwrite the same memory area C and D were busy copying? It seems like F needs some way to know that C and D are done with the old shared snapshot before it starts writing a new one. Furthermore, it's hard to understand how this could be a net improvement in the general case, because now both B and F are copying everything twice (once to the shared area and one to backend-local memory) instead of just once (to backend-local memory) and C and D are sleeping (yikes!). That could maybe be a gain at very high concurrency when spinlock contention is eating us alive, but it doesn't seem like a good idea in general. Maybe I'm missing something; did you mean to attach a patch? I had a different idea for avoiding repeated snapshot computation: save the snapshot in backend-private memory. When a process takes ProcArrayLock in exclusive mode, it increments a 64-byte counter. When a process takes ProcArrayLock in shared mode, it can check whether the counter has changed; if not, it can reuse the snapshot it took before. I think there might be away to tinker with the snapshot management so as to do this without any more copying than we do presently, since CurrentSnapshotData and SecondarySnapshotData are basically treated as scratch-pads anyhow. Another idea would be to try to avoid taking ProcArrayLock at all. For example, suppose we again have a 64-bit counter in shared memory. A transaction wishing to do ProcArrayEndTransaction() takes ProcArrayLock in exclusive mode, increments the counter, memory barrier, clears its XID, memory barrier, increments the counter, releases ProcArrayLock. A transaction wishing to take a snapshot first reads the counter. If the value read from the counter is even, then we continue like this: memory barrier, take snapshot, memory barrier, recheck counter. If we find that the counter is unchanged, then nobody did ProcArrayEndTransaction() while we were taking the snapshot, and we're good to go. If the counter changed then we take ProcArrayLock in shared mode and retake the snapshot. (We also take ProcArrayLock in shared mode if the initial value we read was odd.) This seems like it would all but eliminate contention on the spinlock protecting ProcArrayLock, but I'm not sure how much overhead it would add in other cases. In particular, if we have to retake the snapshot more than very occasionally, it's not going to be good. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
В списке pgsql-hackers по дате отправления:
Следующее
От: Bruce MomjianДата:
Сообщение: Re: [COMMITTERS] pgsql: Modify pg_dump to use error-free memory allocation macros. This