Обсуждение: Freezing is not WAL-logged
I just noticed that freezing a tuple (in vacuumlazy.c) is not
WAL-logged. That leads to this scenario:
postgres=# CREATE TABLE foo (bar int);
CREATE TABLE
postgres=# INSERT INTO foo VALUES (1);
INSERT 0 1
postgres=# SELECT xmin,xmax, bar FROM foo;
xmin | xmax | bar
------+------+-----
669 | 0 | 1
(1 row)
postgres=# VACUUM FREEZE foo;
VACUUM
killall -9 postgres + restart
postgres=# SELECT xmin,xmax, bar FROM foo;
xmin | xmax | bar
------+------+-----
669 | 0 | 1
(1 row)
postgres=# SELECT relminxid FROM pg_class WHERE relname='foo';
relminxid
-----------
672
(1 row)
It looks like a bug to me.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
"Heikki Linnakangas" <heikki@enterprisedb.com> writes:
> I just noticed that freezing a tuple (in vacuumlazy.c) is not
> WAL-logged.
The theory is that this doesn't matter because the tuple is committed
either way ... it's equivalent to a hint-bit update which we don't
WAL-log either.
regards, tom lane
Tom Lane wrote: > "Heikki Linnakangas" <heikki@enterprisedb.com> writes: > > I just noticed that freezing a tuple (in vacuumlazy.c) is not > > WAL-logged. > > The theory is that this doesn't matter because the tuple is committed > either way ... it's equivalent to a hint-bit update which we don't > WAL-log either. Also it'd be hugely expensive to log each freeze operation. The alternative would be to log a VACUUM FREEZE, but that has the potential to cause enormous recovery runtime. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Tom Lane wrote: > "Heikki Linnakangas" <heikki@enterprisedb.com> writes: >> I just noticed that freezing a tuple (in vacuumlazy.c) is not >> WAL-logged. > > The theory is that this doesn't matter because the tuple is committed > either way ... it's equivalent to a hint-bit update which we don't > WAL-log either. Because the relminxid-update is WAL-logged, you can get into situation where relminxid > the real smallest xid of the table, as demonstrated by the example I gave. relminxid is used to determine the safe clog cut-off point, so that's not harmless. Granted, the chances of getting data corruption from this are small, but it's possible. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Alvaro Herrera wrote: > Tom Lane wrote: >> "Heikki Linnakangas" <heikki@enterprisedb.com> writes: >>> I just noticed that freezing a tuple (in vacuumlazy.c) is not >>> WAL-logged. >> The theory is that this doesn't matter because the tuple is committed >> either way ... it's equivalent to a hint-bit update which we don'ton >> WAL-log either. > > Also it'd be hugely expensive to log each freeze operation. The > alternative would be to log a VACUUM FREEZE, but that has the potential > to cause enormous recovery runtime. Freezing isn't very common, and a single WAL record per page would be enough. I can write the patch. It does have the potential to increase recovery times, but I don't think we can just cross our fingers and hope that no crash happens after freezing some tuples. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas wrote: > Alvaro Herrera wrote: > >Tom Lane wrote: > >>"Heikki Linnakangas" <heikki@enterprisedb.com> writes: > >>>I just noticed that freezing a tuple (in vacuumlazy.c) is not > >>>WAL-logged. > >>The theory is that this doesn't matter because the tuple is committed > >>either way ... it's equivalent to a hint-bit update which we don'ton > >>WAL-log either. > > > >Also it'd be hugely expensive to log each freeze operation. The > >alternative would be to log a VACUUM FREEZE, but that has the potential > >to cause enormous recovery runtime. > > Freezing isn't very common, and a single WAL record per page would be > enough. I can write the patch. Yeah, you are right, I was thinking that we would freeze a lot of tuples a lot of the time, but this is obviously not true. > It does have the potential to increase recovery times, but I don't think > we can just cross our fingers and hope that no crash happens after > freezing some tuples. Agreed. My idea was to move relminxid to a new pg_class_nt non-transactional catalog in 8.3, but we certainly need to give a solution for the problem in 8.2. I don't remember why I concluded that we were safe against this problem, but you are right that we aren't. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Alvaro Herrera <alvherre@commandprompt.com> writes:
> My idea was to move relminxid to a new pg_class_nt non-transactional
> catalog in 8.3, but we certainly need to give a solution for the problem
> in 8.2.
Didn't we pretty much shoot down the idea of a nontransactional
extension catalog as being not worth the trouble?
> I don't remember why I concluded that we were safe against this
> problem, but you are right that we aren't.
I think there may be another set of issues here too: what about PITR?
As things stand, neither hint-bit settings nor FrozenXID replacement
are guaranteed to get propagated to a PITR slave.
We don't propagate CLOG truncation to a slave either, which means that
to some extent it'd have the ability to re-set hint bits for itself
after coming up. This isn't bulletproof though; most obviously in
the case where the PITR replay (or hot-slave lifespan) exceeded
4G transactions.
Can we fix it so it's sufficient to propagate FrozenXID replacement
to the slave? If we have to turn hint-bit setting into a logged
operation, that *will* be expensive.
regards, tom lane
On Fri, Oct 20, 2006 at 09:49:47AM +0100, Heikki Linnakangas wrote: > Alvaro Herrera wrote: > >Tom Lane wrote: > >>"Heikki Linnakangas" <heikki@enterprisedb.com> writes: > >>>I just noticed that freezing a tuple (in vacuumlazy.c) is not > >>>WAL-logged. > >>The theory is that this doesn't matter because the tuple is committed > >>either way ... it's equivalent to a hint-bit update which we don'ton > >>WAL-log either. > > > >Also it'd be hugely expensive to log each freeze operation. The > >alternative would be to log a VACUUM FREEZE, but that has the potential > >to cause enormous recovery runtime. > > Freezing isn't very common, and a single WAL record per page would be > enough. I can write the patch. > > It does have the potential to increase recovery times, but I don't think > we can just cross our fingers and hope that no crash happens after > freezing some tuples. If we're worried enough about the recovery time, AFAIK it should be safe to fire off a background process to freeze the table and let the rest of recovery proceed. Perhaps it wouldn't be too difficult to have autovacuum do this, though we'd have to decide what to do if it was disabled. -- Jim Nasby jim@nasby.net EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
Tom Lane wrote: > Alvaro Herrera <alvherre@commandprompt.com> writes: > > My idea was to move relminxid to a new pg_class_nt non-transactional > > catalog in 8.3, but we certainly need to give a solution for the problem > > in 8.2. > > Didn't we pretty much shoot down the idea of a nontransactional > extension catalog as being not worth the trouble? Yeah, for 8.2, but we said there were some things to be gained by doing things that way and agreed to do'em in 8.3. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.