[PATCH 8/8] Introduce wal decoding via catalog timetravel
От | Andres Freund |
---|---|
Тема | [PATCH 8/8] Introduce wal decoding via catalog timetravel |
Дата | |
Msg-id | 1347669575-14371-8-git-send-email-andres@2ndquadrant.com обсуждение исходный текст |
Ответ на | [RFC][PATCH] wal decoding, attempt #2 (Andres Freund <andres@2ndquadrant.com>) |
Ответы |
Re: [PATCH 8/8] Introduce wal decoding via catalog timetravel
|
Список | pgsql-hackers |
This introduces several things: * applycache module which reassembles transactions from a stream of interspersed changes * snapbuilder which builds catalog snapshots so that tuples from wal can be understood * wal decoding into an applycache * decode_xlog(lsn, lsn) debugging function The applycache provides 3 major callbacks: * apply_begin * apply_change * apply_commit It is missing several parts: - spill-to-disk - resource usage controls - command id handling - passing of the correct mvcc snapshot (already has it, just doesn't pass) The snapshot building has the most critical infrastructure but misses several important features: * loads of docs about the internals * improve snapshot building/distributions * don't build them all the time, cache them * don't increase ->xmax so slowly, its inefficient * refcount * actually free them * proper cache handling * we can probably reuse xl_xact_commit->nmsgs * generate new local inval messages from catalog changes? * handle transactions with both ddl, and changes * command_id handling * combocid loggin/handling * Add support for declaring tables as catalog tables that are not pg_catalog.* * properly distribute new SnapshotNow snapshots after a transaction commits * loads of testing/edge cases * provision of a consistent snapshot for pg_dump * spill state to disk at checkpoints * xmin handling The xlog decoding also misses several parts: - HEAP_NEWPAGE support - HEAP2_MULTI_INSERT support - handling of table rewrites --- src/backend/replication/Makefile | 2 + src/backend/replication/logical/Makefile | 19 + src/backend/replication/logical/applycache.c | 574 +++++++++++++ src/backend/replication/logical/decode.c | 366 +++++++++ src/backend/replication/logical/logicalfuncs.c | 237 ++++++ src/backend/replication/logical/snapbuild.c | 1045 ++++++++++++++++++++++++ src/backend/utils/time/tqual.c | 161 ++++ src/include/access/transam.h | 5 + src/include/catalog/pg_proc.h | 3 + src/include/replication/applycache.h | 239 ++++++ src/include/replication/decode.h | 26 + src/include/replication/snapbuild.h | 119 +++ src/include/utils/tqual.h | 21 +- 13 files changed, 2816 insertions(+), 1 deletion(-) create mode 100644 src/backend/replication/logical/Makefile create mode 100644 src/backend/replication/logical/applycache.c create mode 100644 src/backend/replication/logical/decode.c create mode 100644 src/backend/replication/logical/logicalfuncs.c create mode 100644 src/backend/replication/logical/snapbuild.c create mode 100644 src/include/replication/applycache.h create mode 100644 src/include/replication/decode.h create mode 100644 src/include/replication/snapbuild.h
Вложения
В списке pgsql-hackers по дате отправления: