Applying logical replication changes by more than one process
От | Konstantin Knizhnik |
---|---|
Тема | Applying logical replication changes by more than one process |
Дата | |
Msg-id | 56ED2DE8.1080707@postgrespro.ru обсуждение исходный текст |
Ответы |
Re: Applying logical replication changes by more than one
process
Re: Applying logical replication changes by more than one process |
Список | pgsql-hackers |
Hi, I am trying to use logical replication mechanism in implementation of PostgreSQL multimaster and faced with one conceptualproblem. Originally logical replication was intended to support asynchronous replication. In this case applying changes by singleprocess should not be a bottleneck. But if we are using distributed transaction manager to provide global consistency, then applying transaction by one worker leads to very bad performance and what is worser: cause unintended serialization of transactions, which is not takenin account by distributed deadlock detection algorithm and so can cause undetected deadlocks. So I have implemented pool of background workers which can apply transactions concurrently. It works and shows acceptable performance. But now I am thinking about HA and tracking origin LSNs which are needed to correctlyspecify slot position in case of recovery. And there is a problem: as far as I understand to correctly record originLSN in WAL and advance slot position it is necessary to setup session using replorigin_session_setup. It is not so convenient in case of using pool of background workers, because we have to setupsession for each commit. But the main problem is that for each slot session can be associated only with one process: else if (curstate->acquired_by != 0) { ereport(ERROR, (errcode(ERRCODE_OBJECT_IN_USE), errmsg("replication identifier %d is already active for PID %d", curstate->roident, curstate->acquired_by))); } Which once again means that there can be only one process applying changes. To provide correct state of replication node it is necessary to enforce that each logical replication record is replayedexactly once: we should not loose some change or try to apply it twice. So operation of recording original LSN positionin WAL and adjusting slot should be atomic. And during recovery we should restore slot current position based on the origin values extracted from WAL. I wonder if it can be done using current logical replication mechanism when changes of each slot areapplied by more than one process? Or the only alternative is to write/read origin LSNs in WAL myself, for example using custom WAL records? Thanks in advance! -- Konstantin Knizhnik Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
В списке pgsql-hackers по дате отправления: