Re: Replication Node Identifiers and crashsafe Apply Progress

<div dir="ltr"><div class="gmail_extra"><br /><div class="gmail_quote">On Thu, Nov 14, 2013 at 5:26 PM, Andres Freund
<spandir="ltr"><<a href="mailto:andres@2ndquadrant.com" target="_blank">andres@2ndquadrant.com</a>></span>
wrote:<br/><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div
id=":6au"style="overflow:hidden"> But for that the receiving side needs to know up to where changes have<br /> been
applied.One relatively easy solution for that is that the<br /> receiving side does something like:<br /> UPDATE
replication_progressSET lsn = '<a href="tel:0%2F10000600" value="+35310000600">0/10000600</a>' WHERE source_id =
...;<br/> before the end of every replayed transaction. But that obviously will<br /> quickly cause bloat.<br /><br />
Oursolution to that is that a replaying process can tell the backend<br /> that it is currently doing so and setup
threevariables for every<br /> transaction:</div></blockquote></div><br /></div><div class="gmail_extra">This is a
prettymassive design decision to hinge on such a minor implementation detail of table bloat (which I don't think would
actuallybe an issue anyway -- isn't that what we have HOT for?)<br /><br /></div><div class="gmail_extra">Fundamentally
thequestion here is where to keep all the book-keeping state about replicas, in a central repository in the master or
locallyin each replica. At first blush it seems obvious to me that locally in each replica is the more flexible
choice.<br/><br /></div><div class="gmail_extra">Replication systems become complex when you start restoring from old
backupsand not every node has the same view of the topology as every other node. I fear what will happen to a central
repositorywhen you fail over the master and it's out of sync with where the slaves have actually restored up to. Or
whereyou fail over a slave to a standby of the slave and it needs to redo some of the logical replication to catch up.
Orwhere you restore all your nodes, both master and slaves from backups taken at different points in time (presumably
withthe master ahead of the slaves). <br /><br /></div><div class="gmail_extra">Having a central repository makes the
wholesystem simpler but it also makes it much more fragile. It's nice to have a single place to go to find out what the
stateof every replica is but it should do that by actually asking the replicas, not by maintaining state that might be
outof sync.<br /><br clear="all" /></div><div class="gmail_extra"><br />-- <br />greg<br /></div></div>