Re: Replication identifiers, take 3

On 09/26/2014 10:21 AM, Andres Freund wrote:
> On 2014-09-26 09:53:09 -0400, Robert Haas wrote:
>> On Fri, Sep 26, 2014 at 5:05 AM, Andres Freund <andres@2ndquadrant.com> wrote:
>>>> Let me try to summarize the information requirements for each of these
>>>> things.  For #1, you need to know, after crash recovery, for each
>>>> standby, the last commit LSN which the client has confirmed via a
>>>> feedback message.
>>> I'm not sure I understand what you mean here? This is all happening on
>>> the *standby*. The standby needs to know, after crash recovery, the
>>> latest commit LSN from the primary that it has successfully replayed.
>> Ah, sorry, you're right: so, you need to know, after crash recovery,
>> for each machine you are replicating *from*, the last transaction (in
>> terms of LSN) from that server that you successfully replayed.
> Precisely.
>
>>> I don't think a solution which logs the change of origin will be
>>> simpler. When the origin is in every record, you can filter without keep
>>> track of any state. That's different if you can switch the origin per
>>> tx. At the very least you need a in memory entry for the origin.
>> But again, that complexity pertains only to logical decoding.
>> Somebody who wants to tweak the WAL format for an UPDATE in the future
>> doesn't need to understand how this works, or care.
> I agree that that's a worthy goal. But I don't see how this isn't the
> case with what I propose? This isn't happening on the level of
> individual rmgrs/ams - there've been two padding bytes after 'xl_rmid'
> in struct XLogRecord for a long time.
>
> There's also the significant advantage that not basing this on the xid
> allows it to work correctly with records not tied to a
> transaction. There's not that much of that happening yet, but I've
> several features in mind:
>
> * separately replicate 2PC commits. 2PC commits don't have an xid
>    anymore... With some tooling on top replication 2PC in two phases
>    allow for very cool stuff. Like optionally synchronous multimaster
>    replication.
> * I have a pending patch that allows to send 'messages' through logical
>    decoding - yielding a really fast and persistent queue. For that it's
>    useful have transactional *and* nontransactional messages.
> * Sanely replicating CONCURRENTLY stuff gets harder if you tie things to
>    the xid.

At what point in the decoding stream should something related to a 
CONCURRENTLY command show up?
Also, for a logical message queue why couldn't you have a xid associated 
with the message that had nothing else in the transaction?

l