Re: [RFC] CREATE QUEUE (log-only table) for londiste/pgQ ccompatibility

On 10/18/2012 09:18 PM, Christopher Browne wrote:
> On Thu, Oct 18, 2012 at 2:56 PM, Hannu Krosing <hannu@2ndquadrant.com> wrote:
>> * works as table on INSERTS up to inserting logical WAL record describing
>> the
>> insert but no data is inserted locally.
>>
>> with all things that follow from the local table having no data
>>    - unique constraints don't make sense
>>    - indexes make no sense
>>    -  updates and deletes hit no data
>>    - etc. . .
> Yep, I think I was understanding those aspects.
>
> I think I disagree that "indexes make no sense."
>
> I think that it would be meaningful to have an index type for this,
> one that is a pointer at WAL records, to enable efficiently jumping to
> the right WAL log to start accessing a data stream, given an XID.
> That's a fundamentally different sort of index than we have today
> (much the way that hash indexes, GiST indexes, and BTrees differ from
> one another).
>
> I'm having a hard time thinking about what happens if you have
> cascaded replication, and want to carry records downstream.
I'd try to keep it as similar as possible to how the "real" tables
behave in this multi-master (or "bidirectional" as the original
logical wal case was named) scenario.

I assume that the current thinking is that the replicated changes
will carry original (node id, transaxtion id) info which is used to
determine when to stop replicating in case there is more than
one node in the replication ring.

In case any changes to the resulting table are performed due to
conflict resolution this "original (node id, transaxtion id)" gets
replaced (or added ?) by the info from the node that  did the
latest changes so that the original origin node gets a chance
to examine the changes too.

This has to be pondered carefully so that the conflict resolution
chain will end at some point.

(I guess that the whole logrep design is something that should be discussed in Prague . Simon and Andres are doing a
presentation on it there and in case this ignites more discussion
it may be something warranting a separate discussion session
among all interested parties)

Hannu

> In that
> case, the XIDs from the original system aren't miscible with the XIDs
> in a message queue on a downstream database, and I'm not sure what
> we'd want to do.  Keep the original XIDs in a side attribute, maybe?
> It seems weird, at any rate.  Or perhaps data from foreign sources has
> got to go into a separate queue/'sorta-table', and thereby have two
> XIDs, the "source system XID" and the "when we loaded it in locally
> XID."