Re: [RFC][PATCH] wal decoding, attempt #2 - Design Documents (really attached)

On 10/15/2012 3:25 PM, Andres Freund wrote:
> On Monday, October 15, 2012 09:18:57 PM Peter Geoghegan wrote:
>> On 15 October 2012 19:19, Bruce Momjian <bruce@momjian.us> wrote:
>> > I think Robert is right that if Slony can't use the API, it is unlikely
>> > any other replication system could use it.
>>
>> I don't accept that. Clearly there is a circular dependency, and
>> someone has to go first - why should the Slony guys invest in adopting
>> this technology if it is going to necessitate using a forked Postgres
>> with an uncertain future?
>
> Well. I don't think (hope) anybody proposed making something release worthy for
> slony but rather a POC patch that proofs the API is generic enough to be used
> by them. If I (or somebody else familiar with this) work together with somebody
> familiar with with slony internals I think such a POC shouldn't be too hard to
> do.
> I think some more input from that side is a good idea. I plan to send out an
> email to possibly interested parties in about two weeks...

What Slony essentially sends to the receiver node is a COPY stream in 
the format, Christopher described. That stream is directly copied into 
the receiving node's sl_log_N table and picked up there by an apply 
trigger BEFORE INSERT, that performs the corresponding 
INSERT/UPDATE/DELETE operation via prepared plans to the user tables.

For a POC I think it is sufficient to demonstrate that this copy stream 
can be generated out of the WAL decoding.

Note that Slony today does not touch columns in an UPDATE, that have not 
changed in the original UPDATE on the origin. Sending toasted column 
values, that haven't changed, would be a substantial change to the 
storage efficiency on the replica. The consequence of this is that the 
number of colums that need to be in the UPDATE's SET clause varies. The 
log_cmdupdncols is to separate the new column/value pairs from the 
column/key pairs of the updated row. The old row "key" in Slony is based 
on a unique index (preferably a PK, but any unique key will do). This 
makes that cmdupdncols simply the number of column/value pairs minus the 
number of key columns. So it isn't too hard to figure out.


Jan

-- 
Anyone who trades liberty for security deserves neither
liberty nor security. -- Benjamin Franklin