Re: Some questions about mammoth replication

Ühel kenal päeval, N, 2007-10-11 kell 18:25, kirjutas Alexey Klyukin:
> Hello,
> 
> Hannu Krosing wrote:
> > 
> > Here come my questions :
> > 
> > >From looking at http://www.commandprompt.com/images/MR_components.jpg it
> > seems that you don't do replication just from WAL logs, but also collect
> > some extra info inside postgreSQL server. Is this so ?
> >
> > If it is, then in what way does it differ from simple trigger-based
> > change logging ?
> 
> We have hooks in executor calling our own collecting functions, so we
> don't need the trigger machinery to launch replication.

But where do you store the collected info - in your own replication_log
table, or do reuse data in WAL you extract it on master befor
replication to slave (or on slave after moving the WAL) ?

> > Do you make use of snapshot data, to make sure, what parts of WAL log
> > are worth migrating to slaves , or do you just apply everything in WAL
> > in separate transactions and abort if you find out that original
> > transaction aborted ?
> 
> We check if a data transaction is recorded in WAL before sending
> it to a slave. For an aborted transaction we just discard all data collected 
> from that transaction.

Do you duplicate postgresql's MVCC code for that, or will this happen
automatically via using MVCC itself for collected data ?

How do you handle really large inserts/updates/deletes, which change say 10M 
rows in one transaction ?

> > Are your slaves a) standby b) read-only or c) read-write ?
> 
> Replicated relations are read-only on slaves.
> 
> > 
> > Do you extract / generate full sql DML queries from data in WAL logs, or
> > do you apply the changes at some lower level ?
> 
> We replicate the binary data along with a command type. Only the data
> necessary to replay the command on a slave are replicated.

Do you replay it as SQL insert/update/delete commands, or directly on
heap/indexes ?

------------
Hannu