Re: GDQ iimplementation

On 5/11/2010 9:36 AM, Marko Kreen wrote:
> Seems we are in agreement.

That's always a good point to start from.

> And although PgQ can operate with any N >= 2 segments, it queries
> on 2 at a time, same as Slony.  Rest are just there to give admins
> some safety room for "OH F*CK" moments.  With short rotation times,
> it starts to seem useful..

Agreed. The rotation time should actually reflect the longest running
transactions experienced on a frequent base from the application. And
there needs to be a safeguard against rotating over even longer running
transactions.

The problem with a long running transaction is that it could have
written into log segment 1 before we switched to segment 2. We can only
TRUNCATE segment 1 after that transaction committed AND the log has been
consumed by everyone interested in it.

I am not familiar with how PgQ/Londiste do this. Slony specifically
remembers the highest XID in progress at the time of switching, waits
until the lowest XID in progress is higher than that (so all log that
ever went into that segment is now visible or aborted), then waits for
all log in that segment to be confirmed and finally truncates the log.
All this time, it needs to do the UNION query over both log segments.

 > There does not seem any advantage for querying more than 2 segments.

I didn't experiment with such implementation yet. I'll theorize about
that in a separate thread later.

>>  No. I don't know how you read 1) into the above and 2) was my
>> misunderstanding reading the Wiki. I don't want either.
>
> Oh sorry, I got that impression from wiki, not from you.
>
> As there are some ideas from you on the wiki, I assumed
> you are involved, so used 'you' very liberally.

No problem. I misinterpreted stuff there as "the currently favored idea"
too.


Jan

--
Anyone who trades liberty for security deserves neither
liberty nor security. -- Benjamin Franklin