Re: Processing long AND/OR lists

Поиск
Список
Период
Сортировка
От Christopher Browne
Тема Re: Processing long AND/OR lists
Дата
Msg-id CAFNqd5X0pRWzzz=56Pee+35de+cZZqRQ=h7mwQ32Hr8yW1jMTQ@mail.gmail.com
обсуждение исходный текст
Ответ на Re: Processing long AND/OR lists  (Tom Lane <tgl@sss.pgh.pa.us>)
Ответы Re: Processing long AND/OR lists
Re: Processing long AND/OR lists
Список pgsql-hackers
Дерево обсуждения
This situation falls from a problem that we noticed a mighty long time ago in Slony, where the set of XIDs outstanding gets very large, and, attendant to that, the set of "action id" values by which tuples are being filtered, gets correspondingly large.

It happens when there is a long pause in application of replication data, and is commonly the consequence of setting up replication on a very large data table that takes a long time for the initial data copy.

At the time, Neil Conway observed this query breakage with a query that was roughly 640K in size, from whence fell jokes to the effect, "who would ever need a query larger than 640K?"

The resolution that I introduced at the time was to write a little parser that would recognize sequences of adjacent values and merge them into "BETWEEN A and B" clauses, which would bring the query size back to a reasonable size.

In Slony 2.1, the issue re-emerged because the ordering of the "action id" values was lost; the query had previously been implicitly forcing them into order; we had to add an "ORDER BY" clause, to make the "compressor" work again.
http://git.postgresql.org/gitweb/?p=slony1-engine.git;a=blobdiff;f=src/slon/remote_worker.c;h=b1f48043f8e25b4a74a392b0dbceeae8f3e18c27;hp=7fbf67c16f97cb7c3f209cf3be903ea52c4490a9;hb=c4ac435308a78a2db63bf267d401d842c169e87d;hpb=d4612aab78bac5a9836e3e2425c403878f7091c8

Joking about "640K" aside, it doesn't seem reasonable to expect a truly enormous query as is generated by the broken forms of this logic to turn out happily.  I'd rather fix Slony (as done in the above patch).


В списке pgsql-hackers по дате отправления: