Обсуждение: Re: [HACKERS] [COMMITTERS] pgsql: Release note updates.
Tom Lane wrote: > Release note updates. > > Add item for last-minute CREATE INDEX CONCURRENTLY fix. Hi, Sorry for not noticing earlier, but there is a bug in the notes: + If <command>CREATE INDEX CONCURRENTLY</> was used to build an index + that depends on a column not previously indexed, then rows inserted + or updated by transactions that ran concurrently with + the <command>CREATE INDEX</> command could have received incorrect + index entries. CIC bug does not affect inserted rows, only updated rows, since the bogus bitmap is only used to compute whether to omit index tuples for HOT considerations. Also, the bollixed rows do not receive incorrect index entries -- they just do not receive any index entry at all. May I suggest + If <command>CREATE INDEX CONCURRENTLY</> was used to build an index + that depends on a column not previously indexed, then rows + updated by transactions that ran concurrently with + the <command>CREATE INDEX</> command could have missed receiving + index entries. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Tue, Feb 7, 2017 at 5:23 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
Tom Lane wrote:
> Release note updates.
>
> Add item for last-minute CREATE INDEX CONCURRENTLY fix.
Hi,
Sorry for not noticing earlier, but there is a bug in the notes:
+ If <command>CREATE INDEX CONCURRENTLY</> was used to build an index
+ that depends on a column not previously indexed, then rows inserted
+ or updated by transactions that ran concurrently with
+ the <command>CREATE INDEX</> command could have received incorrect
+ index entries.
CIC bug does not affect inserted rows, only updated rows, since the
bogus bitmap is only used to compute whether to omit index tuples for
HOT considerations.
That's correct.
Also, the bollixed rows do not receive incorrect index entries -- they
just do not receive any index entry at all.
I think it's somewhat both. While it's true that the updated rows may not get new index entries as they deserve, they will be reachable from the older index entries. So while a query such as "SELECT * FROM tab WHERE key = newval" may not return any result, "SELECT * FROM tab WHERE key = oldval" may actually return the updated (and wrong) tuple.
Thanks,
Pavan
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
PostgreSQL Development, 24x7 Support, Training & Services
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> Tom Lane wrote:
>> Release note updates.
> Sorry for not noticing earlier, but there is a bug in the notes:
Mmm, right.
> May I suggest
> + If <command>CREATE INDEX CONCURRENTLY</> was used to build an index
> + that depends on a column not previously indexed, then rows
> + updated by transactions that ran concurrently with
> + the <command>CREATE INDEX</> command could have missed receiving
> + index entries.
Can we say "pre-existing rows that were updated by...", or is that
too optimistic?
(I fear this is too late for the current set of releases; I don't want
to make the packagers redo their work just for this. But we can correct
it for future wraps.)
regards, tom lane
Tom Lane wrote: > Alvaro Herrera <alvherre@2ndquadrant.com> writes: > > May I suggest > > > + If <command>CREATE INDEX CONCURRENTLY</> was used to build an index > > + that depends on a column not previously indexed, then rows > > + updated by transactions that ran concurrently with > > + the <command>CREATE INDEX</> command could have missed receiving > > + index entries. > > Can we say "pre-existing rows that were updated by...", or is that > too optimistic? Hmm. Now that I think about it, it is probably possible to have a transaction started before CIC that inserted a bunch of rows, and then runs the UPDATE during the CIC race window. Maybe there's a reason the bug wouldn't hit in that case but I don't see it, and I'm not able to test it right now to verify. > (I fear this is too late for the current set of releases; I don't want > to make the packagers redo their work just for this. But we can correct > it for future wraps.) I think a large fraction of the readers will grab the release notes from the website anyway, not their local copies. And the "press release" is a source that will get to a large number of readers too. I think it's fine not to re-wrap. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> Tom Lane wrote:
>> (I fear this is too late for the current set of releases; I don't want
>> to make the packagers redo their work just for this. But we can correct
>> it for future wraps.)
> I think a large fraction of the readers will grab the release notes from
> the website anyway, not their local copies. And the "press release" is
> a source that will get to a large number of readers too. I think it's
> fine not to re-wrap.
I was just wondering about that: I know the normal workflow for the
website versions of the manual is to build from the tagged release
points. I intend to put the tags on the commits the official tarballs
correspond to, but maybe we can generate the website manuals from a
later commit this time?
regards, tom lane
Alvaro Herrera wrote: > Tom Lane wrote: > > Alvaro Herrera <alvherre@2ndquadrant.com> writes: > > > > May I suggest > > > > > + If <command>CREATE INDEX CONCURRENTLY</> was used to build an index > > > + that depends on a column not previously indexed, then rows > > > + updated by transactions that ran concurrently with > > > + the <command>CREATE INDEX</> command could have missed receiving > > > + index entries. > > > > Can we say "pre-existing rows that were updated by...", or is that > > too optimistic? > > Hmm. Now that I think about it, it is probably possible to have a > transaction started before CIC that inserted a bunch of rows, and then > runs the UPDATE during the CIC race window. Maybe there's a reason the > bug wouldn't hit in that case but I don't see it, and I'm not able to > test it right now to verify. Pavan adds that it's possible to have a transaction do INSERT while CIC is running, then some other transaction executes the UPDATE. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Tue, Feb 7, 2017 at 9:29 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
Alvaro Herrera wrote:
>
> Hmm. Now that I think about it, it is probably possible to have a
> transaction started before CIC that inserted a bunch of rows, and then
> runs the UPDATE during the CIC race window. Maybe there's a reason the
> bug wouldn't hit in that case but I don't see it, and I'm not able to
> test it right now to verify.
Pavan adds that it's possible to have a transaction do INSERT while CIC
is running, then some other transaction executes the UPDATE.
Just to elaborate on that, once a backend ends up with stale cached bitmaps, AFAICS only a second relcache flush can correct that. This may not come for long time. In fact since CIC is already holding a lock on the table, I think second relcache flush will only happen at the end of phase 2. This can take a long time, especially for very large tables. In meanwhile, the compromised backend can run many transactions with the stale information. Those transactions can see not only the existing rows, but also new rows inserted by other new but now committed-good transactions.
It's all theory and I haven't had time to try this out.
Thanks,
Pavan
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
PostgreSQL Development, 24x7 Support, Training & Services