pgsql: Release note updates.

Tom Lane wrote:

Release note updates.

Add item for last-minute CREATE INDEX CONCURRENTLY fix.

Hi,

Sorry for not noticing earlier, but there is a bug in the notes:

+      If <command>CREATE INDEX CONCURRENTLY</> was used to build an index
+      that depends on a column not previously indexed, then rows inserted
+      or updated by transactions that ran concurrently with
+      the <command>CREATE INDEX</> command could have received incorrect
+      index entries.

CIC bug does not affect inserted rows, only updated rows, since the
bogus bitmap is only used to compute whether to omit index tuples for
HOT considerations.

Also, the bollixed rows do not receive incorrect index entries -- they
just do not receive any index entry at all.

May I suggest

+      If <command>CREATE INDEX CONCURRENTLY</> was used to build an index
+      that depends on a column not previously indexed, then rows
+      updated by transactions that ran concurrently with
+      the <command>CREATE INDEX</> command could have missed receiving
+      index entries.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Feb 7, 2017 at 5:23 PM, Alvaro Herrera <alvherre@2ndquadrant.com>
wrote:

Tom Lane wrote:

Release note updates.

Add item for last-minute CREATE INDEX CONCURRENTLY fix.

Hi,

Sorry for not noticing earlier, but there is a bug in the notes:

+      If <command>CREATE INDEX CONCURRENTLY</> was used to build an index
+      that depends on a column not previously indexed, then rows inserted
+      or updated by transactions that ran concurrently with
+      the <command>CREATE INDEX</> command could have received incorrect
+      index entries.

CIC bug does not affect inserted rows, only updated rows, since the
bogus bitmap is only used to compute whether to omit index tuples for
HOT considerations.

That's correct.

Also, the bollixed rows do not receive incorrect index entries -- they
just do not receive any index entry at all.

I think it's somewhat both. While it's true that the updated rows may not
get new index entries as they deserve, they will be reachable from the
older index entries. So while a query such as "SELECT * FROM tab WHERE key
= newval" may not return any result, "SELECT * FROM tab WHERE key = oldval"
may actually return the updated (and wrong) tuple.

Thanks,
Pavan

--
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

Tom Lane wrote:

Release note updates.

Sorry for not noticing earlier, but there is a bug in the notes:

Mmm, right.

May I suggest

+      If <command>CREATE INDEX CONCURRENTLY</> was used to build an index
+      that depends on a column not previously indexed, then rows
+      updated by transactions that ran concurrently with
+      the <command>CREATE INDEX</> command could have missed receiving
+      index entries.

Can we say "pre-existing rows that were updated by...", or is that
too optimistic?

(I fear this is too late for the current set of releases; I don't want
to make the packagers redo their work just for this. But we can correct
it for future wraps.)

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Tom Lane wrote:

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

May I suggest

+      If <command>CREATE INDEX CONCURRENTLY</> was used to build an index
+      that depends on a column not previously indexed, then rows
+      updated by transactions that ran concurrently with
+      the <command>CREATE INDEX</> command could have missed receiving
+      index entries.

Can we say "pre-existing rows that were updated by...", or is that
too optimistic?

Hmm. Now that I think about it, it is probably possible to have a
transaction started before CIC that inserted a bunch of rows, and then
runs the UPDATE during the CIC race window. Maybe there's a reason the
bug wouldn't hit in that case but I don't see it, and I'm not able to
test it right now to verify.

(I fear this is too late for the current set of releases; I don't want
to make the packagers redo their work just for this. But we can correct
it for future wraps.)

I think a large fraction of the readers will grab the release notes from
the website anyway, not their local copies. And the "press release" is
a source that will get to a large number of readers too. I think it's
fine not to re-wrap.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

Tom Lane wrote:

(I fear this is too late for the current set of releases; I don't want
to make the packagers redo their work just for this. But we can correct
it for future wraps.)

I think a large fraction of the readers will grab the release notes from
the website anyway, not their local copies. And the "press release" is
a source that will get to a large number of readers too. I think it's
fine not to re-wrap.

I was just wondering about that: I know the normal workflow for the
website versions of the manual is to build from the tagged release
points. I intend to put the tags on the commits the official tarballs
correspond to, but maybe we can generate the website manuals from a
later commit this time?

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Alvaro Herrera wrote:

Tom Lane wrote:

Alvaro Herrera <alvherre@2ndquadrant.com> writes:

May I suggest

+      If <command>CREATE INDEX CONCURRENTLY</> was used to build an index
+      that depends on a column not previously indexed, then rows
+      updated by transactions that ran concurrently with
+      the <command>CREATE INDEX</> command could have missed receiving
+      index entries.

Can we say "pre-existing rows that were updated by...", or is that
too optimistic?

Hmm. Now that I think about it, it is probably possible to have a
transaction started before CIC that inserted a bunch of rows, and then
runs the UPDATE during the CIC race window. Maybe there's a reason the
bug wouldn't hit in that case but I don't see it, and I'm not able to
test it right now to verify.

Pavan adds that it's possible to have a transaction do INSERT while CIC
is running, then some other transaction executes the UPDATE.

--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Tue, Feb 7, 2017 at 9:29 PM, Alvaro Herrera <alvherre@2ndquadrant.com>
wrote:

Alvaro Herrera wrote:

Hmm. Now that I think about it, it is probably possible to have a
transaction started before CIC that inserted a bunch of rows, and then
runs the UPDATE during the CIC race window. Maybe there's a reason the
bug wouldn't hit in that case but I don't see it, and I'm not able to
test it right now to verify.

Pavan adds that it's possible to have a transaction do INSERT while CIC
is running, then some other transaction executes the UPDATE.

Just to elaborate on that, once a backend ends up with stale cached
bitmaps, AFAICS only a second relcache flush can correct that. This may not
come for long time. In fact since CIC is already holding a lock on the
table, I think second relcache flush will only happen at the end of phase
2. This can take a long time, especially for very large tables. In
meanwhile, the compromised backend can run many transactions with the stale
information. Those transactions can see not only the existing rows, but
also new rows inserted by other new but now committed-good transactions.

It's all theory and I haven't had time to try this out.

Thanks,
Pavan

--
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services