Issues with ON CONFLICT UPDATE and REINDEX CONCURRENTLY
Hello, hackers.
While testing my work on (1) I was struggling with addressing a strange
issue with ON CONFLICT UPDATE and REINDEX CONCURRENTLY.
After some time, I have realized the same issue persists on the master
branch as well :)
I have prepared two TAP tests to reproduce the issues (2), also in
attachment.
First one, does the next thing:
CREATE UNLOGGED TABLE tbl(i int primary key, updated_at timestamp);
CREATE INDEX idx ON tbl(i, updated_at); -- it is not required to
reproduce but make it to happen faster
Then it runs next scripts with pgbench concurrently:
1) INSERT INTO tbl VALUES(13,now()) on conflict(i) do update set
updated_at = now();
2) INSERT INTO tbl VALUES(42,now()) on conflict(i) do update set
updated_at = now();
3) INSERT INTO tbl VALUES(69,now()) on conflict(i) do update set
updated_at = now();
Also, during pgbench the next command is run in the loop:
REINDEX INDEX CONCURRENTLY tbl_pkey;
For some time, everything looks more-less fine (except live locks, but this
is the issue for the next test).
But after some time, about a minute or so (on ~3000th REINDEX) it just
fails like this:
make -C src/test/modules/test_misc/ check
PROVE_TESTS='t/006_*'
# waiting for an about 3000, now is 2174, seconds passed :
84
# waiting for an about 3000, now is 2175, seconds passed :
84
# waiting for an about 3000, now is 2176, seconds passed :
84
# waiting for an about 3000, now is 2177, seconds passed :
84
# waiting for an about 3000, now is 2178, seconds passed :
84
# waiting for an about 3000, now is 2179, seconds passed :
84
# waiting for an about 3000, now is 2180, seconds passed :
84
# waiting for an about 3000, now is 2181, seconds passed :
84
# waiting for an about 3000, now is 2182, seconds passed :
84
# waiting for an about 3000, now is 2183, seconds passed :
84
# waiting for an about 3000, now is 2184, seconds passed :
84
# Failed test 'concurrent INSERTs, UPDATES and RC status
(got 2 vs expected 0)'
# at t/006_concurrently_unique_fail.pl line 69.
# Failed test 'concurrent INSERTs, UPDATES and RC stderr
/(?^:^$)/'
# at t/006_concurrently_unique_fail.pl line 69.
# 'pgbench: error: pgbench: error: client
4 script 0 aborted in command 1 query 0: ERROR: duplicate key value
violates unique constraint "tbl_pkey_ccnew"
# DETAIL: Key (i)=(13) already exists.
# client 15 script 0 aborted in command 1 query 0: ERROR:
duplicate key value violates unique constraint "tbl_pkey_ccnew"
# DETAIL: Key (i)=(13) already exists.
# pgbench: error: client 9 script 0 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey_ccnew"
# DETAIL: Key (i)=(13) already exists.
# pgbench: error: client 11 script 0 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey_ccnew"
# DETAIL: Key (i)=(13) already exists.
# pgbench: error: client 8 script 0 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey_ccnew"
# DETAIL: Key (i)=(13) already exists.
# pgbench: error: client 3 script 2 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey_ccnew"
# DETAIL: Key (i)=(69) already exists.
# pgbench: error: client 2 script 2 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey_ccnew"
# DETAIL: Key (i)=(69) already exists.
# pgbench: error: client 12 script 0 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey_ccold"
# DETAIL: Key (i)=(13) already exists.
# pgbench: error: client 10 script 0 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey_ccold"
# DETAIL: Key (i)=(13) already exists.
# pgbench: error: client 18 script 2 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey_ccnew"
# DETAIL: Key (i)=(69) already exists.
# pgbench: error: pgbench:client 14 script 0 aborted in
command 1 query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey"
# DETAIL: Key (i)=(13) already exists.
# error: client 1 script 0 aborted in command 1 query 0:
ERROR: duplicate key value violates unique constraint "tbl_pkey"
# DETAIL: Key (i)=(13) already exists.
# pgbench: error: client 0 script 2 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint "tbl_pkey"
# DETAIL: Key (i)=(69) already exists.
# pgbench: error: client 13 script 1 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey_ccnew"
# DETAIL: Key (i)=(42) already exists.
# pgbench: error: client 16 script 1 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey_ccnew"
# DETAIL: Key (i)=(42) already exists.
# pgbench: error: client 5 script 1 aborted in command 1
query 0: ERROR: duplicate key value violates unique constraint
"tbl_pkey_ccnew"
# DETAIL: Key (i)=(42) already exists.
# pgbench: error: Run was aborted; the above results are
incomplete.
# '
Probably something wrong with arbiter index selection for different
backends. I am afraid it could be a symptom of a more serious issue.
-------------------------------------
The second test shows an interesting live lock state in the similar
situation.
CREATE UNLOGGED TABLE tbl(i int primary key, n int);
CREATE INDEX idx ON tbl(i, n);
INSERT INTO tbl VALUES(13,1);
pgbench concurrently runs single command
INSERT INTO tbl VALUES(13,1) on conflict(i) do update set n = tbl.n +
EXCLUDED.n;
And also reindexing in the loop
REINDEX INDEX CONCURRENTLY tbl_pkey;
After the start, a little bit strange issue happens
make -C src/test/modules/test_misc/ check PROVE_TESTS='t/007_*'
# going to start reindex, num tuples in table is 1
# reindex 0 done in 0.00704598426818848 seconds, num inserted
during reindex tuples is 0 speed is 0 per second
# going to start reindex, num tuples in table is 7
# reindex 1 done in 0.453176021575928 seconds, num inserted during
reindex tuples is 632 speed is 1394.60158947115 per second
# going to start reindex, num tuples in table is 647
# current n is 808, 808 per one second
# current n is 808, 0 per one second
# current n is 808, 0 per one second
# current n is 808, 0 per one second
# current n is 808, 0 per one second
# current n is 811, 3 per one second
# current n is 917, 106 per one second
# current n is 1024, 107 per one second
# reindex 2 done in 8.4104950428009 seconds, num inserted during
reindex tuples is 467 speed is 55.5258635340064 per second
# going to start reindex, num tuples in table is 1136
# current n is 1257, 233 per one second
# current n is 1257, 0 per one second
# current n is 1257, 0 per one second
# current n is 1257, 0 per one second
# current n is 1257, 0 per one second
# current n is 1490, 233 per one second
# reindex 3 done in 5.21368479728699 seconds, num inserted during
reindex tuples is 411 speed is 78.8310026363446 per second
# going to start reindex, num tuples in table is 1566
In some moments, all CPUs all hot but 30 connections are unable to do any
upsert. I think it may be somehow caused by two arbiter indexes (old and
new reindexed one).
Best regards,
Mikhail.
[1]: /messages/by-id/CANtu0ogBOtd9ravu1CUbuZWgq6qvn1rny38PGKDPk9zzQPH8_A@mail.gmail.com
/messages/by-id/CANtu0ogBOtd9ravu1CUbuZWgq6qvn1rny38PGKDPk9zzQPH8_A@mail.gmail.com
[2]: https://github.com/michail-nikolaev/postgres/commit/9446f944b415306d9e5d5ab98f69938d8f5ee87f
https://github.com/michail-nikolaev/postgres/commit/9446f944b415306d9e5d5ab98f69938d8f5ee87f
Attachments:
v1-0001-test-for-issue-with-upsert-fail.patchapplication/x-patch; name=v1-0001-test-for-issue-with-upsert-fail.patchDownload
From 9446f944b415306d9e5d5ab98f69938d8f5ee87f Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Sat, 8 Jun 2024 20:54:17 +0200
Subject: [PATCH v1] test for issue with upsert fail
make -C src/test/modules/test_misc/ check PROVE_TESTS='t/006_*'
test for issue with upsert livelock
make -C src/test/modules/test_misc/ check PROVE_TESTS='t/007_*'
---
.../t/006_concurrently_unique_fail.pl | 158 +++++++++++++++++
.../t/007_concurrently_unique_stuck.pl | 165 ++++++++++++++++++
2 files changed, 323 insertions(+)
create mode 100644 src/test/modules/test_misc/t/006_concurrently_unique_fail.pl
create mode 100644 src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl
diff --git a/src/test/modules/test_misc/t/006_concurrently_unique_fail.pl b/src/test/modules/test_misc/t/006_concurrently_unique_fail.pl
new file mode 100644
index 0000000000..6e08ff0812
--- /dev/null
+++ b/src/test/modules/test_misc/t/006_concurrently_unique_fail.pl
@@ -0,0 +1,158 @@
+
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+# Test REINDEX CONCURRENTLY with concurrent modifications and HOT updates
+use strict;
+use warnings;
+
+use Config;
+use Errno;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Time::HiRes qw(usleep);
+use IPC::SysV;
+use threads;
+use Time::HiRes qw( time );
+use Test::More;
+use Test::Builder;
+
+if ($@ || $windows_os)
+{
+ plan skip_all => 'Fork and shared memory are not supported by this platform';
+}
+
+my ($pid, $shmem_id, $shmem_key, $shmem_size);
+eval 'sub IPC_CREAT {0001000}' unless defined &IPC_CREAT;
+$shmem_size = 4;
+$shmem_key = rand(1000000);
+$shmem_id = shmget($shmem_key, $shmem_size, &IPC_CREAT | 0777) or die "Can't shmget: $!";
+shmwrite($shmem_id, "wait", 0, $shmem_size) or die "Can't shmwrite: $!";
+
+my $psql_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default);
+#
+# Test set-up
+#
+my ($node, $result);
+$node = PostgreSQL::Test::Cluster->new('RC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf('postgresql.conf', 'fsync = off');
+$node->append_conf('postgresql.conf', 'autovacuum = off');
+$node->start;
+$node->safe_psql('postgres', q(CREATE UNLOGGED TABLE tbl(i int primary key, updated_at timestamp)));
+
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+# IT IS NOT REQUIRED TO REPRODUCE THE ISSUE BUT MAKES IT TO HAPPEN FASTER
+$node->safe_psql('postgres', q(CREATE INDEX idx ON tbl(i, updated_at)));
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+my $builder = Test::More->builder;
+$builder->use_numbers(0);
+$builder->no_plan();
+
+my $child = $builder->child("pg_bench");
+
+if(!defined($pid = fork())) {
+ # fork returned undef, so unsuccessful
+ die "Cannot fork a child: $!";
+} elsif ($pid == 0) {
+
+ $node->pgbench(
+ '--no-vacuum --client=20 -j 2 --transactions=100000',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent INSERTs, UPDATES and RC',
+ {
+ # Ensure some HOT updates happen
+ '002_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ COMMIT;
+ ),
+ '003_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(42,now()) on conflict(i) do update set updated_at = now();
+ COMMIT;
+ ),
+ '004_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(69,now()) on conflict(i) do update set updated_at = now();
+ COMMIT;
+ ),
+ });
+
+ if ($child->is_passing()) {
+ shmwrite($shmem_id, "done", 0, $shmem_size) or die "Can't shmwrite: $!";
+ } else {
+ shmwrite($shmem_id, "fail", 0, $shmem_size) or die "Can't shmwrite: $!";
+ }
+
+ my $pg_bench_fork_flag;
+ while (1) {
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ sleep(0.1);
+ last if $pg_bench_fork_flag eq "stop";
+ }
+} else {
+ my $pg_bench_fork_flag;
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+
+ subtest 'reindex run subtest' => sub {
+ is($pg_bench_fork_flag, "wait", "pg_bench_fork_flag is correct");
+
+ my %psql = (stdin => '', stdout => '', stderr => '');
+ $psql{run} = IPC::Run::start(
+ [ 'psql', '-XA', '-f', '-', '-d', $node->connstr('postgres') ],
+ '<',
+ \$psql{stdin},
+ '>',
+ \$psql{stdout},
+ '2>',
+ \$psql{stderr},
+ $psql_timeout);
+
+ my ($result, $stdout, $stderr, $n, $begin_time, $end_time);
+
+ # IT IS NOT REQUIRED, JUST FOR CONSISTENCY
+ ($result, $stdout, $stderr) = $node->psql('postgres', q(ALTER TABLE tbl SET (parallel_workers=0);));
+ is($result, '0', 'ALTER TABLE is correct');
+
+ $begin_time = time();
+ while (1)
+ {
+ my $sql = q(REINDEX INDEX CONCURRENTLY tbl_pkey;);
+
+ ($result, $stdout, $stderr) = $node->psql('postgres', $sql);
+ is($result, '0', 'REINDEX INDEX CONCURRENTLY is correct');
+
+ $end_time = time();
+ diag('waiting for an about 3000, now is ' . $n++ . ', seconds passed : ' . int($end_time - $begin_time));
+
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag ne "wait";
+ }
+
+ # explicitly shut down psql instances gracefully
+ $psql{stdin} .= "\\q\n";
+ $psql{run}->finish;
+
+ is($pg_bench_fork_flag, "done", "pg_bench_fork_flag is correct");
+ };
+
+ $child->finalize();
+ $child->summary();
+ $node->stop;
+ done_testing();
+
+ shmwrite($shmem_id, "stop", 0, $shmem_size) or die "Can't shmwrite: $!";
+}
diff --git a/src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl b/src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl
new file mode 100644
index 0000000000..2e04b36825
--- /dev/null
+++ b/src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl
@@ -0,0 +1,165 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+# Test REINDEX CONCURRENTLY with concurrent modifications and HOT updates
+use strict;
+use warnings;
+
+use Config;
+use Errno;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Time::HiRes qw(usleep);
+use IPC::SysV;
+use threads;
+use Time::HiRes qw( time );
+use Test::More;
+use Test::Builder;
+
+if ($@ || $windows_os)
+{
+ plan skip_all => 'Fork and shared memory are not supported by this platform';
+}
+
+my ($pid, $shmem_id, $shmem_key, $shmem_size);
+eval 'sub IPC_CREAT {0001000}' unless defined &IPC_CREAT;
+$shmem_size = 4;
+$shmem_key = rand(1000000);
+$shmem_id = shmget($shmem_key, $shmem_size, &IPC_CREAT | 0777) or die "Can't shmget: $!";
+shmwrite($shmem_id, "wait", 0, $shmem_size) or die "Can't shmwrite: $!";
+
+my $psql_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default);
+#
+# Test set-up
+#
+my ($node, $result);
+$node = PostgreSQL::Test::Cluster->new('RC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf('postgresql.conf', 'fsync = off');
+$node->append_conf('postgresql.conf', 'autovacuum = off');
+$node->start;
+$node->safe_psql('postgres', q(CREATE UNLOGGED TABLE tbl(i int primary key, n int)));
+
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+# IT IS **REQUIRED** TO REPRODUCE THE ISSUE
+$node->safe_psql('postgres', q(CREATE INDEX idx ON tbl(i, n)));
+$node->safe_psql('postgres', q(INSERT INTO tbl VALUES(13,1)));
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+my $builder = Test::More->builder;
+$builder->use_numbers(0);
+$builder->no_plan();
+
+my $child = $builder->child("pg_bench");
+
+if(!defined($pid = fork())) {
+ # fork returned undef, so unsuccessful
+ die "Cannot fork a child: $!";
+} elsif ($pid == 0) {
+
+ $pid = fork();
+ if ($pid == 0) {
+ $node->pgbench(
+ '--no-vacuum --client=30 -j 2 --transactions=1000000',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent INSERTs, UPDATES and RC',
+ {
+ '002_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(13,1) on conflict(i) do update set n = tbl.n + EXCLUDED.n;
+ COMMIT;
+ ),
+ });
+
+ if ($child->is_passing()) {
+ shmwrite($shmem_id, "done", 0, $shmem_size) or die "Can't shmwrite: $!";
+ } else {
+ shmwrite($shmem_id, "fail", 0, $shmem_size) or die "Can't shmwrite: $!";
+ }
+
+ my $pg_bench_fork_flag;
+ while (1) {
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ sleep(0.1);
+ last if $pg_bench_fork_flag eq "stop";
+ }
+ }
+ else {
+ my ($result, $stdout, $stderr, $n, $prev_n, $pg_bench_fork_flag);
+ while (1) {
+ sleep(1);
+ $prev_n = $n;
+ ($result, $n, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+ diag(" current n is " . $n . ', ' . ($n - $prev_n) . ' per one second');
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag eq "stop";
+ }
+ }
+} else {
+ my $pg_bench_fork_flag;
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+
+ subtest 'reindex run subtest' => sub {
+ is($pg_bench_fork_flag, "wait", "pg_bench_fork_flag is correct");
+
+ my %psql = (stdin => '', stdout => '', stderr => '');
+ $psql{run} = IPC::Run::start(
+ [ 'psql', '-XA', '-f', '-', '-d', $node->connstr('postgres') ],
+ '<',
+ \$psql{stdin},
+ '>',
+ \$psql{stdout},
+ '2>',
+ \$psql{stderr},
+ $psql_timeout);
+
+ my ($result, $stdout, $stderr, $n, $begin_time, $end_time, $before_reindex, $after_reindex);
+
+ # IT IS NOT REQUIRED, JUST FOR CONSISTENCY
+ ($result, $stdout, $stderr) = $node->psql('postgres', q(ALTER TABLE tbl SET (parallel_workers=0);));
+ is($result, '0', 'ALTER TABLE is correct');
+
+ while (1)
+ {
+ my $sql = q(REINDEX INDEX CONCURRENTLY tbl_pkey;);
+
+ ($result, $before_reindex, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+
+ diag('going to start reindex, num tuples in table is ' . $before_reindex);
+ $begin_time = time();
+ ($result, $stdout, $stderr) = $node->psql('postgres', $sql);
+ is($result, '0', 'REINDEX INDEX CONCURRENTLY is correct');
+
+ $end_time = time();
+ ($result, $after_reindex, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+ diag('reindex ' . $n++ . ' done in ' . ($end_time - $begin_time) . ' seconds, num inserted during reindex tuples is ' . (int($after_reindex) - int($before_reindex)) . ' speed is ' . ((int($after_reindex) - int($before_reindex)) / ($end_time - $begin_time)) . ' per second');
+
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag ne "wait";
+ }
+
+ # explicitly shut down psql instances gracefully
+ $psql{stdin} .= "\\q\n";
+ $psql{run}->finish;
+
+ is($pg_bench_fork_flag, "done", "pg_bench_fork_flag is correct");
+ };
+
+ $child->finalize();
+ $child->summary();
+ $node->stop;
+ done_testing();
+
+ shmwrite($shmem_id, "stop", 0, $shmem_size) or die "Can't shmwrite: $!";
+}
--
2.34.1
On Tue, Jun 11, 2024 at 01:00:00PM +0200, Michail Nikolaev wrote:
Probably something wrong with arbiter index selection for different
backends. I am afraid it could be a symptom of a more serious issue.
ON CONFLICT selects an index that may be rebuilt in parallel of the
REINDEX happening, and its contents may be incomplete. Isn't the
issue that we may select as arbiter indexes stuff that's !indisvalid?
Using the ccnew or ccold indexes would not be correct for the conflict
resolutions.
--
Michael
Hello, Michael.
Isn't the issue that we may select as arbiter indexes stuff that's
!indisvalid?
As far as I can see (1) !indisvalid indexes are filtered out.
But... It looks like this choice is not locked in any way (2), so
index_concurrently_swap or index_concurrently_set_dead can change this
index after the decision is made, even despite WaitForLockersMultiple (3).
In some cases, it may cause a data loss...
But I was unable to reproduce that using some random usleep(), however -
maybe it is a wrong assumption.
[1]: https://github.com/postgres/postgres/blob/915de706d28c433283e9dc63701e8f978488a2b9/src/backend/optimizer/util/plancat.c#L804
https://github.com/postgres/postgres/blob/915de706d28c433283e9dc63701e8f978488a2b9/src/backend/optimizer/util/plancat.c#L804
[2]: https://github.com/postgres/postgres/blob/915de706d28c433283e9dc63701e8f978488a2b9/src/backend/optimizer/util/plancat.c#L924-L928
https://github.com/postgres/postgres/blob/915de706d28c433283e9dc63701e8f978488a2b9/src/backend/optimizer/util/plancat.c#L924-L928
[3]: https://github.com/postgres/postgres/blob/8aee330af55d8a759b2b73f5a771d9d34a7b887f/src/backend/commands/indexcmds.c#L4153
https://github.com/postgres/postgres/blob/8aee330af55d8a759b2b73f5a771d9d34a7b887f/src/backend/commands/indexcmds.c#L4153
Hello.
But I was unable to reproduce that using some random usleep(), however -
maybe it is a wrong assumption.
It seems like the assumption is correct - we may use an invalid index as
arbiter due to race condition.
The attached patch adds a check for that case, and now the test fails like
this:
# pgbench: error: client 16 script 1 aborted in command 1 query 0:
ERROR: duplicate key value violates unique constraint "tbl_pkey_ccold"
# DETAIL: Key (i)=(42) already exists.
# pgbench: error: client 9 script 1 aborted in command 1 query 0:
ERROR: ON CONFLICT does not support invalid indexes as arbiters
# pgbench: error: client 0 script 2 aborted in command 1 query 0:
ERROR: duplicate key value violates unique constraint "tbl_pkey"
# DETAIL: Key (i)=(69) already exists.
# pgbench: error: client 7 script 0 aborted in command 1 query 0:
ERROR: ON CONFLICT does not support invalid indexes as arbiters
# pgbench: error: client 10 script 0 aborted in command 1 query 0:
ERROR: ON CONFLICT does not support invalid indexes as arbiters
# pgbench: error: client 11 script 0 aborted in command 1 query 0:
ERROR: ON CONFLICT does not support invalid indexes as arbiters
I think It is even possible to see !alive index in the same situation (it
is worse case), but I was unable to reproduce it so far.
Best regards,
Mikhail.
Attachments:
upsert_issue.patchtext/x-patch; charset=US-ASCII; name=upsert_issue.patchDownload
Subject: [PATCH] error report in case of invalid index used as arbiter
test for issue with upsert fail
---
Index: src/test/modules/test_misc/t/006_concurrently_unique_fail.pl
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/test_misc/t/006_concurrently_unique_fail.pl b/src/test/modules/test_misc/t/006_concurrently_unique_fail.pl
new file mode 100644
--- /dev/null (revision 9446f944b415306d9e5d5ab98f69938d8f5ee87f)
+++ b/src/test/modules/test_misc/t/006_concurrently_unique_fail.pl (revision 9446f944b415306d9e5d5ab98f69938d8f5ee87f)
@@ -0,0 +1,158 @@
+
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+# Test REINDEX CONCURRENTLY with concurrent modifications and HOT updates
+use strict;
+use warnings;
+
+use Config;
+use Errno;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Time::HiRes qw(usleep);
+use IPC::SysV;
+use threads;
+use Time::HiRes qw( time );
+use Test::More;
+use Test::Builder;
+
+if ($@ || $windows_os)
+{
+ plan skip_all => 'Fork and shared memory are not supported by this platform';
+}
+
+my ($pid, $shmem_id, $shmem_key, $shmem_size);
+eval 'sub IPC_CREAT {0001000}' unless defined &IPC_CREAT;
+$shmem_size = 4;
+$shmem_key = rand(1000000);
+$shmem_id = shmget($shmem_key, $shmem_size, &IPC_CREAT | 0777) or die "Can't shmget: $!";
+shmwrite($shmem_id, "wait", 0, $shmem_size) or die "Can't shmwrite: $!";
+
+my $psql_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default);
+#
+# Test set-up
+#
+my ($node, $result);
+$node = PostgreSQL::Test::Cluster->new('RC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf('postgresql.conf', 'fsync = off');
+$node->append_conf('postgresql.conf', 'autovacuum = off');
+$node->start;
+$node->safe_psql('postgres', q(CREATE UNLOGGED TABLE tbl(i int primary key, updated_at timestamp)));
+
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+# IT IS NOT REQUIRED TO REPRODUCE THE ISSUE BUT MAKES IT TO HAPPEN FASTER
+$node->safe_psql('postgres', q(CREATE INDEX idx ON tbl(i, updated_at)));
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+my $builder = Test::More->builder;
+$builder->use_numbers(0);
+$builder->no_plan();
+
+my $child = $builder->child("pg_bench");
+
+if(!defined($pid = fork())) {
+ # fork returned undef, so unsuccessful
+ die "Cannot fork a child: $!";
+} elsif ($pid == 0) {
+
+ $node->pgbench(
+ '--no-vacuum --client=20 -j 2 --transactions=100000',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent INSERTs, UPDATES and RC',
+ {
+ # Ensure some HOT updates happen
+ '002_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ COMMIT;
+ ),
+ '003_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(42,now()) on conflict(i) do update set updated_at = now();
+ COMMIT;
+ ),
+ '004_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(69,now()) on conflict(i) do update set updated_at = now();
+ COMMIT;
+ ),
+ });
+
+ if ($child->is_passing()) {
+ shmwrite($shmem_id, "done", 0, $shmem_size) or die "Can't shmwrite: $!";
+ } else {
+ shmwrite($shmem_id, "fail", 0, $shmem_size) or die "Can't shmwrite: $!";
+ }
+
+ my $pg_bench_fork_flag;
+ while (1) {
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ sleep(0.1);
+ last if $pg_bench_fork_flag eq "stop";
+ }
+} else {
+ my $pg_bench_fork_flag;
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+
+ subtest 'reindex run subtest' => sub {
+ is($pg_bench_fork_flag, "wait", "pg_bench_fork_flag is correct");
+
+ my %psql = (stdin => '', stdout => '', stderr => '');
+ $psql{run} = IPC::Run::start(
+ [ 'psql', '-XA', '-f', '-', '-d', $node->connstr('postgres') ],
+ '<',
+ \$psql{stdin},
+ '>',
+ \$psql{stdout},
+ '2>',
+ \$psql{stderr},
+ $psql_timeout);
+
+ my ($result, $stdout, $stderr, $n, $begin_time, $end_time);
+
+ # IT IS NOT REQUIRED, JUST FOR CONSISTENCY
+ ($result, $stdout, $stderr) = $node->psql('postgres', q(ALTER TABLE tbl SET (parallel_workers=0);));
+ is($result, '0', 'ALTER TABLE is correct');
+
+ $begin_time = time();
+ while (1)
+ {
+ my $sql = q(REINDEX INDEX CONCURRENTLY tbl_pkey;);
+
+ ($result, $stdout, $stderr) = $node->psql('postgres', $sql);
+ is($result, '0', 'REINDEX INDEX CONCURRENTLY is correct');
+
+ $end_time = time();
+ diag('waiting for an about 3000, now is ' . $n++ . ', seconds passed : ' . int($end_time - $begin_time));
+
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag ne "wait";
+ }
+
+ # explicitly shut down psql instances gracefully
+ $psql{stdin} .= "\\q\n";
+ $psql{run}->finish;
+
+ is($pg_bench_fork_flag, "done", "pg_bench_fork_flag is correct");
+ };
+
+ $child->finalize();
+ $child->summary();
+ $node->stop;
+ done_testing();
+
+ shmwrite($shmem_id, "stop", 0, $shmem_size) or die "Can't shmwrite: $!";
+}
Index: src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl b/src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl
new file mode 100644
--- /dev/null (revision 9446f944b415306d9e5d5ab98f69938d8f5ee87f)
+++ b/src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl (revision 9446f944b415306d9e5d5ab98f69938d8f5ee87f)
@@ -0,0 +1,165 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+# Test REINDEX CONCURRENTLY with concurrent modifications and HOT updates
+use strict;
+use warnings;
+
+use Config;
+use Errno;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Time::HiRes qw(usleep);
+use IPC::SysV;
+use threads;
+use Time::HiRes qw( time );
+use Test::More;
+use Test::Builder;
+
+if ($@ || $windows_os)
+{
+ plan skip_all => 'Fork and shared memory are not supported by this platform';
+}
+
+my ($pid, $shmem_id, $shmem_key, $shmem_size);
+eval 'sub IPC_CREAT {0001000}' unless defined &IPC_CREAT;
+$shmem_size = 4;
+$shmem_key = rand(1000000);
+$shmem_id = shmget($shmem_key, $shmem_size, &IPC_CREAT | 0777) or die "Can't shmget: $!";
+shmwrite($shmem_id, "wait", 0, $shmem_size) or die "Can't shmwrite: $!";
+
+my $psql_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default);
+#
+# Test set-up
+#
+my ($node, $result);
+$node = PostgreSQL::Test::Cluster->new('RC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf('postgresql.conf', 'fsync = off');
+$node->append_conf('postgresql.conf', 'autovacuum = off');
+$node->start;
+$node->safe_psql('postgres', q(CREATE UNLOGGED TABLE tbl(i int primary key, n int)));
+
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+# IT IS **REQUIRED** TO REPRODUCE THE ISSUE
+$node->safe_psql('postgres', q(CREATE INDEX idx ON tbl(i, n)));
+$node->safe_psql('postgres', q(INSERT INTO tbl VALUES(13,1)));
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+my $builder = Test::More->builder;
+$builder->use_numbers(0);
+$builder->no_plan();
+
+my $child = $builder->child("pg_bench");
+
+if(!defined($pid = fork())) {
+ # fork returned undef, so unsuccessful
+ die "Cannot fork a child: $!";
+} elsif ($pid == 0) {
+
+ $pid = fork();
+ if ($pid == 0) {
+ $node->pgbench(
+ '--no-vacuum --client=30 -j 2 --transactions=1000000',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent INSERTs, UPDATES and RC',
+ {
+ '002_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(13,1) on conflict(i) do update set n = tbl.n + EXCLUDED.n;
+ COMMIT;
+ ),
+ });
+
+ if ($child->is_passing()) {
+ shmwrite($shmem_id, "done", 0, $shmem_size) or die "Can't shmwrite: $!";
+ } else {
+ shmwrite($shmem_id, "fail", 0, $shmem_size) or die "Can't shmwrite: $!";
+ }
+
+ my $pg_bench_fork_flag;
+ while (1) {
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ sleep(0.1);
+ last if $pg_bench_fork_flag eq "stop";
+ }
+ }
+ else {
+ my ($result, $stdout, $stderr, $n, $prev_n, $pg_bench_fork_flag);
+ while (1) {
+ sleep(1);
+ $prev_n = $n;
+ ($result, $n, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+ diag(" current n is " . $n . ', ' . ($n - $prev_n) . ' per one second');
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag eq "stop";
+ }
+ }
+} else {
+ my $pg_bench_fork_flag;
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+
+ subtest 'reindex run subtest' => sub {
+ is($pg_bench_fork_flag, "wait", "pg_bench_fork_flag is correct");
+
+ my %psql = (stdin => '', stdout => '', stderr => '');
+ $psql{run} = IPC::Run::start(
+ [ 'psql', '-XA', '-f', '-', '-d', $node->connstr('postgres') ],
+ '<',
+ \$psql{stdin},
+ '>',
+ \$psql{stdout},
+ '2>',
+ \$psql{stderr},
+ $psql_timeout);
+
+ my ($result, $stdout, $stderr, $n, $begin_time, $end_time, $before_reindex, $after_reindex);
+
+ # IT IS NOT REQUIRED, JUST FOR CONSISTENCY
+ ($result, $stdout, $stderr) = $node->psql('postgres', q(ALTER TABLE tbl SET (parallel_workers=0);));
+ is($result, '0', 'ALTER TABLE is correct');
+
+ while (1)
+ {
+ my $sql = q(REINDEX INDEX CONCURRENTLY tbl_pkey;);
+
+ ($result, $before_reindex, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+
+ diag('going to start reindex, num tuples in table is ' . $before_reindex);
+ $begin_time = time();
+ ($result, $stdout, $stderr) = $node->psql('postgres', $sql);
+ is($result, '0', 'REINDEX INDEX CONCURRENTLY is correct');
+
+ $end_time = time();
+ ($result, $after_reindex, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+ diag('reindex ' . $n++ . ' done in ' . ($end_time - $begin_time) . ' seconds, num inserted during reindex tuples is ' . (int($after_reindex) - int($before_reindex)) . ' speed is ' . ((int($after_reindex) - int($before_reindex)) / ($end_time - $begin_time)) . ' per second');
+
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag ne "wait";
+ }
+
+ # explicitly shut down psql instances gracefully
+ $psql{stdin} .= "\\q\n";
+ $psql{run}->finish;
+
+ is($pg_bench_fork_flag, "done", "pg_bench_fork_flag is correct");
+ };
+
+ $child->finalize();
+ $child->summary();
+ $node->stop;
+ done_testing();
+
+ shmwrite($shmem_id, "stop", 0, $shmem_size) or die "Can't shmwrite: $!";
+}
Index: src/backend/executor/execIndexing.c
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
--- a/src/backend/executor/execIndexing.c (revision cb460d43c12db970056b2e994e023de06eabc494)
+++ b/src/backend/executor/execIndexing.c (revision dd72fa154aed98254b47637fed62a1d584131dbf)
@@ -587,6 +587,13 @@
indexRelation->rd_index->indexrelid))
continue;
+ if (!indexRelation->rd_index->indisvalid)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("ON CONFLICT does not support invalid indexes as arbiters"),
+ errtableconstraint(heapRelation,
+ RelationGetRelationName(indexRelation))));
+
if (!indexRelation->rd_index->indimmediate)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
Hello, everyone.
I think It is even possible to see !alive index in the same situation (it
is worse case), but I was unable to reproduce it so far.
Fortunately, it is not possible.
So, seems like I have found the source of the problem:
1) infer_arbiter_indexes calls RelationGetIndexList to get the list of
candidates.
It does no lock selected indexes in any additional way which
prevents index_concurrently_swap changing them (set and clear validity).
RelationGetIndexList relcache.c:4857
infer_arbiter_indexes plancat.c:780
make_modifytable createplan.c:7097 ----------
node->arbiterIndexes = infer_arbiter_indexes(root);
create_modifytable_plan createplan.c:2826
create_plan_recurse createplan.c:532
create_plan createplan.c:349
standard_planner planner.c:421
planner planner.c:282
pg_plan_query postgres.c:904
pg_plan_queries postgres.c:996
exec_simple_query postgres.c:1193
2) other backend marks some index as invalid and commits
index_concurrently_swap index.c:1600
ReindexRelationConcurrently indexcmds.c:4115
ReindexIndex indexcmds.c:2814
ExecReindex indexcmds.c:2743
ProcessUtilitySlow utility.c:1567
standard_ProcessUtility utility.c:1067
ProcessUtility utility.c:523
PortalRunUtility pquery.c:1158
PortalRunMulti pquery.c:1315
PortalRun pquery.c:791
exec_simple_query postgres.c:1274
3) first backend invalidates catalog snapshot because transactional snapshot
InvalidateCatalogSnapshot snapmgr.c:426
GetTransactionSnapshot snapmgr.c:278
PortalRunMulti pquery.c:1244
PortalRun pquery.c:791
exec_simple_query postgres.c:1274
4) first backend copies indexes selected using previous catalog snapshot
ExecInitModifyTable nodeModifyTable.c:4499 --------
resultRelInfo->ri_onConflictArbiterIndexes = node->arbiterIndexes;
ExecInitNode execProcnode.c:177
InitPlan execMain.c:966
standard_ExecutorStart execMain.c:261
ExecutorStart execMain.c:137
ProcessQuery pquery.c:155
PortalRunMulti pquery.c:1277
PortalRun pquery.c:791
exec_simple_query postgres.c:1274
5) then reads indexes using new fresh snapshot
RelationGetIndexList relcache.c:4816
ExecOpenIndices execIndexing.c:175
ExecInsert nodeModifyTable.c:792 -------------
ExecOpenIndices(resultRelInfo, onconflict != ONCONFLICT_NONE);
ExecModifyTable nodeModifyTable.c:4059
ExecProcNodeFirst execProcnode.c:464
ExecProcNode executor.h:274
ExecutePlan execMain.c:1646
standard_ExecutorRun execMain.c:363
ExecutorRun execMain.c:304
ProcessQuery pquery.c:160
PortalRunMulti pquery.c:1277
PortalRun pquery.c:791
exec_simple_query postgres.c:1274
5) and uses arbiter selected with stale snapshot with new index view
(marked as invalid)
ExecInsert nodeModifyTable.c:1016 -------------- arbiterIndexes
= resultRelInfo->ri_onConflictArbiterIndexes;
............
ExecInsert nodeModifyTable.c:1048 ---------------if
(!ExecCheckIndexConstraints(resultRelInfo, slot, estate, conflictTid,
arbiterIndexes))
ExecModifyTable nodeModifyTable.c:4059
ExecProcNodeFirst execProcnode.c:464
ExecProcNode executor.h:274
ExecutePlan execMain.c:1646
standard_ExecutorRun execMain.c:363
ExecutorRun execMain.c:304
ProcessQuery pquery.c:160
PortalRunMulti pquery.c:1277
PortalRun pquery.c:791
exec_simple_query postgres.c:1274
I have attached an updated test for the issue (it fails on assert quickly
and uses only 2 backends).
The same issue may happen in case of CREATE/DROP INDEX CONCURRENTLY as well.
The simplest possible fix is to use ShareLock
instead ShareUpdateExclusiveLock in the index_concurrently_swap
oldClassRel = relation_open(oldIndexId, ShareLock);
newClassRel = relation_open(newIndexId, ShareLock);
But this is not a "concurrent" way. But such update should be fast enough
as far as I understand.
Best regards,
Mikhail.
Attachments:
better_test_error_report_in_case_of_invalid_index_used_as_arbiter_test_for_issue_with_upse.patchtext/x-patch; charset=US-ASCII; name=better_test_error_report_in_case_of_invalid_index_used_as_arbiter_test_for_issue_with_upse.patchDownload
Subject: [PATCH] better test
error report in case of invalid index used as arbiter
test for issue with upsert fail
---
Index: src/test/modules/test_misc/t/006_concurrently_unique_fail.pl
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/test_misc/t/006_concurrently_unique_fail.pl b/src/test/modules/test_misc/t/006_concurrently_unique_fail.pl
new file mode 100644
--- /dev/null (revision 5c5bcc5b7cc83381a6e479decd4252f3897b69bd)
+++ b/src/test/modules/test_misc/t/006_concurrently_unique_fail.pl (revision 5c5bcc5b7cc83381a6e479decd4252f3897b69bd)
@@ -0,0 +1,44 @@
+
+# Copyright (c) 2024-2024, PostgreSQL Global Development Group
+
+# Test REINDEX INDEX CONCURRENTLY with concurrent INSERT
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my ($node, $result);
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->start;
+$node->safe_psql('postgres', q(CREATE UNLOGGED TABLE tbl(i int primary key, updated_at timestamp)));
+
+$node->pgbench(
+ '--no-vacuum --client=2 -j 2 --transactions=1000',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent INSERTs, UPDATES and RC',
+ {
+ '01_updates' => q(
+ INSERT INTO tbl VALUES(13,now()) ON CONFLICT(i) DO UPDATE SET updated_at = now();
+ ),
+ '02_reindex' => q(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \gset
+ \if :gotlock
+ REINDEX INDEX CONCURRENTLY tbl_pkey;
+ SELECT pg_advisory_unlock(42);
+ \endif
+ ),
+ });
+$node->stop;
+done_testing();
\ No newline at end of file
Index: src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl b/src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl
new file mode 100644
--- /dev/null (revision 9446f944b415306d9e5d5ab98f69938d8f5ee87f)
+++ b/src/test/modules/test_misc/t/007_concurrently_unique_stuck.pl (revision 9446f944b415306d9e5d5ab98f69938d8f5ee87f)
@@ -0,0 +1,165 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+# Test REINDEX CONCURRENTLY with concurrent modifications and HOT updates
+use strict;
+use warnings;
+
+use Config;
+use Errno;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Time::HiRes qw(usleep);
+use IPC::SysV;
+use threads;
+use Time::HiRes qw( time );
+use Test::More;
+use Test::Builder;
+
+if ($@ || $windows_os)
+{
+ plan skip_all => 'Fork and shared memory are not supported by this platform';
+}
+
+my ($pid, $shmem_id, $shmem_key, $shmem_size);
+eval 'sub IPC_CREAT {0001000}' unless defined &IPC_CREAT;
+$shmem_size = 4;
+$shmem_key = rand(1000000);
+$shmem_id = shmget($shmem_key, $shmem_size, &IPC_CREAT | 0777) or die "Can't shmget: $!";
+shmwrite($shmem_id, "wait", 0, $shmem_size) or die "Can't shmwrite: $!";
+
+my $psql_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default);
+#
+# Test set-up
+#
+my ($node, $result);
+$node = PostgreSQL::Test::Cluster->new('RC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf('postgresql.conf', 'fsync = off');
+$node->append_conf('postgresql.conf', 'autovacuum = off');
+$node->start;
+$node->safe_psql('postgres', q(CREATE UNLOGGED TABLE tbl(i int primary key, n int)));
+
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+# IT IS **REQUIRED** TO REPRODUCE THE ISSUE
+$node->safe_psql('postgres', q(CREATE INDEX idx ON tbl(i, n)));
+$node->safe_psql('postgres', q(INSERT INTO tbl VALUES(13,1)));
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+my $builder = Test::More->builder;
+$builder->use_numbers(0);
+$builder->no_plan();
+
+my $child = $builder->child("pg_bench");
+
+if(!defined($pid = fork())) {
+ # fork returned undef, so unsuccessful
+ die "Cannot fork a child: $!";
+} elsif ($pid == 0) {
+
+ $pid = fork();
+ if ($pid == 0) {
+ $node->pgbench(
+ '--no-vacuum --client=30 -j 2 --transactions=1000000',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent INSERTs, UPDATES and RC',
+ {
+ '002_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(13,1) on conflict(i) do update set n = tbl.n + EXCLUDED.n;
+ COMMIT;
+ ),
+ });
+
+ if ($child->is_passing()) {
+ shmwrite($shmem_id, "done", 0, $shmem_size) or die "Can't shmwrite: $!";
+ } else {
+ shmwrite($shmem_id, "fail", 0, $shmem_size) or die "Can't shmwrite: $!";
+ }
+
+ my $pg_bench_fork_flag;
+ while (1) {
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ sleep(0.1);
+ last if $pg_bench_fork_flag eq "stop";
+ }
+ }
+ else {
+ my ($result, $stdout, $stderr, $n, $prev_n, $pg_bench_fork_flag);
+ while (1) {
+ sleep(1);
+ $prev_n = $n;
+ ($result, $n, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+ diag(" current n is " . $n . ', ' . ($n - $prev_n) . ' per one second');
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag eq "stop";
+ }
+ }
+} else {
+ my $pg_bench_fork_flag;
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+
+ subtest 'reindex run subtest' => sub {
+ is($pg_bench_fork_flag, "wait", "pg_bench_fork_flag is correct");
+
+ my %psql = (stdin => '', stdout => '', stderr => '');
+ $psql{run} = IPC::Run::start(
+ [ 'psql', '-XA', '-f', '-', '-d', $node->connstr('postgres') ],
+ '<',
+ \$psql{stdin},
+ '>',
+ \$psql{stdout},
+ '2>',
+ \$psql{stderr},
+ $psql_timeout);
+
+ my ($result, $stdout, $stderr, $n, $begin_time, $end_time, $before_reindex, $after_reindex);
+
+ # IT IS NOT REQUIRED, JUST FOR CONSISTENCY
+ ($result, $stdout, $stderr) = $node->psql('postgres', q(ALTER TABLE tbl SET (parallel_workers=0);));
+ is($result, '0', 'ALTER TABLE is correct');
+
+ while (1)
+ {
+ my $sql = q(REINDEX INDEX CONCURRENTLY tbl_pkey;);
+
+ ($result, $before_reindex, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+
+ diag('going to start reindex, num tuples in table is ' . $before_reindex);
+ $begin_time = time();
+ ($result, $stdout, $stderr) = $node->psql('postgres', $sql);
+ is($result, '0', 'REINDEX INDEX CONCURRENTLY is correct');
+
+ $end_time = time();
+ ($result, $after_reindex, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+ diag('reindex ' . $n++ . ' done in ' . ($end_time - $begin_time) . ' seconds, num inserted during reindex tuples is ' . (int($after_reindex) - int($before_reindex)) . ' speed is ' . ((int($after_reindex) - int($before_reindex)) / ($end_time - $begin_time)) . ' per second');
+
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag ne "wait";
+ }
+
+ # explicitly shut down psql instances gracefully
+ $psql{stdin} .= "\\q\n";
+ $psql{run}->finish;
+
+ is($pg_bench_fork_flag, "done", "pg_bench_fork_flag is correct");
+ };
+
+ $child->finalize();
+ $child->summary();
+ $node->stop;
+ done_testing();
+
+ shmwrite($shmem_id, "stop", 0, $shmem_size) or die "Can't shmwrite: $!";
+}
Index: src/backend/executor/execIndexing.c
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
--- a/src/backend/executor/execIndexing.c (revision cb460d43c12db970056b2e994e023de06eabc494)
+++ b/src/backend/executor/execIndexing.c (revision 5c5bcc5b7cc83381a6e479decd4252f3897b69bd)
@@ -587,6 +587,9 @@
indexRelation->rd_index->indexrelid))
continue;
+ Assert(indexRelation->rd_index->indislive);
+ Assert(indexRelation->rd_index->indisvalid);
+
if (!indexRelation->rd_index->indimmediate)
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
On Mon, Jun 17, 2024 at 07:00:51PM +0200, Michail Nikolaev wrote:
The simplest possible fix is to use ShareLock
instead ShareUpdateExclusiveLock in the index_concurrently_swapoldClassRel = relation_open(oldIndexId, ShareLock);
newClassRel = relation_open(newIndexId, ShareLock);But this is not a "concurrent" way. But such update should be fast enough
as far as I understand.
Nope, that won't fly far. We should not use a ShareLock in this step
or we are going to conflict with row exclusive locks, impacting all
workloads when doing a REINDEX CONCURRENTLY.
That may be a long shot, but the issue is that we do the swap of all
the indexes in a single transaction, but do not wait for them to
complete when committing the swap's transaction in phase 4. Your
report is telling us that we really have a good reason to wait for all
the transactions that may use these indexes to finish. One thing
coming on top of my mind to keep things concurrent-safe while allowing
a clean use of the arbiter indexes would be to stick a
WaitForLockersMultiple() on AccessExclusiveLock just *before* the
transaction commit of phase 4, say, lacking the progress report part:
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -4131,6 +4131,8 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
CommandCounterIncrement();
}
+ WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+
/* Commit this transaction and make index swaps visible */
CommitTransactionCommand();
StartTransactionCommand();
This is a non-fresh Friday-afternoon idea, but it would make sure that
we don't have any transactions using the indexes switched to _ccold
with indisvalid that are waiting for a drop in phase 5. Your tests
seem to pass with that, and that keeps the operation intact
concurrent-wise (I'm really wishing for isolation tests with injection
points just now, because I could use them here).
+ Assert(indexRelation->rd_index->indislive); + Assert(indexRelation->rd_index->indisvalid); + if (!indexRelation->rd_index->indimmediate) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
This kind of validation check may be a good idea in the long term.
That seems incredibly useful to me if we were to add more code paths
that do concurrent index rebuilds, to make sure that we don't rely on
an index we should not use at all. That's a HEAD-only thing IMO,
though.
--
Michael
Hello, Michael!
This is a non-fresh Friday-afternoon idea, but it would make sure that
we don't have any transactions using the indexes switched to _ccold
with indisvalid that are waiting for a drop in phase 5. Your tests
seem to pass with that, and that keeps the operation intact
concurrent-wise (I'm really wishing for isolation tests with injection
points just now, because I could use them here).
Yes, I also have tried that approach, but it doesn't work, unfortunately.
You may fail test increasing number of connections:
'--no-vacuum --client=10 -j 2 --transactions=1000',
The source of the issue is not the swap of the indexes (and not related to
REINDEX CONCURRENTLY only), but the fact that indexes are fetched once
during planning (to find the arbiter), but then later reread with a new
catalog snapshot for the the actual execution.
So, other possible fixes I see:
* fallback to replanning in case we see something changed during the
execution
* select arbiter indexes during actual execution
That's a HEAD-only thing IMO,
though.
Do you mean that it needs to be moved to a separate patch?
Best regards,
Mikhail.
On Mon, Jun 17, 2024 at 07:00:51PM +0200, Michail Nikolaev wrote:
The same issue may happen in case of CREATE/DROP INDEX CONCURRENTLY as well.
While looking at all that, I've been also curious about this specific
point, and it is indeed possible to finish in a state where a
duplicate key would be found in one of indexes selected by the
executor during an INSERT ON CONFLICT while a concurrent set of CICs
and DICs are run, so you don't really need a REINDEX. See for example
the attached test.
--
Michael
Attachments:
test_cic_dic.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/test/modules/test_misc/t/009_cdic_concurrently_unique_fail.pl b/src/test/modules/test_misc/t/009_cdic_concurrently_unique_fail.pl
new file mode 100644
index 0000000000..bd37c797a3
--- /dev/null
+++ b/src/test/modules/test_misc/t/009_cdic_concurrently_unique_fail.pl
@@ -0,0 +1,46 @@
+
+# Copyright (c) 2024-2024, PostgreSQL Global Development Group
+
+# Test CREATE/DROP INDEX CONCURRENTLY with concurrent INSERT
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my ($node, $result);
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->start;
+$node->safe_psql('postgres', q(CREATE UNLOGGED TABLE tbl(i int primary key, updated_at timestamp)));
+$node->safe_psql('postgres', q(CREATE UNIQUE INDEX tbl_pkey_2 ON tbl (i)));
+
+$node->pgbench(
+ '--no-vacuum --client=100 -j 2 --transactions=1000',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent INSERTs, UPDATES and CIC/DIC',
+ {
+ '01_updates' => q(
+ INSERT INTO tbl VALUES(13,now()) ON CONFLICT(i) DO UPDATE SET updated_at = now();
+ ),
+ '02_reindex' => q(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \gset
+ \if :gotlock
+ DROP INDEX CONCURRENTLY tbl_pkey_2;
+ CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_2 ON tbl (i);
+ SELECT pg_advisory_unlock(42);
+ \endif
+ ),
+ });
+$node->stop;
+done_testing();
On Fri, Jun 21, 2024 at 11:31:21AM +0200, Michail Nikolaev wrote:
Yes, I also have tried that approach, but it doesn't work, unfortunately.
You may fail test increasing number of connections:'--no-vacuum --client=10 -j 2 --transactions=1000',
The source of the issue is not the swap of the indexes (and not related to
REINDEX CONCURRENTLY only), but the fact that indexes are fetched once
during planning (to find the arbiter), but then later reread with a new
catalog snapshot for the the actual execution.
When I first saw this report, my main worry was that I have somewhat
managed to break the state of the indexes leading to data corruption
because of an incorrect step in the concurrent operations. However,
as far as I can see this is not the case, as an effect of two
properties we rely on for concurrent index operations, that hold in
the executor and the planner. Simply put:
- The planner ignores indexes with !indisvalid.
- The executor ignores indexes with !indislive.
The critical point is that we wait in DROP INDEX CONC and REINDEX CONC
for any transactions still using an index that's waiting to be marked
as !indislive, because such indexes *must* not be used in the
executor.
So, other possible fixes I see:
* fallback to replanning in case we see something changed during the
execution
* select arbiter indexes during actual execution
These two properties make ON CONFLICT react the way it should
depending on the state of the indexes selected by the planner based on
the query clauses, with changes reflecting when executing, with two
patterns involved:
- An index may be created in a concurrent operation after the planner
has selected the arbiter indexes (the index may be defined, still not
valid yet, or just created after), then the query execution would need
to handle the extra index created available at execution, with a
failure on a ccnew index.
- An index may be selected at planning phase, then a different index
could be used by a constraint once both indexes swap, with a failure
on a ccold index.
As far as I can see, it depends on what kind of query semantics and
the amount of transparency you are looking for here in your
application. An error in the query itself can also be defined as
useful so as your application is aware of what happens as an effect of
the concurrent index build (reindex or CIC/DIC), and it is not really
clear to me why silently falling back to a re-selection of the arbiter
indexes would be always better. Replanning could be actually
dangerous if a workload is heavily concurrently REINDEX'd, as we could
fall into a trap where a query can never decide which index to use.
I'm not saying that it cannot be improved, but it's not completely
clear to me what query semantics are the best for all users because
the behavior of HEAD and your suggestions have merits and demerits.
Anything we could come up with would be an improvement anyway, AFAIU.
That's a HEAD-only thing IMO,
though.Do you mean that it needs to be moved to a separate patch?
It should, but I'm wondering if that's necessary for two reasons.
First, a check on indisvalid would be incorrect, because indexes
marked as !indisvalid && indislive mean that there is a concurrent
operation happening, and that this concurrent operation is waiting for
all transactions working with a lock on this index to finish before
flipping the live flag and make this index invalid for decisions taken
in the executor, like HOT updates, etc.
A check on indislive may be an idea, still I'm slightly biased
regarding its additional value because any indexes opened for a
relation are fetched from the relcache with RelationGetIndexList()
explaining why indislive indexes cannot be fetched, and we rely on
that in the executor for the indexes opened by a relation.
--
Michael
Hello, Michael!
As far as I can see, it depends on what kind of query semantics and
the amount of transparency you are looking for here in your
application. An error in the query itself can also be defined as
useful so as your application is aware of what happens as an effect of
the concurrent index build (reindex or CIC/DIC), and it is not really
clear to me why silently falling back to a re-selection of the arbiter
indexes would be always better.
From my point of view, INSERT ON CONFLICT UPDATE should never fail with
"ERROR: duplicate key value violates unique constraint" because the main
idea of upsert is to avoid such situations.
So, it is expected by majority and, probably, is even documented.
On the other side, REINDEX CONCURRENTLY should not cause any queries to
fail accidentally without any clear reason.
Also, as you can see from the topic starter letter, we could see errors
like this:
* ERROR: duplicate key value violates unique constraint "tbl_pkey"
* ERROR: duplicate key value violates unique constraint "tbl_pkey_ccnew"
* ERROR: duplicate key value violates unique constraint "tbl_pkey_ccold"
So, the first error message does not provide any clue for the developer to
understand what happened.
- The planner ignores indexes with !indisvalid.
- The executor ignores indexes with !indislive.
Yes, and it feels like we need one more flag here to distinguish
!indisvalid indexes which are going to become valid and which are going to
become !indislive.
For example, let name it as indiscorrect (it means it contains all the
data). In such case, we may use the following logic:
1) !indisvalid && !indiscorrect - index in validation phase probably, do
not use it as arbiter because it does not contain all the data yet
2) !indisvalid && indiscorrect - index will be dropped most likely. Do not
plan new queries with it, but it still may be used by other queries
(including upserts). So, we still need to include it to the arbiters.
And, during the reindex concurrently:
1) begin; mark new index as indisvalid and indiscorrect; mark old one as
!indisvalid but still indiscorrect. invalidate relcache; commit;
Currently, some queries are still using the old one as arbiter, some
queries use both.
2) WaitForLockersMultiple
Now all queries use both indexes as arbiter.
3) begin; mark old index as !indiscorrect, additionally to !indisvalid;
invalidate cache; commit;
Now, some queries use only the new index, both some still use both.
4) WaitForLockersMultiple;
Now, all queries use only the new index - we are safe to mark the old
one it as !indislive.
It should, but I'm wondering if that's necessary for two reasons.
In that case, it becomes:
Assert(indexRelation->rd_index->indiscorrect);
Assert(indexRelation->rd_index->indislive);
and it is always the valid check.
Best regards,
Mikhail.
Hello, Noah!
Answering
/messages/by-id/20240612194857.1c.nmisch@google.com
On your other thread, it would be useful to see stack traces from the
high-CPU
processes once the live lock has ended all query completion.
I was wrong, it is not a livelock, it is a deadlock, actually. I missed it
because pgbench retries deadlocks automatically.
It looks like this:
2024-06-25 17:16:17.447 CEST [711743] 007_concurrently_unique_stuck.pl
ERROR: deadlock detected
2024-06-25 17:16:17.447 CEST [711743] 007_concurrently_unique_stuck.pl
DETAIL: Process 711743 waits for ShareLock on transaction 3633; blocked by
process 711749.
Process 711749 waits for ShareLock on speculative token 2 of transaction
3622; blocked by process 711743.
Process 711743: INSERT INTO tbl VALUES(13,89318) on conflict(i) do update
set n = tbl.n + 1 RETURNING n
Process 711749: INSERT INTO tbl VALUES(13,41011) on conflict(i) do update
set n = tbl.n + 1 RETURNING n
2024-06-25 17:16:17.447 CEST [711743] 007_concurrently_unique_stuck.pl
HINT: See server log for query details.
2024-06-25 17:16:17.447 CEST [711743] 007_concurrently_unique_stuck.pl
CONTEXT: while inserting index tuple (15,145) in relation "tbl_pkey_ccnew"
2024-06-25 17:16:17.447 CEST [711743] 007_concurrently_unique_stuck.pl
STATEMENT: INSERT INTO tbl VALUES(13,89318) on conflict(i) do update set n
= tbl.n + 1 RETURNING n
Stacktraces:
-------------------------
INSERT INTO tbl VALUES(13,41011) on conflict(i) do update set n = tbl.n + 1
RETURNING n
#0 in epoll_wait (epfd=5, events=0x1203328, maxevents=1, timeout=-1) at
../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1 in WaitEventSetWaitBlock (set=0x12032c0, cur_timeout=-1,
occurred_events=0x7ffcc4e38e30, nevents=1) at latch.c:1570
#2 in WaitEventSetWait (set=0x12032c0, timeout=-1,
occurred_events=0x7ffcc4e38e30, nevents=1, wait_event_info=50331655) at
latch.c:1516
#3 in WaitLatch (latch=0x7acb2a2f5f14, wakeEvents=33, timeout=0,
wait_event_info=50331655) at latch.c:538
#4 in ProcSleep (locallock=0x122f778, lockMethodTable=0x1037340
<default_lockmethod>, dontWait=false) at proc.c:1355
#5 in WaitOnLock (locallock=0x122f778, owner=0x1247408, dontWait=false) at
lock.c:1833
#6 in LockAcquireExtended (locktag=0x7ffcc4e39220, lockmode=5,
sessionLock=false, dontWait=false, reportMemoryError=true, locallockp=0x0)
at lock.c:1046
#7 in LockAcquire (locktag=0x7ffcc4e39220, lockmode=5, sessionLock=false,
dontWait=false) at lock.c:739
#8 in SpeculativeInsertionWait (xid=3622, token=2) at lmgr.c:833
#9 in _bt_doinsert (rel=0x7acb2dbb12e8, itup=0x12f1308,
checkUnique=UNIQUE_CHECK_YES, indexUnchanged=true, heapRel=0x7acb2dbb0f08)
at nbtinsert.c:225
#10 in btinsert (rel=0x7acb2dbb12e8, values=0x7ffcc4e39440,
isnull=0x7ffcc4e39420, ht_ctid=0x12ebe20, heapRel=0x7acb2dbb0f08,
checkUnique=UNIQUE_CHECK_YES, indexUnchanged=true, indexInfo=0x12f08a8) at
nbtree.c:195
#11 in index_insert (indexRelation=0x7acb2dbb12e8, values=0x7ffcc4e39440,
isnull=0x7ffcc4e39420, heap_t_ctid=0x12ebe20, heapRelation=0x7acb2dbb0f08,
checkUnique=UNIQUE_CHECK_YES, indexUnchanged=true, indexInfo=0x12f08a8) at
indexam.c:230
#12 in ExecInsertIndexTuples (resultRelInfo=0x12eaa00, slot=0x12ebdf0,
estate=0x12ea560, update=true, noDupErr=false, specConflict=0x0,
arbiterIndexes=0x0, onlySummarizing=false) at execIndexing.c:438
#13 in ExecUpdateEpilogue (context=0x7ffcc4e39870,
updateCxt=0x7ffcc4e3962c, resultRelInfo=0x12eaa00, tupleid=0x7ffcc4e39732,
oldtuple=0x0, slot=0x12ebdf0) at nodeModifyTable.c:2130
#14 in ExecUpdate (context=0x7ffcc4e39870, resultRelInfo=0x12eaa00,
tupleid=0x7ffcc4e39732, oldtuple=0x0, slot=0x12ebdf0, canSetTag=true) at
nodeModifyTable.c:2478
#15 in ExecOnConflictUpdate (context=0x7ffcc4e39870,
resultRelInfo=0x12eaa00, conflictTid=0x7ffcc4e39732,
excludedSlot=0x12f05b8, canSetTag=true, returning=0x7ffcc4e39738) at
nodeModifyTable.c:2694
#16 in ExecInsert (context=0x7ffcc4e39870, resultRelInfo=0x12eaa00,
slot=0x12f05b8, canSetTag=true, inserted_tuple=0x0, insert_destrel=0x0) at
nodeModifyTable.c:1048
#17 in ExecModifyTable (pstate=0x12ea7f0) at nodeModifyTable.c:4059
#18 in ExecProcNodeFirst (node=0x12ea7f0) at execProcnode.c:464
#19 in ExecProcNode (node=0x12ea7f0) at
../../../src/include/executor/executor.h:274
#20 in ExecutePlan (estate=0x12ea560, planstate=0x12ea7f0,
use_parallel_mode=false, operation=CMD_INSERT, sendTuples=true,
numberTuples=0, direction=ForwardScanDirection, dest=0x12daac8,
execute_once=true) at execMain.c:1646
#21 in standard_ExecutorRun (queryDesc=0x12dab58,
direction=ForwardScanDirection, count=0, execute_once=true) at
execMain.c:363
#22 in ExecutorRun (queryDesc=0x12dab58, direction=ForwardScanDirection,
count=0, execute_once=true) at execMain.c:304
#23 in ProcessQuery (plan=0x12e1360, sourceText=0x12083b0 "INSERT INTO tbl
VALUES(13,41011) on conflict(i) do update set n = tbl.n + 1 RETURNING n ",
params=0x0, queryEnv=0x0, dest=0x12daac8, qc=0x7ffcc4e39ae0) at pquery.c:160
#24 in PortalRunMulti (portal=0x1289c90, isTopLevel=true,
setHoldSnapshot=true, dest=0x12daac8, altdest=0x10382a0 <donothingDR>,
qc=0x7ffcc4e39ae0) at pquery.c:1277
#25 in FillPortalStore (portal=0x1289c90, isTopLevel=true) at pquery.c:1026
#26 in PortalRun (portal=0x1289c90, count=9223372036854775807,
isTopLevel=true, run_once=true, dest=0x12e14c0, altdest=0x12e14c0,
qc=0x7ffcc4e39d30) at pquery.c:763
#27 in exec_simple_query (query_string=0x12083b0 "INSERT INTO tbl
VALUES(13,41011) on conflict(i) do update set n = tbl.n + 1 RETURNING n ")
at postgres.c:1274
-------------------------
INSERT INTO tbl VALUES(13,89318) on conflict(i) do update set n = tbl.n + 1
RETURNING n
#0 in epoll_wait (epfd=5, events=0x1203328, maxevents=1, timeout=-1) at
../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1 in WaitEventSetWaitBlock (set=0x12032c0, cur_timeout=-1,
occurred_events=0x7ffcc4e38f60, nevents=1) at latch.c:1570
#2 in WaitEventSetWait (set=0x12032c0, timeout=-1,
occurred_events=0x7ffcc4e38f60, nevents=1, wait_event_info=50331653) at
latch.c:1516
#3 in WaitLatch (latch=0x7acb2a2f4dbc, wakeEvents=33, timeout=0,
wait_event_info=50331653) at latch.c:538
#4 in ProcSleep (locallock=0x122f670, lockMethodTable=0x1037340
<default_lockmethod>, dontWait=false) at proc.c:1355
#5 in WaitOnLock (locallock=0x122f670, owner=0x1247408, dontWait=false) at
lock.c:1833
#6 in LockAcquireExtended (locktag=0x7ffcc4e39370, lockmode=5,
sessionLock=false, dontWait=false, reportMemoryError=true, locallockp=0x0)
at lock.c:1046
#7 in LockAcquire (locktag=0x7ffcc4e39370, lockmode=5, sessionLock=false,
dontWait=false) at lock.c:739
#8 in XactLockTableWait (xid=3633, rel=0x7acb2dba66d8, ctid=0x1240a68,
oper=XLTW_InsertIndex) at lmgr.c:701
#9 in _bt_doinsert (rel=0x7acb2dba66d8, itup=0x1240a68,
checkUnique=UNIQUE_CHECK_YES, indexUnchanged=false, heapRel=0x7acb2dbb0f08)
at nbtinsert.c:227
#10 in btinsert (rel=0x7acb2dba66d8, values=0x7ffcc4e395c0,
isnull=0x7ffcc4e395a0, ht_ctid=0x12400e8, heapRel=0x7acb2dbb0f08,
checkUnique=UNIQUE_CHECK_YES, indexUnchanged=false, indexInfo=0x1240500) at
nbtree.c:195
#11 in index_insert (indexRelation=0x7acb2dba66d8, values=0x7ffcc4e395c0,
isnull=0x7ffcc4e395a0, heap_t_ctid=0x12400e8, heapRelation=0x7acb2dbb0f08,
checkUnique=UNIQUE_CHECK_YES, indexUnchanged=false, indexInfo=0x1240500) at
indexam.c:230
#12 in ExecInsertIndexTuples (resultRelInfo=0x12eaa00, slot=0x12400b8,
estate=0x12ea560, update=false, noDupErr=true, specConflict=0x7ffcc4e39722,
arbiterIndexes=0x12e0998, onlySummarizing=false) at execIndexing.c:438
#13 in ExecInsert (context=0x7ffcc4e39870, resultRelInfo=0x12eaa00,
slot=0x12400b8, canSetTag=true, inserted_tuple=0x0, insert_destrel=0x0) at
nodeModifyTable.c:1095
#14 in ExecModifyTable (pstate=0x12ea7f0) at nodeModifyTable.c:4059
#15 in ExecProcNodeFirst (node=0x12ea7f0) at execProcnode.c:464
#16 in ExecProcNode (node=0x12ea7f0) at
../../../src/include/executor/executor.h:274
#17 in ExecutePlan (estate=0x12ea560, planstate=0x12ea7f0,
use_parallel_mode=false, operation=CMD_INSERT, sendTuples=true,
numberTuples=0, direction=ForwardScanDirection, dest=0x12daac8,
execute_once=true) at execMain.c:1646
#18 in standard_ExecutorRun (queryDesc=0x12dab58,
direction=ForwardScanDirection, count=0, execute_once=true) at
execMain.c:363
#19 in ExecutorRun (queryDesc=0x12dab58, direction=ForwardScanDirection,
count=0, execute_once=true) at execMain.c:304
#20 in ProcessQuery (plan=0x12e1360, sourceText=0x12083b0 "INSERT INTO tbl
VALUES(13,89318) on conflict(i) do update set n = tbl.n + 1 RETURNING n ",
params=0x0, queryEnv=0x0, dest=0x12daac8, qc=0x7ffcc4e39ae0) at pquery.c:160
#21 in PortalRunMulti (portal=0x1289c90, isTopLevel=true,
setHoldSnapshot=true, dest=0x12daac8, altdest=0x10382a0 <donothingDR>,
qc=0x7ffcc4e39ae0) at pquery.c:1277
#22 in FillPortalStore (portal=0x1289c90, isTopLevel=true) at pquery.c:1026
#23 in PortalRun (portal=0x1289c90, count=9223372036854775807,
isTopLevel=true, run_once=true, dest=0x12e14c0, altdest=0x12e14c0,
qc=0x7ffcc4e39d30) at pquery.c:763
#24 in exec_simple_query (query_string=0x12083b0 "INSERT INTO tbl
VALUES(13,89318) on conflict(i) do update set n = tbl.n + 1 RETURNING n ")
at postgres.c:1274
-------------------------
Also, at that time (but not reported in deadlock) reindex is happening.
Without reindex I am unable to reproduce deadlock.
#0 in epoll_wait (epfd=5, events=0x1203328, maxevents=1, timeout=-1) at
../sysdeps/unix/sysv/linux/epoll_wait.c:30
#1 in WaitEventSetWaitBlock (set=0x12032c0, cur_timeout=-1,
occurred_events=0x7ffcc4e38cd0, nevents=1) at latch.c:1570
#2 in WaitEventSetWait (set=0x12032c0, timeout=-1,
occurred_events=0x7ffcc4e38cd0, nevents=1, wait_event_info=50331654) at
latch.c:1516
#3 in WaitLatch (latch=0x7acb2a2ff0c4, wakeEvents=33, timeout=0,
wait_event_info=50331654) at latch.c:538
#4 in ProcSleep (locallock=0x122f358, lockMethodTable=0x1037340
<default_lockmethod>, dontWait=false) at proc.c:1355
#5 in WaitOnLock (locallock=0x122f358, owner=0x12459f0, dontWait=false) at
lock.c:1833
#6 in LockAcquireExtended (locktag=0x7ffcc4e390e0, lockmode=5,
sessionLock=false, dontWait=false, reportMemoryError=true, locallockp=0x0)
at lock.c:1046
#7 in LockAcquire (locktag=0x7ffcc4e390e0, lockmode=5, sessionLock=false,
dontWait=false) at lock.c:739
#8 in VirtualXactLock (vxid=..., wait=true) at lock.c:4627
#9 in WaitForLockersMultiple (locktags=0x12327a8, lockmode=8,
progress=true) at lmgr.c:955
#10 in ReindexRelationConcurrently (stmt=0x1208e08, relationOid=16401,
params=0x7ffcc4e39528) at indexcmds.c:4154
#11 in ReindexIndex (stmt=0x1208e08, params=0x7ffcc4e39528,
isTopLevel=true) at indexcmds.c:2814
#12 in ExecReindex (pstate=0x12329f0, stmt=0x1208e08, isTopLevel=true) at
indexcmds.c:2743
#13 in ProcessUtilitySlow (pstate=0x12329f0, pstmt=0x1208f58,
queryString=0x12083b0 "REINDEX INDEX CONCURRENTLY tbl_pkey;",
context=PROCESS_UTILITY_TOPLEVEL, params=0x0, queryEnv=0x0, dest=0x1209318,
qc=0x7ffcc4e39d30) at utility.c:1567
#14 in standard_ProcessUtility (pstmt=0x1208f58, queryString=0x12083b0
"REINDEX INDEX CONCURRENTLY tbl_pkey;", readOnlyTree=false,
context=PROCESS_UTILITY_TOPLEVEL, params=0x0, queryEnv=0x0, dest=0x1209318,
qc=0x7ffcc4e39d30) at utility.c:1067
#15 in ProcessUtility (pstmt=0x1208f58, queryString=0x12083b0 "REINDEX
INDEX CONCURRENTLY tbl_pkey;", readOnlyTree=false,
context=PROCESS_UTILITY_TOPLEVEL, params=0x0, queryEnv=0x0, dest=0x1209318,
qc=0x7ffcc4e39d30) at utility.c:523
#16 in PortalRunUtility (portal=0x1289c90, pstmt=0x1208f58,
isTopLevel=true, setHoldSnapshot=false, dest=0x1209318, qc=0x7ffcc4e39d30)
at pquery.c:1158
#17 in PortalRunMulti (portal=0x1289c90, isTopLevel=true,
setHoldSnapshot=false, dest=0x1209318, altdest=0x1209318,
qc=0x7ffcc4e39d30) at pquery.c:1315
#18 in PortalRun (portal=0x1289c90, count=9223372036854775807,
isTopLevel=true, run_once=true, dest=0x1209318, altdest=0x1209318,
qc=0x7ffcc4e39d30) at pquery.c:791
#19 in exec_simple_query (query_string=0x12083b0 "REINDEX INDEX
CONCURRENTLY tbl_pkey;") at postgres.c:1274
It looks like a deadlock caused by different set of indexes being used as
arbiter indexes (or by the different order).
Best regards,
Mikhail.
Hell, everyone!
Using the brand-new injection points support in specs, I created a spec to
reproduce the issue.
It fails like this currently:
make -C src/test/modules/injection_points/ check
@@ -64,6 +64,7 @@
step s3_s1: <... completed>
step s2_s1: <... completed>
+ERROR: duplicate key value violates unique constraint "tbl_pkey_ccold"
starting permutation: s3_s1 s2_s1 s4_s1 s1_s1 s4_s2 s4_s3
injection_points_attach
@@ -129,3 +130,4 @@
step s3_s1: <... completed>
step s2_s1: <... completed>
+ERROR: duplicate key value violates unique constraint "tbl_pkey"
Best regards,
Mikhail.
Attachments:
v1-0001-spec-to-reproduce-issue-with-REINDEX-CONCURRENTLY.patchtext/x-patch; charset=US-ASCII; name=v1-0001-spec-to-reproduce-issue-with-REINDEX-CONCURRENTLY.patchDownload
From 6731d0a2df03eb291b05b5424f8e4aa63c2216ee Mon Sep 17 00:00:00 2001
From: nkey <nkey@toloka.ai>
Date: Sun, 4 Aug 2024 21:24:49 +0200
Subject: [PATCH v1] spec to reproduce issue with REINDEX CONCURRENTLY and
INSERT ON CONFLICT UPDATE
---
src/backend/commands/indexcmds.c | 3 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 3 +
src/test/modules/injection_points/Makefile | 2 +-
.../expected/reindex_concurrently_upsert.out | 131 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 1 +
.../specs/reindex_concurrently_upsert.spec | 70 ++++++++++
7 files changed, 211 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 2caab88aa5..9437654b11 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -69,6 +69,7 @@
#include "utils/regproc.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/* non-export function prototypes */
@@ -4078,7 +4079,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap");
StartTransactionCommand();
/*
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 9f05b3654c..1d451a329a 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -115,6 +115,7 @@
#include "nodes/nodeFuncs.h"
#include "storage/lmgr.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -901,6 +902,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict");
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4913e49319..d91b0a1cc7 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -69,6 +69,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1085,6 +1086,8 @@ ExecInsert(ModifyTableContext *context,
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative");
+
/*
* Before we start insertion proper, acquire our "speculative
* insertion lock". Others can use that to wait for us to decide
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index 2ffd2f77ed..f8a2a7630d 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -9,7 +9,7 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = inplace
+ISOLATION = inplace reindex_concurrently_upsert
# The injection points are cluster-wide, so disable installcheck
NO_INSTALLCHECK = 1
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 0000000000..3e855e2223
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,131 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_s1 s1_s1 s4_s1 s2_s1 s4_s2 s4_s3
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_s1: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_s1: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_s1:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_s1: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_s2:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_s1: <... completed>
+step s4_s3:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_s1: <... completed>
+step s2_s1: <... completed>
+
+starting permutation: s3_s1 s2_s1 s4_s1 s1_s1 s4_s2 s4_s3
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_s1: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s2_s1: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_s1:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_s1: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_s2:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_s1: <... completed>
+step s4_s3:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_s1: <... completed>
+step s2_s1: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 3c23c14d81..7caa9769b2 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -40,6 +40,7 @@ tests += {
'isolation': {
'specs': [
'inplace',
+ 'reindex_concurrently_upsert',
],
},
}
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 0000000000..fcaee1a21c
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,70 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_s1 { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_s1 { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_s1 { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_s1 {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+}
+step s4_s2 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_s3 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+
+permutation
+ s3_s1
+ s1_s1
+ s4_s1
+ s2_s1
+ s4_s2
+ s4_s3
+
+permutation
+ s3_s1
+ s2_s1
+ s4_s1
+ s1_s1
+ s4_s2
+ s4_s3
\ No newline at end of file
--
2.34.1
Hello, everyone.
I have updated the spec to reproduce the issue, now it includes cases with
both CREATE INDEX and REINDEX.
To run:
make -C src/test/modules/injection_points/ check
Issue reproduced on empty index, but it may happen on index of any with the
same probability.
It is not critical, of course, but in production system indexes are
regularly rebuilt using REINDEX CONCURRENTLY as recommended in
documentation [1]https://www.postgresql.org/docs/current/routine-reindex.html.
In most of the cases it is done using pg_repack as far as I know.
So, in these production systems, there is no guarantee what INSERT ON
CONFLICT DO NOTHING/UPDATE will not fail with a "duplicate key value
violates unique constraint" error.
Best regards,
Mikhail.
[1]: https://www.postgresql.org/docs/current/routine-reindex.html
Show quoted text
Attachments:
v2-0001-specs-to-reproduce-issues-with-CREATE-INDEX-REIND.patchtext/x-patch; charset=US-ASCII; name=v2-0001-specs-to-reproduce-issues-with-CREATE-INDEX-REIND.patchDownload
From e41389c4a873fbf7dd28907cb4624dffc56cb3d9 Mon Sep 17 00:00:00 2001
From: nkey <nkey@toloka.ai>
Date: Fri, 16 Aug 2024 11:19:37 +0200
Subject: [PATCH v2] specs to reproduce issues with CREATE INDEX/REINDEX
CONCURRENTLY for UNIQUE indexes and INSERT ON CONFLICT UPDATE using injection
points
---
src/backend/commands/indexcmds.c | 5 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 2 +
src/test/modules/injection_points/Makefile | 2 +-
.../expected/index_concurrently_upsert.out | 80 ++++++
.../expected/reindex_concurrently_upsert.out | 238 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 2 +
.../specs/index_concurrently_upsert.spec | 68 +++++
.../specs/reindex_concurrently_upsert.spec | 86 +++++++
10 files changed, 486 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 2caab88aa5..822467bdd5 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -69,6 +69,7 @@
#include "utils/regproc.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/* non-export function prototypes */
@@ -1775,6 +1776,7 @@ DefineIndex(Oid tableId,
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ INJECTION_POINT("define_index_before_set_valid");
/*
* Index can now be marked valid -- update its pg_index entry
@@ -4078,7 +4080,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap");
StartTransactionCommand();
/*
@@ -4152,6 +4154,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead");
foreach(lc, indexIds)
{
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 9f05b3654c..1d451a329a 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -115,6 +115,7 @@
#include "nodes/nodeFuncs.h"
#include "storage/lmgr.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -901,6 +902,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict");
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4913e49319..65bc63c612 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -69,6 +69,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1084,6 +1085,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative");
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 7d2b34d4f2..3a7357a050 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -64,6 +64,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -426,6 +427,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("invalidate_catalog_snapshot_end");
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index 2ffd2f77ed..9777c48367 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -9,7 +9,7 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = inplace
+ISOLATION = inplace reindex_concurrently_upsert index_concurrently_upsert
# The injection points are cluster-wide, so disable installcheck
NO_INSTALLCHECK = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 0000000000..f39a6d452a
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,80 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ SELECT injection_points_detach('define_index_before_set_valid');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: <... completed>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 0000000000..b7639ff7e6
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,238 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+step s2_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 3c23c14d81..f2aa60702b 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -40,6 +40,8 @@ tests += {
'isolation': {
'specs': [
'inplace',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
],
},
}
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 0000000000..5d6aba9073
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,68 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ SELECT injection_points_detach('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 0000000000..c6ad2c4198
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,86 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s2_start_upsert
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.34.1
Hello, everyone!
This patch set addresses the issues discussed in this thread.
The main idea behind this fix is that it is safe to consider indisready
indexes alongside indisvalid indexes as arbiter indexes. However, it's
crucial that at least one fully valid index is present.
Why is it necessary to consider indisready during the planning phase?
The reason is that these indexes are required for correct processing during
the execution phase.
If "ready" indexes are skipped as arbiters by one transaction, they may
already have become "valid" for another concurrent transaction during its
planning phase.
As a result, both transactions could concurrently process the UPSERT
command with different sets of arbiters (while using the same set of
indexes for tuple insertion later).
This can lead to unexpected "duplicate key value violates unique
constraint" errors and deadlocks.
Is it safe to use a "ready" but not yet "valid" index as an arbiter?
Yes, as long as at least one "valid" index is also used as an arbiter.
The valid index ensures the correctness of the UPSERT logic, while the
"ready" index contains an equal or lesser number of tuples, making it safe
for speculative insertion.
In any case, the insert to that index will be processed during
ExecInsertIndexTuples one way or another (with applyNoDupErr or without).
Fix is divided into a few patches, each following this logic:
1) The first patch provides specs (and injection points) for the various
scenarios related to the issue.
2) The second patch introduces a straightforward change—adding indisready
indexes to arbiters alongside indisvalid. However, at least one indisvalid
is still required. This resolves simple cases involving REINDEX
CONCURRENTLY and CREATE INDEX CONCURRENTLY.
3) The third patch deals with named constraints. Instead of relying solely
on the index with the specified name, we attempt to find other indexes that
are equivalent in terms of being used as an arbiter.
4) This patch fixes a scenario involving partitioned tables. Special checks
are required for partitioned indexes, which may be processed by REINDEX
CONCURRENTLY.
Additionally, a patch with three extra TAP specifications for stress
testing is attached. This patch is not intended for commitment, so I
renamed the extension to prevent accidental application in some CI/DI jobs.
Also, it is possible to look at the patches on GitHub:
https://github.com/postgres/postgres/compare/master...michail-nikolaev:postgres:reindex_concurrently_with_upsert
Best regards,
Mikhail.
Attachments:
v3-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchtext/x-patch; charset=US-ASCII; name=v3-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchDownload
From 9a4d01018e21d491c967b3378e61abdb06f52c74 Mon Sep 17 00:00:00 2001
From: nkey <nkey@toloka.ai>
Date: Sat, 24 Aug 2024 14:04:02 +0200
Subject: [PATCH v3 2/4] Modify the infer_arbiter_indexes function to consider
both indisvalid and indisready indexes. Ensure that at least one indisvalid
index is still required.
The change ensures that all concurrent transactions utilize the same set of indexes as arbiters. This uniformity is required to avoid conditions that could lead to "duplicate key value violates unique constraint" errors during UPSERT operations.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert
* index_concurrently_upsert
* index_concurrently_upsert_predicate
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
* reindex_concurrently_upsert_on_constraint
---
src/backend/optimizer/util/plancat.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 78a3cfafde..5ffc815ae4 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -720,6 +720,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Results */
List *results = NIL;
+ bool foundValid = false;
/*
* Quickly return NIL for ON CONFLICT DO NOTHING without an inference
@@ -813,7 +814,13 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
- if (!idxForm->indisvalid)
+ /*
+ * We need to consider both indisvalid and indisready indexes because
+ * them may become indisvalid before execution phase. It is required
+ * to keep set of indexes used as arbiter to be the same for all
+ * concurrent transactions.
+ */
+ if (!idxForm->indisready)
goto next;
/*
@@ -835,10 +842,9 @@ infer_arbiter_indexes(PlannerInfo *root)
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
results = lappend_oid(results, idxForm->indexrelid);
- list_free(indexList);
+ foundValid |= idxForm->indisvalid;
index_close(idxRel, NoLock);
- table_close(relation, NoLock);
- return results;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
@@ -932,6 +938,7 @@ infer_arbiter_indexes(PlannerInfo *root)
goto next;
results = lappend_oid(results, idxForm->indexrelid);
+ foundValid |= idxForm->indisvalid;
next:
index_close(idxRel, NoLock);
}
@@ -939,7 +946,8 @@ next:
list_free(indexList);
table_close(relation, NoLock);
- if (results == NIL)
+ /* It is required to have at least one indisvalid index during the planning. */
+ if (results == NIL || !foundValid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
errmsg("there is no unique or exclusion constraint matching the ON CONFLICT specification")));
--
2.34.1
v3-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX-.patchtext/x-patch; charset=US-ASCII; name=v3-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX-.patchDownload
From c2566ce93a8e24410ae89c1838a57e310e33cf88 Mon Sep 17 00:00:00 2001
From: nkey <nkey@toloka.ai>
Date: Sat, 24 Aug 2024 13:44:32 +0200
Subject: [PATCH v3 1/4] Specs top reproduce the issues with CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY in scenarios involving INSERT ON
CONFLICT DO UPDATE. These tests reproduce different error cases related to
"duplicate key value violates unique constraint" where this error should not
occur by design.
* REINDEX CONCURRENTLY and UPSERT with inferred index
* CREATE INDEX CONCURRENTLY and UPSERT with inferred indexes
* REINDEX CONCURRENTLY on partitioned table
* REINDEX CONCURRENTLY with specified constraint name
* CREATE INDEX CONCURRENTLY with predicates
In each of these scenarios, the expected behavior is that the INSERT ON CONFLICT DO UPDATE should handle conflicts gracefully without raising a "duplicate key value violates unique constraint" error. However, due to the concurrent operations on the indexes, this error is encountered.
---
src/backend/commands/indexcmds.c | 5 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 2 +
src/test/modules/injection_points/Makefile | 7 +-
.../expected/index_concurrently_upsert.out | 80 ++++++
.../index_concurrently_upsert_predicate.out | 80 ++++++
.../expected/reindex_concurrently_upsert.out | 238 ++++++++++++++++++
...ndex_concurrently_upsert_on_constraint.out | 238 ++++++++++++++++++
...eindex_concurrently_upsert_partitioned.out | 238 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 5 +
.../specs/index_concurrently_upsert.spec | 68 +++++
.../index_concurrently_upsert_predicate.spec | 70 ++++++
.../specs/reindex_concurrently_upsert.spec | 86 +++++++
...dex_concurrently_upsert_on_constraint.spec | 86 +++++++
...index_concurrently_upsert_partitioned.spec | 88 +++++++
16 files changed, 1294 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index c5a56c75f6..ed7ed0f640 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -69,6 +69,7 @@
#include "utils/regproc.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/* non-export function prototypes */
@@ -1776,6 +1777,7 @@ DefineIndex(Oid tableId,
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ INJECTION_POINT("define_index_before_set_valid");
/*
* Index can now be marked valid -- update its pg_index entry
@@ -4082,7 +4084,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap");
StartTransactionCommand();
/*
@@ -4156,6 +4158,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead");
foreach(lc, indexIds)
{
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 403a3f4055..5f0957e53f 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -115,6 +115,7 @@
#include "nodes/nodeFuncs.h"
#include "storage/lmgr.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -906,6 +907,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict");
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 8bf4c80d4a..cdbf0b4d4f 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -69,6 +69,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1087,6 +1088,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative");
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 7d2b34d4f2..3a7357a050 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -64,6 +64,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -426,6 +427,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("invalidate_catalog_snapshot_end");
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index ed28cd13a8..0d5005be1f 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -13,7 +13,12 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = inplace
+ISOLATION = inplace \
+ reindex_concurrently_upsert \
+ index_concurrently_upsert \
+ reindex_concurrently_upsert_partitioned \
+ reindex_concurrently_upsert_on_constraint \
+ index_concurrently_upsert_predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 0000000000..f39a6d452a
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,80 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ SELECT injection_points_detach('define_index_before_set_valid');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: <... completed>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
new file mode 100644
index 0000000000..014d94d7ec
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
@@ -0,0 +1,80 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ SELECT injection_points_detach('define_index_before_set_valid');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: <... completed>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 0000000000..b7639ff7e6
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,238 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+step s2_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
new file mode 100644
index 0000000000..dbbb9691cc
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
@@ -0,0 +1,238 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+step s2_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
new file mode 100644
index 0000000000..b4cedf820c
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
@@ -0,0 +1,238 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+step s2_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index c9e357f644..6391417f4b 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -42,6 +42,11 @@ tests += {
'isolation': {
'specs': [
'inplace',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
+ 'reindex_concurrently_upsert_partitioned',
+ 'reindex_concurrently_upsert_on_constraint',
+ 'index_concurrently_upsert_predicate',
],
},
'tap': {
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 0000000000..5d6aba9073
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,68 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ SELECT injection_points_detach('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
new file mode 100644
index 0000000000..2bc3cfd775
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
@@ -0,0 +1,70 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+
+ CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;}
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ SELECT injection_points_detach('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 0000000000..c6ad2c4198
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,86 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s2_start_upsert
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
new file mode 100644
index 0000000000..fb030f8575
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
@@ -0,0 +1,86 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s2_start_upsert
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
new file mode 100644
index 0000000000..efa55809ae
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
@@ -0,0 +1,88 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s2_start_upsert
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.34.1
v3-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchtext/x-patch; charset=US-ASCII; name=v3-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchDownload
From fd2dad875688f2d8f465c4a689940edb1824c39d Mon Sep 17 00:00:00 2001
From: nkey <nkey@toloka.ai>
Date: Sat, 24 Aug 2024 14:33:47 +0200
Subject: [PATCH v3 3/4] Modify the infer_arbiter_indexes function to also look
for indexes that match the specified named constraint to be used as arbiters.
This ensures that the same set of arbiter indexes is used for all concurrent
transactions in cases where REINDEX CONCURRENTLY processes an index used as a
named constraint.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_on_constraint
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
---
src/backend/optimizer/util/plancat.c | 121 +++++++++++++++++++--------
1 file changed, 88 insertions(+), 33 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 5ffc815ae4..6712efe4dd 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -714,9 +714,10 @@ infer_arbiter_indexes(PlannerInfo *root)
List *indexList;
ListCell *l;
- /* Normalized inference attributes and inference expressions: */
- Bitmapset *inferAttrs = NULL;
- List *inferElems = NIL;
+ /* Normalized required attributes and expressions: */
+ Bitmapset *requiredArbiterAttrs = NULL;
+ List *requiredArbiterElems = NIL;
+ List *requiredIndexPredExprs = (List *) onconflict->arbiterWhere;
/* Results */
List *results = NIL;
@@ -755,8 +756,8 @@ infer_arbiter_indexes(PlannerInfo *root)
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
- inferElems = lappend(inferElems, elem->expr);
+ /* If not a plain Var, just shove it in requiredArbiterElems for now */
+ requiredArbiterElems = lappend(requiredArbiterElems, elem->expr);
continue;
}
@@ -768,30 +769,76 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("whole row unique index inference specifications are not supported")));
- inferAttrs = bms_add_member(inferAttrs,
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
+ indexList = RelationGetIndexList(relation);
+
/*
* Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * because some additional sanity checks are required. Additionally, we
+ * need to process other indexes as potential arbiters to account for
+ * cases where REINDEX CONCURRENTLY is processing an index used as a
+ * named constraint.
*/
if (onconflict->constraint != InvalidOid)
{
indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
+ {
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
- errmsg("constraint in ON CONFLICT clause has no associated index")));
+ errmsg("constraint in ON CONFLICT clause has no associated index")));
+ }
+
+ /*
+ * Find the named constraint index to extract its attributes and predicates.
+ * We open all indexes in the loop to avoid deadlock of changed order of locks.
+ * */
+ foreach(l, indexList)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation idxRel;
+ Form_pg_index idxForm;
+ AttrNumber natt;
+
+ idxRel = index_open(indexoid, rte->rellockmode);
+ idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Prepare requirements for other indexes to be used as arbiter together
+ * with indexOidFromConstraint. It is required to involve both equals indexes
+ * in case of REINDEX CONCURRENTLY.
+ */
+ for (natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno = idxRel->rd_index->indkey.values[natt];
+
+ if (attno != 0)
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+ requiredArbiterElems = RelationGetIndexExpressions(idxRel);
+ requiredIndexPredExprs = RelationGetIndexPredicate(idxRel);
+ /* We are done, so, quite the loop. */
+ index_close(idxRel, NoLock);
+ break;
+ }
+ }
+ index_close(idxRel, NoLock);
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
foreach(l, indexList)
{
Oid indexoid = lfirst_oid(l);
@@ -840,26 +887,23 @@ infer_arbiter_indexes(PlannerInfo *root)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
-
- results = lappend_oid(results, idxForm->indexrelid);
- foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ goto found;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /* In the case of "ON constraint_name DO UPDATE" we need to skip non-unique candidates. */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ goto next;
+ } else {
+ /*
+ * Only considering conventional inference at this point (not named
+ * constraints), so index under consideration can be immediately
+ * skipped if it's not unique
+ */
+ if (!idxForm->indisunique)
+ goto next;
}
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
-
/* Build BMS representation of plain (non expression) index attrs */
indexedAttrs = NULL;
for (natt = 0; natt < idxForm->indnkeyatts; natt++)
@@ -872,7 +916,7 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/* Non-expression attributes (if any) must match */
- if (!bms_equal(indexedAttrs, inferAttrs))
+ if (!bms_equal(indexedAttrs, requiredArbiterAttrs))
goto next;
/* Expression attributes (if any) must match */
@@ -880,6 +924,10 @@ infer_arbiter_indexes(PlannerInfo *root)
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
+ /*
+ * If arbiterElems are present, check them. If name >constraint is
+ * present arbiterElems == NIL.
+ */
foreach(el, onconflict->arbiterElems)
{
InferenceElem *elem = (InferenceElem *) lfirst(el);
@@ -917,26 +965,33 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of the conventional inference involved ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In the case of named constraint ensure candidate has equal set
+ * of expressions as the named constraint index.
*/
- if (list_difference(idxExprs, inferElems) != NIL)
+ if (list_difference(idxExprs, requiredArbiterElems) != NIL)
goto next;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
+ /*
+ * If it's a partial index and conventional inference, its predicate must be implied
+ * by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid && !predicate_implied_by(predExprs, requiredIndexPredExprs, false))
+ goto next;
+ /* If it's a partial index and named constraint predicates must be equal. */
+ if (indexOidFromConstraint != InvalidOid && list_difference(predExprs, requiredIndexPredExprs) != NIL)
goto next;
+found:
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
next:
--
2.34.1
v3-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchtext/x-patch; charset=US-ASCII; name=v3-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchDownload
From 0c51f52341205b52fefa2ce0e7f3513835abf49e Mon Sep 17 00:00:00 2001
From: nkey <nkey@toloka.ai>
Date: Sat, 24 Aug 2024 15:19:53 +0200
Subject: [PATCH v3 4/4] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY as
arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_partitioned
---
src/backend/executor/execPartition.c | 119 ++++++++++++++++++++++++---
1 file changed, 107 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 7651886229..aeeee41d5f 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -483,6 +483,48 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+ /* it is not supported for cases of exclusion constraints. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+ if (arbiterIndexRelation->rd_index->indnkeyatts != indexRelation->rd_index->indnkeyatts)
+ return false;
+
+ for (i = 0; i < indexRelation->rd_index->indnkeyatts; i++)
+ {
+ int arbiterAttoNo = arbiterIndexRelation->rd_index->indkey.values[i];
+ int attoNo = indexRelation->rd_index->indkey.values[i];
+ if (arbiterAttoNo != attoNo)
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -693,6 +735,8 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
{
List *childIdxs;
+ List *nonAncestorIdxs = NIL;
+ int i, j, additional_arbiters = 0;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
@@ -703,23 +747,74 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, lfirst_oid(lc2)))
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxs = lappend_oid(nonAncestorIdxs, childIdx);
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxs)
+ {
+ for (i = 0; i < leaf_part_rri->ri_NumIndices; i++)
+ {
+ if (list_member_oid(nonAncestorIdxs, leaf_part_rri->ri_IndexRelationDescs[i]->rd_index->indexrelid))
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[i];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[i];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to us non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ for (j = 0; j < leaf_part_rri->ri_NumIndices; j++)
+ {
+ if (list_member_oid(arbiterIndexes,
+ leaf_part_rri->ri_IndexRelationDescs[j]->rd_index->indexrelid))
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[j];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[j];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxs);
+
+ /*
+ * If the resulting lists are of inequal length, something is wrong.
+ * (This shouldn't happen, since arbiter index selection should not
+ * pick up a non-ready index.)
+ *
+ * But we need to consider an additional arbiter indexes also.
+ */
+ if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
+ list_length(arbiterIndexes) - additional_arbiters)
+ elog(ERROR, "invalid arbiter index list");
}
-
- /*
- * If the resulting lists are of inequal length, something is wrong.
- * (This shouldn't happen, since arbiter index selection should not
- * pick up an invalid index.)
- */
- if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
- elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
/*
--
2.34.1
stress_test.__patch__application/octet-stream; name=stress_test.__patch__Download
Subject: [PATCH] Stress test for issues form https://www.postgresql.org/message-id/flat/ZnoZ6GNwkJmq-gTh%40paquier.xyz#4d13f826fb1e62860cc3ae30067bd23a
To run:
make -C src/test/modules/test_misc/ check PROVE_TESTS='t/007_*'
make -C src/test/modules/test_misc/ check PROVE_TESTS='t/008_*'
make -C src/test/modules/test_misc/ check PROVE_TESTS='t/009_*'
---
Index: src/test/modules/test_misc/t/007_concurrently_unique_deadlock.pl
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/test_misc/t/007_concurrently_unique_deadlock.pl b/src/test/modules/test_misc/t/007_concurrently_unique_deadlock.pl
new file mode 100644
--- /dev/null (date 1724509948524)
+++ b/src/test/modules/test_misc/t/007_concurrently_unique_deadlock.pl (date 1724509948524)
@@ -0,0 +1,163 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+# Test REINDEX CONCURRENTLY with concurrent modifications and HOT updates
+use strict;
+use warnings;
+
+use Config;
+use Errno;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Time::HiRes qw(usleep);
+use IPC::SysV;
+use threads;
+use Time::HiRes qw( time );
+use Test::More;
+use Test::Builder;
+
+if ($@ || $windows_os)
+{
+ plan skip_all => 'Fork and shared memory are not supported by this platform';
+}
+
+my ($pid, $shmem_id, $shmem_key, $shmem_size);
+eval 'sub IPC_CREAT {0001000}' unless defined &IPC_CREAT;
+$shmem_size = 4;
+$shmem_key = rand(1000000);
+$shmem_id = shmget($shmem_key, $shmem_size, &IPC_CREAT | 0777) or die "Can't shmget: $!";
+shmwrite($shmem_id, "wait", 0, $shmem_size) or die "Can't shmwrite: $!";
+
+my $psql_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default);
+#
+# Test set-up
+#
+my ($node, $result);
+$node = PostgreSQL::Test::Cluster->new('RC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf('postgresql.conf', 'fsync = off');
+$node->append_conf('postgresql.conf', 'autovacuum = off');
+$node->start;
+$node->safe_psql('postgres', q(CREATE UNLOGGED TABLE tbl(i int primary key, n int)));
+
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+# IT IS **REQUIRED** TO REPRODUCE THE ISSUE
+$node->safe_psql('postgres', q(CREATE INDEX idx ON tbl(i, n)));
+$node->safe_psql('postgres', q(INSERT INTO tbl VALUES(13,1)));
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+my $builder = Test::More->builder;
+$builder->use_numbers(0);
+$builder->no_plan();
+
+my $child = $builder->child("pg_bench");
+
+if(!defined($pid = fork())) {
+ # fork returned undef, so unsuccessful
+ die "Cannot fork a child: $!";
+} elsif ($pid == 0) {
+
+ $pid = fork();
+ if ($pid == 0) {
+ $node->pgbench(
+ '--no-vacuum --client=30 -j 2 --transactions=10000 --exit-on-abort --verbose-errors',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent INSERTs, UPDATES and RC',
+ {
+ '002_pgbench_concurrent_transaction_updates' => q(
+ INSERT INTO tbl VALUES(13,1) on conflict(i) do update set n = tbl.n + EXCLUDED.n;
+ ),
+ });
+
+ if ($child->is_passing()) {
+ shmwrite($shmem_id, "done", 0, $shmem_size) or die "Can't shmwrite: $!";
+ } else {
+ shmwrite($shmem_id, "fail", 0, $shmem_size) or die "Can't shmwrite: $!";
+ }
+
+ my $pg_bench_fork_flag;
+ while (1) {
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ sleep(0.1);
+ last if $pg_bench_fork_flag eq "stop";
+ }
+ }
+ else {
+ my ($result, $stdout, $stderr, $n, $prev_n, $pg_bench_fork_flag);
+ while (1) {
+ sleep(1);
+ $prev_n = $n;
+ ($result, $n, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+ diag(" current n is " . $n . ', ' . ($n - $prev_n) . ' per one second');
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag eq "stop";
+ }
+ }
+} else {
+ my $pg_bench_fork_flag;
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+
+ subtest 'reindex run subtest' => sub {
+ is($pg_bench_fork_flag, "wait", "pg_bench_fork_flag is correct");
+
+ my %psql = (stdin => '', stdout => '', stderr => '');
+ $psql{run} = IPC::Run::start(
+ [ 'psql', '-XA', '-f', '-', '-d', $node->connstr('postgres') ],
+ '<',
+ \$psql{stdin},
+ '>',
+ \$psql{stdout},
+ '2>',
+ \$psql{stderr},
+ $psql_timeout);
+
+ my ($result, $stdout, $stderr, $n, $begin_time, $end_time, $before_reindex, $after_reindex);
+
+ # IT IS NOT REQUIRED, JUST FOR CONSISTENCY
+ ($result, $stdout, $stderr) = $node->psql('postgres', q(ALTER TABLE tbl SET (parallel_workers=0);));
+ is($result, '0', 'ALTER TABLE is correct');
+
+ while (1)
+ {
+ my $sql = q(REINDEX INDEX CONCURRENTLY tbl_pkey;);
+
+ ($result, $before_reindex, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+
+ diag('going to start reindex, num tuples in table is ' . $before_reindex);
+ $begin_time = time();
+ ($result, $stdout, $stderr) = $node->psql('postgres', $sql);
+ is($result, '0', 'REINDEX INDEX CONCURRENTLY is correct');
+
+ $end_time = time();
+ ($result, $after_reindex, $stderr) = $node->psql('postgres', q(SELECT n from tbl WHERE i = 13;));
+ diag('reindex ' . $n++ . ' done in ' . ($end_time - $begin_time) . ' seconds, num inserted during reindex tuples is ' . (int($after_reindex) - int($before_reindex)) . ' speed is ' . ((int($after_reindex) - int($before_reindex)) / ($end_time - $begin_time)) . ' per second');
+
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag ne "wait";
+ }
+
+ # explicitly shut down psql instances gracefully
+ $psql{stdin} .= "\\q\n";
+ $psql{run}->finish;
+
+ is($pg_bench_fork_flag, "done", "pg_bench_fork_flag is correct");
+ };
+
+ $child->finalize();
+ $child->summary();
+ $node->stop;
+ done_testing();
+
+ shmwrite($shmem_id, "stop", 0, $shmem_size) or die "Can't shmwrite: $!";
+}
Index: src/test/modules/test_misc/t/009_concurrently_unique_fail_reindex.pl
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/test_misc/t/009_concurrently_unique_fail_reindex.pl b/src/test/modules/test_misc/t/009_concurrently_unique_fail_reindex.pl
new file mode 100644
--- /dev/null (date 1724509958148)
+++ b/src/test/modules/test_misc/t/009_concurrently_unique_fail_reindex.pl (date 1724509958148)
@@ -0,0 +1,152 @@
+
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+# Test REINDEX CONCURRENTLY with concurrent modifications and HOT updates
+use strict;
+use warnings;
+
+use Config;
+use Errno;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Time::HiRes qw(usleep);
+use IPC::SysV;
+use threads;
+use Time::HiRes qw( time );
+use Test::More;
+use Test::Builder;
+
+if ($@ || $windows_os)
+{
+ plan skip_all => 'Fork and shared memory are not supported by this platform';
+}
+
+my ($pid, $shmem_id, $shmem_key, $shmem_size);
+eval 'sub IPC_CREAT {0001000}' unless defined &IPC_CREAT;
+$shmem_size = 4;
+$shmem_key = rand(1000000);
+$shmem_id = shmget($shmem_key, $shmem_size, &IPC_CREAT | 0777) or die "Can't shmget: $!";
+shmwrite($shmem_id, "wait", 0, $shmem_size) or die "Can't shmwrite: $!";
+
+my $psql_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default);
+#
+# Test set-up
+#
+my ($node, $result);
+$node = PostgreSQL::Test::Cluster->new('RC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf('postgresql.conf', 'fsync = off');
+$node->append_conf('postgresql.conf', 'autovacuum = off');
+$node->start;
+$node->safe_psql('postgres', q(CREATE UNLOGGED TABLE tbl(i int primary key, updated_at timestamp)));
+
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+# IT IS NOT REQUIRED TO REPRODUCE THE ISSUE BUT MAKES IT TO HAPPEN FASTER
+$node->safe_psql('postgres', q(CREATE INDEX idx ON tbl(i, updated_at)));
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+my $builder = Test::More->builder;
+$builder->use_numbers(0);
+$builder->no_plan();
+
+my $child = $builder->child("pg_bench");
+
+if(!defined($pid = fork())) {
+ # fork returned undef, so unsuccessful
+ die "Cannot fork a child: $!";
+} elsif ($pid == 0) {
+
+ $node->pgbench(
+ '--no-vacuum --client=20 -j 2 --transactions=10000 --exit-on-abort --verbose-errors',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent INSERTs, UPDATES and RC',
+ {
+ # Ensure some HOT updates happen
+ '002_pgbench_concurrent_transaction_updates' => q(
+ INSERT INTO tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ ),
+ '003_pgbench_concurrent_transaction_updates' => q(
+ INSERT INTO tbl VALUES(42,now()) on conflict(i) do update set updated_at = now();
+ ),
+ '004_pgbench_concurrent_transaction_updates' => q(
+ INSERT INTO tbl VALUES(69,now()) on conflict(i) do update set updated_at = now();
+ ),
+ });
+
+ if ($child->is_passing()) {
+ shmwrite($shmem_id, "done", 0, $shmem_size) or die "Can't shmwrite: $!";
+ } else {
+ shmwrite($shmem_id, "fail", 0, $shmem_size) or die "Can't shmwrite: $!";
+ }
+
+ my $pg_bench_fork_flag;
+ while (1) {
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ sleep(0.1);
+ last if $pg_bench_fork_flag eq "stop";
+ }
+} else {
+ my $pg_bench_fork_flag;
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+
+ subtest 'reindex run subtest' => sub {
+ is($pg_bench_fork_flag, "wait", "pg_bench_fork_flag is correct");
+
+ my %psql = (stdin => '', stdout => '', stderr => '');
+ $psql{run} = IPC::Run::start(
+ [ 'psql', '-XA', '-f', '-', '-d', $node->connstr('postgres') ],
+ '<',
+ \$psql{stdin},
+ '>',
+ \$psql{stdout},
+ '2>',
+ \$psql{stderr},
+ $psql_timeout);
+
+ my ($result, $stdout, $stderr, $n, $begin_time, $end_time);
+
+ # IT IS NOT REQUIRED, JUST FOR CONSISTENCY
+ ($result, $stdout, $stderr) = $node->psql('postgres', q(ALTER TABLE tbl SET (parallel_workers=0);));
+ is($result, '0', 'ALTER TABLE is correct');
+
+ $begin_time = time();
+ while (1)
+ {
+ my $sql = q(REINDEX INDEX CONCURRENTLY tbl_pkey;);
+
+ ($result, $stdout, $stderr) = $node->psql('postgres', $sql);
+ is($result, '0', 'REINDEX INDEX CONCURRENTLY is correct');
+
+ $end_time = time();
+ diag('waiting, now is ' . $n++ . ', seconds passed : ' . int($end_time - $begin_time));
+
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag ne "wait";
+ }
+
+ # explicitly shut down psql instances gracefully
+ $psql{stdin} .= "\\q\n";
+ $psql{run}->finish;
+
+ is($pg_bench_fork_flag, "done", "pg_bench_fork_flag is correct");
+ };
+
+ $child->finalize();
+ $child->summary();
+ $node->stop;
+ done_testing();
+
+ shmwrite($shmem_id, "stop", 0, $shmem_size) or die "Can't shmwrite: $!";
+}
Index: src/test/modules/test_misc/t/008_concurrently_unique_fail_create_index.pl
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/test_misc/t/008_concurrently_unique_fail_create_index.pl b/src/test/modules/test_misc/t/008_concurrently_unique_fail_create_index.pl
new file mode 100644
--- /dev/null (date 1724509958155)
+++ b/src/test/modules/test_misc/t/008_concurrently_unique_fail_create_index.pl (date 1724509958155)
@@ -0,0 +1,162 @@
+
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+# Test REINDEX CONCURRENTLY with concurrent modifications and HOT updates
+use strict;
+use warnings;
+
+use Config;
+use Errno;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Time::HiRes qw(usleep);
+use IPC::SysV;
+use threads;
+use Time::HiRes qw( time );
+use Test::More;
+use Test::Builder;
+
+if ($@ || $windows_os)
+{
+ plan skip_all => 'Fork and shared memory are not supported by this platform';
+}
+
+my ($pid, $shmem_id, $shmem_key, $shmem_size);
+eval 'sub IPC_CREAT {0001000}' unless defined &IPC_CREAT;
+$shmem_size = 4;
+$shmem_key = rand(1000000);
+$shmem_id = shmget($shmem_key, $shmem_size, &IPC_CREAT | 0777) or die "Can't shmget: $!";
+shmwrite($shmem_id, "wait", 0, $shmem_size) or die "Can't shmwrite: $!";
+
+my $psql_timeout = IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default);
+#
+# Test set-up
+#
+my ($node, $result);
+$node = PostgreSQL::Test::Cluster->new('RC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf('postgresql.conf', 'fsync = off');
+$node->append_conf('postgresql.conf', 'autovacuum = off');
+$node->start;
+$node->safe_psql('postgres', q(CREATE UNLOGGED TABLE tbl(i int primary key, updated_at timestamp)));
+
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+# IT IS NOT REQUIRED TO REPRODUCE THE ISSUE BUT MAKES IT TO HAPPEN FASTER
+$node->safe_psql('postgres', q(CREATE INDEX idx ON tbl(i, updated_at)));
+
+#######################################################################################################################
+#######################################################################################################################
+#######################################################################################################################
+
+my $builder = Test::More->builder;
+$builder->use_numbers(0);
+$builder->no_plan();
+
+my $child = $builder->child("pg_bench");
+
+if(!defined($pid = fork())) {
+ # fork returned undef, so unsuccessful
+ die "Cannot fork a child: $!";
+} elsif ($pid == 0) {
+
+ $node->pgbench(
+ '--no-vacuum --client=20 -j 2 --transactions=10000 --exit-on-abort --verbose-errors',
+ 0,
+ [qr{actually processed}],
+ [qr{^$}],
+ 'concurrent INSERTs, UPDATES and RC',
+ {
+ # Ensure some HOT updates happen
+ '002_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ COMMIT;
+ ),
+ '003_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(42,now()) on conflict(i) do update set updated_at = now();
+ COMMIT;
+ ),
+ '004_pgbench_concurrent_transaction_updates' => q(
+ BEGIN;
+ INSERT INTO tbl VALUES(69,now()) on conflict(i) do update set updated_at = now();
+ COMMIT;
+ ),
+ });
+
+ if ($child->is_passing()) {
+ shmwrite($shmem_id, "done", 0, $shmem_size) or die "Can't shmwrite: $!";
+ } else {
+ shmwrite($shmem_id, "fail", 0, $shmem_size) or die "Can't shmwrite: $!";
+ }
+
+ my $pg_bench_fork_flag;
+ while (1) {
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ sleep(0.1);
+ last if $pg_bench_fork_flag eq "stop";
+ }
+} else {
+ my $pg_bench_fork_flag;
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+
+ subtest 'reindex run subtest' => sub {
+ is($pg_bench_fork_flag, "wait", "pg_bench_fork_flag is correct");
+
+ my %psql = (stdin => '', stdout => '', stderr => '');
+ $psql{run} = IPC::Run::start(
+ [ 'psql', '-XA', '-f', '-', '-d', $node->connstr('postgres') ],
+ '<',
+ \$psql{stdin},
+ '>',
+ \$psql{stdout},
+ '2>',
+ \$psql{stderr},
+ $psql_timeout);
+
+ my ($result, $stdout, $stderr, $n, $begin_time, $end_time);
+
+ # IT IS NOT REQUIRED, JUST FOR CONSISTENCY
+ ($result, $stdout, $stderr) = $node->psql('postgres', q(ALTER TABLE tbl SET (parallel_workers=0);));
+ is($result, '0', 'ALTER TABLE is correct');
+
+ $begin_time = time();
+ while (1)
+ {
+ my $sql1 = q(CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey2 ON tbl(i););
+ my $sql2 = q(DROP INDEX CONCURRENTLY tbl_pkey2;);
+
+ ($result, $stdout, $stderr) = $node->psql('postgres', $sql1);
+ is($result, '0', 'CREATE INDEX CONCURRENTLY is correct');
+
+ ($result, $stdout, $stderr) = $node->psql('postgres', $sql2);
+ is($result, '0', 'DROP INDEX CONCURRENTLY is correct');
+
+ $end_time = time();
+ #diag('waiting, now is ' . $n++ . ', seconds passed : ' . int($end_time - $begin_time));
+
+ shmread($shmem_id, $pg_bench_fork_flag, 0, $shmem_size) or die "Can't shmread: $!";
+ last if $pg_bench_fork_flag ne "wait";
+ }
+
+ # explicitly shut down psql instances gracefully
+ $psql{stdin} .= "\\q\n";
+ $psql{run}->finish;
+
+ is($pg_bench_fork_flag, "done", "pg_bench_fork_flag is correct");
+ };
+
+ $child->finalize();
+ $child->summary();
+ $node->stop;
+ done_testing();
+
+ shmwrite($shmem_id, "stop", 0, $shmem_size) or die "Can't shmwrite: $!";
+}
Hello, everyone.
Rebased on master.
Show quoted text
Attachments:
v4-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchtext/x-patch; charset=US-ASCII; name=v4-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchDownload
From 9c85b499793e9a4bcbb55cc5d78cfeacab368b58 Mon Sep 17 00:00:00 2001
From: nkey <nkey@toloka.ai>
Date: Thu, 14 Nov 2024 22:36:26 +0100
Subject: [PATCH v4 2/4] Modify the infer_arbiter_indexes function to consider
both indisvalid and indisready indexes. Ensure that at least one indisvalid
index is still required.
The change ensures that all concurrent transactions utilize the same set of indexes as arbiters. This uniformity is required to avoid conditions that could lead to "duplicate key value violates unique constraint" errors during UPSERT operations.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert
* index_concurrently_upsert
* index_concurrently_upsert_predicate
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
* reindex_concurrently_upsert_on_constraint
---
src/backend/optimizer/util/plancat.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 37b0ca2e43..c835813290 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -719,6 +719,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Results */
List *results = NIL;
+ bool foundValid = false;
/*
* Quickly return NIL for ON CONFLICT DO NOTHING without an inference
@@ -812,7 +813,13 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
- if (!idxForm->indisvalid)
+ /*
+ * We need to consider both indisvalid and indisready indexes because
+ * them may become indisvalid before execution phase. It is required
+ * to keep set of indexes used as arbiter to be the same for all
+ * concurrent transactions.
+ */
+ if (!idxForm->indisready)
goto next;
/*
@@ -834,10 +841,9 @@ infer_arbiter_indexes(PlannerInfo *root)
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
results = lappend_oid(results, idxForm->indexrelid);
- list_free(indexList);
+ foundValid |= idxForm->indisvalid;
index_close(idxRel, NoLock);
- table_close(relation, NoLock);
- return results;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
@@ -938,6 +944,7 @@ infer_arbiter_indexes(PlannerInfo *root)
goto next;
results = lappend_oid(results, idxForm->indexrelid);
+ foundValid |= idxForm->indisvalid;
next:
index_close(idxRel, NoLock);
}
@@ -945,7 +952,8 @@ next:
list_free(indexList);
table_close(relation, NoLock);
- if (results == NIL)
+ /* It is required to have at least one indisvalid index during the planning. */
+ if (results == NIL || !foundValid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
errmsg("there is no unique or exclusion constraint matching the ON CONFLICT specification")));
--
2.43.0
v4-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchtext/x-patch; charset=US-ASCII; name=v4-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchDownload
From 087a6ebaf196487668afef1387ce843de54d5a2d Mon Sep 17 00:00:00 2001
From: nkey <nkey@toloka.ai>
Date: Thu, 14 Nov 2024 22:41:00 +0100
Subject: [PATCH v4 4/4] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY as
arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_partitioned
---
src/backend/executor/execPartition.c | 119 ++++++++++++++++++++++++---
1 file changed, 107 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 7651886229..aeeee41d5f 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -483,6 +483,48 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+ /* it is not supported for cases of exclusion constraints. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+ if (arbiterIndexRelation->rd_index->indnkeyatts != indexRelation->rd_index->indnkeyatts)
+ return false;
+
+ for (i = 0; i < indexRelation->rd_index->indnkeyatts; i++)
+ {
+ int arbiterAttoNo = arbiterIndexRelation->rd_index->indkey.values[i];
+ int attoNo = indexRelation->rd_index->indkey.values[i];
+ if (arbiterAttoNo != attoNo)
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -693,6 +735,8 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
{
List *childIdxs;
+ List *nonAncestorIdxs = NIL;
+ int i, j, additional_arbiters = 0;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
@@ -703,23 +747,74 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, lfirst_oid(lc2)))
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxs = lappend_oid(nonAncestorIdxs, childIdx);
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxs)
+ {
+ for (i = 0; i < leaf_part_rri->ri_NumIndices; i++)
+ {
+ if (list_member_oid(nonAncestorIdxs, leaf_part_rri->ri_IndexRelationDescs[i]->rd_index->indexrelid))
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[i];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[i];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to us non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ for (j = 0; j < leaf_part_rri->ri_NumIndices; j++)
+ {
+ if (list_member_oid(arbiterIndexes,
+ leaf_part_rri->ri_IndexRelationDescs[j]->rd_index->indexrelid))
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[j];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[j];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxs);
+
+ /*
+ * If the resulting lists are of inequal length, something is wrong.
+ * (This shouldn't happen, since arbiter index selection should not
+ * pick up a non-ready index.)
+ *
+ * But we need to consider an additional arbiter indexes also.
+ */
+ if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
+ list_length(arbiterIndexes) - additional_arbiters)
+ elog(ERROR, "invalid arbiter index list");
}
-
- /*
- * If the resulting lists are of inequal length, something is wrong.
- * (This shouldn't happen, since arbiter index selection should not
- * pick up an invalid index.)
- */
- if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
- elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
/*
--
2.43.0
v4-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX.patchtext/x-patch; charset=US-ASCII; name=v4-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX.patchDownload
From b50d6e07fbd3ad7aa231b488c1ede72ac3985219 Mon Sep 17 00:00:00 2001
From: nkey <nkey@toloka.ai>
Date: Thu, 14 Nov 2024 22:35:49 +0100
Subject: [PATCH v4 1/4] Specs top reproduce the issues with CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY in scenarios involving INSERT ON
CONFLICT DO UPDATE. These tests reproduce different error cases related to
"duplicate key value violates unique constraint" where this error should not
occur by design.
* REINDEX CONCURRENTLY and UPSERT with inferred index
* CREATE INDEX CONCURRENTLY and UPSERT with inferred indexes
* REINDEX CONCURRENTLY on partitioned table
* REINDEX CONCURRENTLY with specified constraint name
* CREATE INDEX CONCURRENTLY with predicates
In each of these scenarios, the expected behavior is that the INSERT ON CONFLICT DO UPDATE should handle conflicts gracefully without raising a "duplicate key value violates unique constraint" error. However, due to the concurrent operations on the indexes, this error is encountered.
---
src/backend/commands/indexcmds.c | 5 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 2 +
src/test/modules/injection_points/Makefile | 7 +-
.../expected/index_concurrently_upsert.out | 80 ++++++
.../index_concurrently_upsert_predicate.out | 80 ++++++
.../expected/reindex_concurrently_upsert.out | 238 ++++++++++++++++++
...ndex_concurrently_upsert_on_constraint.out | 238 ++++++++++++++++++
...eindex_concurrently_upsert_partitioned.out | 238 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 5 +
.../specs/index_concurrently_upsert.spec | 68 +++++
.../index_concurrently_upsert_predicate.spec | 70 ++++++
.../specs/reindex_concurrently_upsert.spec | 86 +++++++
...dex_concurrently_upsert_on_constraint.spec | 86 +++++++
...index_concurrently_upsert_partitioned.spec | 88 +++++++
16 files changed, 1294 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index d1134733c1..bb0ea67554 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -72,6 +72,7 @@
#include "utils/regproc.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/* non-export function prototypes */
@@ -1769,6 +1770,7 @@ DefineIndex(Oid tableId,
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ INJECTION_POINT("define_index_before_set_valid");
/*
* Updating pg_index might involve TOAST table access, so ensure we have a
@@ -4206,7 +4208,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap");
StartTransactionCommand();
/*
@@ -4288,6 +4290,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead");
foreach(lc, indexIds)
{
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index f9a2fac79e..5d04f18934 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -117,6 +117,7 @@
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -936,6 +937,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict");
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 1161520f76..23cf4c6b54 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -69,6 +69,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1087,6 +1088,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative");
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 7d2b34d4f2..3a7357a050 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -64,6 +64,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -426,6 +427,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("invalidate_catalog_snapshot_end");
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index 0753a9df58..f8f86e8f3b 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -13,7 +13,12 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points reindex_conc
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic inplace
+ISOLATION = basic inplace \
+ reindex_concurrently_upsert \
+ index_concurrently_upsert \
+ reindex_concurrently_upsert_partitioned \
+ reindex_concurrently_upsert_on_constraint \
+ index_concurrently_upsert_predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 0000000000..f39a6d452a
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,80 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ SELECT injection_points_detach('define_index_before_set_valid');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: <... completed>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
new file mode 100644
index 0000000000..014d94d7ec
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
@@ -0,0 +1,80 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ SELECT injection_points_detach('define_index_before_set_valid');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: <... completed>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 0000000000..b7639ff7e6
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,238 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+step s2_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
new file mode 100644
index 0000000000..dbbb9691cc
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
@@ -0,0 +1,238 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+step s2_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
new file mode 100644
index 0000000000..b4cedf820c
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
@@ -0,0 +1,238 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+step s2_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 58f1900115..eccc686b46 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -44,6 +44,11 @@ tests += {
'specs': [
'basic',
'inplace',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
+ 'reindex_concurrently_upsert_partitioned',
+ 'reindex_concurrently_upsert_on_constraint',
+ 'index_concurrently_upsert_predicate',
],
},
'tap': {
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 0000000000..5d6aba9073
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,68 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ SELECT injection_points_detach('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
new file mode 100644
index 0000000000..2bc3cfd775
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
@@ -0,0 +1,70 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+
+ CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;}
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ SELECT injection_points_detach('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 0000000000..c6ad2c4198
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,86 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s2_start_upsert
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
new file mode 100644
index 0000000000..fb030f8575
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
@@ -0,0 +1,86 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s2_start_upsert
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
new file mode 100644
index 0000000000..efa55809ae
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
@@ -0,0 +1,88 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s2_start_upsert
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
v4-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchtext/x-patch; charset=US-ASCII; name=v4-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchDownload
From 39d5c287183c8baef57b3fc9a2fae537a6d4702e Mon Sep 17 00:00:00 2001
From: nkey <nkey@toloka.ai>
Date: Thu, 14 Nov 2024 22:37:30 +0100
Subject: [PATCH v4 3/4] Modify the infer_arbiter_indexes function to also look
for indexes that match the specified named constraint to be used as
arbiters. This ensures that the same set of arbiter indexes is used for all
concurrent transactions in cases where REINDEX CONCURRENTLY processes an
index used as a named constraint.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_on_constraint
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
---
src/backend/optimizer/util/plancat.c | 121 +++++++++++++++++++--------
1 file changed, 88 insertions(+), 33 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index c835813290..5ffef4595e 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -713,9 +713,10 @@ infer_arbiter_indexes(PlannerInfo *root)
List *indexList;
ListCell *l;
- /* Normalized inference attributes and inference expressions: */
- Bitmapset *inferAttrs = NULL;
- List *inferElems = NIL;
+ /* Normalized required attributes and expressions: */
+ Bitmapset *requiredArbiterAttrs = NULL;
+ List *requiredArbiterElems = NIL;
+ List *requiredIndexPredExprs = (List *) onconflict->arbiterWhere;
/* Results */
List *results = NIL;
@@ -754,8 +755,8 @@ infer_arbiter_indexes(PlannerInfo *root)
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
- inferElems = lappend(inferElems, elem->expr);
+ /* If not a plain Var, just shove it in requiredArbiterElems for now */
+ requiredArbiterElems = lappend(requiredArbiterElems, elem->expr);
continue;
}
@@ -767,30 +768,76 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("whole row unique index inference specifications are not supported")));
- inferAttrs = bms_add_member(inferAttrs,
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
+ indexList = RelationGetIndexList(relation);
+
/*
* Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * because some additional sanity checks are required. Additionally, we
+ * need to process other indexes as potential arbiters to account for
+ * cases where REINDEX CONCURRENTLY is processing an index used as a
+ * named constraint.
*/
if (onconflict->constraint != InvalidOid)
{
indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
+ {
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
- errmsg("constraint in ON CONFLICT clause has no associated index")));
+ errmsg("constraint in ON CONFLICT clause has no associated index")));
+ }
+
+ /*
+ * Find the named constraint index to extract its attributes and predicates.
+ * We open all indexes in the loop to avoid deadlock of changed order of locks.
+ * */
+ foreach(l, indexList)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation idxRel;
+ Form_pg_index idxForm;
+ AttrNumber natt;
+
+ idxRel = index_open(indexoid, rte->rellockmode);
+ idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Prepare requirements for other indexes to be used as arbiter together
+ * with indexOidFromConstraint. It is required to involve both equals indexes
+ * in case of REINDEX CONCURRENTLY.
+ */
+ for (natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno = idxRel->rd_index->indkey.values[natt];
+
+ if (attno != 0)
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+ requiredArbiterElems = RelationGetIndexExpressions(idxRel);
+ requiredIndexPredExprs = RelationGetIndexPredicate(idxRel);
+ /* We are done, so, quite the loop. */
+ index_close(idxRel, NoLock);
+ break;
+ }
+ }
+ index_close(idxRel, NoLock);
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
foreach(l, indexList)
{
Oid indexoid = lfirst_oid(l);
@@ -839,26 +886,23 @@ infer_arbiter_indexes(PlannerInfo *root)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
-
- results = lappend_oid(results, idxForm->indexrelid);
- foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ goto found;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /* In the case of "ON constraint_name DO UPDATE" we need to skip non-unique candidates. */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ goto next;
+ } else {
+ /*
+ * Only considering conventional inference at this point (not named
+ * constraints), so index under consideration can be immediately
+ * skipped if it's not unique
+ */
+ if (!idxForm->indisunique)
+ goto next;
}
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
-
/*
* So-called unique constraints with WITHOUT OVERLAPS are really
* exclusion constraints, so skip those too.
@@ -878,7 +922,7 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/* Non-expression attributes (if any) must match */
- if (!bms_equal(indexedAttrs, inferAttrs))
+ if (!bms_equal(indexedAttrs, requiredArbiterAttrs))
goto next;
/* Expression attributes (if any) must match */
@@ -886,6 +930,10 @@ infer_arbiter_indexes(PlannerInfo *root)
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
+ /*
+ * If arbiterElems are present, check them. If name >constraint is
+ * present arbiterElems == NIL.
+ */
foreach(el, onconflict->arbiterElems)
{
InferenceElem *elem = (InferenceElem *) lfirst(el);
@@ -923,26 +971,33 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of the conventional inference involved ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In the case of named constraint ensure candidate has equal set
+ * of expressions as the named constraint index.
*/
- if (list_difference(idxExprs, inferElems) != NIL)
+ if (list_difference(idxExprs, requiredArbiterElems) != NIL)
goto next;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
+ /*
+ * If it's a partial index and conventional inference, its predicate must be implied
+ * by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid && !predicate_implied_by(predExprs, requiredIndexPredExprs, false))
+ goto next;
+ /* If it's a partial index and named constraint predicates must be equal. */
+ if (indexOidFromConstraint != InvalidOid && list_difference(predExprs, requiredIndexPredExprs) != NIL)
goto next;
+found:
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
next:
--
2.43.0
Hello, everyone!
I've improved the test stability. The revised version should provide
consistent results in all test runs.
Best regards,
Mikhail.
Show quoted text
Attachments:
v5-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchtext/plain; charset=US-ASCII; name=v5-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchDownload
From e9a562587a509ace581d7ccf40eb41eb73b93b6f Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Sun, 24 Nov 2024 14:55:13 +0100
Subject: [PATCH v5 4/4] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY as
arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_partitioned
---
src/backend/executor/execPartition.c | 119 ++++++++++++++++++++++++---
1 file changed, 107 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 76518862291..aeeee41d5f1 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -483,6 +483,48 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+ /* it is not supported for cases of exclusion constraints. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+ if (arbiterIndexRelation->rd_index->indnkeyatts != indexRelation->rd_index->indnkeyatts)
+ return false;
+
+ for (i = 0; i < indexRelation->rd_index->indnkeyatts; i++)
+ {
+ int arbiterAttoNo = arbiterIndexRelation->rd_index->indkey.values[i];
+ int attoNo = indexRelation->rd_index->indkey.values[i];
+ if (arbiterAttoNo != attoNo)
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -693,6 +735,8 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
{
List *childIdxs;
+ List *nonAncestorIdxs = NIL;
+ int i, j, additional_arbiters = 0;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
@@ -703,23 +747,74 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, lfirst_oid(lc2)))
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxs = lappend_oid(nonAncestorIdxs, childIdx);
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxs)
+ {
+ for (i = 0; i < leaf_part_rri->ri_NumIndices; i++)
+ {
+ if (list_member_oid(nonAncestorIdxs, leaf_part_rri->ri_IndexRelationDescs[i]->rd_index->indexrelid))
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[i];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[i];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to us non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ for (j = 0; j < leaf_part_rri->ri_NumIndices; j++)
+ {
+ if (list_member_oid(arbiterIndexes,
+ leaf_part_rri->ri_IndexRelationDescs[j]->rd_index->indexrelid))
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[j];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[j];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxs);
+
+ /*
+ * If the resulting lists are of inequal length, something is wrong.
+ * (This shouldn't happen, since arbiter index selection should not
+ * pick up a non-ready index.)
+ *
+ * But we need to consider an additional arbiter indexes also.
+ */
+ if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
+ list_length(arbiterIndexes) - additional_arbiters)
+ elog(ERROR, "invalid arbiter index list");
}
-
- /*
- * If the resulting lists are of inequal length, something is wrong.
- * (This shouldn't happen, since arbiter index selection should not
- * pick up an invalid index.)
- */
- if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
- elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
/*
--
2.43.0
v5-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX.patchtext/plain; charset=US-ASCII; name=v5-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX.patchDownload
From e781488e7f6f7cd4be9b3b9582c1b9e32b3df946 Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Sun, 24 Nov 2024 14:45:41 +0100
Subject: [PATCH v5 1/4] Specs top reproduce the issues with CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY in scenarios involving INSERT ON
CONFLICT DO UPDATE. These tests reproduce different error cases related to
"duplicate key value violates unique constraint" where this error should not
occur by design.
* REINDEX CONCURRENTLY and UPSERT with inferred index
* CREATE INDEX CONCURRENTLY and UPSERT with inferred indexes
* REINDEX CONCURRENTLY on partitioned table
* REINDEX CONCURRENTLY with specified constraint name
* CREATE INDEX CONCURRENTLY with predicates
In each of these scenarios, the expected behavior is that the INSERT ON CONFLICT DO UPDATE should handle conflicts gracefully without raising a "duplicate key value violates unique constraint" error. However, due to the concurrent operations on the indexes, this error is encountered.
---
src/backend/commands/indexcmds.c | 4 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 2 +
src/test/modules/injection_points/Makefile | 7 +-
.../expected/index_concurrently_upsert.out | 80 ++++++
.../index_concurrently_upsert_predicate.out | 80 ++++++
.../expected/reindex_concurrently_upsert.out | 238 ++++++++++++++++++
...ndex_concurrently_upsert_on_constraint.out | 238 ++++++++++++++++++
...eindex_concurrently_upsert_partitioned.out | 238 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 11 +
.../specs/index_concurrently_upsert.spec | 68 +++++
.../index_concurrently_upsert_predicate.spec | 70 ++++++
.../specs/reindex_concurrently_upsert.spec | 86 +++++++
...dex_concurrently_upsert_on_constraint.spec | 86 +++++++
...index_concurrently_upsert_partitioned.spec | 88 +++++++
16 files changed, 1299 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index d1134733c17..8f48f14eddd 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1766,6 +1766,7 @@ DefineIndex(Oid tableId,
* before the reference snap was taken, we have to wait out any
* transactions that might have older snapshots.
*/
+ INJECTION_POINT("define_index_before_set_valid");
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
@@ -4206,7 +4207,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap");
StartTransactionCommand();
/*
@@ -4285,6 +4286,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* index_drop() for more details.
*/
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead");
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index f9a2fac79e4..5d04f189340 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -117,6 +117,7 @@
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -936,6 +937,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict");
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 1161520f76b..23cf4c6b540 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -69,6 +69,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1087,6 +1088,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative");
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 7d2b34d4f20..3a7357a050d 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -64,6 +64,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -426,6 +427,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("invalidate_catalog_snapshot_end");
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index 0753a9df58c..f8f86e8f3b6 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -13,7 +13,12 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points reindex_conc
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic inplace
+ISOLATION = basic inplace \
+ reindex_concurrently_upsert \
+ index_concurrently_upsert \
+ reindex_concurrently_upsert_partitioned \
+ reindex_concurrently_upsert_on_constraint \
+ index_concurrently_upsert_predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 00000000000..7f0659e8369
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,80 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: <... completed>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
new file mode 100644
index 00000000000..2300d5165e9
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
@@ -0,0 +1,80 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: <... completed>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 00000000000..24bbbcbdd88
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,238 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+step s2_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
new file mode 100644
index 00000000000..d1cfd1731c8
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
@@ -0,0 +1,238 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+step s2_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
new file mode 100644
index 00000000000..c95ff264f12
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
@@ -0,0 +1,238 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s2_start_upsert: INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+step s2_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 58f19001157..91fc8ce687f 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -44,7 +44,16 @@ tests += {
'specs': [
'basic',
'inplace',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
+ 'reindex_concurrently_upsert_partitioned',
+ 'reindex_concurrently_upsert_on_constraint',
+ 'index_concurrently_upsert_predicate',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
+ # We waiting for all snapshots, so, avoid parallel test executions
+ 'runningcheck-parallel': false,
},
'tap': {
'env': {
@@ -53,5 +62,7 @@ tests += {
'tests': [
't/001_stats.pl',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
},
}
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 00000000000..075450935b6
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,68 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
new file mode 100644
index 00000000000..70a27475e10
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
@@ -0,0 +1,70 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+
+ CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;}
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 00000000000..38b86d84345
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,86 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s2_start_upsert
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
new file mode 100644
index 00000000000..7d8e371bb0a
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
@@ -0,0 +1,86 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s2_start_upsert
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
new file mode 100644
index 00000000000..b9253463039
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
@@ -0,0 +1,88 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert { INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now(); }
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s2_start_upsert
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
v5-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchtext/plain; charset=US-ASCII; name=v5-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchDownload
From 20f6ddb8c43463b0fc9ee47505a114fe44e6d4ff Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Sun, 24 Nov 2024 14:54:13 +0100
Subject: [PATCH v5 3/4] Modify the infer_arbiter_indexes function to also look
for indexes that match the specified named constraint to be used as
arbiters. This ensures that the same set of arbiter indexes is used for all
concurrent transactions in cases where REINDEX CONCURRENTLY processes an
index used as a named constraint.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_on_constraint
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
---
src/backend/optimizer/util/plancat.c | 121 +++++++++++++++++++--------
1 file changed, 88 insertions(+), 33 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index c835813290a..5ffef4595e2 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -713,9 +713,10 @@ infer_arbiter_indexes(PlannerInfo *root)
List *indexList;
ListCell *l;
- /* Normalized inference attributes and inference expressions: */
- Bitmapset *inferAttrs = NULL;
- List *inferElems = NIL;
+ /* Normalized required attributes and expressions: */
+ Bitmapset *requiredArbiterAttrs = NULL;
+ List *requiredArbiterElems = NIL;
+ List *requiredIndexPredExprs = (List *) onconflict->arbiterWhere;
/* Results */
List *results = NIL;
@@ -754,8 +755,8 @@ infer_arbiter_indexes(PlannerInfo *root)
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
- inferElems = lappend(inferElems, elem->expr);
+ /* If not a plain Var, just shove it in requiredArbiterElems for now */
+ requiredArbiterElems = lappend(requiredArbiterElems, elem->expr);
continue;
}
@@ -767,30 +768,76 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("whole row unique index inference specifications are not supported")));
- inferAttrs = bms_add_member(inferAttrs,
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
+ indexList = RelationGetIndexList(relation);
+
/*
* Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * because some additional sanity checks are required. Additionally, we
+ * need to process other indexes as potential arbiters to account for
+ * cases where REINDEX CONCURRENTLY is processing an index used as a
+ * named constraint.
*/
if (onconflict->constraint != InvalidOid)
{
indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
+ {
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
- errmsg("constraint in ON CONFLICT clause has no associated index")));
+ errmsg("constraint in ON CONFLICT clause has no associated index")));
+ }
+
+ /*
+ * Find the named constraint index to extract its attributes and predicates.
+ * We open all indexes in the loop to avoid deadlock of changed order of locks.
+ * */
+ foreach(l, indexList)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation idxRel;
+ Form_pg_index idxForm;
+ AttrNumber natt;
+
+ idxRel = index_open(indexoid, rte->rellockmode);
+ idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Prepare requirements for other indexes to be used as arbiter together
+ * with indexOidFromConstraint. It is required to involve both equals indexes
+ * in case of REINDEX CONCURRENTLY.
+ */
+ for (natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno = idxRel->rd_index->indkey.values[natt];
+
+ if (attno != 0)
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+ requiredArbiterElems = RelationGetIndexExpressions(idxRel);
+ requiredIndexPredExprs = RelationGetIndexPredicate(idxRel);
+ /* We are done, so, quite the loop. */
+ index_close(idxRel, NoLock);
+ break;
+ }
+ }
+ index_close(idxRel, NoLock);
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
foreach(l, indexList)
{
Oid indexoid = lfirst_oid(l);
@@ -839,26 +886,23 @@ infer_arbiter_indexes(PlannerInfo *root)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
-
- results = lappend_oid(results, idxForm->indexrelid);
- foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ goto found;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /* In the case of "ON constraint_name DO UPDATE" we need to skip non-unique candidates. */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ goto next;
+ } else {
+ /*
+ * Only considering conventional inference at this point (not named
+ * constraints), so index under consideration can be immediately
+ * skipped if it's not unique
+ */
+ if (!idxForm->indisunique)
+ goto next;
}
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
-
/*
* So-called unique constraints with WITHOUT OVERLAPS are really
* exclusion constraints, so skip those too.
@@ -878,7 +922,7 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/* Non-expression attributes (if any) must match */
- if (!bms_equal(indexedAttrs, inferAttrs))
+ if (!bms_equal(indexedAttrs, requiredArbiterAttrs))
goto next;
/* Expression attributes (if any) must match */
@@ -886,6 +930,10 @@ infer_arbiter_indexes(PlannerInfo *root)
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
+ /*
+ * If arbiterElems are present, check them. If name >constraint is
+ * present arbiterElems == NIL.
+ */
foreach(el, onconflict->arbiterElems)
{
InferenceElem *elem = (InferenceElem *) lfirst(el);
@@ -923,26 +971,33 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of the conventional inference involved ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In the case of named constraint ensure candidate has equal set
+ * of expressions as the named constraint index.
*/
- if (list_difference(idxExprs, inferElems) != NIL)
+ if (list_difference(idxExprs, requiredArbiterElems) != NIL)
goto next;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
+ /*
+ * If it's a partial index and conventional inference, its predicate must be implied
+ * by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid && !predicate_implied_by(predExprs, requiredIndexPredExprs, false))
+ goto next;
+ /* If it's a partial index and named constraint predicates must be equal. */
+ if (indexOidFromConstraint != InvalidOid && list_difference(predExprs, requiredIndexPredExprs) != NIL)
goto next;
+found:
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
next:
--
2.43.0
v5-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchtext/plain; charset=US-ASCII; name=v5-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchDownload
From ddfc5d199626759695f4d7f1fa78c58fafae5e54 Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Sun, 24 Nov 2024 14:49:35 +0100
Subject: [PATCH v5 2/4] Modify the infer_arbiter_indexes function to consider
both indisvalid and indisready indexes. Ensure that at least one indisvalid
index is still required.
The change ensures that all concurrent transactions utilize the same set of indexes as arbiters. This uniformity is required to avoid conditions that could lead to "duplicate key value violates unique constraint" errors during UPSERT operations.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert
* index_concurrently_upsert
* index_concurrently_upsert_predicate
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
* reindex_concurrently_upsert_on_constraint
---
src/backend/optimizer/util/plancat.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 37b0ca2e439..c835813290a 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -719,6 +719,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Results */
List *results = NIL;
+ bool foundValid = false;
/*
* Quickly return NIL for ON CONFLICT DO NOTHING without an inference
@@ -812,7 +813,13 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
- if (!idxForm->indisvalid)
+ /*
+ * We need to consider both indisvalid and indisready indexes because
+ * them may become indisvalid before execution phase. It is required
+ * to keep set of indexes used as arbiter to be the same for all
+ * concurrent transactions.
+ */
+ if (!idxForm->indisready)
goto next;
/*
@@ -834,10 +841,9 @@ infer_arbiter_indexes(PlannerInfo *root)
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
results = lappend_oid(results, idxForm->indexrelid);
- list_free(indexList);
+ foundValid |= idxForm->indisvalid;
index_close(idxRel, NoLock);
- table_close(relation, NoLock);
- return results;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
@@ -938,6 +944,7 @@ infer_arbiter_indexes(PlannerInfo *root)
goto next;
results = lappend_oid(results, idxForm->indexrelid);
+ foundValid |= idxForm->indisvalid;
next:
index_close(idxRel, NoLock);
}
@@ -945,7 +952,8 @@ next:
list_free(indexList);
table_close(relation, NoLock);
- if (results == NIL)
+ /* It is required to have at least one indisvalid index during the planning. */
+ if (results == NIL || !foundValid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
errmsg("there is no unique or exclusion constraint matching the ON CONFLICT specification")));
--
2.43.0
Hello, everyone!
I have noticed tests are still flapping a little bit on FreeBSD.
Now I have added some markers to isolation specs to avoid possible race
conditions.
Best regards,
Mikhail.
Show quoted text
Attachments:
v6-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchapplication/octet-stream; name=v6-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchDownload
From cb8129a5baf491a8b9e24df819ebc6cdf67a19b9 Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Sun, 24 Nov 2024 14:54:13 +0100
Subject: [PATCH v6 3/4] Modify the infer_arbiter_indexes function to also look
for indexes that match the specified named constraint to be used as arbiters.
This ensures that the same set of arbiter indexes is used for all concurrent
transactions in cases where REINDEX CONCURRENTLY processes an index used as a
named constraint.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_on_constraint
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
---
src/backend/optimizer/util/plancat.c | 121 +++++++++++++++++++--------
1 file changed, 88 insertions(+), 33 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 34f205109ae..f91203dd353 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -714,9 +714,10 @@ infer_arbiter_indexes(PlannerInfo *root)
List *indexList;
ListCell *l;
- /* Normalized inference attributes and inference expressions: */
- Bitmapset *inferAttrs = NULL;
- List *inferElems = NIL;
+ /* Normalized required attributes and expressions: */
+ Bitmapset *requiredArbiterAttrs = NULL;
+ List *requiredArbiterElems = NIL;
+ List *requiredIndexPredExprs = (List *) onconflict->arbiterWhere;
/* Results */
List *results = NIL;
@@ -755,8 +756,8 @@ infer_arbiter_indexes(PlannerInfo *root)
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
- inferElems = lappend(inferElems, elem->expr);
+ /* If not a plain Var, just shove it in requiredArbiterElems for now */
+ requiredArbiterElems = lappend(requiredArbiterElems, elem->expr);
continue;
}
@@ -768,30 +769,76 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("whole row unique index inference specifications are not supported")));
- inferAttrs = bms_add_member(inferAttrs,
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
+ indexList = RelationGetIndexList(relation);
+
/*
* Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * because some additional sanity checks are required. Additionally, we
+ * need to process other indexes as potential arbiters to account for
+ * cases where REINDEX CONCURRENTLY is processing an index used as a
+ * named constraint.
*/
if (onconflict->constraint != InvalidOid)
{
indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
+ {
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
- errmsg("constraint in ON CONFLICT clause has no associated index")));
+ errmsg("constraint in ON CONFLICT clause has no associated index")));
+ }
+
+ /*
+ * Find the named constraint index to extract its attributes and predicates.
+ * We open all indexes in the loop to avoid deadlock of changed order of locks.
+ * */
+ foreach(l, indexList)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation idxRel;
+ Form_pg_index idxForm;
+ AttrNumber natt;
+
+ idxRel = index_open(indexoid, rte->rellockmode);
+ idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Prepare requirements for other indexes to be used as arbiter together
+ * with indexOidFromConstraint. It is required to involve both equals indexes
+ * in case of REINDEX CONCURRENTLY.
+ */
+ for (natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno = idxRel->rd_index->indkey.values[natt];
+
+ if (attno != 0)
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+ requiredArbiterElems = RelationGetIndexExpressions(idxRel);
+ requiredIndexPredExprs = RelationGetIndexPredicate(idxRel);
+ /* We are done, so, quite the loop. */
+ index_close(idxRel, NoLock);
+ break;
+ }
+ }
+ index_close(idxRel, NoLock);
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
foreach(l, indexList)
{
Oid indexoid = lfirst_oid(l);
@@ -840,26 +887,23 @@ infer_arbiter_indexes(PlannerInfo *root)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
-
- results = lappend_oid(results, idxForm->indexrelid);
- foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ goto found;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /* In the case of "ON constraint_name DO UPDATE" we need to skip non-unique candidates. */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ goto next;
+ } else {
+ /*
+ * Only considering conventional inference at this point (not named
+ * constraints), so index under consideration can be immediately
+ * skipped if it's not unique
+ */
+ if (!idxForm->indisunique)
+ goto next;
}
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
-
/*
* So-called unique constraints with WITHOUT OVERLAPS are really
* exclusion constraints, so skip those too.
@@ -879,7 +923,7 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/* Non-expression attributes (if any) must match */
- if (!bms_equal(indexedAttrs, inferAttrs))
+ if (!bms_equal(indexedAttrs, requiredArbiterAttrs))
goto next;
/* Expression attributes (if any) must match */
@@ -887,6 +931,10 @@ infer_arbiter_indexes(PlannerInfo *root)
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
+ /*
+ * If arbiterElems are present, check them. If name >constraint is
+ * present arbiterElems == NIL.
+ */
foreach(el, onconflict->arbiterElems)
{
InferenceElem *elem = (InferenceElem *) lfirst(el);
@@ -924,26 +972,33 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of the conventional inference involved ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In the case of named constraint ensure candidate has equal set
+ * of expressions as the named constraint index.
*/
- if (list_difference(idxExprs, inferElems) != NIL)
+ if (list_difference(idxExprs, requiredArbiterElems) != NIL)
goto next;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
+ /*
+ * If it's a partial index and conventional inference, its predicate must be implied
+ * by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid && !predicate_implied_by(predExprs, requiredIndexPredExprs, false))
+ goto next;
+ /* If it's a partial index and named constraint predicates must be equal. */
+ if (indexOidFromConstraint != InvalidOid && list_difference(predExprs, requiredIndexPredExprs) != NIL)
goto next;
+found:
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
next:
--
2.43.0
v6-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX-.patchapplication/octet-stream; name=v6-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX-.patchDownload
From 066cde03a949b7951206dd2247ee607ab1192ceb Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Sun, 24 Nov 2024 14:45:41 +0100
Subject: [PATCH v6 1/4] Specs top reproduce the issues with CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY in scenarios involving INSERT ON
CONFLICT DO UPDATE. These tests reproduce different error cases related to
"duplicate key value violates unique constraint" where this error should not
occur by design.
* REINDEX CONCURRENTLY and UPSERT with inferred index
* CREATE INDEX CONCURRENTLY and UPSERT with inferred indexes
* REINDEX CONCURRENTLY on partitioned table
* REINDEX CONCURRENTLY with specified constraint name
* CREATE INDEX CONCURRENTLY with predicates
In each of these scenarios, the expected behavior is that the INSERT ON CONFLICT DO UPDATE should handle conflicts gracefully without raising a "duplicate key value violates unique constraint" error. However, due to the concurrent operations on the indexes, this error is encountered.
---
src/backend/commands/indexcmds.c | 4 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 2 +
src/test/modules/injection_points/Makefile | 7 +-
.../expected/index_concurrently_upsert.out | 106 ++++++
.../index_concurrently_upsert_predicate.out | 106 ++++++
.../expected/reindex_concurrently_upsert.out | 316 ++++++++++++++++++
...ndex_concurrently_upsert_on_constraint.out | 316 ++++++++++++++++++
...eindex_concurrently_upsert_partitioned.out | 316 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 11 +
.../specs/index_concurrently_upsert.spec | 96 ++++++
.../index_concurrently_upsert_predicate.spec | 98 ++++++
.../specs/reindex_concurrently_upsert.spec | 114 +++++++
...dex_concurrently_upsert_on_constraint.spec | 115 +++++++
...index_concurrently_upsert_partitioned.spec | 116 +++++++
16 files changed, 1726 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index d6e23caef17..0ff498c4e14 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1766,6 +1766,7 @@ DefineIndex(Oid tableId,
* before the reference snap was taken, we have to wait out any
* transactions that might have older snapshots.
*/
+ INJECTION_POINT("define_index_before_set_valid");
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
@@ -4206,7 +4207,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap");
StartTransactionCommand();
/*
@@ -4285,6 +4286,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* index_drop() for more details.
*/
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead");
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 7c87f012c30..ae11c1dd463 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -117,6 +117,7 @@
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -936,6 +937,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict");
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 1af8c9caf6c..8a1a085b106 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -69,6 +69,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1087,6 +1088,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative");
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 8f1508b1ee2..3d018c3a1e8 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -64,6 +64,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -388,6 +389,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("invalidate_catalog_snapshot_end");
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index 0753a9df58c..f8f86e8f3b6 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -13,7 +13,12 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points reindex_conc
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic inplace
+ISOLATION = basic inplace \
+ reindex_concurrently_upsert \
+ index_concurrently_upsert \
+ reindex_concurrently_upsert_partitioned \
+ reindex_concurrently_upsert_on_constraint \
+ index_concurrently_upsert_predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 00000000000..4a65abbb55e
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,106 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+step s1_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ call raise_notice('wakeup before set valid done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before set valid done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ call raise_notice('wakeup 1 done again');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done again
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 2 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s2: NOTICE: upsert 2 done
+step s2_start_upsert: <... completed>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: <... completed>
+s1: NOTICE: upsert 1 done
+step s1_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
new file mode 100644
index 00000000000..7c7ddc7a7f6
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
@@ -0,0 +1,106 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000; <waiting ...>
+step s1_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ call raise_notice('wakeup before set valid done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before set valid done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ call raise_notice('wakeup 1 done again');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done again
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 2 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s2: NOTICE: upsert 2 done
+step s2_start_upsert: <... completed>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: <... completed>
+s1: NOTICE: upsert 1 done
+step s1_start_upsert: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 00000000000..cb597620fe3
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,316 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before swap done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: upsert 1 done
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 2 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s2: NOTICE: upsert 2 done
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before set dead done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s2_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before swap done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: upsert 1 done
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 2 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s2: NOTICE: upsert 2 done
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before set dead done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before swap done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s2_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: upsert 1 done
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before set dead done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 2 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s2: NOTICE: upsert 2 done
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
new file mode 100644
index 00000000000..b29caa94e50
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
@@ -0,0 +1,316 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before swap done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: upsert 1 done
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 2 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s2: NOTICE: upsert 2 done
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before set dead done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s2_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before swap done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: upsert 1 done
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 2 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s2: NOTICE: upsert 2 done
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before set dead done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before swap done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s2_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: upsert 1 done
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before set dead done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 2 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s2: NOTICE: upsert 2 done
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
new file mode 100644
index 00000000000..16aebaa2e12
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
@@ -0,0 +1,316 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before swap done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: upsert 1 done
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 2 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s2: NOTICE: upsert 2 done
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before set dead done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s2_start_upsert s4_wakeup_to_swap s1_start_upsert s4_wakeup_s1 s4_wakeup_s2 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s2_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before swap done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: upsert 1 done
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 2 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s2: NOTICE: upsert 2 done
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before set dead done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before swap done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s2_start_upsert:
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 1 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: upsert 1 done
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup before set dead done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+s4: NOTICE: wakeup 2 done
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s2: NOTICE: upsert 2 done
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 989b4db226b..428f1219349 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -44,7 +44,16 @@ tests += {
'specs': [
'basic',
'inplace',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
+ 'reindex_concurrently_upsert_partitioned',
+ 'reindex_concurrently_upsert_on_constraint',
+ 'index_concurrently_upsert_predicate',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
+ # We waiting for all snapshots, so, avoid parallel test executions
+ 'runningcheck-parallel': false,
},
'tap': {
'env': {
@@ -53,5 +62,7 @@ tests += {
'tests': [
't/001_stats.pl',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
},
}
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 00000000000..a33e3e40b41
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,96 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+
+ CREATE PROCEDURE raise_notice (s text) LANGUAGE plpgsql AS
+ $$
+ BEGIN
+ RAISE NOTICE '%', s;
+ END;
+ $$;
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+ DROP PROCEDURE raise_notice;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ call raise_notice('wakeup 1 done again');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ call raise_notice('wakeup before set valid done');
+}
+
+permutation
+ s3_start_create_index(s2_start_upsert notices 1)
+ s1_start_upsert(s2_start_upsert notices 1)
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
new file mode 100644
index 00000000000..5dc7a71d7d4
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
@@ -0,0 +1,98 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+
+ CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+
+ CREATE PROCEDURE raise_notice (s text) LANGUAGE plpgsql AS
+ $$
+ BEGIN
+ RAISE NOTICE '%', s;
+ END;
+ $$;
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+ DROP PROCEDURE raise_notice;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;}
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+ call raise_notice('wakeup 1 done again');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+ call raise_notice('wakeup before set valid done');
+}
+
+permutation
+ s3_start_create_index(s2_start_upsert notices 1)
+ s1_start_upsert(s2_start_upsert notices 1)
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 00000000000..733a83c91d2
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,114 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+
+ CREATE PROCEDURE raise_notice (s text) LANGUAGE plpgsql AS
+ $$
+ BEGIN
+ RAISE NOTICE '%', s;
+ END;
+ $$;
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+ DROP PROCEDURE raise_notice;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+}
+
+permutation
+ s3_start_reindex(s2_start_upsert notices 1)
+ s1_start_upsert(s4_wakeup_s2 notices 1)
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert notices 1)
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert notices 1)
+ s2_start_upsert(s1_start_upsert notices 1)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s2_start_upsert notices 1)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert notices 1)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
new file mode 100644
index 00000000000..7181fee8031
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
@@ -0,0 +1,115 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+
+ CREATE PROCEDURE raise_notice (s text) LANGUAGE plpgsql AS
+ $$
+ BEGIN
+ RAISE NOTICE '%', s;
+ END;
+ $$;
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+ DROP PROCEDURE raise_notice;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+}
+
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+}
+
+permutation
+ s3_start_reindex(s2_start_upsert notices 1)
+ s1_start_upsert(s4_wakeup_s2 notices 1)
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert notices 1)
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert notices 1)
+ s2_start_upsert(s1_start_upsert notices 1)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s2_start_upsert notices 1)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert notices 1)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
new file mode 100644
index 00000000000..d577889e9c9
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
@@ -0,0 +1,116 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+
+ CREATE PROCEDURE raise_notice (s text) LANGUAGE plpgsql AS
+ $$
+ BEGIN
+ RAISE NOTICE '%', s;
+ END;
+ $$;
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+ DROP PROCEDURE raise_notice;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 1 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ DO $$
+ BEGIN
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ RAISE NOTICE 'upsert 2 done';
+ EXCEPTION WHEN OTHERS THEN
+ RAISE NOTICE '% %', SQLERRM, SQLSTATE;
+ END; $$
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+ call raise_notice('wakeup before swap done');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+ call raise_notice('wakeup 1 done');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+ call raise_notice('wakeup 2 done');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+ call raise_notice('wakeup before set dead done');
+}
+
+permutation
+ s3_start_reindex(s2_start_upsert notices 1)
+ s1_start_upsert(s4_wakeup_s2 notices 1)
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert notices 1)
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert notices 1)
+ s2_start_upsert(s1_start_upsert notices 1)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s4_wakeup_s1
+ s4_wakeup_s2
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s2_start_upsert notices 1)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert notices 1)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
v6-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchapplication/octet-stream; name=v6-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchDownload
From ca3c8144eec8c8cb8636a5c1c18190234ae4a96b Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Sun, 24 Nov 2024 14:55:13 +0100
Subject: [PATCH v6 4/4] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY as
arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_partitioned
---
src/backend/executor/execPartition.c | 119 ++++++++++++++++++++++++---
1 file changed, 107 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 7e71d422a62..3922ae39681 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -483,6 +483,48 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+ /* it is not supported for cases of exclusion constraints. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+ if (arbiterIndexRelation->rd_index->indnkeyatts != indexRelation->rd_index->indnkeyatts)
+ return false;
+
+ for (i = 0; i < indexRelation->rd_index->indnkeyatts; i++)
+ {
+ int arbiterAttoNo = arbiterIndexRelation->rd_index->indkey.values[i];
+ int attoNo = indexRelation->rd_index->indkey.values[i];
+ if (arbiterAttoNo != attoNo)
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -693,6 +735,8 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
{
List *childIdxs;
+ List *nonAncestorIdxs = NIL;
+ int i, j, additional_arbiters = 0;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
@@ -703,23 +747,74 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, lfirst_oid(lc2)))
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxs = lappend_oid(nonAncestorIdxs, childIdx);
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxs)
+ {
+ for (i = 0; i < leaf_part_rri->ri_NumIndices; i++)
+ {
+ if (list_member_oid(nonAncestorIdxs, leaf_part_rri->ri_IndexRelationDescs[i]->rd_index->indexrelid))
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[i];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[i];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to us non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ for (j = 0; j < leaf_part_rri->ri_NumIndices; j++)
+ {
+ if (list_member_oid(arbiterIndexes,
+ leaf_part_rri->ri_IndexRelationDescs[j]->rd_index->indexrelid))
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[j];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[j];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxs);
+
+ /*
+ * If the resulting lists are of inequal length, something is wrong.
+ * (This shouldn't happen, since arbiter index selection should not
+ * pick up a non-ready index.)
+ *
+ * But we need to consider an additional arbiter indexes also.
+ */
+ if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
+ list_length(arbiterIndexes) - additional_arbiters)
+ elog(ERROR, "invalid arbiter index list");
}
-
- /*
- * If the resulting lists are of inequal length, something is wrong.
- * (This shouldn't happen, since arbiter index selection should not
- * pick up an invalid index.)
- */
- if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
- elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
/*
--
2.43.0
v6-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchapplication/octet-stream; name=v6-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchDownload
From 5792be833e4a8204962fbe24294ced6b8113d0a0 Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Sun, 24 Nov 2024 14:49:35 +0100
Subject: [PATCH v6 2/4] Modify the infer_arbiter_indexes function to consider
both indisvalid and indisready indexes. Ensure that at least one indisvalid
index is still required.
The change ensures that all concurrent transactions utilize the same set of indexes as arbiters. This uniformity is required to avoid conditions that could lead to "duplicate key value violates unique constraint" errors during UPSERT operations.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert
* index_concurrently_upsert
* index_concurrently_upsert_predicate
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
* reindex_concurrently_upsert_on_constraint
---
src/backend/optimizer/util/plancat.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index b9759c31252..34f205109ae 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -720,6 +720,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Results */
List *results = NIL;
+ bool foundValid = false;
/*
* Quickly return NIL for ON CONFLICT DO NOTHING without an inference
@@ -813,7 +814,13 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
- if (!idxForm->indisvalid)
+ /*
+ * We need to consider both indisvalid and indisready indexes because
+ * them may become indisvalid before execution phase. It is required
+ * to keep set of indexes used as arbiter to be the same for all
+ * concurrent transactions.
+ */
+ if (!idxForm->indisready)
goto next;
/*
@@ -835,10 +842,9 @@ infer_arbiter_indexes(PlannerInfo *root)
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
results = lappend_oid(results, idxForm->indexrelid);
- list_free(indexList);
+ foundValid |= idxForm->indisvalid;
index_close(idxRel, NoLock);
- table_close(relation, NoLock);
- return results;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
@@ -939,6 +945,7 @@ infer_arbiter_indexes(PlannerInfo *root)
goto next;
results = lappend_oid(results, idxForm->indexrelid);
+ foundValid |= idxForm->indisvalid;
next:
index_close(idxRel, NoLock);
}
@@ -946,7 +953,8 @@ next:
list_free(indexList);
table_close(relation, NoLock);
- if (results == NIL)
+ /* It is required to have at least one indisvalid index during the planning. */
+ if (results == NIL || !foundValid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
errmsg("there is no unique or exclusion constraint matching the ON CONFLICT specification")));
--
2.43.0
Hello, everyone!
This is just small test refactoring after the last stabilization.
Best regards,
Mikhail.
Show quoted text
Attachments:
v7-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX.patchapplication/octet-stream; name=v7-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX.patchDownload
From 73ee9ba10ed157365876e4ba1efa1d529f86b6f6 Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Fri, 17 Jan 2025 21:14:32 +0100
Subject: [PATCH v7 1/4] Specs top reproduce the issues with CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY in scenarios involving INSERT ON
CONFLICT DO UPDATE. These tests reproduce different error cases related to
"duplicate key value violates unique constraint" where this error should not
occur by design.
* REINDEX CONCURRENTLY and UPSERT with inferred index
* CREATE INDEX CONCURRENTLY and UPSERT with inferred indexes
* REINDEX CONCURRENTLY on partitioned table
* REINDEX CONCURRENTLY with specified constraint name
* CREATE INDEX CONCURRENTLY with predicates
In each of these scenarios, the expected behavior is that the INSERT ON CONFLICT DO UPDATE should handle conflicts gracefully without raising a "duplicate key value violates unique constraint" error. However, due to the concurrent operations on the indexes, this error is encountered.
---
src/backend/commands/indexcmds.c | 4 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 2 +
src/test/modules/injection_points/Makefile | 7 +-
.../expected/index_concurrently_upsert.out | 84 ++++++
.../index_concurrently_upsert_predicate.out | 84 ++++++
.../expected/reindex_concurrently_upsert.out | 250 ++++++++++++++++++
...ndex_concurrently_upsert_on_constraint.out | 250 ++++++++++++++++++
...eindex_concurrently_upsert_partitioned.out | 250 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 11 +
.../specs/index_concurrently_upsert.spec | 72 +++++
.../index_concurrently_upsert_predicate.spec | 74 ++++++
.../specs/reindex_concurrently_upsert.spec | 90 +++++++
...dex_concurrently_upsert_on_constraint.spec | 91 +++++++
...index_concurrently_upsert_partitioned.spec | 92 +++++++
16 files changed, 1364 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index d6e23caef17..0ff498c4e14 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1766,6 +1766,7 @@ DefineIndex(Oid tableId,
* before the reference snap was taken, we have to wait out any
* transactions that might have older snapshots.
*/
+ INJECTION_POINT("define_index_before_set_valid");
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
@@ -4206,7 +4207,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap");
StartTransactionCommand();
/*
@@ -4285,6 +4286,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* index_drop() for more details.
*/
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead");
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 7c87f012c30..ae11c1dd463 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -117,6 +117,7 @@
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -936,6 +937,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict");
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 1af8c9caf6c..8a1a085b106 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -69,6 +69,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1087,6 +1088,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative");
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 8f1508b1ee2..3d018c3a1e8 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -64,6 +64,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -388,6 +389,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("invalidate_catalog_snapshot_end");
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index 0753a9df58c..f8f86e8f3b6 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -13,7 +13,12 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points reindex_conc
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic inplace
+ISOLATION = basic inplace \
+ reindex_concurrently_upsert \
+ index_concurrently_upsert \
+ reindex_concurrently_upsert_partitioned \
+ reindex_concurrently_upsert_on_constraint \
+ index_concurrently_upsert_predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 00000000000..e7612e065f4
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
new file mode 100644
index 00000000000..0ef2f3a681c
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 00000000000..1bd8041289e
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,250 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
new file mode 100644
index 00000000000..68288812de2
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
@@ -0,0 +1,250 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
new file mode 100644
index 00000000000..f8c81e2bea2
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
@@ -0,0 +1,250 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 989b4db226b..428f1219349 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -44,7 +44,16 @@ tests += {
'specs': [
'basic',
'inplace',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
+ 'reindex_concurrently_upsert_partitioned',
+ 'reindex_concurrently_upsert_on_constraint',
+ 'index_concurrently_upsert_predicate',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
+ # We waiting for all snapshots, so, avoid parallel test executions
+ 'runningcheck-parallel': false,
},
'tap': {
'env': {
@@ -53,5 +62,7 @@ tests += {
'tests': [
't/001_stats.pl',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
},
}
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 00000000000..473b0408f55
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,72 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
new file mode 100644
index 00000000000..c8644a82d57
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
@@ -0,0 +1,74 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+
+ CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;}
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 00000000000..08d0deaf58b
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,90 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
new file mode 100644
index 00000000000..97622388e71
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
@@ -0,0 +1,91 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
new file mode 100644
index 00000000000..41f10c8736d
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
@@ -0,0 +1,92 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
v7-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchapplication/octet-stream; name=v7-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchDownload
From d5e3a336869ff2f0abe299440358c70a2c6a5d34 Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Fri, 17 Jan 2025 21:16:10 +0100
Subject: [PATCH v7 4/4] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY as
arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_partitioned
---
src/backend/executor/execPartition.c | 119 ++++++++++++++++++++++++---
1 file changed, 107 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 7e71d422a62..3922ae39681 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -483,6 +483,48 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+ /* it is not supported for cases of exclusion constraints. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+ if (arbiterIndexRelation->rd_index->indnkeyatts != indexRelation->rd_index->indnkeyatts)
+ return false;
+
+ for (i = 0; i < indexRelation->rd_index->indnkeyatts; i++)
+ {
+ int arbiterAttoNo = arbiterIndexRelation->rd_index->indkey.values[i];
+ int attoNo = indexRelation->rd_index->indkey.values[i];
+ if (arbiterAttoNo != attoNo)
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -693,6 +735,8 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
{
List *childIdxs;
+ List *nonAncestorIdxs = NIL;
+ int i, j, additional_arbiters = 0;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
@@ -703,23 +747,74 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, lfirst_oid(lc2)))
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxs = lappend_oid(nonAncestorIdxs, childIdx);
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxs)
+ {
+ for (i = 0; i < leaf_part_rri->ri_NumIndices; i++)
+ {
+ if (list_member_oid(nonAncestorIdxs, leaf_part_rri->ri_IndexRelationDescs[i]->rd_index->indexrelid))
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[i];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[i];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to us non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ for (j = 0; j < leaf_part_rri->ri_NumIndices; j++)
+ {
+ if (list_member_oid(arbiterIndexes,
+ leaf_part_rri->ri_IndexRelationDescs[j]->rd_index->indexrelid))
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[j];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[j];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxs);
+
+ /*
+ * If the resulting lists are of inequal length, something is wrong.
+ * (This shouldn't happen, since arbiter index selection should not
+ * pick up a non-ready index.)
+ *
+ * But we need to consider an additional arbiter indexes also.
+ */
+ if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
+ list_length(arbiterIndexes) - additional_arbiters)
+ elog(ERROR, "invalid arbiter index list");
}
-
- /*
- * If the resulting lists are of inequal length, something is wrong.
- * (This shouldn't happen, since arbiter index selection should not
- * pick up an invalid index.)
- */
- if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
- elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
/*
--
2.43.0
v7-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchapplication/octet-stream; name=v7-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchDownload
From 02cf315bb0bdb2784a311ea28c52f13d52e120c2 Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Fri, 17 Jan 2025 21:15:10 +0100
Subject: [PATCH v7 2/4] Modify the infer_arbiter_indexes function to consider
both indisvalid and indisready indexes. Ensure that at least one indisvalid
index is still required.
The change ensures that all concurrent transactions utilize the same set of indexes as arbiters. This uniformity is required to avoid conditions that could lead to "duplicate key value violates unique constraint" errors during UPSERT operations.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert
* index_concurrently_upsert
* index_concurrently_upsert_predicate
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
* reindex_concurrently_upsert_on_constraint
---
src/backend/optimizer/util/plancat.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index f2d319101d3..a4c937da3d1 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -720,6 +720,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Results */
List *results = NIL;
+ bool foundValid = false;
/*
* Quickly return NIL for ON CONFLICT DO NOTHING without an inference
@@ -813,7 +814,13 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
- if (!idxForm->indisvalid)
+ /*
+ * We need to consider both indisvalid and indisready indexes because
+ * them may become indisvalid before execution phase. It is required
+ * to keep set of indexes used as arbiter to be the same for all
+ * concurrent transactions.
+ */
+ if (!idxForm->indisready)
goto next;
/*
@@ -835,10 +842,9 @@ infer_arbiter_indexes(PlannerInfo *root)
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
results = lappend_oid(results, idxForm->indexrelid);
- list_free(indexList);
+ foundValid |= idxForm->indisvalid;
index_close(idxRel, NoLock);
- table_close(relation, NoLock);
- return results;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
@@ -939,6 +945,7 @@ infer_arbiter_indexes(PlannerInfo *root)
goto next;
results = lappend_oid(results, idxForm->indexrelid);
+ foundValid |= idxForm->indisvalid;
next:
index_close(idxRel, NoLock);
}
@@ -946,7 +953,8 @@ next:
list_free(indexList);
table_close(relation, NoLock);
- if (results == NIL)
+ /* It is required to have at least one indisvalid index during the planning. */
+ if (results == NIL || !foundValid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
errmsg("there is no unique or exclusion constraint matching the ON CONFLICT specification")));
--
2.43.0
v7-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchapplication/octet-stream; name=v7-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchDownload
From 063d1e1ae4b9c77c2b8b580838e0ece0d2df7d1d Mon Sep 17 00:00:00 2001
From: nkey <michail.nikolaev@gmail.com>
Date: Fri, 17 Jan 2025 21:15:49 +0100
Subject: [PATCH v7 3/4] Modify the infer_arbiter_indexes function to also look
for indexes that match the specified named constraint to be used as
arbiters. This ensures that the same set of arbiter indexes is used for all
concurrent transactions in cases where REINDEX CONCURRENTLY processes an
index used as a named constraint.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_on_constraint
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
---
src/backend/optimizer/util/plancat.c | 121 +++++++++++++++++++--------
1 file changed, 88 insertions(+), 33 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index a4c937da3d1..71b61b69c1f 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -714,9 +714,10 @@ infer_arbiter_indexes(PlannerInfo *root)
List *indexList;
ListCell *l;
- /* Normalized inference attributes and inference expressions: */
- Bitmapset *inferAttrs = NULL;
- List *inferElems = NIL;
+ /* Normalized required attributes and expressions: */
+ Bitmapset *requiredArbiterAttrs = NULL;
+ List *requiredArbiterElems = NIL;
+ List *requiredIndexPredExprs = (List *) onconflict->arbiterWhere;
/* Results */
List *results = NIL;
@@ -755,8 +756,8 @@ infer_arbiter_indexes(PlannerInfo *root)
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
- inferElems = lappend(inferElems, elem->expr);
+ /* If not a plain Var, just shove it in requiredArbiterElems for now */
+ requiredArbiterElems = lappend(requiredArbiterElems, elem->expr);
continue;
}
@@ -768,30 +769,76 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("whole row unique index inference specifications are not supported")));
- inferAttrs = bms_add_member(inferAttrs,
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
+ indexList = RelationGetIndexList(relation);
+
/*
* Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * because some additional sanity checks are required. Additionally, we
+ * need to process other indexes as potential arbiters to account for
+ * cases where REINDEX CONCURRENTLY is processing an index used as a
+ * named constraint.
*/
if (onconflict->constraint != InvalidOid)
{
indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
+ {
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
- errmsg("constraint in ON CONFLICT clause has no associated index")));
+ errmsg("constraint in ON CONFLICT clause has no associated index")));
+ }
+
+ /*
+ * Find the named constraint index to extract its attributes and predicates.
+ * We open all indexes in the loop to avoid deadlock of changed order of locks.
+ * */
+ foreach(l, indexList)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation idxRel;
+ Form_pg_index idxForm;
+ AttrNumber natt;
+
+ idxRel = index_open(indexoid, rte->rellockmode);
+ idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Prepare requirements for other indexes to be used as arbiter together
+ * with indexOidFromConstraint. It is required to involve both equals indexes
+ * in case of REINDEX CONCURRENTLY.
+ */
+ for (natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno = idxRel->rd_index->indkey.values[natt];
+
+ if (attno != 0)
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+ requiredArbiterElems = RelationGetIndexExpressions(idxRel);
+ requiredIndexPredExprs = RelationGetIndexPredicate(idxRel);
+ /* We are done, so, quite the loop. */
+ index_close(idxRel, NoLock);
+ break;
+ }
+ }
+ index_close(idxRel, NoLock);
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
foreach(l, indexList)
{
Oid indexoid = lfirst_oid(l);
@@ -840,26 +887,23 @@ infer_arbiter_indexes(PlannerInfo *root)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
-
- results = lappend_oid(results, idxForm->indexrelid);
- foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ goto found;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /* In the case of "ON constraint_name DO UPDATE" we need to skip non-unique candidates. */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ goto next;
+ } else {
+ /*
+ * Only considering conventional inference at this point (not named
+ * constraints), so index under consideration can be immediately
+ * skipped if it's not unique
+ */
+ if (!idxForm->indisunique)
+ goto next;
}
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
-
/*
* So-called unique constraints with WITHOUT OVERLAPS are really
* exclusion constraints, so skip those too.
@@ -879,7 +923,7 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/* Non-expression attributes (if any) must match */
- if (!bms_equal(indexedAttrs, inferAttrs))
+ if (!bms_equal(indexedAttrs, requiredArbiterAttrs))
goto next;
/* Expression attributes (if any) must match */
@@ -887,6 +931,10 @@ infer_arbiter_indexes(PlannerInfo *root)
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
+ /*
+ * If arbiterElems are present, check them. If name >constraint is
+ * present arbiterElems == NIL.
+ */
foreach(el, onconflict->arbiterElems)
{
InferenceElem *elem = (InferenceElem *) lfirst(el);
@@ -924,26 +972,33 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of the conventional inference involved ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In the case of named constraint ensure candidate has equal set
+ * of expressions as the named constraint index.
*/
- if (list_difference(idxExprs, inferElems) != NIL)
+ if (list_difference(idxExprs, requiredArbiterElems) != NIL)
goto next;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
+ /*
+ * If it's a partial index and conventional inference, its predicate must be implied
+ * by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid && !predicate_implied_by(predExprs, requiredIndexPredExprs, false))
+ goto next;
+ /* If it's a partial index and named constraint predicates must be equal. */
+ if (indexOidFromConstraint != InvalidOid && list_difference(predExprs, requiredIndexPredExprs) != NIL)
goto next;
+found:
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
next:
--
2.43.0
Hello!
Just rebased.
Attachments:
v8-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchtext/x-patch; charset=US-ASCII; name=v8-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchDownload
From 2db0af6a7b5ba485464ad5a59ce106a6e438d41a Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:58 +0300
Subject: [PATCH v8 2/4] Modify the infer_arbiter_indexes function to consider
both indisvalid and indisready indexes. Ensure that at least one indisvalid
index is still required.
The change ensures that all concurrent transactions utilize the same set of indexes as arbiters. This uniformity is required to avoid conditions that could lead to "duplicate key value violates unique constraint" errors during UPSERT operations.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert
* index_concurrently_upsert
* index_concurrently_upsert_predicate
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
* reindex_concurrently_upsert_on_constraint
---
src/backend/optimizer/util/plancat.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 71abb01f655..b6b55b7cbff 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -720,6 +720,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Results */
List *results = NIL;
+ bool foundValid = false;
/*
* Quickly return NIL for ON CONFLICT DO NOTHING without an inference
@@ -813,7 +814,13 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
- if (!idxForm->indisvalid)
+ /*
+ * We need to consider both indisvalid and indisready indexes because
+ * them may become indisvalid before execution phase. It is required
+ * to keep set of indexes used as arbiter to be the same for all
+ * concurrent transactions.
+ */
+ if (!idxForm->indisready)
goto next;
/*
@@ -835,10 +842,9 @@ infer_arbiter_indexes(PlannerInfo *root)
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
results = lappend_oid(results, idxForm->indexrelid);
- list_free(indexList);
+ foundValid |= idxForm->indisvalid;
index_close(idxRel, NoLock);
- table_close(relation, NoLock);
- return results;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
@@ -939,6 +945,7 @@ infer_arbiter_indexes(PlannerInfo *root)
goto next;
results = lappend_oid(results, idxForm->indexrelid);
+ foundValid |= idxForm->indisvalid;
next:
index_close(idxRel, NoLock);
}
@@ -946,7 +953,8 @@ next:
list_free(indexList);
table_close(relation, NoLock);
- if (results == NIL)
+ /* It is required to have at least one indisvalid index during the planning. */
+ if (results == NIL || !foundValid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
errmsg("there is no unique or exclusion constraint matching the ON CONFLICT specification")));
--
2.43.0
v8-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchtext/x-patch; charset=US-ASCII; name=v8-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchDownload
From e7ad602f92185f373c3ab1af3baac9cee0416191 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:52:23 +0300
Subject: [PATCH v8 4/4] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY as
arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_partitioned
---
src/backend/executor/execPartition.c | 119 ++++++++++++++++++++++++---
1 file changed, 107 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index b6e89d0620d..50e58be657f 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -486,6 +486,48 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+ /* it is not supported for cases of exclusion constraints. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+ if (arbiterIndexRelation->rd_index->indnkeyatts != indexRelation->rd_index->indnkeyatts)
+ return false;
+
+ for (i = 0; i < indexRelation->rd_index->indnkeyatts; i++)
+ {
+ int arbiterAttoNo = arbiterIndexRelation->rd_index->indkey.values[i];
+ int attoNo = indexRelation->rd_index->indkey.values[i];
+ if (arbiterAttoNo != attoNo)
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -696,6 +738,8 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
{
List *childIdxs;
+ List *nonAncestorIdxs = NIL;
+ int i, j, additional_arbiters = 0;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
@@ -706,23 +750,74 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, lfirst_oid(lc2)))
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxs = lappend_oid(nonAncestorIdxs, childIdx);
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxs)
+ {
+ for (i = 0; i < leaf_part_rri->ri_NumIndices; i++)
+ {
+ if (list_member_oid(nonAncestorIdxs, leaf_part_rri->ri_IndexRelationDescs[i]->rd_index->indexrelid))
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[i];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[i];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to us non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ for (j = 0; j < leaf_part_rri->ri_NumIndices; j++)
+ {
+ if (list_member_oid(arbiterIndexes,
+ leaf_part_rri->ri_IndexRelationDescs[j]->rd_index->indexrelid))
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[j];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[j];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxs);
+
+ /*
+ * If the resulting lists are of inequal length, something is wrong.
+ * (This shouldn't happen, since arbiter index selection should not
+ * pick up a non-ready index.)
+ *
+ * But we need to consider an additional arbiter indexes also.
+ */
+ if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
+ list_length(arbiterIndexes) - additional_arbiters)
+ elog(ERROR, "invalid arbiter index list");
}
-
- /*
- * If the resulting lists are of inequal length, something is wrong.
- * (This shouldn't happen, since arbiter index selection should not
- * pick up an invalid index.)
- */
- if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
- elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
/*
--
2.43.0
v8-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX.patchtext/x-patch; charset=US-ASCII; name=v8-0001-Specs-top-reproduce-the-issues-with-CREATE-INDEX.patchDownload
From 0e0e528b92eb7d9cc68f7af05a6cc9b2a39297f2 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:10 +0300
Subject: [PATCH v8 1/4] Specs top reproduce the issues with CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY in scenarios involving INSERT ON
CONFLICT DO UPDATE. These tests reproduce different error cases related to
"duplicate key value violates unique constraint" where this error should not
occur by design.
* REINDEX CONCURRENTLY and UPSERT with inferred index
* CREATE INDEX CONCURRENTLY and UPSERT with inferred indexes
* REINDEX CONCURRENTLY on partitioned table
* REINDEX CONCURRENTLY with specified constraint name
* CREATE INDEX CONCURRENTLY with predicates
In each of these scenarios, the expected behavior is that the INSERT ON CONFLICT DO UPDATE should handle conflicts gracefully without raising a "duplicate key value violates unique constraint" error. However, due to the concurrent operations on the indexes, this error is encountered.
---
src/backend/commands/indexcmds.c | 4 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 2 +
src/test/modules/injection_points/Makefile | 7 +-
.../expected/index_concurrently_upsert.out | 84 ++++++
.../index_concurrently_upsert_predicate.out | 84 ++++++
.../expected/reindex_concurrently_upsert.out | 250 ++++++++++++++++++
...ndex_concurrently_upsert_on_constraint.out | 250 ++++++++++++++++++
...eindex_concurrently_upsert_partitioned.out | 250 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 8 +
.../specs/index_concurrently_upsert.spec | 72 +++++
.../index_concurrently_upsert_predicate.spec | 74 ++++++
.../specs/reindex_concurrently_upsert.spec | 90 +++++++
...dex_concurrently_upsert_on_constraint.spec | 91 +++++++
...index_concurrently_upsert_partitioned.spec | 92 +++++++
16 files changed, 1361 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index f8d3ea820e1..47c509ceb3e 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1796,6 +1796,7 @@ DefineIndex(Oid tableId,
* before the reference snap was taken, we have to wait out any
* transactions that might have older snapshots.
*/
+ INJECTION_POINT("define_index_before_set_valid");
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
@@ -4201,7 +4202,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap");
StartTransactionCommand();
/*
@@ -4280,6 +4281,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* index_drop() for more details.
*/
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead");
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 7c87f012c30..ae11c1dd463 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -117,6 +117,7 @@
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -936,6 +937,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict");
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index b0fe50075ad..d5ad73f6f69 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -69,6 +69,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1158,6 +1159,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative");
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 8f1508b1ee2..3d018c3a1e8 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -64,6 +64,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -388,6 +389,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("invalidate_catalog_snapshot_end");
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index e680991f8d4..5aa53f03049 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -14,7 +14,12 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points hashagg reindex_conc
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic inplace syscache-update-pruned
+ISOLATION = basic inplace syscache-update-pruned \
+ reindex_concurrently_upsert \
+ index_concurrently_upsert \
+ reindex_concurrently_upsert_partitioned \
+ reindex_concurrently_upsert_on_constraint \
+ index_concurrently_upsert_predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 00000000000..e7612e065f4
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
new file mode 100644
index 00000000000..0ef2f3a681c
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 00000000000..1bd8041289e
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,250 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
new file mode 100644
index 00000000000..68288812de2
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
@@ -0,0 +1,250 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
new file mode 100644
index 00000000000..f8c81e2bea2
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
@@ -0,0 +1,250 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index d61149712fd..e1443664c70 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -47,8 +47,14 @@ tests += {
'basic',
'inplace',
'syscache-update-pruned',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
+ 'reindex_concurrently_upsert_partitioned',
+ 'reindex_concurrently_upsert_on_constraint',
+ 'index_concurrently_upsert_predicate',
],
'runningcheck': false, # see syscache-update-pruned
+ 'runningcheck-parallel': false, # We waiting for all snapshots, so, avoid parallel test executions
},
'tap': {
'env': {
@@ -57,5 +63,7 @@ tests += {
'tests': [
't/001_stats.pl',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
},
}
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 00000000000..473b0408f55
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,72 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
new file mode 100644
index 00000000000..c8644a82d57
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
@@ -0,0 +1,74 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+
+ CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;}
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 00000000000..08d0deaf58b
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,90 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
new file mode 100644
index 00000000000..97622388e71
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
@@ -0,0 +1,91 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
new file mode 100644
index 00000000000..41f10c8736d
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
@@ -0,0 +1,92 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
v8-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchtext/x-patch; charset=US-ASCII; name=v8-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchDownload
From b813b41f58d4e5a97eba97d2bacfd34e280de271 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:51:19 +0300
Subject: [PATCH v8 3/4] Modify the infer_arbiter_indexes function to also look
for indexes that match the specified named constraint to be used as
arbiters. This ensures that the same set of arbiter indexes is used for all
concurrent transactions in cases where REINDEX CONCURRENTLY processes an
index used as a named constraint.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_on_constraint
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
---
src/backend/optimizer/util/plancat.c | 121 +++++++++++++++++++--------
1 file changed, 88 insertions(+), 33 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index b6b55b7cbff..af7586a428f 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -714,9 +714,10 @@ infer_arbiter_indexes(PlannerInfo *root)
List *indexList;
ListCell *l;
- /* Normalized inference attributes and inference expressions: */
- Bitmapset *inferAttrs = NULL;
- List *inferElems = NIL;
+ /* Normalized required attributes and expressions: */
+ Bitmapset *requiredArbiterAttrs = NULL;
+ List *requiredArbiterElems = NIL;
+ List *requiredIndexPredExprs = (List *) onconflict->arbiterWhere;
/* Results */
List *results = NIL;
@@ -755,8 +756,8 @@ infer_arbiter_indexes(PlannerInfo *root)
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
- inferElems = lappend(inferElems, elem->expr);
+ /* If not a plain Var, just shove it in requiredArbiterElems for now */
+ requiredArbiterElems = lappend(requiredArbiterElems, elem->expr);
continue;
}
@@ -768,30 +769,76 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("whole row unique index inference specifications are not supported")));
- inferAttrs = bms_add_member(inferAttrs,
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
+ indexList = RelationGetIndexList(relation);
+
/*
* Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * because some additional sanity checks are required. Additionally, we
+ * need to process other indexes as potential arbiters to account for
+ * cases where REINDEX CONCURRENTLY is processing an index used as a
+ * named constraint.
*/
if (onconflict->constraint != InvalidOid)
{
indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
+ {
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
- errmsg("constraint in ON CONFLICT clause has no associated index")));
+ errmsg("constraint in ON CONFLICT clause has no associated index")));
+ }
+
+ /*
+ * Find the named constraint index to extract its attributes and predicates.
+ * We open all indexes in the loop to avoid deadlock of changed order of locks.
+ * */
+ foreach(l, indexList)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation idxRel;
+ Form_pg_index idxForm;
+ AttrNumber natt;
+
+ idxRel = index_open(indexoid, rte->rellockmode);
+ idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Prepare requirements for other indexes to be used as arbiter together
+ * with indexOidFromConstraint. It is required to involve both equals indexes
+ * in case of REINDEX CONCURRENTLY.
+ */
+ for (natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno = idxRel->rd_index->indkey.values[natt];
+
+ if (attno != 0)
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+ requiredArbiterElems = RelationGetIndexExpressions(idxRel);
+ requiredIndexPredExprs = RelationGetIndexPredicate(idxRel);
+ /* We are done, so, quite the loop. */
+ index_close(idxRel, NoLock);
+ break;
+ }
+ }
+ index_close(idxRel, NoLock);
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
foreach(l, indexList)
{
Oid indexoid = lfirst_oid(l);
@@ -840,26 +887,23 @@ infer_arbiter_indexes(PlannerInfo *root)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
-
- results = lappend_oid(results, idxForm->indexrelid);
- foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ goto found;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /* In the case of "ON constraint_name DO UPDATE" we need to skip non-unique candidates. */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ goto next;
+ } else {
+ /*
+ * Only considering conventional inference at this point (not named
+ * constraints), so index under consideration can be immediately
+ * skipped if it's not unique
+ */
+ if (!idxForm->indisunique)
+ goto next;
}
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
-
/*
* So-called unique constraints with WITHOUT OVERLAPS are really
* exclusion constraints, so skip those too.
@@ -879,7 +923,7 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/* Non-expression attributes (if any) must match */
- if (!bms_equal(indexedAttrs, inferAttrs))
+ if (!bms_equal(indexedAttrs, requiredArbiterAttrs))
goto next;
/* Expression attributes (if any) must match */
@@ -887,6 +931,10 @@ infer_arbiter_indexes(PlannerInfo *root)
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
+ /*
+ * If arbiterElems are present, check them. If name >constraint is
+ * present arbiterElems == NIL.
+ */
foreach(el, onconflict->arbiterElems)
{
InferenceElem *elem = (InferenceElem *) lfirst(el);
@@ -924,26 +972,33 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of the conventional inference involved ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In the case of named constraint ensure candidate has equal set
+ * of expressions as the named constraint index.
*/
- if (list_difference(idxExprs, inferElems) != NIL)
+ if (list_difference(idxExprs, requiredArbiterElems) != NIL)
goto next;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
+ /*
+ * If it's a partial index and conventional inference, its predicate must be implied
+ * by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid && !predicate_implied_by(predExprs, requiredIndexPredExprs, false))
+ goto next;
+ /* If it's a partial index and named constraint predicates must be equal. */
+ if (indexOidFromConstraint != InvalidOid && list_difference(predExprs, requiredIndexPredExprs) != NIL)
goto next;
+found:
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
next:
--
2.43.0
Hello!
Rebased version.
Attachments:
v9-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchapplication/octet-stream; name=v9-0004-Modify-the-ExecInitPartitionInfo-function-to-cons.patchDownload
From e9634bc128203f2f8f86249765ad9a945dfbb3de Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:52:23 +0300
Subject: [PATCH v9 4/4] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY as
arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_partitioned
---
src/backend/executor/execPartition.c | 119 ++++++++++++++++++++++++---
1 file changed, 107 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 3f8a4cb5244..f1757d02f1c 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -487,6 +487,48 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+ /* it is not supported for cases of exclusion constraints. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+ if (arbiterIndexRelation->rd_index->indnkeyatts != indexRelation->rd_index->indnkeyatts)
+ return false;
+
+ for (i = 0; i < indexRelation->rd_index->indnkeyatts; i++)
+ {
+ int arbiterAttoNo = arbiterIndexRelation->rd_index->indkey.values[i];
+ int attoNo = indexRelation->rd_index->indkey.values[i];
+ if (arbiterAttoNo != attoNo)
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -697,6 +739,8 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
{
List *childIdxs;
+ List *nonAncestorIdxs = NIL;
+ int i, j, additional_arbiters = 0;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
@@ -707,23 +751,74 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, lfirst_oid(lc2)))
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxs = lappend_oid(nonAncestorIdxs, childIdx);
list_free(ancestors);
}
- }
- /*
- * If the resulting lists are of inequal length, something is wrong.
- * (This shouldn't happen, since arbiter index selection should not
- * pick up an invalid index.)
- */
- if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
- elog(ERROR, "invalid arbiter index list");
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxs)
+ {
+ for (i = 0; i < leaf_part_rri->ri_NumIndices; i++)
+ {
+ if (list_member_oid(nonAncestorIdxs, leaf_part_rri->ri_IndexRelationDescs[i]->rd_index->indexrelid))
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[i];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[i];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to us non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ for (j = 0; j < leaf_part_rri->ri_NumIndices; j++)
+ {
+ if (list_member_oid(arbiterIndexes,
+ leaf_part_rri->ri_IndexRelationDescs[j]->rd_index->indexrelid))
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[j];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[j];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxs);
+
+ /*
+ * If the resulting lists are of inequal length, something is wrong.
+ * (This shouldn't happen, since arbiter index selection should not
+ * pick up a non-ready index.)
+ *
+ * But we need to consider an additional arbiter indexes also.
+ */
+ if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
+ list_length(arbiterIndexes) - additional_arbiters)
+ elog(ERROR, "invalid arbiter index list");
+ }
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
/*
--
2.43.0
v9-0001-Specs-to-reproduce-the-issues-with-CREATE-INDEX-C.patchapplication/octet-stream; name=v9-0001-Specs-to-reproduce-the-issues-with-CREATE-INDEX-C.patchDownload
From ff3a8b82137723c025772051b65561d1bd469a54 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:10 +0300
Subject: [PATCH v9 1/4] Specs to reproduce the issues with CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY in scenarios involving INSERT ON
CONFLICT DO UPDATE. These tests reproduce different error cases related to
"duplicate key value violates unique constraint" where this error should not
occur by design.
* REINDEX CONCURRENTLY and UPSERT with inferred index
* CREATE INDEX CONCURRENTLY and UPSERT with inferred indexes
* REINDEX CONCURRENTLY on a partitioned table
* REINDEX CONCURRENTLY with specified constraint name
* CREATE INDEX CONCURRENTLY with predicates
In each of these scenarios, the expected behavior is that the INSERT ON CONFLICT DO UPDATE should handle conflicts gracefully without raising a "duplicate key value violates unique constraint" error. However, due to the concurrent operations on the indexes, this error is encountered.
---
src/backend/commands/indexcmds.c | 4 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 2 +
src/test/modules/injection_points/Makefile | 7 +-
.../expected/index_concurrently_upsert.out | 84 ++++++
.../index_concurrently_upsert_predicate.out | 84 ++++++
.../expected/reindex_concurrently_upsert.out | 250 ++++++++++++++++++
...ndex_concurrently_upsert_on_constraint.out | 250 ++++++++++++++++++
...eindex_concurrently_upsert_partitioned.out | 250 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 8 +
.../specs/index_concurrently_upsert.spec | 72 +++++
.../index_concurrently_upsert_predicate.spec | 74 ++++++
.../specs/reindex_concurrently_upsert.spec | 90 +++++++
...dex_concurrently_upsert_on_constraint.spec | 91 +++++++
...index_concurrently_upsert_partitioned.spec | 92 +++++++
16 files changed, 1361 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index d962fe392cd..0f75debe7f1 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1790,6 +1790,7 @@ DefineIndex(Oid tableId,
* before the reference snap was taken, we have to wait out any
* transactions that might have older snapshots.
*/
+ INJECTION_POINT("define_index_before_set_valid", NULL);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
@@ -4195,7 +4196,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap", NULL);
StartTransactionCommand();
/*
@@ -4274,6 +4275,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* index_drop() for more details.
*/
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead", NULL);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index bdf862b2406..499cba145dd 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -117,6 +117,7 @@
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -942,6 +943,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict", NULL);
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 46d533b7288..566dbecb390 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -69,6 +69,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1178,6 +1179,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative", NULL);
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index ea35f30f494..ad440ff024c 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -123,6 +123,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -447,6 +448,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("invalidate_catalog_snapshot_end", NULL);
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index e680991f8d4..5aa53f03049 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -14,7 +14,12 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points hashagg reindex_conc
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic inplace syscache-update-pruned
+ISOLATION = basic inplace syscache-update-pruned \
+ reindex_concurrently_upsert \
+ index_concurrently_upsert \
+ reindex_concurrently_upsert_partitioned \
+ reindex_concurrently_upsert_on_constraint \
+ index_concurrently_upsert_predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 00000000000..e7612e065f4
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
new file mode 100644
index 00000000000..0ef2f3a681c
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 00000000000..1bd8041289e
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,250 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
new file mode 100644
index 00000000000..68288812de2
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
@@ -0,0 +1,250 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
new file mode 100644
index 00000000000..f8c81e2bea2
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
@@ -0,0 +1,250 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1 s4_wakeup_to_set_dead
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_start_reindex s4_wakeup_to_swap s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index d61149712fd..e1443664c70 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -47,8 +47,14 @@ tests += {
'basic',
'inplace',
'syscache-update-pruned',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
+ 'reindex_concurrently_upsert_partitioned',
+ 'reindex_concurrently_upsert_on_constraint',
+ 'index_concurrently_upsert_predicate',
],
'runningcheck': false, # see syscache-update-pruned
+ 'runningcheck-parallel': false, # We waiting for all snapshots, so, avoid parallel test executions
},
'tap': {
'env': {
@@ -57,5 +63,7 @@ tests += {
'tests': [
't/001_stats.pl',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
},
}
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 00000000000..473b0408f55
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,72 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
new file mode 100644
index 00000000000..c8644a82d57
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
@@ -0,0 +1,74 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+
+ CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;}
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 00000000000..08d0deaf58b
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,90 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
new file mode 100644
index 00000000000..97622388e71
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
@@ -0,0 +1,91 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
new file mode 100644
index 00000000000..41f10c8736d
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
@@ -0,0 +1,92 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+
+permutation
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s4_wakeup_to_swap
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
v9-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchapplication/octet-stream; name=v9-0002-Modify-the-infer_arbiter_indexes-function-to-cons.patchDownload
From ec818ec7d9e06239efe381df7ead72831b035eaf Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:58 +0300
Subject: [PATCH v9 2/4] Modify the infer_arbiter_indexes function to consider
both indisvalid and indisready indexes. Ensure that at least one indisvalid
index is still required.
The change ensures that all concurrent transactions utilize the same set of indexes as arbiters. This uniformity is required to avoid conditions that could lead to "duplicate key value violates unique constraint" errors during UPSERT operations.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert
* index_concurrently_upsert
* index_concurrently_upsert_predicate
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
* reindex_concurrently_upsert_on_constraint
---
src/backend/optimizer/util/plancat.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 59233b64730..63f2c90340a 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -722,6 +722,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Results */
List *results = NIL;
+ bool foundValid = false;
/*
* Quickly return NIL for ON CONFLICT DO NOTHING without an inference
@@ -815,7 +816,13 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
- if (!idxForm->indisvalid)
+ /*
+ * We need to consider both indisvalid and indisready indexes because
+ * them may become indisvalid before execution phase. It is required
+ * to keep set of indexes used as arbiter to be the same for all
+ * concurrent transactions.
+ */
+ if (!idxForm->indisready)
goto next;
/*
@@ -837,10 +844,9 @@ infer_arbiter_indexes(PlannerInfo *root)
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
results = lappend_oid(results, idxForm->indexrelid);
- list_free(indexList);
+ foundValid |= idxForm->indisvalid;
index_close(idxRel, NoLock);
- table_close(relation, NoLock);
- return results;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
@@ -941,6 +947,7 @@ infer_arbiter_indexes(PlannerInfo *root)
goto next;
results = lappend_oid(results, idxForm->indexrelid);
+ foundValid |= idxForm->indisvalid;
next:
index_close(idxRel, NoLock);
}
@@ -948,7 +955,8 @@ next:
list_free(indexList);
table_close(relation, NoLock);
- if (results == NIL)
+ /* It is required to have at least one indisvalid index during the planning. */
+ if (results == NIL || !foundValid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
errmsg("there is no unique or exclusion constraint matching the ON CONFLICT specification")));
--
2.43.0
v9-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchapplication/octet-stream; name=v9-0003-Modify-the-infer_arbiter_indexes-function-to-also.patchDownload
From 25b1a6a215434881effc5a3355397a446d2dcaf8 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:51:19 +0300
Subject: [PATCH v9 3/4] Modify the infer_arbiter_indexes function to also look
for indexes that match the specified named constraint to be used as arbiters.
This ensures that the same set of arbiter indexes is used for all concurrent
transactions in cases where REINDEX CONCURRENTLY processes an index used as a
named constraint.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_on_constraint
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
---
src/backend/optimizer/util/plancat.c | 121 +++++++++++++++++++--------
1 file changed, 88 insertions(+), 33 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 63f2c90340a..0c720e450e9 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -716,9 +716,10 @@ infer_arbiter_indexes(PlannerInfo *root)
List *indexList;
ListCell *l;
- /* Normalized inference attributes and inference expressions: */
- Bitmapset *inferAttrs = NULL;
- List *inferElems = NIL;
+ /* Normalized required attributes and expressions: */
+ Bitmapset *requiredArbiterAttrs = NULL;
+ List *requiredArbiterElems = NIL;
+ List *requiredIndexPredExprs = (List *) onconflict->arbiterWhere;
/* Results */
List *results = NIL;
@@ -757,8 +758,8 @@ infer_arbiter_indexes(PlannerInfo *root)
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
- inferElems = lappend(inferElems, elem->expr);
+ /* If not a plain Var, just shove it in requiredArbiterElems for now */
+ requiredArbiterElems = lappend(requiredArbiterElems, elem->expr);
continue;
}
@@ -770,30 +771,76 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("whole row unique index inference specifications are not supported")));
- inferAttrs = bms_add_member(inferAttrs,
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
+ indexList = RelationGetIndexList(relation);
+
/*
* Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * because some additional sanity checks are required. Additionally, we
+ * need to process other indexes as potential arbiters to account for
+ * cases where REINDEX CONCURRENTLY is processing an index used as a
+ * named constraint.
*/
if (onconflict->constraint != InvalidOid)
{
indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
+ {
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
- errmsg("constraint in ON CONFLICT clause has no associated index")));
+ errmsg("constraint in ON CONFLICT clause has no associated index")));
+ }
+
+ /*
+ * Find the named constraint index to extract its attributes and predicates.
+ * We open all indexes in the loop to avoid deadlock of changed order of locks.
+ * */
+ foreach(l, indexList)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation idxRel;
+ Form_pg_index idxForm;
+ AttrNumber natt;
+
+ idxRel = index_open(indexoid, rte->rellockmode);
+ idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Prepare requirements for other indexes to be used as arbiter together
+ * with indexOidFromConstraint. It is required to involve both equals indexes
+ * in case of REINDEX CONCURRENTLY.
+ */
+ for (natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno = idxRel->rd_index->indkey.values[natt];
+
+ if (attno != 0)
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+ requiredArbiterElems = RelationGetIndexExpressions(idxRel);
+ requiredIndexPredExprs = RelationGetIndexPredicate(idxRel);
+ /* We are done, so, quite the loop. */
+ index_close(idxRel, NoLock);
+ break;
+ }
+ }
+ index_close(idxRel, NoLock);
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
foreach(l, indexList)
{
Oid indexoid = lfirst_oid(l);
@@ -842,26 +889,23 @@ infer_arbiter_indexes(PlannerInfo *root)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
-
- results = lappend_oid(results, idxForm->indexrelid);
- foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ goto found;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /* In the case of "ON constraint_name DO UPDATE" we need to skip non-unique candidates. */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ goto next;
+ } else {
+ /*
+ * Only considering conventional inference at this point (not named
+ * constraints), so index under consideration can be immediately
+ * skipped if it's not unique
+ */
+ if (!idxForm->indisunique)
+ goto next;
}
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
-
/*
* So-called unique constraints with WITHOUT OVERLAPS are really
* exclusion constraints, so skip those too.
@@ -881,7 +925,7 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/* Non-expression attributes (if any) must match */
- if (!bms_equal(indexedAttrs, inferAttrs))
+ if (!bms_equal(indexedAttrs, requiredArbiterAttrs))
goto next;
/* Expression attributes (if any) must match */
@@ -889,6 +933,10 @@ infer_arbiter_indexes(PlannerInfo *root)
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
+ /*
+ * If arbiterElems are present, check them. If name >constraint is
+ * present arbiterElems == NIL.
+ */
foreach(el, onconflict->arbiterElems)
{
InferenceElem *elem = (InferenceElem *) lfirst(el);
@@ -926,26 +974,33 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of the conventional inference involved ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In the case of named constraint ensure candidate has equal set
+ * of expressions as the named constraint index.
*/
- if (list_difference(idxExprs, inferElems) != NIL)
+ if (list_difference(idxExprs, requiredArbiterElems) != NIL)
goto next;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
+ /*
+ * If it's a partial index and conventional inference, its predicate must be implied
+ * by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid && !predicate_implied_by(predExprs, requiredIndexPredExprs, false))
+ goto next;
+ /* If it's a partial index and named constraint predicates must be equal. */
+ if (indexOidFromConstraint != InvalidOid && list_difference(predExprs, requiredIndexPredExprs) != NIL)
goto next;
+found:
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
next:
--
2.43.0
Some tests of stabilization, discussed in [0]/messages/by-id/CADzfLwUc=jtSUEaQCtyt8zTeOJ-gHZ8=w_KJsVjDOYSLqaY9Lg@mail.gmail.com.
Also, an issue known for more then 1.5year... Should we at least document it?
[0]: /messages/by-id/CADzfLwUc=jtSUEaQCtyt8zTeOJ-gHZ8=w_KJsVjDOYSLqaY9Lg@mail.gmail.com
Attachments:
v10-0002-Modify-the-infer_arbiter_indexes-function-to-con.patchapplication/octet-stream; name=v10-0002-Modify-the-infer_arbiter_indexes-function-to-con.patchDownload
From 24833f7a8336812a1bb0f0d8262732582b71c01b Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:58 +0300
Subject: [PATCH v10 2/4] Modify the infer_arbiter_indexes function to consider
both indisvalid and indisready indexes. Ensure that at least one indisvalid
index is still required.
The change ensures that all concurrent transactions utilize the same set of indexes as arbiters. This uniformity is required to avoid conditions that could lead to "duplicate key value violates unique constraint" errors during UPSERT operations.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert
* index_concurrently_upsert
* index_concurrently_upsert_predicate
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
* reindex_concurrently_upsert_on_constraint
---
src/backend/optimizer/util/plancat.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index da5d901ec3c..d3f26396aef 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -809,6 +809,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Results */
List *results = NIL;
+ bool foundValid = false;
/*
* Quickly return NIL for ON CONFLICT DO NOTHING without an inference
@@ -902,7 +903,13 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
- if (!idxForm->indisvalid)
+ /*
+ * We need to consider both indisvalid and indisready indexes because
+ * them may become indisvalid before execution phase. It is required
+ * to keep set of indexes used as arbiter to be the same for all
+ * concurrent transactions.
+ */
+ if (!idxForm->indisready)
goto next;
/*
@@ -924,10 +931,9 @@ infer_arbiter_indexes(PlannerInfo *root)
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
results = lappend_oid(results, idxForm->indexrelid);
- list_free(indexList);
+ foundValid |= idxForm->indisvalid;
index_close(idxRel, NoLock);
- table_close(relation, NoLock);
- return results;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
@@ -1028,6 +1034,7 @@ infer_arbiter_indexes(PlannerInfo *root)
goto next;
results = lappend_oid(results, idxForm->indexrelid);
+ foundValid |= idxForm->indisvalid;
next:
index_close(idxRel, NoLock);
}
@@ -1035,7 +1042,8 @@ next:
list_free(indexList);
table_close(relation, NoLock);
- if (results == NIL)
+ /* It is required to have at least one indisvalid index during the planning. */
+ if (results == NIL || !foundValid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
errmsg("there is no unique or exclusion constraint matching the ON CONFLICT specification")));
--
2.43.0
v10-0004-Modify-the-ExecInitPartitionInfo-function-to-con.patchapplication/octet-stream; name=v10-0004-Modify-the-ExecInitPartitionInfo-function-to-con.patchDownload
From 8d836fbede940fb672a643966db0f6cf565ef0e0 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:52:23 +0300
Subject: [PATCH v10 4/4] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY as
arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_partitioned
---
src/backend/executor/execPartition.c | 119 ++++++++++++++++++++++++---
1 file changed, 107 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 1f2da072632..f77fe42a2a9 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -490,6 +490,48 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+ /* it is not supported for cases of exclusion constraints. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+ if (arbiterIndexRelation->rd_index->indnkeyatts != indexRelation->rd_index->indnkeyatts)
+ return false;
+
+ for (i = 0; i < indexRelation->rd_index->indnkeyatts; i++)
+ {
+ int arbiterAttoNo = arbiterIndexRelation->rd_index->indkey.values[i];
+ int attoNo = indexRelation->rd_index->indkey.values[i];
+ if (arbiterAttoNo != attoNo)
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -701,6 +743,8 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
{
List *childIdxs;
+ List *nonAncestorIdxs = NIL;
+ int i, j, additional_arbiters = 0;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
@@ -711,23 +755,74 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, lfirst_oid(lc2)))
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxs = lappend_oid(nonAncestorIdxs, childIdx);
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxs)
+ {
+ for (i = 0; i < leaf_part_rri->ri_NumIndices; i++)
+ {
+ if (list_member_oid(nonAncestorIdxs, leaf_part_rri->ri_IndexRelationDescs[i]->rd_index->indexrelid))
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[i];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[i];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to us non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ for (j = 0; j < leaf_part_rri->ri_NumIndices; j++)
+ {
+ if (list_member_oid(arbiterIndexes,
+ leaf_part_rri->ri_IndexRelationDescs[j]->rd_index->indexrelid))
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[j];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[j];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxs);
+
+ /*
+ * If the resulting lists are of inequal length, something is wrong.
+ * (This shouldn't happen, since arbiter index selection should not
+ * pick up a non-ready index.)
+ *
+ * But we need to consider an additional arbiter indexes also.
+ */
+ if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
+ list_length(arbiterIndexes) - additional_arbiters)
+ elog(ERROR, "invalid arbiter index list");
}
-
- /*
- * If the resulting lists are of inequal length, something is wrong.
- * (This shouldn't happen, since arbiter index selection should not
- * pick up an invalid index.)
- */
- if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
- elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
/*
--
2.43.0
v10-0001-Specs-to-reproduce-the-issues-with-CREATE-INDEX-.patchapplication/octet-stream; name=v10-0001-Specs-to-reproduce-the-issues-with-CREATE-INDEX-.patchDownload
From d7a570ffa1426632f10f0e4126406541622002d5 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:10 +0300
Subject: [PATCH v10 1/4] Specs to reproduce the issues with CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY in scenarios involving INSERT ON
CONFLICT DO UPDATE. These tests reproduce different error cases related to
"duplicate key value violates unique constraint" where this error should not
occur by design.
* REINDEX CONCURRENTLY and UPSERT with inferred index
* CREATE INDEX CONCURRENTLY and UPSERT with inferred indexes
* REINDEX CONCURRENTLY on a partitioned table
* REINDEX CONCURRENTLY with specified constraint name
* CREATE INDEX CONCURRENTLY with predicates
In each of these scenarios, the expected behavior is that the INSERT ON CONFLICT DO UPDATE should handle conflicts gracefully without raising a "duplicate key value violates unique constraint" error. However, due to the concurrent operations on the indexes, this error is encountered.
---
src/backend/commands/indexcmds.c | 4 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 2 +
src/test/modules/injection_points/Makefile | 7 +-
.../expected/index_concurrently_upsert.out | 84 +++++++
.../index_concurrently_upsert_predicate.out | 84 +++++++
.../expected/reindex_concurrently_upsert.out | 232 ++++++++++++++++++
...ndex_concurrently_upsert_on_constraint.out | 232 ++++++++++++++++++
...eindex_concurrently_upsert_partitioned.out | 232 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 8 +
.../specs/index_concurrently_upsert.spec | 72 ++++++
.../index_concurrently_upsert_predicate.spec | 74 ++++++
.../specs/reindex_concurrently_upsert.spec | 94 +++++++
...dex_concurrently_upsert_on_constraint.spec | 95 +++++++
...index_concurrently_upsert_partitioned.spec | 96 ++++++++
16 files changed, 1319 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 5712fac3697..974243c5c60 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1789,6 +1789,7 @@ DefineIndex(Oid tableId,
* before the reference snap was taken, we have to wait out any
* transactions that might have older snapshots.
*/
+ INJECTION_POINT("define_index_before_set_valid", NULL);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
@@ -4228,7 +4229,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap", NULL);
StartTransactionCommand();
/*
@@ -4307,6 +4308,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* index_drop() for more details.
*/
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead", NULL);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index ca33a854278..0edf54e852d 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -117,6 +117,7 @@
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -942,6 +943,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict", NULL);
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4c5647ac38a..f6d2a6ede93 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -70,6 +70,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1179,6 +1180,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative", NULL);
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 65561cc6bc3..8e1a918f130 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -123,6 +123,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -458,6 +459,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("invalidate_catalog_snapshot_end", NULL);
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index fc82cd67f6c..6a03024b5ce 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -14,7 +14,12 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points hashagg reindex_conc vacuum
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic inplace syscache-update-pruned
+ISOLATION = basic inplace syscache-update-pruned \
+ reindex_concurrently_upsert \
+ index_concurrently_upsert \
+ reindex_concurrently_upsert_partitioned \
+ reindex_concurrently_upsert_on_constraint \
+ index_concurrently_upsert_predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 00000000000..e7612e065f4
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
new file mode 100644
index 00000000000..0ef2f3a681c
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 00000000000..3a2292696f1
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
new file mode 100644
index 00000000000..fa898f2e21b
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
new file mode 100644
index 00000000000..d6340d60738
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 20390d6b4bf..c1d0b93a58a 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -48,8 +48,14 @@ tests += {
'basic',
'inplace',
'syscache-update-pruned',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
+ 'reindex_concurrently_upsert_partitioned',
+ 'reindex_concurrently_upsert_on_constraint',
+ 'index_concurrently_upsert_predicate',
],
'runningcheck': false, # see syscache-update-pruned
+ 'runningcheck-parallel': false, # We waiting for all snapshots, so, avoid parallel test executions
},
'tap': {
'env': {
@@ -58,5 +64,7 @@ tests += {
'tests': [
't/001_stats.pl',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
},
}
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 00000000000..473b0408f55
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,72 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
new file mode 100644
index 00000000000..c8644a82d57
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
@@ -0,0 +1,74 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+
+ CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;}
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 00000000000..a4d1c1ea232
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,94 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
new file mode 100644
index 00000000000..82ee940cfa5
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
@@ -0,0 +1,95 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
new file mode 100644
index 00000000000..322848c808b
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
@@ -0,0 +1,96 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
v10-0003-Modify-the-infer_arbiter_indexes-function-to-als.patchapplication/octet-stream; name=v10-0003-Modify-the-infer_arbiter_indexes-function-to-als.patchDownload
From 3ef8d27719350e92203f11afc1d59780edd447ce Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:51:19 +0300
Subject: [PATCH v10 3/4] Modify the infer_arbiter_indexes function to also
look for indexes that match the specified named constraint to be used as
arbiters. This ensures that the same set of arbiter indexes is used for all
concurrent transactions in cases where REINDEX CONCURRENTLY processes an
index used as a named constraint.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_on_constraint
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
---
src/backend/optimizer/util/plancat.c | 121 +++++++++++++++++++--------
1 file changed, 88 insertions(+), 33 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d3f26396aef..d0c4386f798 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -803,9 +803,10 @@ infer_arbiter_indexes(PlannerInfo *root)
List *indexList;
ListCell *l;
- /* Normalized inference attributes and inference expressions: */
- Bitmapset *inferAttrs = NULL;
- List *inferElems = NIL;
+ /* Normalized required attributes and expressions: */
+ Bitmapset *requiredArbiterAttrs = NULL;
+ List *requiredArbiterElems = NIL;
+ List *requiredIndexPredExprs = (List *) onconflict->arbiterWhere;
/* Results */
List *results = NIL;
@@ -844,8 +845,8 @@ infer_arbiter_indexes(PlannerInfo *root)
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
- inferElems = lappend(inferElems, elem->expr);
+ /* If not a plain Var, just shove it in requiredArbiterElems for now */
+ requiredArbiterElems = lappend(requiredArbiterElems, elem->expr);
continue;
}
@@ -857,30 +858,76 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("whole row unique index inference specifications are not supported")));
- inferAttrs = bms_add_member(inferAttrs,
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
+ indexList = RelationGetIndexList(relation);
+
/*
* Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * because some additional sanity checks are required. Additionally, we
+ * need to process other indexes as potential arbiters to account for
+ * cases where REINDEX CONCURRENTLY is processing an index used as a
+ * named constraint.
*/
if (onconflict->constraint != InvalidOid)
{
indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
+ {
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
- errmsg("constraint in ON CONFLICT clause has no associated index")));
+ errmsg("constraint in ON CONFLICT clause has no associated index")));
+ }
+
+ /*
+ * Find the named constraint index to extract its attributes and predicates.
+ * We open all indexes in the loop to avoid deadlock of changed order of locks.
+ * */
+ foreach(l, indexList)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation idxRel;
+ Form_pg_index idxForm;
+ AttrNumber natt;
+
+ idxRel = index_open(indexoid, rte->rellockmode);
+ idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Prepare requirements for other indexes to be used as arbiter together
+ * with indexOidFromConstraint. It is required to involve both equals indexes
+ * in case of REINDEX CONCURRENTLY.
+ */
+ for (natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno = idxRel->rd_index->indkey.values[natt];
+
+ if (attno != 0)
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+ requiredArbiterElems = RelationGetIndexExpressions(idxRel);
+ requiredIndexPredExprs = RelationGetIndexPredicate(idxRel);
+ /* We are done, so, quite the loop. */
+ index_close(idxRel, NoLock);
+ break;
+ }
+ }
+ index_close(idxRel, NoLock);
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
foreach(l, indexList)
{
Oid indexoid = lfirst_oid(l);
@@ -929,26 +976,23 @@ infer_arbiter_indexes(PlannerInfo *root)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
-
- results = lappend_oid(results, idxForm->indexrelid);
- foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ goto found;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /* In the case of "ON constraint_name DO UPDATE" we need to skip non-unique candidates. */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ goto next;
+ } else {
+ /*
+ * Only considering conventional inference at this point (not named
+ * constraints), so index under consideration can be immediately
+ * skipped if it's not unique
+ */
+ if (!idxForm->indisunique)
+ goto next;
}
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
-
/*
* So-called unique constraints with WITHOUT OVERLAPS are really
* exclusion constraints, so skip those too.
@@ -968,7 +1012,7 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/* Non-expression attributes (if any) must match */
- if (!bms_equal(indexedAttrs, inferAttrs))
+ if (!bms_equal(indexedAttrs, requiredArbiterAttrs))
goto next;
/* Expression attributes (if any) must match */
@@ -976,6 +1020,10 @@ infer_arbiter_indexes(PlannerInfo *root)
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
+ /*
+ * If arbiterElems are present, check them. If name >constraint is
+ * present arbiterElems == NIL.
+ */
foreach(el, onconflict->arbiterElems)
{
InferenceElem *elem = (InferenceElem *) lfirst(el);
@@ -1013,26 +1061,33 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of the conventional inference involved ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In the case of named constraint ensure candidate has equal set
+ * of expressions as the named constraint index.
*/
- if (list_difference(idxExprs, inferElems) != NIL)
+ if (list_difference(idxExprs, requiredArbiterElems) != NIL)
goto next;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
+ /*
+ * If it's a partial index and conventional inference, its predicate must be implied
+ * by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid && !predicate_implied_by(predExprs, requiredIndexPredExprs, false))
+ goto next;
+ /* If it's a partial index and named constraint predicates must be equal. */
+ if (indexOidFromConstraint != InvalidOid && list_difference(predExprs, requiredIndexPredExprs) != NIL)
goto next;
+found:
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
next:
--
2.43.0
On Mon, Oct 20, 2025 at 09:27:00PM +0200, Mihail Nikalayeu wrote:
Some tests of stabilization, discussed in [0].
Also, an issue known for more then 1.5year... Should we at least document it?
Yes, I'm happy to push a patch documenting it. Would you like to propose the
specific doc patch? I regret lacking the bandwidth to review the fix patches.
Show quoted text
[0]: /messages/by-id/CADzfLwUc=jtSUEaQCtyt8zTeOJ-gHZ8=w_KJsVjDOYSLqaY9Lg@mail.gmail.com
Hello, Noah!
Thanks for the attention!
On Mon, Oct 27, 2025 at 7:06 PM Noah Misch <noah@leadboat.com> wrote:
Yes, I'm happy to push a patch documenting it. Would you like to propose the
specific doc patch? I regret lacking the bandwidth to review the fix patches.
Of course!
First version in attachment, waiting for your comment.
Best regards,
Mikhail.
Attachments:
nocfbot-v1-0001-doc-Document-potential-failure-scenarios-for-conc.patchapplication/octet-stream; name=nocfbot-v1-0001-doc-Document-potential-failure-scenarios-for-conc.patchDownload
From 0b2717f70e45899d43049f16cb74c8381ab9e2ec Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Tue, 28 Oct 2025 01:16:13 +0100
Subject: [PATCH v1] doc: Document potential failure scenarios for concurrent
`INSERT ... ON CONFLICT` during CONCURRENTLY index operation on the same
table.
Author: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CANtu0ojXmqjmEzp-=aJSxjsdE76iAsRgHBoK0QtYHimb_mEfsg@mail.gmail.com
---
doc/src/sgml/ref/create_index.sgml | 10 ++++++++++
doc/src/sgml/ref/insert.sgml | 17 +++++++++++++++++
doc/src/sgml/ref/reindex.sgml | 11 +++++++++++
3 files changed, 38 insertions(+)
diff --git a/doc/src/sgml/ref/create_index.sgml b/doc/src/sgml/ref/create_index.sgml
index b9c679c41e8..5ae7f1a1520 100644
--- a/doc/src/sgml/ref/create_index.sgml
+++ b/doc/src/sgml/ref/create_index.sgml
@@ -707,6 +707,16 @@ Indexes:
partitioned index is a metadata only operation.
</para>
+ <warning>
+ <para>
+ While <command>CREATE INDEX CONCURRENTLY</command> is running on a unique
+ index, concurrent <command>INSERT ... ON CONFLICT</command> statement on
+ the same table that uses an index compatible with the index being built
+ may unexpectedly fail with duplicate key constraint violation error.
+ See <xref linkend="sql-on-conflict"/> for more details.
+ </para>
+ </warning>
+
</refsect2>
</refsect1>
diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml
index 3f139917790..dc4b341ca3e 100644
--- a/doc/src/sgml/ref/insert.sgml
+++ b/doc/src/sgml/ref/insert.sgml
@@ -594,6 +594,23 @@ INSERT INTO <replaceable class="parameter">table_name</replaceable> [ AS <replac
</para>
</tip>
+ <warning>
+ <para>
+ If <command>CREATE INDEX CONCURRENTLY</command> or <command>REINDEX
+ CONCURRENTLY</command> is running on a unique index (or on a table
+ containing unique indexes), concurrent <command>INSERT ... ON CONFLICT</command>
+ statements on the same table may unexpectedly fail with duplicate key
+ constraint violation errors. This can occur when the statement uses an
+ index that is either compatible with the index being built or rebuilt,
+ or is itself being reindexed. Both <literal>DO UPDATE</literal> and
+ <literal>DO NOTHING</literal> conflict actions are affected.
+ </para>
+ <para>
+ These failures only occur during a brief window when the system catalogs
+ are being modified, not throughout the entire index build or rebuild process.
+ </para>
+ </warning>
+
</refsect2>
</refsect1>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c4055397146..9b65b81615e 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -513,6 +513,17 @@ Indexes:
in the <structname>pg_stat_progress_create_index</structname> view. See
<xref linkend="create-index-progress-reporting"/> for details.
</para>
+
+ <warning>
+ <para>
+ While <command>REINDEX CONCURRENTLY</command> is running on a unique
+ index or on a table containing unique indexes, concurrent
+ <command>INSERT ... ON CONFLICT</command> statement on the same table
+ that uses an index compatible with any of the indexes being rebuilt
+ may unexpectedly fail with duplicate key constraint violation errors.
+ See <xref linkend="sql-on-conflict"/> for more details.
+ </para>
+ </warning>
</refsect2>
</refsect1>
--
2.43.0
On Tue, Oct 28, 2025 at 02:19:00AM +0100, Mihail Nikalayeu wrote:
On Mon, Oct 27, 2025 at 7:06 PM Noah Misch <noah@leadboat.com> wrote:
Yes, I'm happy to push a patch documenting it. Would you like to propose the
specific doc patch? I regret lacking the bandwidth to review the fix patches.Of course!
First version in attachment, waiting for your comment.
Thanks. Does "ON CONFLICT ON CONSTRAINT constraint_name" avoid the problem w/
concurrent REINDEX CONCURRENTLY? A search of the thread found no mention of
"ON CONSTRAINT". It seems safe to assume that clause would avoid problems w/
CREATE INDEX CONCURRENTLY, but that's less certain for REINDEX.
The attached version has these changes:
- Mention ON CONSTRAINT as a workaround. Will remove if you find or suspect
it's not effective.
- Limit the doc change to ON CONFLICT. I think mentioning it at the INDEX
commands is undue emphasis.
- Use term "unique violation", a term used earlier on the same page, instead
of "duplicate key ...".
- Remove the internals-focused point about the brief window.
- Remove some detail I considered insufficiently surprising, e.g. the point
about "compatible with the index being built".
Attachments:
doc-concurrently-on-conflict-v2nm.patchtext/plain; charset=us-asciiDownload
From: Noah Misch <noah@leadboat.com>
Doc: cover index CONCURRENTLY causing errors in INSERT ... ON CONFLICT.
Author: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CANtu0ojXmqjmEzp-=aJSxjsdE76iAsRgHBoK0QtYHimb_mEfsg@mail.gmail.com
Backpatch-through: 13
diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml
index 3f13991..f88ea78 100644
--- a/doc/src/sgml/ref/insert.sgml
+++ b/doc/src/sgml/ref/insert.sgml
@@ -594,6 +594,17 @@ INSERT INTO <replaceable class="parameter">table_name</replaceable> [ AS <replac
</para>
</tip>
+ <warning>
+ <para>
+ While <command>CREATE INDEX CONCURRENTLY</command> or <command>REINDEX
+ CONCURRENTLY</command> is running on a unique index, <command>INSERT
+ ... ON CONFLICT</command> statements on the same table may unexpectedly
+ fail with a unique violation. Using <literal>ON CONFLICT ON
+ CONSTRAINT</literal> <replaceable class="parameter">
+ constraint_name</replaceable> avoids this, by disabling inference.
+ </para>
+ </warning>
+
</refsect2>
</refsect1>
Hello!
On Mon, Nov 3, 2025 at 12:21 AM Noah Misch <noah@leadboat.com> wrote:
Thanks. Does "ON CONFLICT ON CONSTRAINT constraint_name" avoid the problem w/
concurrent REINDEX CONCURRENTLY? A search of the thread found no mention of
"ON CONSTRAINT". It seems safe to assume that clause would avoid problems w/
CREATE INDEX CONCURRENTLY, but that's less certain for REINDEX.
It is also affected. There is a special
reindex_concurrently_upsert_on_constraint spec in the patch.
And even a special commit (0004) to fix it :)
But yes, it happens only in the case of REINDEX.
I removed the mention of "ON CONSTRAINT" and added a small comment
near infer_arbiter_indexes.
Doc patch is 0001, other - specs and fixes for future.
Best regards,
Mikhail.
Attachments:
v11-0002-Specs-to-reproduce-the-issues-with-CREATE-INDEX-.patchapplication/octet-stream; name=v11-0002-Specs-to-reproduce-the-issues-with-CREATE-INDEX-.patchDownload
From 1e4056a0766ad350b2c232dac6e85a6bb0a3456f Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:10 +0300
Subject: [PATCH v11 2/6] Specs to reproduce the issues with CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY in scenarios involving INSERT ON
CONFLICT DO UPDATE. These tests reproduce different error cases related to
"duplicate key value violates unique constraint" where this error should not
occur by design.
* REINDEX CONCURRENTLY and UPSERT with inferred index
* CREATE INDEX CONCURRENTLY and UPSERT with inferred indexes
* REINDEX CONCURRENTLY on a partitioned table
* REINDEX CONCURRENTLY with specified constraint name
* CREATE INDEX CONCURRENTLY with predicates
In each of these scenarios, the expected behavior is that the INSERT ON CONFLICT DO UPDATE should handle conflicts gracefully without raising a "duplicate key value violates unique constraint" error. However, due to the concurrent operations on the indexes, this error is encountered.
---
src/backend/commands/indexcmds.c | 4 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 2 +
src/test/modules/injection_points/Makefile | 7 +-
.../expected/index_concurrently_upsert.out | 84 +++++++
.../index_concurrently_upsert_predicate.out | 84 +++++++
.../expected/reindex_concurrently_upsert.out | 232 ++++++++++++++++++
...ndex_concurrently_upsert_on_constraint.out | 232 ++++++++++++++++++
...eindex_concurrently_upsert_partitioned.out | 232 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 8 +
.../specs/index_concurrently_upsert.spec | 72 ++++++
.../index_concurrently_upsert_predicate.spec | 74 ++++++
.../specs/reindex_concurrently_upsert.spec | 94 +++++++
...dex_concurrently_upsert_on_constraint.spec | 95 +++++++
...index_concurrently_upsert_partitioned.spec | 96 ++++++++
16 files changed, 1319 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 5712fac3697..974243c5c60 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1789,6 +1789,7 @@ DefineIndex(Oid tableId,
* before the reference snap was taken, we have to wait out any
* transactions that might have older snapshots.
*/
+ INJECTION_POINT("define_index_before_set_valid", NULL);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
@@ -4228,7 +4229,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap", NULL);
StartTransactionCommand();
/*
@@ -4307,6 +4308,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* index_drop() for more details.
*/
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead", NULL);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index ca33a854278..0edf54e852d 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -117,6 +117,7 @@
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -942,6 +943,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict", NULL);
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4c5647ac38a..f6d2a6ede93 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -70,6 +70,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1179,6 +1180,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative", NULL);
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 65561cc6bc3..8e1a918f130 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -123,6 +123,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -458,6 +459,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("invalidate_catalog_snapshot_end", NULL);
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index fc82cd67f6c..6a03024b5ce 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -14,7 +14,12 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points hashagg reindex_conc vacuum
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic inplace syscache-update-pruned
+ISOLATION = basic inplace syscache-update-pruned \
+ reindex_concurrently_upsert \
+ index_concurrently_upsert \
+ reindex_concurrently_upsert_partitioned \
+ reindex_concurrently_upsert_on_constraint \
+ index_concurrently_upsert_predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 00000000000..e7612e065f4
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
new file mode 100644
index 00000000000..0ef2f3a681c
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 00000000000..3a2292696f1
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
new file mode 100644
index 00000000000..fa898f2e21b
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
new file mode 100644
index 00000000000..d6340d60738
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 20390d6b4bf..c1d0b93a58a 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -48,8 +48,14 @@ tests += {
'basic',
'inplace',
'syscache-update-pruned',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
+ 'reindex_concurrently_upsert_partitioned',
+ 'reindex_concurrently_upsert_on_constraint',
+ 'index_concurrently_upsert_predicate',
],
'runningcheck': false, # see syscache-update-pruned
+ 'runningcheck-parallel': false, # We waiting for all snapshots, so, avoid parallel test executions
},
'tap': {
'env': {
@@ -58,5 +64,7 @@ tests += {
'tests': [
't/001_stats.pl',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
},
}
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 00000000000..473b0408f55
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,72 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
new file mode 100644
index 00000000000..c8644a82d57
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
@@ -0,0 +1,74 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+
+ CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;}
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 00000000000..a4d1c1ea232
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,94 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
new file mode 100644
index 00000000000..82ee940cfa5
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
@@ -0,0 +1,95 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
new file mode 100644
index 00000000000..322848c808b
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
@@ -0,0 +1,96 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
v11-0005-Modify-the-ExecInitPartitionInfo-function-to-con.patchapplication/octet-stream; name=v11-0005-Modify-the-ExecInitPartitionInfo-function-to-con.patchDownload
From 32ff7943bd411fde36906b7b2a941ac4aafa853d Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:52:23 +0300
Subject: [PATCH v11 5/6] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY as
arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_partitioned
---
src/backend/executor/execPartition.c | 119 ++++++++++++++++++++++++---
1 file changed, 107 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 1f2da072632..f77fe42a2a9 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -490,6 +490,48 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+ /* it is not supported for cases of exclusion constraints. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+ if (arbiterIndexRelation->rd_index->indnkeyatts != indexRelation->rd_index->indnkeyatts)
+ return false;
+
+ for (i = 0; i < indexRelation->rd_index->indnkeyatts; i++)
+ {
+ int arbiterAttoNo = arbiterIndexRelation->rd_index->indkey.values[i];
+ int attoNo = indexRelation->rd_index->indkey.values[i];
+ if (arbiterAttoNo != attoNo)
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -701,6 +743,8 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
{
List *childIdxs;
+ List *nonAncestorIdxs = NIL;
+ int i, j, additional_arbiters = 0;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
@@ -711,23 +755,74 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, lfirst_oid(lc2)))
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxs = lappend_oid(nonAncestorIdxs, childIdx);
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxs)
+ {
+ for (i = 0; i < leaf_part_rri->ri_NumIndices; i++)
+ {
+ if (list_member_oid(nonAncestorIdxs, leaf_part_rri->ri_IndexRelationDescs[i]->rd_index->indexrelid))
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[i];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[i];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to us non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ for (j = 0; j < leaf_part_rri->ri_NumIndices; j++)
+ {
+ if (list_member_oid(arbiterIndexes,
+ leaf_part_rri->ri_IndexRelationDescs[j]->rd_index->indexrelid))
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[j];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[j];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxs);
+
+ /*
+ * If the resulting lists are of inequal length, something is wrong.
+ * (This shouldn't happen, since arbiter index selection should not
+ * pick up a non-ready index.)
+ *
+ * But we need to consider an additional arbiter indexes also.
+ */
+ if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
+ list_length(arbiterIndexes) - additional_arbiters)
+ elog(ERROR, "invalid arbiter index list");
}
-
- /*
- * If the resulting lists are of inequal length, something is wrong.
- * (This shouldn't happen, since arbiter index selection should not
- * pick up an invalid index.)
- */
- if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
- elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
/*
--
2.43.0
v11-0003-Modify-the-infer_arbiter_indexes-function-to-con.patchapplication/octet-stream; name=v11-0003-Modify-the-infer_arbiter_indexes-function-to-con.patchDownload
From 2b1320129a3687f3c2244e8b17da837a4bdf6ce1 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:58 +0300
Subject: [PATCH v11 3/6] Modify the infer_arbiter_indexes function to consider
both indisvalid and indisready indexes. Ensure that at least one indisvalid
index is still required.
The change ensures that all concurrent transactions utilize the same set of indexes as arbiters. This uniformity is required to avoid conditions that could lead to "duplicate key value violates unique constraint" errors during UPSERT operations.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert
* index_concurrently_upsert
* index_concurrently_upsert_predicate
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
* reindex_concurrently_upsert_on_constraint
---
src/backend/optimizer/util/plancat.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 4b62a9756c1..b21e19290ab 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -814,6 +814,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Results */
List *results = NIL;
+ bool foundValid = false;
/*
* Quickly return NIL for ON CONFLICT DO NOTHING without an inference
@@ -907,7 +908,13 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
- if (!idxForm->indisvalid)
+ /*
+ * We need to consider both indisvalid and indisready indexes because
+ * them may become indisvalid before execution phase. It is required
+ * to keep set of indexes used as arbiter to be the same for all
+ * concurrent transactions.
+ */
+ if (!idxForm->indisready)
goto next;
/*
@@ -929,10 +936,9 @@ infer_arbiter_indexes(PlannerInfo *root)
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
results = lappend_oid(results, idxForm->indexrelid);
- list_free(indexList);
+ foundValid |= idxForm->indisvalid;
index_close(idxRel, NoLock);
- table_close(relation, NoLock);
- return results;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
@@ -1033,6 +1039,7 @@ infer_arbiter_indexes(PlannerInfo *root)
goto next;
results = lappend_oid(results, idxForm->indexrelid);
+ foundValid |= idxForm->indisvalid;
next:
index_close(idxRel, NoLock);
}
@@ -1040,7 +1047,8 @@ next:
list_free(indexList);
table_close(relation, NoLock);
- if (results == NIL)
+ /* It is required to have at least one indisvalid index during the planning. */
+ if (results == NIL || !foundValid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
errmsg("there is no unique or exclusion constraint matching the ON CONFLICT specification")));
--
2.43.0
v11-0006-Revert-Doc-cover-index-CONCURRENTLY-causing-erro.patchapplication/octet-stream; name=v11-0006-Revert-Doc-cover-index-CONCURRENTLY-causing-erro.patchDownload
From b62b94c0c51c310f4f237d78ef5db61cff77a501 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Mon, 3 Nov 2025 20:34:33 +0100
Subject: [PATCH v11 6/6] Revert "Doc: cover index CONCURRENTLY causing errors
in INSERT ... ON CONFLICT."
Issue fixed, this reverts commit 606df58881bfcdf50eea93d34308591bd3997fb5 with warnings.
---
doc/src/sgml/ref/insert.sgml | 9 ---------
src/backend/optimizer/util/plancat.c | 5 -----
2 files changed, 14 deletions(-)
diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml
index b337f2ee555..3f139917790 100644
--- a/doc/src/sgml/ref/insert.sgml
+++ b/doc/src/sgml/ref/insert.sgml
@@ -594,15 +594,6 @@ INSERT INTO <replaceable class="parameter">table_name</replaceable> [ AS <replac
</para>
</tip>
- <warning>
- <para>
- While <command>CREATE INDEX CONCURRENTLY</command> or <command>REINDEX
- CONCURRENTLY</command> is running on a unique index, <command>INSERT
- ... ON CONFLICT</command> statements on the same table may unexpectedly
- fail with a unique violation.
- </para>
- </warning>
-
</refsect2>
</refsect1>
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d5f6ac8c16a..d0c4386f798 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -789,11 +789,6 @@ find_relation_notnullatts(PlannerInfo *root, Oid relid)
* the purposes of inference. If no opclass (or collation) is specified, then
* all matching indexes (that may or may not match the default in terms of
* each attribute opclass/collation) are used for inference.
- *
- * Note: during index CONCURRENTLY operations, different transactions
- * may reference different sets of arbiter indexes. This can lead to false
- * unique constraint violations that wouldn't occur during normal operations.
- * For more information, see insert.sgml.
*/
List *
infer_arbiter_indexes(PlannerInfo *root)
--
2.43.0
v11-0004-Modify-the-infer_arbiter_indexes-function-to-als.patchapplication/octet-stream; name=v11-0004-Modify-the-infer_arbiter_indexes-function-to-als.patchDownload
From 23e0efd495f0e2d3d719718d320bb551016dda65 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:51:19 +0300
Subject: [PATCH v11 4/6] Modify the infer_arbiter_indexes function to also
look for indexes that match the specified named constraint to be used as
arbiters. This ensures that the same set of arbiter indexes is used for all
concurrent transactions in cases where REINDEX CONCURRENTLY processes an
index used as a named constraint.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_on_constraint
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
---
src/backend/optimizer/util/plancat.c | 121 +++++++++++++++++++--------
1 file changed, 88 insertions(+), 33 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index b21e19290ab..d5f6ac8c16a 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -808,9 +808,10 @@ infer_arbiter_indexes(PlannerInfo *root)
List *indexList;
ListCell *l;
- /* Normalized inference attributes and inference expressions: */
- Bitmapset *inferAttrs = NULL;
- List *inferElems = NIL;
+ /* Normalized required attributes and expressions: */
+ Bitmapset *requiredArbiterAttrs = NULL;
+ List *requiredArbiterElems = NIL;
+ List *requiredIndexPredExprs = (List *) onconflict->arbiterWhere;
/* Results */
List *results = NIL;
@@ -849,8 +850,8 @@ infer_arbiter_indexes(PlannerInfo *root)
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
- inferElems = lappend(inferElems, elem->expr);
+ /* If not a plain Var, just shove it in requiredArbiterElems for now */
+ requiredArbiterElems = lappend(requiredArbiterElems, elem->expr);
continue;
}
@@ -862,30 +863,76 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("whole row unique index inference specifications are not supported")));
- inferAttrs = bms_add_member(inferAttrs,
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
+ indexList = RelationGetIndexList(relation);
+
/*
* Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * because some additional sanity checks are required. Additionally, we
+ * need to process other indexes as potential arbiters to account for
+ * cases where REINDEX CONCURRENTLY is processing an index used as a
+ * named constraint.
*/
if (onconflict->constraint != InvalidOid)
{
indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
+ {
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
- errmsg("constraint in ON CONFLICT clause has no associated index")));
+ errmsg("constraint in ON CONFLICT clause has no associated index")));
+ }
+
+ /*
+ * Find the named constraint index to extract its attributes and predicates.
+ * We open all indexes in the loop to avoid deadlock of changed order of locks.
+ * */
+ foreach(l, indexList)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation idxRel;
+ Form_pg_index idxForm;
+ AttrNumber natt;
+
+ idxRel = index_open(indexoid, rte->rellockmode);
+ idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Prepare requirements for other indexes to be used as arbiter together
+ * with indexOidFromConstraint. It is required to involve both equals indexes
+ * in case of REINDEX CONCURRENTLY.
+ */
+ for (natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno = idxRel->rd_index->indkey.values[natt];
+
+ if (attno != 0)
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+ requiredArbiterElems = RelationGetIndexExpressions(idxRel);
+ requiredIndexPredExprs = RelationGetIndexPredicate(idxRel);
+ /* We are done, so, quite the loop. */
+ index_close(idxRel, NoLock);
+ break;
+ }
+ }
+ index_close(idxRel, NoLock);
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
foreach(l, indexList)
{
Oid indexoid = lfirst_oid(l);
@@ -934,26 +981,23 @@ infer_arbiter_indexes(PlannerInfo *root)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
-
- results = lappend_oid(results, idxForm->indexrelid);
- foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ goto found;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /* In the case of "ON constraint_name DO UPDATE" we need to skip non-unique candidates. */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ goto next;
+ } else {
+ /*
+ * Only considering conventional inference at this point (not named
+ * constraints), so index under consideration can be immediately
+ * skipped if it's not unique
+ */
+ if (!idxForm->indisunique)
+ goto next;
}
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
-
/*
* So-called unique constraints with WITHOUT OVERLAPS are really
* exclusion constraints, so skip those too.
@@ -973,7 +1017,7 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/* Non-expression attributes (if any) must match */
- if (!bms_equal(indexedAttrs, inferAttrs))
+ if (!bms_equal(indexedAttrs, requiredArbiterAttrs))
goto next;
/* Expression attributes (if any) must match */
@@ -981,6 +1025,10 @@ infer_arbiter_indexes(PlannerInfo *root)
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
+ /*
+ * If arbiterElems are present, check them. If name >constraint is
+ * present arbiterElems == NIL.
+ */
foreach(el, onconflict->arbiterElems)
{
InferenceElem *elem = (InferenceElem *) lfirst(el);
@@ -1018,26 +1066,33 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of the conventional inference involved ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In the case of named constraint ensure candidate has equal set
+ * of expressions as the named constraint index.
*/
- if (list_difference(idxExprs, inferElems) != NIL)
+ if (list_difference(idxExprs, requiredArbiterElems) != NIL)
goto next;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
+ /*
+ * If it's a partial index and conventional inference, its predicate must be implied
+ * by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid && !predicate_implied_by(predExprs, requiredIndexPredExprs, false))
+ goto next;
+ /* If it's a partial index and named constraint predicates must be equal. */
+ if (indexOidFromConstraint != InvalidOid && list_difference(predExprs, requiredIndexPredExprs) != NIL)
goto next;
+found:
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
next:
--
2.43.0
v11-0001-Doc-cover-index-CONCURRENTLY-causing-errors-in-I.patchapplication/octet-stream; name=v11-0001-Doc-cover-index-CONCURRENTLY-causing-errors-in-I.patchDownload
From 606df58881bfcdf50eea93d34308591bd3997fb5 Mon Sep 17 00:00:00 2001
From: Noah Misch <noah@leadboat.com>
Date: Mon, 3 Nov 2025 20:19:10 +0100
Subject: [PATCH v11 1/6] Doc: cover index CONCURRENTLY causing errors in
INSERT ... ON CONFLICT.
Author: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/CANtu0ojXmqjmEzp-=aJSxjsdE76iAsRgHBoK0QtYHimb_mEfsg@mail.gmail.com
Backpatch-through: 13
---
doc/src/sgml/ref/insert.sgml | 9 +++++++++
src/backend/optimizer/util/plancat.c | 5 +++++
2 files changed, 14 insertions(+)
diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml
index 3f139917790..b337f2ee555 100644
--- a/doc/src/sgml/ref/insert.sgml
+++ b/doc/src/sgml/ref/insert.sgml
@@ -594,6 +594,15 @@ INSERT INTO <replaceable class="parameter">table_name</replaceable> [ AS <replac
</para>
</tip>
+ <warning>
+ <para>
+ While <command>CREATE INDEX CONCURRENTLY</command> or <command>REINDEX
+ CONCURRENTLY</command> is running on a unique index, <command>INSERT
+ ... ON CONFLICT</command> statements on the same table may unexpectedly
+ fail with a unique violation.
+ </para>
+ </warning>
+
</refsect2>
</refsect1>
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index da5d901ec3c..4b62a9756c1 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -789,6 +789,11 @@ find_relation_notnullatts(PlannerInfo *root, Oid relid)
* the purposes of inference. If no opclass (or collation) is specified, then
* all matching indexes (that may or may not match the default in terms of
* each attribute opclass/collation) are used for inference.
+ *
+ * Note: during index CONCURRENTLY operations, different transactions
+ * may reference different sets of arbiter indexes. This can lead to false
+ * unique constraint violations that wouldn't occur during normal operations.
+ * For more information, see insert.sgml.
*/
List *
infer_arbiter_indexes(PlannerInfo *root)
--
2.43.0
On Mon, Nov 03, 2025 at 09:41:00PM +0100, Mihail Nikalayeu wrote:
On Mon, Nov 3, 2025 at 12:21 AM Noah Misch <noah@leadboat.com> wrote:
Thanks. Does "ON CONFLICT ON CONSTRAINT constraint_name" avoid the problem w/
concurrent REINDEX CONCURRENTLY? A search of the thread found no mention of
"ON CONSTRAINT". It seems safe to assume that clause would avoid problems w/
CREATE INDEX CONCURRENTLY, but that's less certain for REINDEX.It is also affected. There is a special
reindex_concurrently_upsert_on_constraint spec in the patch.
And even a special commit (0004) to fix it :)
My mistake. I've redone the mbox search that I thought I had done. That
search should have found it yesterday, so I must have made a typo then.
I removed the mention of "ON CONSTRAINT" and added a small comment
near infer_arbiter_indexes.Doc patch is 0001, other - specs and fixes for future.
I re-flowed the new comment to the standard 78 columns and pushed 0001.
Thanks.
Hello, Noah!
I re-flowed the new comment to the standard 78 columns and pushed 0001.
Thanks again!
Show quoted text
Hello!
Rebased, excluded committed part, updated revering part.
Best regards,
Mikhail.
Attachments:
v12-0002-Modify-the-infer_arbiter_indexes-function-to-con.patchapplication/x-patch; name=v12-0002-Modify-the-infer_arbiter_indexes-function-to-con.patchDownload
From 7ae8d431ec01c1c89a8de675dd05d036e37f714c Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:58 +0300
Subject: [PATCH v12 2/5] Modify the infer_arbiter_indexes function to consider
both indisvalid and indisready indexes. Ensure that at least one indisvalid
index is still required.
The change ensures that all concurrent transactions utilize the same set of indexes as arbiters. This uniformity is required to avoid conditions that could lead to "duplicate key value violates unique constraint" errors during UPSERT operations.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert
* index_concurrently_upsert
* index_concurrently_upsert_predicate
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
* reindex_concurrently_upsert_on_constraint
---
src/backend/optimizer/util/plancat.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d950bd93002..a8ae4401006 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -814,6 +814,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Results */
List *results = NIL;
+ bool foundValid = false;
/*
* Quickly return NIL for ON CONFLICT DO NOTHING without an inference
@@ -907,7 +908,13 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
- if (!idxForm->indisvalid)
+ /*
+ * We need to consider both indisvalid and indisready indexes because
+ * them may become indisvalid before execution phase. It is required
+ * to keep set of indexes used as arbiter to be the same for all
+ * concurrent transactions.
+ */
+ if (!idxForm->indisready)
goto next;
/*
@@ -929,10 +936,9 @@ infer_arbiter_indexes(PlannerInfo *root)
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
results = lappend_oid(results, idxForm->indexrelid);
- list_free(indexList);
+ foundValid |= idxForm->indisvalid;
index_close(idxRel, NoLock);
- table_close(relation, NoLock);
- return results;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
@@ -1033,6 +1039,7 @@ infer_arbiter_indexes(PlannerInfo *root)
goto next;
results = lappend_oid(results, idxForm->indexrelid);
+ foundValid |= idxForm->indisvalid;
next:
index_close(idxRel, NoLock);
}
@@ -1040,7 +1047,8 @@ next:
list_free(indexList);
table_close(relation, NoLock);
- if (results == NIL)
+ /* It is required to have at least one indisvalid index during the planning. */
+ if (results == NIL || !foundValid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
errmsg("there is no unique or exclusion constraint matching the ON CONFLICT specification")));
--
2.43.0
v12-0004-Modify-the-ExecInitPartitionInfo-function-to-con.patchapplication/x-patch; name=v12-0004-Modify-the-ExecInitPartitionInfo-function-to-con.patchDownload
From d2dd53972a6df00468248927b58d8d01d7e93673 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:52:23 +0300
Subject: [PATCH v12 4/5] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY as
arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_partitioned
---
src/backend/executor/execPartition.c | 119 ++++++++++++++++++++++++---
1 file changed, 107 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index aa12e9ad2ea..066686483f0 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -490,6 +490,48 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+ /* it is not supported for cases of exclusion constraints. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+ if (arbiterIndexRelation->rd_index->indnkeyatts != indexRelation->rd_index->indnkeyatts)
+ return false;
+
+ for (i = 0; i < indexRelation->rd_index->indnkeyatts; i++)
+ {
+ int arbiterAttoNo = arbiterIndexRelation->rd_index->indkey.values[i];
+ int attoNo = indexRelation->rd_index->indkey.values[i];
+ if (arbiterAttoNo != attoNo)
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -701,6 +743,8 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
{
List *childIdxs;
+ List *nonAncestorIdxs = NIL;
+ int i, j, additional_arbiters = 0;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
@@ -711,23 +755,74 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, lfirst_oid(lc2)))
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxs = lappend_oid(nonAncestorIdxs, childIdx);
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxs)
+ {
+ for (i = 0; i < leaf_part_rri->ri_NumIndices; i++)
+ {
+ if (list_member_oid(nonAncestorIdxs, leaf_part_rri->ri_IndexRelationDescs[i]->rd_index->indexrelid))
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[i];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[i];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to us non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ for (j = 0; j < leaf_part_rri->ri_NumIndices; j++)
+ {
+ if (list_member_oid(arbiterIndexes,
+ leaf_part_rri->ri_IndexRelationDescs[j]->rd_index->indexrelid))
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[j];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[j];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxs);
+
+ /*
+ * If the resulting lists are of inequal length, something is wrong.
+ * (This shouldn't happen, since arbiter index selection should not
+ * pick up a non-ready index.)
+ *
+ * But we need to consider an additional arbiter indexes also.
+ */
+ if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
+ list_length(arbiterIndexes) - additional_arbiters)
+ elog(ERROR, "invalid arbiter index list");
}
-
- /*
- * If the resulting lists are of inequal length, something is wrong.
- * (This shouldn't happen, since arbiter index selection should not
- * pick up an invalid index.)
- */
- if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
- elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
/*
--
2.43.0
v12-0005-Revert-Doc-cover-index-CONCURRENTLY-causing-erro.patchapplication/x-patch; name=v12-0005-Revert-Doc-cover-index-CONCURRENTLY-causing-erro.patchDownload
From 5bacecbc5bed4f6cd0eb6827c5c4e3c14692678e Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sun, 9 Nov 2025 11:48:34 +0100
Subject: [PATCH v12 5/5] Revert "Doc: cover index CONCURRENTLY causing errors
in INSERT ... ON CONFLICT."
This reverts commit 8b18ed6dfbb8b3e4483801b513fea6b429140569.
---
doc/src/sgml/ref/insert.sgml | 9 ---------
src/backend/optimizer/util/plancat.c | 5 -----
2 files changed, 14 deletions(-)
diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml
index 0598b8dea34..04962e39e12 100644
--- a/doc/src/sgml/ref/insert.sgml
+++ b/doc/src/sgml/ref/insert.sgml
@@ -594,15 +594,6 @@ INSERT INTO <replaceable class="parameter">table_name</replaceable> [ AS <replac
</para>
</tip>
- <warning>
- <para>
- While <command>CREATE INDEX CONCURRENTLY</command> or <command>REINDEX
- CONCURRENTLY</command> is running on a unique index, <command>INSERT
- ... ON CONFLICT</command> statements on the same table may unexpectedly
- fail with a unique violation.
- </para>
- </warning>
-
</refsect2>
</refsect1>
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index ff416f0522c..02c732f8817 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -789,11 +789,6 @@ find_relation_notnullatts(PlannerInfo *root, Oid relid)
* the purposes of inference. If no opclass (or collation) is specified, then
* all matching indexes (that may or may not match the default in terms of
* each attribute opclass/collation) are used for inference.
- *
- * Note: during index CONCURRENTLY operations, different transactions may
- * reference different sets of arbiter indexes. This can lead to false unique
- * constraint violations that wouldn't occur during normal operations. For
- * more information, see insert.sgml.
*/
List *
infer_arbiter_indexes(PlannerInfo *root)
--
2.43.0
v12-0003-Modify-the-infer_arbiter_indexes-function-to-als.patchapplication/x-patch; name=v12-0003-Modify-the-infer_arbiter_indexes-function-to-als.patchDownload
From e81de553df67a38ae3710169700c0cdb542de73b Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:51:19 +0300
Subject: [PATCH v12 3/5] Modify the infer_arbiter_indexes function to also
look for indexes that match the specified named constraint to be used as
arbiters. This ensures that the same set of arbiter indexes is used for all
concurrent transactions in cases where REINDEX CONCURRENTLY processes an
index used as a named constraint.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_on_constraint
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
---
src/backend/optimizer/util/plancat.c | 121 +++++++++++++++++++--------
1 file changed, 88 insertions(+), 33 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index a8ae4401006..ff416f0522c 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -808,9 +808,10 @@ infer_arbiter_indexes(PlannerInfo *root)
List *indexList;
ListCell *l;
- /* Normalized inference attributes and inference expressions: */
- Bitmapset *inferAttrs = NULL;
- List *inferElems = NIL;
+ /* Normalized required attributes and expressions: */
+ Bitmapset *requiredArbiterAttrs = NULL;
+ List *requiredArbiterElems = NIL;
+ List *requiredIndexPredExprs = (List *) onconflict->arbiterWhere;
/* Results */
List *results = NIL;
@@ -849,8 +850,8 @@ infer_arbiter_indexes(PlannerInfo *root)
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
- inferElems = lappend(inferElems, elem->expr);
+ /* If not a plain Var, just shove it in requiredArbiterElems for now */
+ requiredArbiterElems = lappend(requiredArbiterElems, elem->expr);
continue;
}
@@ -862,30 +863,76 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("whole row unique index inference specifications are not supported")));
- inferAttrs = bms_add_member(inferAttrs,
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
+ indexList = RelationGetIndexList(relation);
+
/*
* Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * because some additional sanity checks are required. Additionally, we
+ * need to process other indexes as potential arbiters to account for
+ * cases where REINDEX CONCURRENTLY is processing an index used as a
+ * named constraint.
*/
if (onconflict->constraint != InvalidOid)
{
indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
+ {
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
- errmsg("constraint in ON CONFLICT clause has no associated index")));
+ errmsg("constraint in ON CONFLICT clause has no associated index")));
+ }
+
+ /*
+ * Find the named constraint index to extract its attributes and predicates.
+ * We open all indexes in the loop to avoid deadlock of changed order of locks.
+ * */
+ foreach(l, indexList)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation idxRel;
+ Form_pg_index idxForm;
+ AttrNumber natt;
+
+ idxRel = index_open(indexoid, rte->rellockmode);
+ idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Prepare requirements for other indexes to be used as arbiter together
+ * with indexOidFromConstraint. It is required to involve both equals indexes
+ * in case of REINDEX CONCURRENTLY.
+ */
+ for (natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno = idxRel->rd_index->indkey.values[natt];
+
+ if (attno != 0)
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+ requiredArbiterElems = RelationGetIndexExpressions(idxRel);
+ requiredIndexPredExprs = RelationGetIndexPredicate(idxRel);
+ /* We are done, so, quite the loop. */
+ index_close(idxRel, NoLock);
+ break;
+ }
+ }
+ index_close(idxRel, NoLock);
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
foreach(l, indexList)
{
Oid indexoid = lfirst_oid(l);
@@ -934,26 +981,23 @@ infer_arbiter_indexes(PlannerInfo *root)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
-
- results = lappend_oid(results, idxForm->indexrelid);
- foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ goto found;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /* In the case of "ON constraint_name DO UPDATE" we need to skip non-unique candidates. */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ goto next;
+ } else {
+ /*
+ * Only considering conventional inference at this point (not named
+ * constraints), so index under consideration can be immediately
+ * skipped if it's not unique
+ */
+ if (!idxForm->indisunique)
+ goto next;
}
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
-
/*
* So-called unique constraints with WITHOUT OVERLAPS are really
* exclusion constraints, so skip those too.
@@ -973,7 +1017,7 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/* Non-expression attributes (if any) must match */
- if (!bms_equal(indexedAttrs, inferAttrs))
+ if (!bms_equal(indexedAttrs, requiredArbiterAttrs))
goto next;
/* Expression attributes (if any) must match */
@@ -981,6 +1025,10 @@ infer_arbiter_indexes(PlannerInfo *root)
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
+ /*
+ * If arbiterElems are present, check them. If name >constraint is
+ * present arbiterElems == NIL.
+ */
foreach(el, onconflict->arbiterElems)
{
InferenceElem *elem = (InferenceElem *) lfirst(el);
@@ -1018,26 +1066,33 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of the conventional inference involved ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In the case of named constraint ensure candidate has equal set
+ * of expressions as the named constraint index.
*/
- if (list_difference(idxExprs, inferElems) != NIL)
+ if (list_difference(idxExprs, requiredArbiterElems) != NIL)
goto next;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
+ /*
+ * If it's a partial index and conventional inference, its predicate must be implied
+ * by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid && !predicate_implied_by(predExprs, requiredIndexPredExprs, false))
+ goto next;
+ /* If it's a partial index and named constraint predicates must be equal. */
+ if (indexOidFromConstraint != InvalidOid && list_difference(predExprs, requiredIndexPredExprs) != NIL)
goto next;
+found:
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
next:
--
2.43.0
v12-0001-Specs-to-reproduce-the-issues-with-CREATE-INDEX-.patchapplication/x-patch; name=v12-0001-Specs-to-reproduce-the-issues-with-CREATE-INDEX-.patchDownload
From e9ca7ed8905d89350ee1af1459eb1f8f7fec3989 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:10 +0300
Subject: [PATCH v12 1/5] Specs to reproduce the issues with CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY in scenarios involving INSERT ON
CONFLICT DO UPDATE (8b18ed6dfbb8b3e4483801b513fea6b429140569).
These tests reproduce different error cases related to "duplicate key value violates unique constraint" where this error should not occur by design.
* REINDEX CONCURRENTLY and UPSERT with inferred index
* CREATE INDEX CONCURRENTLY and UPSERT with inferred indexes
* REINDEX CONCURRENTLY on a partitioned table
* REINDEX CONCURRENTLY with specified constraint name
* CREATE INDEX CONCURRENTLY with predicates
In each of these scenarios, the expected behavior is that the INSERT ON CONFLICT DO UPDATE should handle conflicts gracefully without raising a "duplicate key value violates unique constraint" error. However, due to the concurrent operations on the indexes, this error is encountered.
---
src/backend/commands/indexcmds.c | 4 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 2 +
src/test/modules/injection_points/Makefile | 7 +-
.../expected/index_concurrently_upsert.out | 84 +++++++
.../index_concurrently_upsert_predicate.out | 84 +++++++
.../expected/reindex_concurrently_upsert.out | 232 ++++++++++++++++++
...ndex_concurrently_upsert_on_constraint.out | 232 ++++++++++++++++++
...eindex_concurrently_upsert_partitioned.out | 232 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 8 +
.../specs/index_concurrently_upsert.spec | 72 ++++++
.../index_concurrently_upsert_predicate.spec | 74 ++++++
.../specs/reindex_concurrently_upsert.spec | 94 +++++++
...dex_concurrently_upsert_on_constraint.spec | 95 +++++++
...index_concurrently_upsert_partitioned.spec | 96 ++++++++
16 files changed, 1319 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 5712fac3697..974243c5c60 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1789,6 +1789,7 @@ DefineIndex(Oid tableId,
* before the reference snap was taken, we have to wait out any
* transactions that might have older snapshots.
*/
+ INJECTION_POINT("define_index_before_set_valid", NULL);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
@@ -4228,7 +4229,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap", NULL);
StartTransactionCommand();
/*
@@ -4307,6 +4308,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* index_drop() for more details.
*/
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead", NULL);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 401606f840a..df7e7bce86d 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -117,6 +117,7 @@
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -942,6 +943,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check_exclusion_or_unique_constraint_no_conflict", NULL);
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4c5647ac38a..f6d2a6ede93 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -70,6 +70,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1179,6 +1180,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec_insert_before_insert_speculative", NULL);
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 65561cc6bc3..8e1a918f130 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -123,6 +123,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -458,6 +459,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("invalidate_catalog_snapshot_end", NULL);
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index fc82cd67f6c..6a03024b5ce 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -14,7 +14,12 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points hashagg reindex_conc vacuum
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic inplace syscache-update-pruned
+ISOLATION = basic inplace syscache-update-pruned \
+ reindex_concurrently_upsert \
+ index_concurrently_upsert \
+ reindex_concurrently_upsert_partitioned \
+ reindex_concurrently_upsert_on_constraint \
+ index_concurrently_upsert_predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 00000000000..e7612e065f4
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
new file mode 100644
index 00000000000..0ef2f3a681c
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
@@ -0,0 +1,84 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 00000000000..3a2292696f1
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
new file mode 100644
index 00000000000..fa898f2e21b
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
new file mode 100644
index 00000000000..d6340d60738
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 20390d6b4bf..c1d0b93a58a 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -48,8 +48,14 @@ tests += {
'basic',
'inplace',
'syscache-update-pruned',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
+ 'reindex_concurrently_upsert_partitioned',
+ 'reindex_concurrently_upsert_on_constraint',
+ 'index_concurrently_upsert_predicate',
],
'runningcheck': false, # see syscache-update-pruned
+ 'runningcheck-parallel': false, # We waiting for all snapshots, so, avoid parallel test executions
},
'tap': {
'env': {
@@ -58,5 +64,7 @@ tests += {
'tests': [
't/001_stats.pl',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
},
}
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 00000000000..473b0408f55
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,72 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
new file mode 100644
index 00000000000..c8644a82d57
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
@@ -0,0 +1,74 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+
+ CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+ SELECT injection_points_attach('invalidate_catalog_snapshot_end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define_index_before_set_valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;}
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate_catalog_snapshot_end');
+ SELECT injection_points_wakeup('invalidate_catalog_snapshot_end');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define_index_before_set_valid');
+ SELECT injection_points_wakeup('define_index_before_set_valid');
+}
+
+permutation
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 00000000000..a4d1c1ea232
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,94 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
new file mode 100644
index 00000000000..82ee940cfa5
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
@@ -0,0 +1,95 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
new file mode 100644
index 00000000000..322848c808b
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
@@ -0,0 +1,96 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check_exclusion_or_unique_constraint_no_conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec_insert_before_insert_speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check_exclusion_or_unique_constraint_no_conflict');
+ SELECT injection_points_wakeup('check_exclusion_or_unique_constraint_no_conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec_insert_before_insert_speculative');
+ SELECT injection_points_wakeup('exec_insert_before_insert_speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
On 2024-Aug-24, Michail Nikolaev wrote:
Hello, I'm working on getting the 0002 fix committed, with the tests it
fixes. I ran across this comment:
@@ -813,7 +814,13 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;- if (!idxForm->indisvalid) + /* + * We need to consider both indisvalid and indisready indexes because + * them may become indisvalid before execution phase. It is required + * to keep set of indexes used as arbiter to be the same for all + * concurrent transactions. + */ + if (!idxForm->indisready) goto next;/*
I think this comment is wrong, or at least confusing. It says "we need
to consider both indisvalid and indisready indexes", which is somewhat
dubious: perhaps it wants to say "we need to consider indexes that have
indisvalid and indisready". But that is also wrong: I mean, if we
wanted to consider indexes that are marked indisready=false, then we
wouldn't "next" here (which essentially ignores such indexes). So,
really, I think what this wants to say is "we need to consider indexes
that are indisready regardless of whether or not they are indisvalid,
because they may become valid later".
Also, I think this comment lacks an explanation of _why_ indexes with
indisvalid=false are required. You wrote earlier[1]/messages/by-id/CANtu0ogv+6wqRzPK241jik4U95s1pW3MCZ3rX5ZqbFdUysz7Qw@mail.gmail.com in the thread the
following:
The reason is that these indexes are required for correct processing during
the execution phase.
If "ready" indexes are skipped as arbiters by one transaction, they may
already have become "valid" for another concurrent transaction during its
planning phase.
As a result, both transactions could concurrently process the UPSERT
command with different sets of arbiters (while using the same set of
indexes for tuple insertion later).
This can lead to unexpected "duplicate key value violates unique
constraint" errors and deadlocks.
I think this text is also confusing or wrong. I think you meant 'If
"not valid" indexes are skipped as arbiters, they may have become
"valid" for another concurrent transaction'. The text in catalogs.sgml
for these columns is:
: indisvalid bool
:
: If true, the index is currently valid for queries. False means the
: index is possibly incomplete: it must still be modified by
: INSERT/UPDATE operations, but it cannot safely be used for queries.
: If it is unique, the uniqueness property is not guaranteed true
: either.
: indisready bool
:
: If true, the index is currently ready for inserts. False means the
: index must be ignored by INSERT/UPDATE operations.
It makes sense that indisready indexes must be ignored, because
basically the catalog state is not ready yet. So that part seems
correct.
The other critical point is that the uniqueness must hold, and an index
with indisvalid=false would not necessarily detect that because the tree
might not be built completely yet, so the heap tuple that would be a
duplicate of the one we're writing might not have been scanned yet. But
that's not a problem, because we require that a valid index exists, even
if we don't "infer" that one -- which means the uniqueness clause is
being enforced by that other index.
Given all of that, I have rewritten the comment thusly:
/*
* Ignore indexes that aren't indisready, because we cannot trust their
* catalog structure yet. However, if any indexes are marked
* indisready but not yet indisvalid, we still consider them, because
* they might turn valid while we're running. Doing it this way
* allows a concurrent transaction with a slightly later catalog
* snapshot infer the same set of indexes, which is critical to
* prevent spurious 'duplicate key' errors.
*
* However, another critical aspect is that a unique index that isn't
* yet marked indisvalid=true might not be complete yet, meaning it
* wouldn't detect possible duplicate rows. In order to prevent false
* negatives, we require that we include in the set of inferred indexes
* at least one index that is marked valid.
*/
if (!idxForm->indisready)
goto next;
Am I understanding this correctly?
Thanks,
[1]: /messages/by-id/CANtu0ogv+6wqRzPK241jik4U95s1pW3MCZ3rX5ZqbFdUysz7Qw@mail.gmail.com
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"Learn about compilers. Then everything looks like either a compiler or
a database, and now you have two problems but one of them is fun."
https://twitter.com/thingskatedid/status/1456027786158776329
Hello, Álvaro!
Hello, I'm working on getting the 0002 fix committed, with the tests it
fixes. I ran across this comment:
Nice!
Am I understanding this correctly?
Yes, you are right. Looks like I mixed up everything while writing the comment.
I attached an updated version, changes are:
1) your fix for comment
2) some updates for 0001 test stability (related to [0]/messages/by-id/CADzfLwUc=jtSUEaQCtyt8zTeOJ-gHZ8=w_KJsVjDOYSLqaY9Lg@mail.gmail.com).
I'll recheck other patches too, maybe something needs to be adjusted also.
Best regards,
Mikhail.
[0]: /messages/by-id/CADzfLwUc=jtSUEaQCtyt8zTeOJ-gHZ8=w_KJsVjDOYSLqaY9Lg@mail.gmail.com
Attachments:
v13-0001-Specs-to-reproduce-the-issues-with-CREATE-INDEX-.patchapplication/x-patch; name=v13-0001-Specs-to-reproduce-the-issues-with-CREATE-INDEX-.patchDownload
From d8ff71490497bea9d746daebeb95e3025df907f8 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:10 +0300
Subject: [PATCH v13 1/5] Specs to reproduce the issues with CREATE INDEX
CONCURRENTLY and REINDEX CONCURRENTLY in scenarios involving INSERT ON
CONFLICT DO UPDATE (8b18ed6dfbb8b3e4483801b513fea6b429140569).
These tests reproduce different error cases related to "duplicate key value violates unique constraint" where this error should not occur by design.
* REINDEX CONCURRENTLY and UPSERT with inferred index
* CREATE INDEX CONCURRENTLY and UPSERT with inferred indexes
* REINDEX CONCURRENTLY on a partitioned table
* REINDEX CONCURRENTLY with specified constraint name
* CREATE INDEX CONCURRENTLY with predicates
In each of these scenarios, the expected behavior is that the INSERT ON CONFLICT DO UPDATE should handle conflicts gracefully without raising a "duplicate key value violates unique constraint" error. However, due to the concurrent operations on the indexes, this error is encountered.
---
src/backend/commands/indexcmds.c | 4 +-
src/backend/executor/execIndexing.c | 3 +
src/backend/executor/nodeModifyTable.c | 2 +
src/backend/utils/time/snapmgr.c | 3 +
src/test/modules/injection_points/Makefile | 7 +-
.../expected/index_concurrently_upsert.out | 88 +++++++
.../index_concurrently_upsert_predicate.out | 88 +++++++
.../expected/reindex_concurrently_upsert.out | 232 ++++++++++++++++++
...ndex_concurrently_upsert_on_constraint.out | 232 ++++++++++++++++++
...eindex_concurrently_upsert_partitioned.out | 232 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 8 +
.../specs/index_concurrently_upsert.spec | 77 ++++++
.../index_concurrently_upsert_predicate.spec | 79 ++++++
.../specs/reindex_concurrently_upsert.spec | 94 +++++++
...dex_concurrently_upsert_on_constraint.spec | 95 +++++++
...index_concurrently_upsert_partitioned.spec | 96 ++++++++
16 files changed, 1338 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
create mode 100644 src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
create mode 100644 src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 5712fac3697..316a94b59bc 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -1789,6 +1789,7 @@ DefineIndex(Oid tableId,
* before the reference snap was taken, we have to wait out any
* transactions that might have older snapshots.
*/
+ INJECTION_POINT("define-index-before-set-valid", NULL);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
@@ -4228,7 +4229,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* the same time to make sure we only get constraint violations from the
* indexes with the correct names.
*/
-
+ INJECTION_POINT("reindex_relation_concurrently_before_swap", NULL);
StartTransactionCommand();
/*
@@ -4307,6 +4308,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* index_drop() for more details.
*/
+ INJECTION_POINT("reindex_relation_concurrently_before_set_dead", NULL);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c
index 401606f840a..d973ecd8f8e 100644
--- a/src/backend/executor/execIndexing.c
+++ b/src/backend/executor/execIndexing.c
@@ -117,6 +117,7 @@
#include "utils/multirangetypes.h"
#include "utils/rangetypes.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
/* waitMode argument to check_exclusion_or_unique_constraint() */
typedef enum
@@ -942,6 +943,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;
ExecDropSingleTupleTableSlot(existing_slot);
+ if (!conflict)
+ INJECTION_POINT("check-exclusion-or-unique-constraint-no-conflict", NULL);
return !conflict;
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 4c5647ac38a..2f91a5912cf 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -70,6 +70,7 @@
#include "utils/datum.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/injection_point.h"
typedef struct MTTargetRelLookup
@@ -1179,6 +1180,7 @@ ExecInsert(ModifyTableContext *context,
return NULL;
}
}
+ INJECTION_POINT("exec-insert-before-insert-speculative", NULL);
/*
* Before we start insertion proper, acquire our "speculative
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 65561cc6bc3..e3bdabbe2f3 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -123,6 +123,7 @@
#include "utils/resowner.h"
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+#include "utils/injection_point.h"
/*
@@ -458,6 +459,8 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("pre-invalidate-catalog-snapshot-end", NULL);
+ INJECTION_POINT("invalidate-catalog-snapshot-end", NULL);
}
}
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index fc82cd67f6c..6a03024b5ce 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -14,7 +14,12 @@ PGFILEDESC = "injection_points - facility for injection points"
REGRESS = injection_points hashagg reindex_conc vacuum
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic inplace syscache-update-pruned
+ISOLATION = basic inplace syscache-update-pruned \
+ reindex_concurrently_upsert \
+ index_concurrently_upsert \
+ reindex_concurrently_upsert_partitioned \
+ reindex_concurrently_upsert_on_constraint \
+ index_concurrently_upsert_predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert.out b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
new file mode 100644
index 00000000000..d26c3afc177
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert.out
@@ -0,0 +1,88 @@
+Parsed test spec with 5 sessions
+
+starting permutation: s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s5_noop: <waiting ...>
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); <waiting ...>
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s5_noop: <... completed>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define-index-before-set-valid');
+ SELECT injection_points_wakeup('define-index-before-set-valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s5_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate-catalog-snapshot-end');
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
new file mode 100644
index 00000000000..ed76de113ed
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index_concurrently_upsert_predicate.out
@@ -0,0 +1,88 @@
+Parsed test spec with 5 sessions
+
+starting permutation: s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s5_noop: <waiting ...>
+step s3_start_create_index: CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000; <waiting ...>
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s5_noop: <... completed>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define-index-before-set-valid');
+ SELECT injection_points_wakeup('define-index-before-set-valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+ <waiting ...>
+step s5_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate-catalog-snapshot-end');
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
new file mode 100644
index 00000000000..f2ff16aff2a
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
new file mode 100644
index 00000000000..5bf1430ae10
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_on_constraint.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
new file mode 100644
index 00000000000..45ffdf94c1d
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex_concurrently_upsert_partitioned.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 20390d6b4bf..c1d0b93a58a 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -48,8 +48,14 @@ tests += {
'basic',
'inplace',
'syscache-update-pruned',
+ 'reindex_concurrently_upsert',
+ 'index_concurrently_upsert',
+ 'reindex_concurrently_upsert_partitioned',
+ 'reindex_concurrently_upsert_on_constraint',
+ 'index_concurrently_upsert_predicate',
],
'runningcheck': false, # see syscache-update-pruned
+ 'runningcheck-parallel': false, # We waiting for all snapshots, so, avoid parallel test executions
},
'tap': {
'env': {
@@ -58,5 +64,7 @@ tests += {
'tests': [
't/001_stats.pl',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
},
}
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
new file mode 100644
index 00000000000..01e0e4a5e3e
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert.spec
@@ -0,0 +1,77 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+ SELECT injection_points_attach('pre-invalidate-catalog-snapshot-end', 'notice');
+ SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define-index-before-set-valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i); }
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define-index-before-set-valid');
+ SELECT injection_points_wakeup('define-index-before-set-valid');
+}
+
+session s5
+step s5_noop {}
+step s5_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate-catalog-snapshot-end');
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+}
+
+permutation
+ s5_noop(s1_start_upsert notices 1)
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s5_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
new file mode 100644
index 00000000000..476e002cccb
--- /dev/null
+++ b/src/test/modules/injection_points/specs/index_concurrently_upsert_predicate.spec
@@ -0,0 +1,79 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: CREATE UNIQUE INDEX CONCURRENTLY
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+
+ CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+ SELECT injection_points_attach('pre-invalidate-catalog-snapshot-end', 'notice');
+ SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(abs(i)) where i < 100 do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('define-index-before-set-valid', 'wait');
+}
+step s3_start_create_index { CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;}
+
+session s4
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+}
+step s4_wakeup_define_index_before_set_valid {
+ SELECT injection_points_detach('define-index-before-set-valid');
+ SELECT injection_points_wakeup('define-index-before-set-valid');
+}
+
+session s5
+step s5_noop {}
+step s5_wakeup_s1_from_invalidate_catalog_snapshot {
+ SELECT injection_points_detach('invalidate-catalog-snapshot-end');
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+}
+
+permutation
+ s5_noop(s1_start_upsert notices 1)
+ s3_start_create_index(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_define_index_before_set_valid
+ s2_start_upsert(s1_start_upsert)
+ s5_wakeup_s1_from_invalidate_catalog_snapshot
+ s4_wakeup_s2
+ s4_wakeup_s1
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
new file mode 100644
index 00000000000..967033197ec
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert.spec
@@ -0,0 +1,94 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
new file mode 100644
index 00000000000..4defd6ccffe
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_on_constraint.spec
@@ -0,0 +1,95 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict on constraint tbl_pkey do update set updated_at = now();
+}
+
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
diff --git a/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
new file mode 100644
index 00000000000..8a2e172f3f1
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex_concurrently_upsert_partitioned.spec
@@ -0,0 +1,96 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_set_dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex_relation_concurrently_before_swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_swap');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex_relation_concurrently_before_set_dead');
+ SELECT injection_points_wakeup('reindex_relation_concurrently_before_set_dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
v13-0005-Revert-Doc-cover-index-CONCURRENTLY-causing-erro.patchapplication/x-patch; name=v13-0005-Revert-Doc-cover-index-CONCURRENTLY-causing-erro.patchDownload
From 5bfac887269c239c91284b147dfb5ea80489c732 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sun, 9 Nov 2025 11:48:34 +0100
Subject: [PATCH v13 5/5] Revert "Doc: cover index CONCURRENTLY causing errors
in INSERT ... ON CONFLICT."
This reverts commit 8b18ed6dfbb8b3e4483801b513fea6b429140569.
---
doc/src/sgml/ref/insert.sgml | 9 ---------
src/backend/optimizer/util/plancat.c | 5 -----
2 files changed, 14 deletions(-)
diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml
index 0598b8dea34..04962e39e12 100644
--- a/doc/src/sgml/ref/insert.sgml
+++ b/doc/src/sgml/ref/insert.sgml
@@ -594,15 +594,6 @@ INSERT INTO <replaceable class="parameter">table_name</replaceable> [ AS <replac
</para>
</tip>
- <warning>
- <para>
- While <command>CREATE INDEX CONCURRENTLY</command> or <command>REINDEX
- CONCURRENTLY</command> is running on a unique index, <command>INSERT
- ... ON CONFLICT</command> statements on the same table may unexpectedly
- fail with a unique violation.
- </para>
- </warning>
-
</refsect2>
</refsect1>
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 4cb6eb9269c..08ae828e83b 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -789,11 +789,6 @@ find_relation_notnullatts(PlannerInfo *root, Oid relid)
* the purposes of inference. If no opclass (or collation) is specified, then
* all matching indexes (that may or may not match the default in terms of
* each attribute opclass/collation) are used for inference.
- *
- * Note: during index CONCURRENTLY operations, different transactions may
- * reference different sets of arbiter indexes. This can lead to false unique
- * constraint violations that wouldn't occur during normal operations. For
- * more information, see insert.sgml.
*/
List *
infer_arbiter_indexes(PlannerInfo *root)
--
2.43.0
v13-0003-Modify-the-infer_arbiter_indexes-function-to-als.patchapplication/x-patch; name=v13-0003-Modify-the-infer_arbiter_indexes-function-to-als.patchDownload
From 00c32a088c533f294eb927788e1f78750d2d34e6 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:51:19 +0300
Subject: [PATCH v13 3/5] Modify the infer_arbiter_indexes function to also
look for indexes that match the specified named constraint to be used as
arbiters. This ensures that the same set of arbiter indexes is used for all
concurrent transactions in cases where REINDEX CONCURRENTLY processes an
index used as a named constraint.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_on_constraint
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
---
src/backend/optimizer/util/plancat.c | 121 +++++++++++++++++++--------
1 file changed, 88 insertions(+), 33 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index f1ad164847d..4cb6eb9269c 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -808,9 +808,10 @@ infer_arbiter_indexes(PlannerInfo *root)
List *indexList;
ListCell *l;
- /* Normalized inference attributes and inference expressions: */
- Bitmapset *inferAttrs = NULL;
- List *inferElems = NIL;
+ /* Normalized required attributes and expressions: */
+ Bitmapset *requiredArbiterAttrs = NULL;
+ List *requiredArbiterElems = NIL;
+ List *requiredIndexPredExprs = (List *) onconflict->arbiterWhere;
/* Results */
List *results = NIL;
@@ -849,8 +850,8 @@ infer_arbiter_indexes(PlannerInfo *root)
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
- inferElems = lappend(inferElems, elem->expr);
+ /* If not a plain Var, just shove it in requiredArbiterElems for now */
+ requiredArbiterElems = lappend(requiredArbiterElems, elem->expr);
continue;
}
@@ -862,30 +863,76 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("whole row unique index inference specifications are not supported")));
- inferAttrs = bms_add_member(inferAttrs,
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
+ indexList = RelationGetIndexList(relation);
+
/*
* Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * because some additional sanity checks are required. Additionally, we
+ * need to process other indexes as potential arbiters to account for
+ * cases where REINDEX CONCURRENTLY is processing an index used as a
+ * named constraint.
*/
if (onconflict->constraint != InvalidOid)
{
indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
+ {
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
- errmsg("constraint in ON CONFLICT clause has no associated index")));
+ errmsg("constraint in ON CONFLICT clause has no associated index")));
+ }
+
+ /*
+ * Find the named constraint index to extract its attributes and predicates.
+ * We open all indexes in the loop to avoid deadlock of changed order of locks.
+ * */
+ foreach(l, indexList)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation idxRel;
+ Form_pg_index idxForm;
+ AttrNumber natt;
+
+ idxRel = index_open(indexoid, rte->rellockmode);
+ idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Prepare requirements for other indexes to be used as arbiter together
+ * with indexOidFromConstraint. It is required to involve both equals indexes
+ * in case of REINDEX CONCURRENTLY.
+ */
+ for (natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno = idxRel->rd_index->indkey.values[natt];
+
+ if (attno != 0)
+ requiredArbiterAttrs = bms_add_member(requiredArbiterAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+ requiredArbiterElems = RelationGetIndexExpressions(idxRel);
+ requiredIndexPredExprs = RelationGetIndexPredicate(idxRel);
+ /* We are done, so, quite the loop. */
+ index_close(idxRel, NoLock);
+ break;
+ }
+ }
+ index_close(idxRel, NoLock);
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
foreach(l, indexList)
{
Oid indexoid = lfirst_oid(l);
@@ -943,26 +990,23 @@ infer_arbiter_indexes(PlannerInfo *root)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
-
- results = lappend_oid(results, idxForm->indexrelid);
- foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ goto found;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /* In the case of "ON constraint_name DO UPDATE" we need to skip non-unique candidates. */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ goto next;
+ } else {
+ /*
+ * Only considering conventional inference at this point (not named
+ * constraints), so index under consideration can be immediately
+ * skipped if it's not unique
+ */
+ if (!idxForm->indisunique)
+ goto next;
}
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
-
/*
* So-called unique constraints with WITHOUT OVERLAPS are really
* exclusion constraints, so skip those too.
@@ -982,7 +1026,7 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/* Non-expression attributes (if any) must match */
- if (!bms_equal(indexedAttrs, inferAttrs))
+ if (!bms_equal(indexedAttrs, requiredArbiterAttrs))
goto next;
/* Expression attributes (if any) must match */
@@ -990,6 +1034,10 @@ infer_arbiter_indexes(PlannerInfo *root)
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
+ /*
+ * If arbiterElems are present, check them. If name >constraint is
+ * present arbiterElems == NIL.
+ */
foreach(el, onconflict->arbiterElems)
{
InferenceElem *elem = (InferenceElem *) lfirst(el);
@@ -1027,26 +1075,33 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of the conventional inference involved ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In the case of named constraint ensure candidate has equal set
+ * of expressions as the named constraint index.
*/
- if (list_difference(idxExprs, inferElems) != NIL)
+ if (list_difference(idxExprs, requiredArbiterElems) != NIL)
goto next;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
+ /*
+ * If it's a partial index and conventional inference, its predicate must be implied
+ * by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid && !predicate_implied_by(predExprs, requiredIndexPredExprs, false))
+ goto next;
+ /* If it's a partial index and named constraint predicates must be equal. */
+ if (indexOidFromConstraint != InvalidOid && list_difference(predExprs, requiredIndexPredExprs) != NIL)
goto next;
+found:
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
next:
--
2.43.0
v13-0004-Modify-the-ExecInitPartitionInfo-function-to-con.patchapplication/x-patch; name=v13-0004-Modify-the-ExecInitPartitionInfo-function-to-con.patchDownload
From d1241cb9d5f4afa52b4ddf8abbc3ce5b125cdb7a Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:52:23 +0300
Subject: [PATCH v13 4/5] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY as
arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert_partitioned
---
src/backend/executor/execPartition.c | 119 ++++++++++++++++++++++++---
1 file changed, 107 insertions(+), 12 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index aa12e9ad2ea..066686483f0 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -490,6 +490,48 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+ /* it is not supported for cases of exclusion constraints. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+ if (arbiterIndexRelation->rd_index->indnkeyatts != indexRelation->rd_index->indnkeyatts)
+ return false;
+
+ for (i = 0; i < indexRelation->rd_index->indnkeyatts; i++)
+ {
+ int arbiterAttoNo = arbiterIndexRelation->rd_index->indkey.values[i];
+ int attoNo = indexRelation->rd_index->indkey.values[i];
+ if (arbiterAttoNo != attoNo)
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -701,6 +743,8 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
{
List *childIdxs;
+ List *nonAncestorIdxs = NIL;
+ int i, j, additional_arbiters = 0;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
@@ -711,23 +755,74 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, lfirst_oid(lc2)))
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxs = lappend_oid(nonAncestorIdxs, childIdx);
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxs)
+ {
+ for (i = 0; i < leaf_part_rri->ri_NumIndices; i++)
+ {
+ if (list_member_oid(nonAncestorIdxs, leaf_part_rri->ri_IndexRelationDescs[i]->rd_index->indexrelid))
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[i];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[i];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to us non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ for (j = 0; j < leaf_part_rri->ri_NumIndices; j++)
+ {
+ if (list_member_oid(arbiterIndexes,
+ leaf_part_rri->ri_IndexRelationDescs[j]->rd_index->indexrelid))
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[j];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[j];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxs);
+
+ /*
+ * If the resulting lists are of inequal length, something is wrong.
+ * (This shouldn't happen, since arbiter index selection should not
+ * pick up a non-ready index.)
+ *
+ * But we need to consider an additional arbiter indexes also.
+ */
+ if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
+ list_length(arbiterIndexes) - additional_arbiters)
+ elog(ERROR, "invalid arbiter index list");
}
-
- /*
- * If the resulting lists are of inequal length, something is wrong.
- * (This shouldn't happen, since arbiter index selection should not
- * pick up an invalid index.)
- */
- if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
- elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
/*
--
2.43.0
v13-0002-Modify-the-infer_arbiter_indexes-function-to-con.patchapplication/x-patch; name=v13-0002-Modify-the-infer_arbiter_indexes-function-to-con.patchDownload
From 384b38b86c412540a2f9471224273d24700e5b7a Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 20 Feb 2025 14:50:58 +0300
Subject: [PATCH v13 2/5] Modify the infer_arbiter_indexes function to consider
both indisvalid and indisready indexes. Ensure that at least one indisvalid
index is still required.
The change ensures that all concurrent transactions utilize the same set of indexes as arbiters. This uniformity is required to avoid conditions that could lead to "duplicate key value violates unique constraint" errors during UPSERT operations.
The patch resolves the issues in the following specs:
* reindex_concurrently_upsert
* index_concurrently_upsert
* index_concurrently_upsert_predicate
Despite the patch, the following specs are still affected:
* reindex_concurrently_upsert_partitioned
* reindex_concurrently_upsert_on_constraint
---
src/backend/optimizer/util/plancat.c | 27 ++++++++++++++++++++++-----
1 file changed, 22 insertions(+), 5 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d950bd93002..f1ad164847d 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -814,6 +814,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Results */
List *results = NIL;
+ bool foundValid = false;
/*
* Quickly return NIL for ON CONFLICT DO NOTHING without an inference
@@ -907,7 +908,22 @@ infer_arbiter_indexes(PlannerInfo *root)
idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
- if (!idxForm->indisvalid)
+ /*
+ * Ignore indexes that aren't indisready, because we cannot trust their
+ * catalog structure yet. However, if any indexes are marked
+ * indisready but not yet indisvalid, we still consider them, because
+ * they might turn valid while we're running. Doing it this way
+ * allows a concurrent transaction with a slightly later catalog
+ * snapshot infer the same set of indexes, which is critical to
+ * prevent spurious 'duplicate key' errors.
+ *
+ * However, another critical aspect is that a unique index that isn't
+ * yet marked indisvalid=true might not be complete yet, meaning it
+ * wouldn't detect possible duplicate rows. In order to prevent false
+ * negatives, we require that we include in the set of inferred indexes
+ * at least one index that is marked valid.
+ */
+ if (!idxForm->indisready)
goto next;
/*
@@ -929,10 +945,9 @@ infer_arbiter_indexes(PlannerInfo *root)
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
results = lappend_oid(results, idxForm->indexrelid);
- list_free(indexList);
+ foundValid |= idxForm->indisvalid;
index_close(idxRel, NoLock);
- table_close(relation, NoLock);
- return results;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
@@ -1033,6 +1048,7 @@ infer_arbiter_indexes(PlannerInfo *root)
goto next;
results = lappend_oid(results, idxForm->indexrelid);
+ foundValid |= idxForm->indisvalid;
next:
index_close(idxRel, NoLock);
}
@@ -1040,7 +1056,8 @@ next:
list_free(indexList);
table_close(relation, NoLock);
- if (results == NIL)
+ /* It is required to have at least one indisvalid index during the planning. */
+ if (results == NIL || !foundValid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
errmsg("there is no unique or exclusion constraint matching the ON CONFLICT specification")));
--
2.43.0
Hello,
On 2025-Nov-22, Mihail Nikalayeu wrote:
I attached an updated version, changes are:
1) your fix for comment
2) some updates for 0001 test stability (related to [0]).
Thank you. I've been looking at this part of 0001 too:
@@ -942,6 +943,8 @@ retry:
econtext->ecxt_scantuple = save_scantuple;ExecDropSingleTupleTableSlot(existing_slot); + if (!conflict) + INJECTION_POINT("check-exclusion-or-unique-constraint-no-conflict", NULL);
and wondering whether it would make sense to pass the 'conflict' as an
argument to the injection point instead of being conditional on it.
That is, do it like
ExecDropSingleTupleTableSlot(existing_slot);
+ INJECTION_POINT("check-exclusion-or-unique-constraint-no-conflict", &conflict);
However, I don't see that the SQL injection point interface has the
ability to receive arguments, so I'm guessing that this is not workable?
Maybe something to consider for the future.
BTW I've split the patches to commit differently: instead of 0001 with a
bunch of failing tests and then 0002 with fixes for some of them, I
intend to push a modified 0002 with the tests in 0001 that it fixes,
then your 0003 with the tests it fixes, and so on. (I have already cut
it this way, so you don't need to resubmit anything at this point. But
I'll verify what changes you have in the v13-0001 compared to the one
before, to be certain I'm not missing any later changes there.)
Thanks!
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"How strange it is to find the words "Perl" and "saner" in such close
proximity, with no apparent sense of irony. I doubt that Larry himself
could have managed it." (ncm, http://lwn.net/Articles/174769/)
On Sat, Nov 22, 2025 at 03:16:44PM +0100, Alvaro Herrera wrote:
ExecDropSingleTupleTableSlot(existing_slot);
+ INJECTION_POINT("check-exclusion-or-unique-constraint-no-conflict", &conflict);However, I don't see that the SQL injection point interface has the
ability to receive arguments, so I'm guessing that this is not workable?
Maybe something to consider for the future.
Are you referring to something like 371f2db8b05e here? This has been
added in 18.
--
Michael
On 2025-Nov-23, Michael Paquier wrote:
On Sat, Nov 22, 2025 at 03:16:44PM +0100, Alvaro Herrera wrote:
ExecDropSingleTupleTableSlot(existing_slot);
+ INJECTION_POINT("check-exclusion-or-unique-constraint-no-conflict", &conflict);However, I don't see that the SQL injection point interface has the
ability to receive arguments, so I'm guessing that this is not workable?
Maybe something to consider for the future.Are you referring to something like 371f2db8b05e here? This has been
added in 18.
Yes, exactly that ... but can this be used by the SQL injection points
functionality? The test is an isolation .spec file, and I didn't find a
way to say "make me sleep when I hit this injection point, but only if
conflict is false". Or maybe I just missed it.
Thanks!
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Estoy de acuerdo contigo en que la verdad absoluta no existe...
El problema es que la mentira sí existe y tu estás mintiendo" (G. Lama)
On Sun, Nov 23, 2025 at 01:14:37PM +0100, Alvaro Herrera wrote:
Yes, exactly that ... but can this be used by the SQL injection points
functionality? The test is an isolation .spec file, and I didn't find a
way to say "make me sleep when I hit this injection point, but only if
conflict is false". Or maybe I just missed it.
(Sorry for the low activity, last week was a crazy conference week and
I'm still recovering.)
Reading through v13-0001, there is currently no direct way with the
existing callbacks to do as you want, which would be to push down a
conditional wait inside the callback itself, based on a run-time
stack. There would be two ways to do that, by extending the facility:
- Simple one: addition of a new dedicated callback, that accepts one
single boolean argument.
- More complicated one: extend the module injection_points so as it is
possible to pass down conditions that should be checked at run-time.
I've mentioned that in the past, folks felt meh.
Saying that, Mihail's patch to just run the injection point only if
conflict == false is OK, and that's what I have seen most hackers do
as a matter of simplicity. This makes the injection point footprint
in the backend slightly larger but it's not that bad, englobed inside
an ifdef. You could also use a secondary point for an else branch
defined in execIndexing.c, with a different name and a different
callback attached to it if you want to take a special action for the
conflict == true case.
--
Michael
On 2025-Nov-22, Mihail Nikalayeu wrote:
2) some updates for 0001 test stability (related to [0]).
This patch would bring the committed test file up to date with what you
last submitted. However, I didn't understand what is the problem with
the original formulation, and I haven't seen the test fail ... can you
explain?
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"El número de instalaciones de UNIX se ha elevado a 10,
y se espera que este número aumente" (UPM, 1972)
Attachments:
0001-Improve-test-case-stability.patch.txttext/plain; charset=utf-8Download
From dede9ad2366fb4987af96a8dd4d6a33e9f832676 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <alvherre@kurilemu.de>
Date: Mon, 24 Nov 2025 18:46:24 +0100
Subject: [PATCH] Improve test case stability
Author: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
---
src/backend/utils/time/snapmgr.c | 1 +
.../index-concurrently-upsert-predicate.out | 11 +++++++---
.../expected/index-concurrently-upsert.out | 11 +++++++---
.../index-concurrently-upsert-predicate.spec | 21 ++++++++++++-------
.../specs/index-concurrently-upsert.spec | 19 +++++++++++------
5 files changed, 44 insertions(+), 19 deletions(-)
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 24f73a49d27..434abbf6b6f 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -459,6 +459,7 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
+ INJECTION_POINT("pre-invalidate-catalog-snapshot-end", NULL);
INJECTION_POINT("invalidate-catalog-snapshot-end", NULL);
}
}
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
index af602bdc048..3d9512c8fc7 100644
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
+++ b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
@@ -1,6 +1,6 @@
-Parsed test spec with 4 sessions
+Parsed test spec with 5 sessions
-starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+starting permutation: s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
injection_points_attach
-----------------------
@@ -16,12 +16,16 @@ injection_points_attach
(1 row)
+step s5_noop:
+ <waiting ...>
step s3_start_create_index:
CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
<waiting ...>
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s1_start_upsert:
INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
<waiting ...>
+step s5_noop: <... completed>
step s4_wakeup_define_index_before_set_valid:
SELECT injection_points_detach('define-index-before-set-valid');
SELECT injection_points_wakeup('define-index-before-set-valid');
@@ -39,7 +43,7 @@ injection_points_wakeup
step s2_start_upsert:
INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
<waiting ...>
-step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+step s5_wakeup_s1_from_invalidate_catalog_snapshot:
SELECT injection_points_detach('invalidate-catalog-snapshot-end');
SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
@@ -81,6 +85,7 @@ injection_points_wakeup
(1 row)
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s1_start_upsert: <... completed>
step s2_start_upsert: <... completed>
step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert.out b/src/test/modules/injection_points/expected/index-concurrently-upsert.out
index eb6fd9592df..a9e8bb5d00e 100644
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert.out
+++ b/src/test/modules/injection_points/expected/index-concurrently-upsert.out
@@ -1,6 +1,6 @@
-Parsed test spec with 4 sessions
+Parsed test spec with 5 sessions
-starting permutation: s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s4_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+starting permutation: s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
injection_points_attach
-----------------------
@@ -16,12 +16,16 @@ injection_points_attach
(1 row)
+step s5_noop:
+ <waiting ...>
step s3_start_create_index:
CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
<waiting ...>
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s1_start_upsert:
INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
<waiting ...>
+step s5_noop: <... completed>
step s4_wakeup_define_index_before_set_valid:
SELECT injection_points_detach('define-index-before-set-valid');
SELECT injection_points_wakeup('define-index-before-set-valid');
@@ -39,7 +43,7 @@ injection_points_wakeup
step s2_start_upsert:
INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
<waiting ...>
-step s4_wakeup_s1_from_invalidate_catalog_snapshot:
+step s5_wakeup_s1_from_invalidate_catalog_snapshot:
SELECT injection_points_detach('invalidate-catalog-snapshot-end');
SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
@@ -81,6 +85,7 @@ injection_points_wakeup
(1 row)
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s1_start_upsert: <... completed>
step s2_start_upsert: <... completed>
step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec b/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
index 13897d88bce..725f6f22295 100644
--- a/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
+++ b/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
@@ -5,7 +5,7 @@
# - s2: UPSERT the same tuple
# - s3: CREATE UNIQUE INDEX CONCURRENTLY (with a predicate)
#
-# - s4: control concurrency via injection points
+# - s4 and s5: control concurrency via injection points
setup
{
@@ -27,6 +27,7 @@ setup
{
SELECT injection_points_set_local();
SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+ SELECT injection_points_attach('pre-invalidate-catalog-snapshot-end', 'notice');
SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
}
step s1_start_upsert
@@ -62,11 +63,6 @@ step s4_wakeup_s1
SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
}
-step s4_wakeup_s1_from_invalidate_catalog_snapshot
-{
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-}
step s4_wakeup_s2
{
SELECT injection_points_detach('exec-insert-before-insert-speculative');
@@ -78,11 +74,22 @@ step s4_wakeup_define_index_before_set_valid
SELECT injection_points_wakeup('define-index-before-set-valid');
}
+session s5
+step s5_noop
+{
+}
+step s5_wakeup_s1_from_invalidate_catalog_snapshot
+{
+ SELECT injection_points_detach('invalidate-catalog-snapshot-end');
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+}
+
permutation
+ s5_noop(s1_start_upsert notices 1)
s3_start_create_index(s1_start_upsert, s2_start_upsert)
s1_start_upsert
s4_wakeup_define_index_before_set_valid
s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s5_wakeup_s1_from_invalidate_catalog_snapshot
s4_wakeup_s2
s4_wakeup_s1
diff --git a/src/test/modules/injection_points/specs/index-concurrently-upsert.spec b/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
index b07a6408b3b..4487834aa8e 100644
--- a/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
+++ b/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
@@ -26,6 +26,7 @@ setup
{
SELECT injection_points_set_local();
SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+ SELECT injection_points_attach('pre-invalidate-catalog-snapshot-end', 'notice');
SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
}
step s1_start_upsert
@@ -61,11 +62,6 @@ step s4_wakeup_s1
SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
}
-step s4_wakeup_s1_from_invalidate_catalog_snapshot
-{
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-}
step s4_wakeup_s2
{
SELECT injection_points_detach('exec-insert-before-insert-speculative');
@@ -77,11 +73,22 @@ step s4_wakeup_define_index_before_set_valid
SELECT injection_points_wakeup('define-index-before-set-valid');
}
+session s5
+step s5_noop
+{
+}
+step s5_wakeup_s1_from_invalidate_catalog_snapshot
+{
+ SELECT injection_points_detach('invalidate-catalog-snapshot-end');
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+}
+
permutation
+ s5_noop(s1_start_upsert notices 1)
s3_start_create_index(s1_start_upsert, s2_start_upsert)
s1_start_upsert
s4_wakeup_define_index_before_set_valid
s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1_from_invalidate_catalog_snapshot
+ s5_wakeup_s1_from_invalidate_catalog_snapshot
s4_wakeup_s2
s4_wakeup_s1
--
2.47.3
On Mon, Nov 24, 2025 at 06:49:47PM +0100, Alvaro Herrera wrote:
This patch would bring the committed test file up to date with what you
last submitted. However, I didn't understand what is the problem with
the original formulation, and I haven't seen the test fail ... can you
explain?
Reading through bc32a12e0db2, I am puzzled by the committed result
here:
+#ifdef USE_INJECTION_POINTS
+ if (conflict)
+ INJECTION_POINT("check-exclusion-or-unique-constraint-conflict", NULL);
+ else
+ INJECTION_POINT("check-exclusion-or-unique-constraint-no-conflict", NULL);
+#endif
The "no-conflict" point is used in the isolation test, but not the
other in the "conflict == true" path:
$ git grep check-exclusion-or-unique-constraint-conflict
src/backend/executor/execIndexing.c:
INJECTION_POINT("check-exclusion-or-unique-constraint-conflict", NULL);
If you have no plans for it in the long-term, I'd rather remove it
from the tree, rather than keep it. Of course, I would keep the
USE_INJECTION_POINTS block to avoid the extra boolean check in
non-USE_INJECTION_POINTS builds.
--
Michael
Hello, Álvaro!
On Mon, Nov 24, 2025 at 6:49 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
This patch would bring the committed test file up to date with what you
last submitted. However, I didn't understand what is the problem with
the original formulation, and I haven't seen the test fail ... can you
explain?
Yes, it looks strange - but it is the best option I have found so far.
I've seen tests fail a few times in CI due to race.
More details are available at [0]/messages/by-id/CADzfLwUc=jtSUEaQCtyt8zTeOJ-gHZ8=w_KJsVjDOYSLqaY9Lg@mail.gmail.com and particularly [1]/messages/by-id/aREW7Qo0GqjfiHn7@paquier.xyz.
In a few words - it is an attempt to make sure the test goes to the
wake-up backend only after it actually enters to wait mode. For that
reason an additional 'notice' point is used by spec.
I have proposed another possible solution for the [0]/messages/by-id/CADzfLwUc=jtSUEaQCtyt8zTeOJ-gHZ8=w_KJsVjDOYSLqaY9Lg@mail.gmail.com thread.
Best regards,
Mikhail.
[0]: /messages/by-id/CADzfLwUc=jtSUEaQCtyt8zTeOJ-gHZ8=w_KJsVjDOYSLqaY9Lg@mail.gmail.com
[1]: /messages/by-id/aREW7Qo0GqjfiHn7@paquier.xyz
On 2025-Nov-24, Mihail Nikalayeu wrote:
In a few words - it is an attempt to make sure the test goes to the
wake-up backend only after it actually enters to wait mode. For that
reason an additional 'notice' point is used by spec.
I have proposed another possible solution for the [0] thread.
That makes sense. I pushed it now, many thanks.
On 2025-Nov-25, Michael Paquier wrote:
Reading through bc32a12e0db2, I am puzzled by the committed result here: +#ifdef USE_INJECTION_POINTS + if (conflict) + INJECTION_POINT("check-exclusion-or-unique-constraint-conflict", NULL); + else + INJECTION_POINT("check-exclusion-or-unique-constraint-no-conflict", NULL); +#endif
If you have no plans for it in the long-term, I'd rather remove it
from the tree, rather than keep it.
Removed.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"We have labored long to build a heaven, only to find it
populated with horrors" (Prof. Milton Glass)
Import Notes
Reply to msg id not found: CADzfLwW2VkOrUujmWa25n_RY3onrFG4apqqyx8as1+F0USMMA@mail.gmail.comaSTY9z_b4PzZNSap@paquier.xyz | Resolved by subject fallback
Hello,
We ran into one more problem with the new test, evidenced by timeouts by
buildfarm member prion. For CATCACHE_FORCE_RELEASE builds on two of the
tests, we get a few invalidations of the catalog snapshot ahead of what
we expect, and because we have an injection point to sleep there, those
tests get stuck.
Here's one possible fix. I had to take the attach operation on
invalidate-catalog-snapshot-end to a new step of s1, instead of
occurring in the setup block. I understand that this is because no step
can run until the setup of all steps completes, so if one setup gets
stuck, we're out of luck. And then, session s4 can do a conditional
wakeup of session s1.
Patch attached. Thoughts?
Maybe there's some other way to go about this -- for instance I
considered the idea of moving the injection point somewhere else from
InvalidateCatalogSnapshot(). I don't have any ideas about that though,
but I'm willing to listen if anybody has any.
The output in both cases is different (we get more notices in the weird
build and also s1 goes to sleep in different order), so I had to add an
alternative expected file. The diffs look okay to me -- essentially
they say, in the normal case, the injection_points_attach() returns
immediately, but in the other case, s1 goes to sleep and is awakened
when it sees the 'case' expression by session 4:
--- expected/index-concurrently-upsert-predicate.out 2025-11-26 19:13:58.702213673 +0100
+++ expected/index-concurrently-upsert-predicate_1.out 2025-11-26 19:04:16.745666956 +0100
@@ -1,8 +1,9 @@
Parsed test spec with 5 sessions
starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
injection_points_attach
-----------------------
(1 row)
@@ -14,18 +15,15 @@
injection_points_attach
-----------------------
(1 row)
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s1_attach_invalidate_catalog_snapshot:
SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
+ <waiting ...>
step s4_wakeup_s1_setup:
select case when
(select pid from pg_stat_activity
where wait_event_type = 'InjectionPoint' and
wait_event = 'invalidate-catalog-snapshot-end') is not null
@@ -35,10 +33,16 @@
case
----
(1 row)
+step s1_attach_invalidate_catalog_snapshot: <... completed>
+injection_points_attach
+-----------------------
+
+(1 row)
+
step s5_noop:
<waiting ...>
step s3_start_create_index:
CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
<waiting ...>
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"Los minutos y los segundos son mercadería de la ciudad, donde un infeliz se
afana por no perder ni siquiera un segundo y no advierto que obrando de ese
modo pierde una vida." ("La vuelta de Don Camilo", G. Guareschi)
Attachments:
0001-Fix-new-test-for-CATCACHE_FORCE_RELEASE-builds.patchtext/x-diff; charset=utf-8Download
From 795427760facc8b3dcbeaa8a19bf0d237d3e630e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <alvherre@kurilemu.de>
Date: Wed, 26 Nov 2025 19:15:12 +0100
Subject: [PATCH] Fix new test for CATCACHE_FORCE_RELEASE builds
---
.../index-concurrently-upsert-predicate.out | 23 +++-
.../index-concurrently-upsert-predicate_1.out | 116 ++++++++++++++++++
.../expected/index-concurrently-upsert.out | 23 +++-
.../expected/index-concurrently-upsert_1.out | 116 ++++++++++++++++++
.../index-concurrently-upsert-predicate.spec | 18 +++
.../specs/index-concurrently-upsert.spec | 18 +++
6 files changed, 312 insertions(+), 2 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
create mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
index 3d9512c8fc7..7adbab1c78e 100644
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
+++ b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
@@ -1,6 +1,6 @@
Parsed test spec with 5 sessions
-starting permutation: s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
injection_points_attach
-----------------------
@@ -16,6 +16,27 @@ injection_points_attach
(1 row)
+step s1_attach_invalidate_catalog_snapshot:
+ SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1_setup:
+ select case when
+ (select pid from pg_stat_activity
+ where wait_event_type = 'InjectionPoint' and
+ wait_event = 'invalidate-catalog-snapshot-end') is not null
+ then injection_points_wakeup('invalidate-catalog-snapshot-end')
+ end;
+
+case
+----
+
+(1 row)
+
step s5_noop:
<waiting ...>
step s3_start_create_index:
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
new file mode 100644
index 00000000000..affc256ff82
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
@@ -0,0 +1,116 @@
+Parsed test spec with 5 sessions
+
+starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+step s1_attach_invalidate_catalog_snapshot:
+ SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+ <waiting ...>
+step s4_wakeup_s1_setup:
+ select case when
+ (select pid from pg_stat_activity
+ where wait_event_type = 'InjectionPoint' and
+ wait_event = 'invalidate-catalog-snapshot-end') is not null
+ then injection_points_wakeup('invalidate-catalog-snapshot-end')
+ end;
+
+case
+----
+
+(1 row)
+
+step s1_attach_invalidate_catalog_snapshot: <... completed>
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s5_noop:
+ <waiting ...>
+step s3_start_create_index:
+ CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
+ <waiting ...>
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s5_noop: <... completed>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define-index-before-set-valid');
+ SELECT injection_points_wakeup('define-index-before-set-valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s5_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate-catalog-snapshot-end');
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert.out b/src/test/modules/injection_points/expected/index-concurrently-upsert.out
index a9e8bb5d00e..cf961378942 100644
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert.out
+++ b/src/test/modules/injection_points/expected/index-concurrently-upsert.out
@@ -1,6 +1,6 @@
Parsed test spec with 5 sessions
-starting permutation: s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
injection_points_attach
-----------------------
@@ -16,6 +16,27 @@ injection_points_attach
(1 row)
+step s1_attach_invalidate_catalog_snapshot:
+ SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1_setup:
+ select case when
+ (select pid from pg_stat_activity
+ where wait_event_type = 'InjectionPoint' and
+ wait_event = 'invalidate-catalog-snapshot-end') is not null
+ then injection_points_wakeup('invalidate-catalog-snapshot-end')
+ end;
+
+case
+----
+
+(1 row)
+
step s5_noop:
<waiting ...>
step s3_start_create_index:
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out b/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
new file mode 100644
index 00000000000..7a77830d56c
--- /dev/null
+++ b/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
@@ -0,0 +1,116 @@
+Parsed test spec with 5 sessions
+
+starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+step s1_attach_invalidate_catalog_snapshot:
+ SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+ <waiting ...>
+step s4_wakeup_s1_setup:
+ select case when
+ (select pid from pg_stat_activity
+ where wait_event_type = 'InjectionPoint' and
+ wait_event = 'invalidate-catalog-snapshot-end') is not null
+ then injection_points_wakeup('invalidate-catalog-snapshot-end')
+ end;
+
+case
+----
+
+(1 row)
+
+step s1_attach_invalidate_catalog_snapshot: <... completed>
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s5_noop:
+ <waiting ...>
+step s3_start_create_index:
+ CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
+ <waiting ...>
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s5_noop: <... completed>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define-index-before-set-valid');
+ SELECT injection_points_wakeup('define-index-before-set-valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s5_wakeup_s1_from_invalidate_catalog_snapshot:
+ SELECT injection_points_detach('invalidate-catalog-snapshot-end');
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec b/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
index 725f6f22295..e65e4a9120c 100644
--- a/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
+++ b/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
@@ -28,6 +28,9 @@ setup
SELECT injection_points_set_local();
SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
SELECT injection_points_attach('pre-invalidate-catalog-snapshot-end', 'notice');
+}
+step s1_attach_invalidate_catalog_snapshot
+{
SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
}
step s1_start_upsert
@@ -58,6 +61,19 @@ step s3_start_create_index
}
session s4
+# Step s1_attach_invalidate_catalog_snapshot sleeps or not depending on
+# build conditions (CATCACHE_FORCE_RELEASE). Here we send a wakeup signal if
+# it's sleeping or do nothing otherwise, and print a null value in either
+# case.
+step s4_wakeup_s1_setup
+{
+ select case when
+ (select pid from pg_stat_activity
+ where wait_event_type = 'InjectionPoint' and
+ wait_event = 'invalidate-catalog-snapshot-end') is not null
+ then injection_points_wakeup('invalidate-catalog-snapshot-end')
+ end;
+}
step s4_wakeup_s1
{
SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
@@ -85,6 +101,8 @@ step s5_wakeup_s1_from_invalidate_catalog_snapshot
}
permutation
+ s1_attach_invalidate_catalog_snapshot
+ s4_wakeup_s1_setup
s5_noop(s1_start_upsert notices 1)
s3_start_create_index(s1_start_upsert, s2_start_upsert)
s1_start_upsert
diff --git a/src/test/modules/injection_points/specs/index-concurrently-upsert.spec b/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
index 4487834aa8e..f78ed16c4f3 100644
--- a/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
+++ b/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
@@ -27,6 +27,9 @@ setup
SELECT injection_points_set_local();
SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
SELECT injection_points_attach('pre-invalidate-catalog-snapshot-end', 'notice');
+}
+step s1_attach_invalidate_catalog_snapshot
+{
SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
}
step s1_start_upsert
@@ -57,6 +60,19 @@ step s3_start_create_index
}
session s4
+# Step s1_attach_invalidate_catalog_snapshot sleeps or not depending on
+# build conditions (CATCACHE_FORCE_RELEASE). Here we send a wakeup signal if
+# it's sleeping or do nothing otherwise, and print a null value in either
+# case.
+step s4_wakeup_s1_setup
+{
+ select case when
+ (select pid from pg_stat_activity
+ where wait_event_type = 'InjectionPoint' and
+ wait_event = 'invalidate-catalog-snapshot-end') is not null
+ then injection_points_wakeup('invalidate-catalog-snapshot-end')
+ end;
+}
step s4_wakeup_s1
{
SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
@@ -84,6 +100,8 @@ step s5_wakeup_s1_from_invalidate_catalog_snapshot
}
permutation
+ s1_attach_invalidate_catalog_snapshot
+ s4_wakeup_s1_setup
s5_noop(s1_start_upsert notices 1)
s3_start_create_index(s1_start_upsert, s2_start_upsert)
s1_start_upsert
--
2.47.3
Hello!
On Wed, Nov 26, 2025 at 7:34 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
We ran into one more problem with the new test, evidenced by timeouts by
buildfarm member prion. For CATCACHE_FORCE_RELEASE builds on two of the
tests, we get a few invalidations of the catalog snapshot ahead of what
we expect, and because we have an injection point to sleep there, those
tests get stuck.
Oh, I missed that. Non-yet pushed tests are probably affected too.
Here's one possible fix. I had to take the attach operation on
invalidate-catalog-snapshot-end to a new step of s1, instead of
occurring in the setup block. I understand that this is because no step
can run until the setup of all steps completes, so if one setup gets
stuck, we're out of luck. And then, session s4 can do a conditional
wakeup of session s1.
I have tried to move the setup of invalidate-catalog-snapshot-end to
s1_start_upsert as the first command - but for some reason it wasn't
working the way I expected. But maybe I missed something.
Patch attached. Thoughts?
Solution seems reasonable to me, another related ideas:
* replace "select case when" with function like
injection_points_wakeup_if_waiting to avoid the possible race between
select and wake up (but AFAIK it is not possible in the current case)
* introduce some injection_points function to enter "ignore all runs,
but still allowed to attach/detach" mode and "normal" mode.. As first
command of setup - enter such "setup mode", as last - back to normal.
Maybe there's some other way to go about this -- for instance I
considered the idea of moving the injection point somewhere else from
InvalidateCatalogSnapshot(). I don't have any ideas about that though,
but I'm willing to listen if anybody has any.
AFAIU it is the only place.
Best regard,
Mikhail.
Hi,
On 2025-Nov-27, Mihail Nikalayeu wrote:
On Wed, Nov 26, 2025 at 7:34 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
We ran into one more problem with the new test, evidenced by timeouts by
buildfarm member prion. For CATCACHE_FORCE_RELEASE builds on two of the
tests, we get a few invalidations of the catalog snapshot ahead of what
we expect, and because we have an injection point to sleep there, those
tests get stuck.Oh, I missed that. Non-yet pushed tests are probably affected too.
Yeah, I suspect as much.
Here's one possible fix.
I have tried to move the setup of invalidate-catalog-snapshot-end to
s1_start_upsert as the first command - but for some reason it wasn't
working the way I expected. But maybe I missed something.
Right, this is why I said that this is one possible fix. I mean, maybe
there are other ways to fix it. I'm not sure it's the simplest or the
most robust, but I don't want to spend too much time looking for other
ways either.
Solution seems reasonable to me, another related ideas:
* replace "select case when" with function like
injection_points_wakeup_if_waiting to avoid the possible race between
select and wake up (but AFAIK it is not possible in the current case)
* introduce some injection_points function to enter "ignore all runs,
but still allowed to attach/detach" mode and "normal" mode.. As first
command of setup - enter such "setup mode", as last - back to normal.
Ah, I had thought about the first one of these ideas, but not the second
one. I noted both in the commit message, in case somebody is motivated
to implement them.
Thanks for reviewing. I have pushed it now. Looking at the next one.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Having your biases confirmed independently is how scientific progress is
made, and hence made our great society what it is today" (Mary Gardiner)
Hello,
Here's a slightly different approach for the fix proposed in your 0003.
I wasn't happy with the idea of opening all indexes twice in
infer_arbiter_indexes(), so I instead made it collect all Relations from
those indexes in an initial loop, then process them in the two places
that wanted them, and we close them all again together. I think this
also makes the code clearer. We no longer have the "next" goto label to
close the index at the bottom of the loop, but instead we can just do
"continue" cleanly.
I also rewrote some comments. I may not have done all the edits I
wanted, but ran out of time today and I think this is in pretty good
shape.
I tried under CATCACHE_FORCE_RELEASE and saw no problems.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Attachments:
v14-0003-ON-CONFLICT-Consider-indexes-matching-constraint.patchtext/x-diff; charset=utf-8Download
From 6b7626dc756125bb2792668185fb1bba01090aea Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <alvherre@kurilemu.de>
Date: Fri, 28 Nov 2025 18:09:40 +0100
Subject: [PATCH v14] ON CONFLICT: Consider indexes matching constraint index
during REINDEX CONCURRENTLY
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This ensures that all transactions doing INSERT ON CONFLICT consider the
same set of indexes during the reindex operation, avoiding spurious
errors about duplicate insertions.
Author: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Reviewed-by: Ãlvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/CANtu0ojXmqjmEzp-=aJSxjsdE76iAsRgHBoK0QtYHimb_mEfsg@mail.gmail.com
---
src/backend/optimizer/util/plancat.c | 192 ++++++++++----
src/backend/parser/parse_clause.c | 16 +-
src/test/modules/injection_points/Makefile | 1 +
...ndex-concurrently-upsert-on-constraint.out | 238 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 1 +
...dex-concurrently-upsert-on-constraint.spec | 110 ++++++++
6 files changed, 506 insertions(+), 52 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out
create mode 100644 src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 7af9a2064e3..daf75612333 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -806,9 +806,15 @@ infer_arbiter_indexes(PlannerInfo *root)
Relation relation;
Oid indexOidFromConstraint = InvalidOid;
List *indexList;
- ListCell *l;
+ List *indexRelList = NIL;
- /* Normalized inference attributes and inference expressions: */
+ /*
+ * Required attributes and expressions used to match indexes to the clause
+ * given by the user. In the case where ON CONFLICT ON CONSTRAINT was
+ * given, we need to compute these things to match other indexes, to
+ * account for the case where the index is under REINDEX CONCURRENTLY.
+ */
+ List *inferIndexExprs = (List *) onconflict->arbiterWhere;
Bitmapset *inferAttrs = NULL;
List *inferElems = NIL;
@@ -841,15 +847,19 @@ infer_arbiter_indexes(PlannerInfo *root)
* well as a separate list of expression items. This simplifies matching
* the cataloged definition of indexes.
*/
- foreach(l, onconflict->arbiterElems)
+ foreach_ptr(InferenceElem, elem, onconflict->arbiterElems)
{
- InferenceElem *elem = (InferenceElem *) lfirst(l);
Var *var;
int attno;
+ /* we cannot also have a constraint name, per grammar */
+ Assert(!OidIsValid(onconflict->constraint));
+
if (!IsA(elem->expr, Var))
{
- /* If not a plain Var, just shove it in inferElems for now */
+ /*
+ * If not a plain Var, just shove it in inferElems for now.
+ */
inferElems = lappend(inferElems, elem->expr);
continue;
}
@@ -867,45 +877,100 @@ infer_arbiter_indexes(PlannerInfo *root)
}
/*
- * Lookup named constraint's index. This is not immediately returned
- * because some additional sanity checks are required.
+ * Next, open all the indexes. We need this list for two things: first,
+ * if an ON CONSTRAINT clause was given, and that constraint's index is
+ * undergoing REINDEX CONCURRENTLY, then we need to consider all matches
+ * for that index. Second, if an attribute list was specified in the ON
+ * CONFLICT clause, we use the list to find the indexes whose attributes
+ * match that list.
+ */
+ indexList = RelationGetIndexList(relation);
+ foreach_oid(indexoid, indexList)
+ {
+ Relation idxRel;
+
+ /*
+ * Must open in this order to avoid deadlock. Obtain the same lock
+ * type that the executor will ultimately use.
+ */
+ idxRel = index_open(indexoid, rte->rellockmode);
+ indexRelList = lappend(indexRelList, idxRel);
+ }
+
+ /*
+ * If a constraint was named in the command, look up its index. We don't
+ * return it immediately because we need some additional sanity checks,
+ * and also because we need to include other indexes as arbiters to
+ * account for REINDEX CONCURRENTLY processing the constraint's index.
*/
if (onconflict->constraint != InvalidOid)
{
- indexOidFromConstraint = get_constraint_index(onconflict->constraint);
+ /* we cannot also have an explicit list of elements, per grammar */
+ Assert(onconflict->arbiterElems == NIL);
+ indexOidFromConstraint = get_constraint_index(onconflict->constraint);
if (indexOidFromConstraint == InvalidOid)
ereport(ERROR,
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("constraint in ON CONFLICT clause has no associated index")));
+
+ /*
+ * Find the named constraint index to extract its attributes and
+ * predicates.
+ */
+ foreach_ptr(RelationData, idxRel, indexRelList)
+ {
+ Form_pg_index idxForm = idxRel->rd_index;
+
+ if (idxForm->indisready)
+ {
+ if (indexOidFromConstraint == idxForm->indexrelid)
+ {
+ /*
+ * Set up inferElems and inferPredExprs to match
+ * the constraint index, so that we can match them
+ * in the loop below.
+ */
+ for (int natt = 0; natt < idxForm->indnkeyatts; natt++)
+ {
+ int attno;
+
+ attno = idxRel->rd_index->indkey.values[natt];
+ if (attno != InvalidAttrNumber)
+ inferAttrs =
+ bms_add_member(inferAttrs,
+ attno - FirstLowInvalidHeapAttributeNumber);
+ }
+
+ /* found it */
+ inferElems = RelationGetIndexExpressions(idxRel);
+ inferIndexExprs = RelationGetIndexPredicate(idxRel);
+ break;
+ }
+ }
+ }
}
/*
* Using that representation, iterate through the list of indexes on the
* target relation to try and find a match
*/
- indexList = RelationGetIndexList(relation);
-
- foreach(l, indexList)
+ foreach_ptr(RelationData, idxRel, indexRelList)
{
- Oid indexoid = lfirst_oid(l);
- Relation idxRel;
Form_pg_index idxForm;
Bitmapset *indexedAttrs;
List *idxExprs;
List *predExprs;
AttrNumber natt;
- ListCell *el;
+ bool match;
/*
- * Extract info from the relation descriptor for the index. Obtain
- * the same lock type that the executor will ultimately use.
+ * Extract info from the relation descriptor for the index.
*
* Let executor complain about !indimmediate case directly, because
* enforcement needs to occur there anyway when an inference clause is
* omitted.
*/
- idxRel = index_open(indexoid, rte->rellockmode);
idxForm = idxRel->rd_index;
/*
@@ -924,7 +989,7 @@ infer_arbiter_indexes(PlannerInfo *root)
* indexes at least one index that is marked valid.
*/
if (!idxForm->indisready)
- goto next;
+ continue;
/*
* Note that we do not perform a check against indcheckxmin (like e.g.
@@ -934,7 +999,7 @@ infer_arbiter_indexes(PlannerInfo *root)
*/
/*
- * Look for match on "ON constraint_name" variant, which may not be
+ * Look for match on "ON constraint_name" variant, which may not be a
* unique constraint. This can only be a constraint name.
*/
if (indexOidFromConstraint == idxForm->indexrelid)
@@ -944,31 +1009,37 @@ infer_arbiter_indexes(PlannerInfo *root)
(errcode(ERRCODE_WRONG_OBJECT_TYPE),
errmsg("ON CONFLICT DO UPDATE not supported with exclusion constraints")));
+ /* Consider this one a match already */
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
- index_close(idxRel, NoLock);
- break;
+ continue;
}
else if (indexOidFromConstraint != InvalidOid)
{
- /* No point in further work for index in named constraint case */
- goto next;
+ /*
+ * In the case of "ON constraint_name DO UPDATE" we need to skip
+ * non-unique candidates.
+ */
+ if (!idxForm->indisunique && onconflict->action == ONCONFLICT_UPDATE)
+ continue;
+ }
+ else
+ {
+ /*
+ * Only considering conventional inference at this point (not
+ * named constraints), so index under consideration can be
+ * immediately skipped if it's not unique.
+ */
+ if (!idxForm->indisunique)
+ continue;
}
-
- /*
- * Only considering conventional inference at this point (not named
- * constraints), so index under consideration can be immediately
- * skipped if it's not unique
- */
- if (!idxForm->indisunique)
- goto next;
/*
* So-called unique constraints with WITHOUT OVERLAPS are really
* exclusion constraints, so skip those too.
*/
if (idxForm->indisexclusion)
- goto next;
+ continue;
/* Build BMS representation of plain (non expression) index attrs */
indexedAttrs = NULL;
@@ -983,17 +1054,20 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Non-expression attributes (if any) must match */
if (!bms_equal(indexedAttrs, inferAttrs))
- goto next;
+ continue;
/* Expression attributes (if any) must match */
idxExprs = RelationGetIndexExpressions(idxRel);
if (idxExprs && varno != 1)
ChangeVarNodes((Node *) idxExprs, 1, varno, 0);
- foreach(el, onconflict->arbiterElems)
+ /*
+ * If arbiterElems are present, check them. (Note that if a
+ * constraint name was given in the command line, this list is NIL.)
+ */
+ match = true;
+ foreach_ptr(InferenceElem, elem, onconflict->arbiterElems)
{
- InferenceElem *elem = (InferenceElem *) lfirst(el);
-
/*
* Ensure that collation/opclass aspects of inference expression
* element match. Even though this loop is primarily concerned
@@ -1002,7 +1076,10 @@ infer_arbiter_indexes(PlannerInfo *root)
* attributes appearing as inference elements.
*/
if (!infer_collation_opclass_match(elem, idxRel, idxExprs))
- goto next;
+ {
+ match = false;
+ break;
+ }
/*
* Plain Vars don't factor into count of expression elements, and
@@ -1023,37 +1100,58 @@ infer_arbiter_indexes(PlannerInfo *root)
list_member(idxExprs, elem->expr))
continue;
- goto next;
+ match = false;
+ break;
}
+ if (!match)
+ continue;
/*
- * Now that all inference elements were matched, ensure that the
+ * In case of inference from an attribute list, ensure that the
* expression elements from inference clause are not missing any
* cataloged expressions. This does the right thing when unique
* indexes redundantly repeat the same attribute, or if attributes
* redundantly appear multiple times within an inference clause.
+ *
+ * In case a constraint was named, ensure the candidate has an equal
+ * set of expressions as the named constraint's index.
*/
if (list_difference(idxExprs, inferElems) != NIL)
- goto next;
+ continue;
- /*
- * If it's a partial index, its predicate must be implied by the ON
- * CONFLICT's WHERE clause.
- */
predExprs = RelationGetIndexPredicate(idxRel);
if (predExprs && varno != 1)
ChangeVarNodes((Node *) predExprs, 1, varno, 0);
- if (!predicate_implied_by(predExprs, (List *) onconflict->arbiterWhere, false))
- goto next;
+ /*
+ * If it's a partial index and conventional inference, its predicate
+ * must be implied by the ON CONFLICT's WHERE clause.
+ */
+ if (indexOidFromConstraint == InvalidOid &&
+ !predicate_implied_by(predExprs, inferIndexExprs, false))
+ continue;
+ /*
+ * If it's a partial index and named constraint predicates must be
+ * equal.
+ */
+ if (indexOidFromConstraint != InvalidOid &&
+ list_difference(predExprs, inferIndexExprs) != NIL)
+ continue;
+
+ /* Consider this a match */
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
-next:
+ }
+
+ /* Close all indexes */
+ foreach_ptr(RelationData, idxRel, indexRelList)
+ {
index_close(idxRel, NoLock);
}
list_free(indexList);
+ list_free(indexRelList);
table_close(relation, NoLock);
/* We require at least one indisvalid index */
diff --git a/src/backend/parser/parse_clause.c b/src/backend/parser/parse_clause.c
index ca26f6f61f2..bee9860c513 100644
--- a/src/backend/parser/parse_clause.c
+++ b/src/backend/parser/parse_clause.c
@@ -3277,11 +3277,11 @@ resolve_unique_index_expr(ParseState *pstate, InferClause *infer,
* Raw grammar re-uses CREATE INDEX infrastructure for unique index
* inference clause, and so will accept opclasses by name and so on.
*
- * Make no attempt to match ASC or DESC ordering or NULLS FIRST/NULLS
- * LAST ordering, since those are not significant for inference
- * purposes (any unique index matching the inference specification in
- * other regards is accepted indifferently). Actively reject this as
- * wrong-headed.
+ * Make no attempt to match ASC or DESC ordering, NULLS FIRST/NULLS
+ * LAST ordering or opclass options, since those are not significant
+ * for inference purposes (any unique index matching the inference
+ * specification in other regards is accepted indifferently). Actively
+ * reject this as wrong-headed.
*/
if (ielem->ordering != SORTBY_DEFAULT)
ereport(ERROR,
@@ -3295,6 +3295,12 @@ resolve_unique_index_expr(ParseState *pstate, InferClause *infer,
errmsg("NULLS FIRST/LAST is not allowed in ON CONFLICT clause"),
parser_errposition(pstate,
exprLocation((Node *) infer))));
+ if (ielem->opclassopts)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_COLUMN_REFERENCE),
+ errmsg("operator class options are not allowed in ON CONFLICT clause"),
+ parser_errposition(pstate,
+ exprLocation((Node *) infer)));
if (!ielem->expr)
{
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index 7b3c0c4b716..0a9716db27c 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -19,6 +19,7 @@ ISOLATION = basic \
syscache-update-pruned \
index-concurrently-upsert \
reindex-concurrently-upsert \
+ reindex-concurrently-upsert-on-constraint \
index-concurrently-upsert-predicate
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out b/src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out
new file mode 100644
index 00000000000..c1ac1f77c61
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out
@@ -0,0 +1,238 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex:
+ REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+ <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex:
+ REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+ <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex:
+ REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+ <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 485b483e3ca..0706cd3d6e9 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -51,6 +51,7 @@ tests += {
'index-concurrently-upsert',
'reindex-concurrently-upsert',
'index-concurrently-upsert-predicate',
+ 'reindex-concurrently-upsert-on-constraint',
],
'runningcheck': false, # see syscache-update-pruned
# Some tests wait for all snapshots, so avoid parallel execution
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec
new file mode 100644
index 00000000000..8126256460c
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec
@@ -0,0 +1,110 @@
+# Test race conditions involving:
+#
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: concurrently REINDEX the primary key
+#
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ ALTER TABLE test.tbl SET (parallel_workers=0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup
+{
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+}
+step s1_start_upsert
+{
+ INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+}
+
+session s2
+setup
+{
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+}
+step s2_start_upsert
+{
+ INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+}
+
+session s3
+setup
+{
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead
+{
+ SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+}
+step s3_setup_wait_before_swap
+{
+ SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+}
+step s3_start_reindex
+{
+ REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+}
+
+session s4
+step s4_wakeup_to_swap
+{
+ SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
+}
+step s4_wakeup_s1
+{
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+}
+step s4_wakeup_s2
+{
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+}
+step s4_wakeup_to_set_dead
+{
+ SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
--
2.47.3
Please also check my (very raw) proposal in the thread "Revisiting
{CREATE INDEX, REINDEX} CONCURRENTLY improvements"
TL;DR - use logical decoding for adding index entries for tuples added
during CIC.
Maybe this also makes the issue with ON CONFLICT UPDATE and REINDEX
CONCURRENTLY go away.
Show quoted text
On Fri, Nov 28, 2025 at 6:30 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
Hello,
Here's a slightly different approach for the fix proposed in your 0003.
I wasn't happy with the idea of opening all indexes twice in
infer_arbiter_indexes(), so I instead made it collect all Relations from
those indexes in an initial loop, then process them in the two places
that wanted them, and we close them all again together. I think this
also makes the code clearer. We no longer have the "next" goto label to
close the index at the bottom of the loop, but instead we can just do
"continue" cleanly.I also rewrote some comments. I may not have done all the edits I
wanted, but ran out of time today and I think this is in pretty good
shape.I tried under CATCACHE_FORCE_RELEASE and saw no problems.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Hello!
On Fri, Nov 28, 2025 at 6:30 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
I wasn't happy with the idea of opening all indexes twice in
infer_arbiter_indexes(), so I instead made it collect all Relations from
those indexes in an initial loop, then process them in the two places
that wanted them, and we close them all again together. I think this
also makes the code clearer. We no longer have the "next" goto label to
close the index at the bottom of the loop, but instead we can just do
"continue" cleanly.
Yes, agreed - that looks better than my version.
Few moments:
Second, if an attribute list was specified in the ON
* CONFLICT clause, we use the list to find the indexes whose attributes
* match that list.
I think we may notice expressions and predicates also.
/*
* Find the named constraint index to extract its attributes and
* predicates.
*/
foreach_ptr(RelationData, idxRel, indexRelList)
Should we consider assert to ensure we have actually found something?
Best regards,
Mikhail.
Hi!
On Fri, Nov 28, 2025 at 7:09 PM Hannu Krosing <hannuk@google.com> wrote:
Please also check my (very raw) proposal in the thread "Revisiting
{CREATE INDEX, REINDEX} CONCURRENTLY improvements"TL;DR - use logical decoding for adding index entries for tuples added
during CIC.
No, the current issue is caused by another reason - it is more about
switching from old to new index.
Best regards,
Mikhail.
Also, I think it is good to mention changes in parse_clause.c in the
commit message.
On 2025-11-28, Mihail Nikalayeu wrote:
Also, I think it is good to mention changes in parse_clause.c in the
commit message.
Oh rats, no, thanks for noticing, that's a mostly unrelated change that I intend to commit separately, mentioned in another thread, that I happened to merge here accidentally.
--
Álvaro Herrera
Hi Mihail,
Looking at 0004, I think IsIndexCompatibleAsArbiter() should map the
attribute numbers, in case the partition has a different column layout
than the parent (e.g. in case there are dropped columns or just
different column orders)
Regards
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"We have labored long to build a heaven, only to find it
populated with horrors" (Prof. Milton Glass)
Hello!
On Sat, Nov 29, 2025 at 6:48 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
Looking at 0004, I think IsIndexCompatibleAsArbiter() should map the
attribute numbers, in case the partition has a different column layout
than the parent (e.g. in case there are dropped columns or just
different column orders)
Oh, yes, you're right. I'll prepare an updated version (also it looks
like some inner loops may be refactored in a more effective way).
Best regards,
Mikhail.
Hello, Álvaro!
On Sun, Nov 30, 2025 at 1:11 PM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:
On Sat, Nov 29, 2025 at 6:48 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
Looking at 0004, I think IsIndexCompatibleAsArbiter() should map the
attribute numbers, in case the partition has a different column layout
than the parent (e.g. in case there are dropped columns or just
different column orders)Oh, yes, you're right. I'll prepare an updated version (also it looks
like some inner loops may be refactored in a more effective way).
I was wrong, function is called only for indexes from the same
relation (actual partition).
But anyway I reworked the commit - it is much clearer now (and a
little bit more effective).
Best regards,
Mikhail.
Attachments:
v15-0005-Revert-Doc-cover-index-CONCURRENTLY-causing-erro.patchtext/plain; charset=US-ASCII; name=v15-0005-Revert-Doc-cover-index-CONCURRENTLY-causing-erro.patchDownload
From 13735733ad82bd7e3963140e6856e458d8eb7156 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sun, 30 Nov 2025 16:49:20 +0100
Subject: [PATCH v15 2/2] Revert "Doc: cover index CONCURRENTLY causing errors
in INSERT ... ON CONFLICT."
This reverts commit 8b18ed6dfbb8b3e4483801b513fea6b429140569.
---
doc/src/sgml/ref/insert.sgml | 9 ---------
src/backend/optimizer/util/plancat.c | 5 -----
2 files changed, 14 deletions(-)
diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml
index 0598b8dea34..04962e39e12 100644
--- a/doc/src/sgml/ref/insert.sgml
+++ b/doc/src/sgml/ref/insert.sgml
@@ -594,15 +594,6 @@ INSERT INTO <replaceable class="parameter">table_name</replaceable> [ AS <replac
</para>
</tip>
- <warning>
- <para>
- While <command>CREATE INDEX CONCURRENTLY</command> or <command>REINDEX
- CONCURRENTLY</command> is running on a unique index, <command>INSERT
- ... ON CONFLICT</command> statements on the same table may unexpectedly
- fail with a unique violation.
- </para>
- </warning>
-
</refsect2>
</refsect1>
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 7af9a2064e3..d5325f0f732 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -789,11 +789,6 @@ find_relation_notnullatts(PlannerInfo *root, Oid relid)
* the purposes of inference. If no opclass (or collation) is specified, then
* all matching indexes (that may or may not match the default in terms of
* each attribute opclass/collation) are used for inference.
- *
- * Note: during index CONCURRENTLY operations, different transactions may
- * reference different sets of arbiter indexes. This can lead to false unique
- * constraint violations that wouldn't occur during normal operations. For
- * more information, see insert.sgml.
*/
List *
infer_arbiter_indexes(PlannerInfo *root)
--
2.43.0
v15-0004-Modify-the-ExecInitPartitionInfo-function-to-con.patchtext/plain; charset=US-ASCII; name=v15-0004-Modify-the-ExecInitPartitionInfo-function-to-con.patchDownload
From 53e41c5bdd89dbaf5022f59169d0de483f82babc Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sun, 30 Nov 2025 16:48:55 +0100
Subject: [PATCH v15 1/2] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY
as arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
---
src/backend/executor/execPartition.c | 127 +++++++++-
src/test/modules/injection_points/Makefile | 3 +-
...eindex-concurrently-upsert-partitioned.out | 232 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 1 +
...index-concurrently-upsert-partitioned.spec | 96 ++++++++
5 files changed, 446 insertions(+), 13 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
create mode 100644 src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 0dcce181f09..bd3527393ec 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -15,6 +15,7 @@
#include "access/table.h"
#include "access/tableam.h"
+#include "catalog/index.h"
#include "catalog/partition.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -490,6 +491,62 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Only indexes of the same relation are supported.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ Assert(arbiterIndexRelation->rd_index->indrelid == indexRelation->rd_index->indrelid);
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+
+ if (arbiterIndexInfo->ii_NullsNotDistinct != indexInfo->ii_NullsNotDistinct)
+ return false;
+
+ /* and same number of key attributes */
+ if (arbiterIndexInfo->ii_NumIndexKeyAttrs != indexInfo->ii_NumIndexKeyAttrs)
+ return false;
+
+ /* No support currently for comparing exclusion indexes. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+
+ for (i = 0; i < arbiterIndexInfo->ii_NumIndexKeyAttrs; i++)
+ {
+ if (arbiterIndexRelation->rd_indcollation[i] != indexRelation->rd_indcollation[i])
+ return false;
+
+ if (arbiterIndexRelation->rd_opfamily[i] != indexRelation->rd_opfamily[i])
+ return false;
+
+ if (arbiterIndexRelation->rd_index->indkey.values[i] != indexRelation->rd_index->indkey.values[i])
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -690,7 +747,9 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
TupleDesc partrelDesc = RelationGetDescr(partrel);
ExprContext *econtext = mtstate->ps.ps_ExprContext;
ListCell *lc;
- List *arbiterIndexes = NIL;
+ List *arbiterIndexes = NIL,
+ *arbiterIndexesOffset = NIL;
+ int additional_arbiters = 0;
/*
* If there is a list of arbiter indexes, map it to a list of indexes
@@ -698,36 +757,80 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
* list and searching for ancestry relationships to each index in the
* ancestor table.
*/
- if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
- {
- List *childIdxs;
+ if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL) {
+ List *childIdxs,
+ *nonAncestorIdxOffset = NIL;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
foreach(lc, childIdxs)
{
Oid childIdx = lfirst_oid(lc);
+ int i = foreach_current_index(lc);
List *ancestors;
- ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach_oid(oid, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, oid))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ arbiterIndexesOffset = lappend_int(arbiterIndexesOffset, i);
+ }
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxOffset = lappend_int(nonAncestorIdxOffset, i);
+
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxOffset && arbiterIndexesOffset)
+ {
+ foreach_int(childIdxOffset, nonAncestorIdxOffset)
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[childIdxOffset];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[childIdxOffset];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to use non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ foreach_int(arbiterIdxOffset, arbiterIndexesOffset)
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[arbiterIdxOffset];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[arbiterIdxOffset];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxOffset);
}
/*
* If the resulting lists are of inequal length, something is wrong.
- * XXX This may happen because we don't match the lists correctly when
- * a partitioned index is being processed by REINDEX CONCURRENTLY.
- * FIXME later.
+ * But we need to consider additional arbiter indexes also.
*/
if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
+ list_length(arbiterIndexes) - additional_arbiters)
elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index 7b3c0c4b716..e10475d2665 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -19,7 +19,8 @@ ISOLATION = basic \
syscache-update-pruned \
index-concurrently-upsert \
reindex-concurrently-upsert \
- index-concurrently-upsert-predicate
+ index-concurrently-upsert-predicate \
+ reindex-concurrently-upsert-partitioned
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out b/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
new file mode 100644
index 00000000000..588ffbd57a5
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 485b483e3ca..3e06220fe31 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -51,6 +51,7 @@ tests += {
'index-concurrently-upsert',
'reindex-concurrently-upsert',
'index-concurrently-upsert-predicate',
+ 'reindex-concurrently-upsert-partitioned'
],
'runningcheck': false, # see syscache-update-pruned
# Some tests wait for all snapshots, so avoid parallel execution
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
new file mode 100644
index 00000000000..c8a8eb9cca5
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
@@ -0,0 +1,96 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
Hello Alvaro,
27.11.2025 14:32, Álvaro Herrera wrote:
Thanks for reviewing. I have pushed it now. Looking at the next one.
I've noticed two failures from skink you could find interesting: [1]https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2025-11-25%2021%3A55%3A00, [2]https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2025-11-29%2021%3A51%3A13.
The difference from [2]https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2025-11-29%2021%3A51%3A13:
ok 3 - syscache-update-pruned 94198 ms
not ok 4 - index-concurrently-upsert 14008 ms
ok 5 - reindex-concurrently-upsert 14379 ms
--- /home/bf/bf-build/skink-master/HEAD/pgsql/src/test/modules/injection_points/expected/index-concurrently-upsert.out
2025-11-27 13:38:19.513528475 +0100
+++
/home/bf/bf-build/skink-master/HEAD/pgsql.build/testrun/injection_points/isolation/results/index-concurrently-upsert.out
2025-11-30 00:10:01.697938769 +0100
@@ -107,6 +107,7 @@
(1 row)
s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s1_start_upsert: <... completed>
step s2_start_upsert: <... completed>
step s3_start_create_index: <... completed>
I've managed to reproduce something similar to this diff when running
multiple test instances under Valgrind with parallel. With the attached
patch applied:
for i in {1..40}; do cp -r src/test/modules/injection_points/ src/test/modules/injection_points_$i/; sed -i.bak
"s|src/test/modules/injection_points|src/test/modules/injection_points_$i|"
src/test/modules/injection_points_$i/Makefile; done
make -s check -C src/test/modules/injection_points
and/tmp/extra.config containing:
log_min_messages = INFO
backtrace_functions = 'injection_notice'
for i in {1..10}; do np=5; echo "ITERATION $i"; parallel -j40 --linebuffer --tag PROVE_TESTS="t/099*" NO_TEMP_INSTALL=1
TEMP_CONFIG=/tmp/extra.config make -s check -C src/test/modules/injection_points_{} ::: `seq $np` || break; done
gives me:
ITERATION 1
...
4 # Failed test 'regression tests pass'
4 # at t/099_isolation_regress.pl line 52.
4 # got: '256'
4 # expected: '0'
src/test/modules/injection_points_4/tmp_check/log/regress_log_099_isolation_regress contains:
...
ok 11 - index-concurrently-upsert 5282 ms
not ok 12 - index-concurrently-upsert 6347 ms
ok 13 - index-concurrently-upsert 5723 ms
...
--- .../src/test/modules/injection_points_4/expected/index-concurrently-upsert.out 2025-11-30 14:24:29.385133831 +0200
+++ .../src/test/modules/injection_points_4/tmp_check/results/index-concurrently-upsert.out 2025-11-30
16:22:29.168920744 +0200
@@ -78,6 +78,7 @@
(1 row)
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s4_wakeup_s2:
SELECT injection_points_detach('exec-insert-before-insert-speculative');
SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
=== EOF ===
I can see in the following stack trace for this extra notice:
2025-11-30 16:22:28.465 EET|law|isolation_regression|692c531f.bb652|NOTICE: notice triggered for injection point
pre-invalidate-catalog-snapshot-end
2025-11-30 16:22:28.465 EET|law|isolation_regression|692c531f.bb652|BACKTRACE:
injection_notice at injection_points.c:278:3
InjectionPointRun at injection_point.c:555:1
InvalidateCatalogSnapshot at snapmgr.c:463:3
LocalExecuteInvalidationMessage at inval.c:831:4
ReceiveSharedInvalidMessages at sinval.c:113:18
AcceptInvalidationMessages at inval.c:970:23
LockRelationOid at lmgr.c:137:3
relation_open at relation.c:58:6
table_open at table.c:44:6
SearchCatCacheMiss at catcache.c:1550:13
SearchCatCacheInternal at catcache.c:1495:9
SearchCatCache1 at catcache.c:1368:1
SearchSysCache1 at syscache.c:227:1
build_coercion_expression at parse_coerce.c:852:8
coerce_type at parse_coerce.c:433:13
coerce_to_target_type at parse_coerce.c:105:11
transformAssignedExpr at parse_target.c:580:4
transformInsertRow at analyze.c:1121:10
transformInsertStmt at analyze.c:988:14
transformStmt at analyze.c:364:13
transformOptionalSelectInto at analyze.c:317:1
transformTopLevelStmt at analyze.c:266:11
parse_analyze_fixedparams at analyze.c:134:10
pg_analyze_and_rewrite_fixedparams at postgres.c:687:10
exec_simple_query at postgres.c:1195:20
PostgresMain at postgres.c:4777:6
BackendInitialize at backend_startup.c:142:1
postmaster_child_launch at launch_backend.c:269:3
BackendStartup at postmaster.c:3598:8
ServerLoop at postmaster.c:1716:10
PostmasterMain at postmaster.c:1403:11
main at main.c:236:2
I also observed:
2025-11-30 15:33:45.746 EET|law|isolation_regression|692c47b4.675e7|NOTICE: notice triggered for injection point
pre-invalidate-catalog-snapshot-end
2025-11-30 15:33:45.746 EET|law|isolation_regression|692c47b4.675e7|BACKTRACE:
injection_notice at injection_points.c:278:3
InjectionPointRun at injection_point.c:555:1
InvalidateCatalogSnapshot at snapmgr.c:463:3
LocalExecuteInvalidationMessage at inval.c:831:4
ReceiveSharedInvalidMessages at sinval.c:113:18
AcceptInvalidationMessages at inval.c:970:23
LockRelationOid at lmgr.c:137:3
relation_open at relation.c:58:6
table_open at table.c:44:6
CatalogCacheInitializeCache at catcache.c:1131:13
ConditionalCatalogCacheInitializeCache at catcache.c:1092:1
SearchCatCacheInternal at catcache.c:1424:15
SearchCatCache1 at catcache.c:1368:1
SearchSysCache1 at syscache.c:227:1
check_enable_rls at rls.c:66:10
get_row_security_policies at rowsecurity.c:130:15
fireRIRrules at rewriteHandler.c:2218:3
QueryRewrite at rewriteHandler.c:4581:11
pg_rewrite_query at postgres.c:822:20
pg_analyze_and_rewrite_fixedparams at postgres.c:696:19
exec_simple_query at postgres.c:1195:20
PostgresMain at postgres.c:4777:6
BackendInitialize at backend_startup.c:142:1
postmaster_child_launch at launch_backend.c:269:3
BackendStartup at postmaster.c:3598:8
ServerLoop at postmaster.c:1716:10
PostmasterMain at postmaster.c:1403:11
main at main.c:236:2
Could you please look if this can be fixed?
[1]: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2025-11-25%2021%3A55%3A00
[2]: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2025-11-29%2021%3A51%3A13
Best regards,
Alexander
Attachments:
injection_points-tests-as-TAP.patchtext/x-patch; charset=UTF-8; name=injection_points-tests-as-TAP.patchDownload
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index 7b3c0c4b716..46566937d81 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -11,10 +11,10 @@ EXTENSION = injection_points
DATA = injection_points--1.0.sql
PGFILEDESC = "injection_points - facility for injection points"
-REGRESS = injection_points hashagg reindex_conc vacuum
+XREGRESS = injection_points hashagg reindex_conc vacuum
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
-ISOLATION = basic \
+XISOLATION = basic \
inplace \
syscache-update-pruned \
index-concurrently-upsert \
@@ -28,6 +28,9 @@ NO_INSTALLCHECK = 1
export enable_injection_points
+REGRESS_SHLIB=$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX)
+export REGRESS_SHLIB
+
ifdef USE_PGXS
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
diff --git a/src/test/modules/injection_points/isolation_schedule b/src/test/modules/injection_points/isolation_schedule
new file mode 100644
index 00000000000..9b71d065309
--- /dev/null
+++ b/src/test/modules/injection_points/isolation_schedule
@@ -0,0 +1,23 @@
+test: basic
+test: inplace
+test: syscache-update-pruned
+test: index-concurrently-upsert
+test: index-concurrently-upsert
+test: index-concurrently-upsert
+test: index-concurrently-upsert
+test: index-concurrently-upsert
+test: index-concurrently-upsert
+test: index-concurrently-upsert
+test: index-concurrently-upsert
+test: index-concurrently-upsert
+test: index-concurrently-upsert
+test: reindex-concurrently-upsert
+test: reindex-concurrently-upsert
+test: reindex-concurrently-upsert
+test: reindex-concurrently-upsert
+test: reindex-concurrently-upsert
+test: reindex-concurrently-upsert
+test: reindex-concurrently-upsert
+test: reindex-concurrently-upsert
+test: reindex-concurrently-upsert
+test: reindex-concurrently-upsert
diff --git a/src/test/modules/injection_points/t/099_isolation_regress.pl b/src/test/modules/injection_points/t/099_isolation_regress.pl
new file mode 100644
index 00000000000..047530c3d6a
--- /dev/null
+++ b/src/test/modules/injection_points/t/099_isolation_regress.pl
@@ -0,0 +1,55 @@
+
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+# Run the standard regression tests with streaming replication
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+use File::Basename;
+
+# Initialize primary node
+my $node_primary = PostgreSQL::Test::Cluster->new('primary');
+$node_primary->init();
+
+# Increase some settings that Cluster->new makes too low by default.
+$node_primary->adjust_conf('postgresql.conf', 'max_connections', '25');
+$node_primary->append_conf('postgresql.conf',
+ 'max_prepared_transactions = 10');
+
+$node_primary->start;
+
+my $dlpath = dirname($ENV{REGRESS_SHLIB});
+my $outputdir = $PostgreSQL::Test::Utils::tmp_check;
+
+# Run the regression tests against the primary.
+my $extra_opts = $ENV{EXTRA_REGRESS_OPTS} || "";
+
+my $rc =
+ system( ($ENV{PG_REGRESS} =~ s/regress\/pg_regress/isolation\/pg_isolation_regress/r)
+ . " $extra_opts "
+ . "--dlpath=\"$dlpath\" "
+ . "--bindir= "
+ . "--host="
+ . $node_primary->host . " "
+ . "--port="
+ . $node_primary->port . " "
+ . "--schedule=./isolation_schedule "
+ . "--inputdir=./ "
+ . "--outputdir=\"$outputdir\"");
+if ($rc != 0)
+{
+ # Dump out the regression diffs file, if there is one
+ my $diffs = "$outputdir/regression.diffs";
+ if (-e $diffs)
+ {
+ print "=== dumping $diffs ===\n";
+ print slurp_file($diffs);
+ print "=== EOF ===\n";
+ }
+}
+is($rc, 0, 'regression tests pass');
+
+
+done_testing();
Hello!
On Sun, Nov 30, 2025 at 6:00 PM Alexander Lakhin <exclusion@gmail.com> wrote:
s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
+s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s1_start_upsert: <... completed>
step s2_start_upsert: <... completed>
step s3_start_create_index: <... completed>
Oh, I'm afraid it may become an endless fight....
I think it is better to implement something in isolationtester to
natively support the case from [0]/messages/by-id/CADzfLwUc=jtSUEaQCtyt8zTeOJ-gHZ8=w_KJsVjDOYSLqaY9Lg@mail.gmail.com (in such case we don't need NOTICE
on .pre-invalidate-catalog-snapshot-end).
Best regards,
Mikhail.
[0]: /messages/by-id/CADzfLwUc=jtSUEaQCtyt8zTeOJ-gHZ8=w_KJsVjDOYSLqaY9Lg@mail.gmail.com
Hello!
On Sun, Nov 30, 2025 at 6:26 PM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:
I think it is better to implement something in isolationtester to
natively support the case from [0] (in such case we don't need NOTICE
on .pre-invalidate-catalog-snapshot-end).
I think I have implemented a better solution - without any additional NOTICE.
It just actually waits for the other backend to hang on the injection point.
Attached (together with other changes).
Attachments:
v16-0002-Revert-Doc-cover-index-CONCURRENTLY-causing-erro.patchtext/plain; charset=US-ASCII; name=v16-0002-Revert-Doc-cover-index-CONCURRENTLY-causing-erro.patchDownload
From 5dc3e4eb50e445a291a13663fc9ce93d0db96b1c Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sun, 30 Nov 2025 16:49:20 +0100
Subject: [PATCH v16 2/3] Revert "Doc: cover index CONCURRENTLY causing errors
in INSERT ... ON CONFLICT."
This reverts commit 8b18ed6dfbb8b3e4483801b513fea6b429140569.
---
doc/src/sgml/ref/insert.sgml | 9 ---------
src/backend/optimizer/util/plancat.c | 5 -----
2 files changed, 14 deletions(-)
diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml
index 0598b8dea34..04962e39e12 100644
--- a/doc/src/sgml/ref/insert.sgml
+++ b/doc/src/sgml/ref/insert.sgml
@@ -594,15 +594,6 @@ INSERT INTO <replaceable class="parameter">table_name</replaceable> [ AS <replac
</para>
</tip>
- <warning>
- <para>
- While <command>CREATE INDEX CONCURRENTLY</command> or <command>REINDEX
- CONCURRENTLY</command> is running on a unique index, <command>INSERT
- ... ON CONFLICT</command> statements on the same table may unexpectedly
- fail with a unique violation.
- </para>
- </warning>
-
</refsect2>
</refsect1>
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 7af9a2064e3..d5325f0f732 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -789,11 +789,6 @@ find_relation_notnullatts(PlannerInfo *root, Oid relid)
* the purposes of inference. If no opclass (or collation) is specified, then
* all matching indexes (that may or may not match the default in terms of
* each attribute opclass/collation) are used for inference.
- *
- * Note: during index CONCURRENTLY operations, different transactions may
- * reference different sets of arbiter indexes. This can lead to false unique
- * constraint violations that wouldn't occur during normal operations. For
- * more information, see insert.sgml.
*/
List *
infer_arbiter_indexes(PlannerInfo *root)
--
2.43.0
v16-0003-Refactor-test-to-avoid-any-additional-injection-.patchtext/plain; charset=US-ASCII; name=v16-0003-Refactor-test-to-avoid-any-additional-injection-.patchDownload
From a09dcb5f2a0fb5801f205d4f5e0448ddbe064d38 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Mon, 1 Dec 2025 12:14:13 +0100
Subject: [PATCH v16 3/3] Refactor test to avoid any additional injection
points and notice.
---
src/backend/utils/time/snapmgr.c | 1 -
.../index-concurrently-upsert-predicate.out | 8 +-
.../index-concurrently-upsert-predicate_1.out | 116 ------------------
.../expected/index-concurrently-upsert.out | 14 +--
.../expected/index-concurrently-upsert_1.out | 116 ------------------
.../index-concurrently-upsert-predicate.spec | 31 ++++-
.../specs/index-concurrently-upsert.spec | 36 +++++-
7 files changed, 64 insertions(+), 258 deletions(-)
delete mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
delete mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 434abbf6b6f..24f73a49d27 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -459,7 +459,6 @@ InvalidateCatalogSnapshot(void)
pairingheap_remove(&RegisteredSnapshots, &CatalogSnapshot->ph_node);
CatalogSnapshot = NULL;
SnapshotResetXmin();
- INJECTION_POINT("pre-invalidate-catalog-snapshot-end", NULL);
INJECTION_POINT("invalidate-catalog-snapshot-end", NULL);
}
}
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
index 5a57cd5fa40..11b23e8721f 100644
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
+++ b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
@@ -1,6 +1,6 @@
Parsed test spec with 5 sessions
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
injection_points_attach
-----------------------
@@ -37,16 +37,12 @@ case
(1 row)
-step s5_noop:
- <waiting ...>
step s3_start_create_index:
CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
<waiting ...>
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s1_start_upsert:
INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
<waiting ...>
-step s5_noop: <... completed>
step s4_wakeup_define_index_before_set_valid:
SELECT injection_points_detach('define-index-before-set-valid');
SELECT injection_points_wakeup('define-index-before-set-valid');
@@ -65,6 +61,7 @@ step s2_start_upsert:
INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
<waiting ...>
step s5_wakeup_s1_from_invalidate_catalog_snapshot:
+ CALL test.wait_for_injection_point();
SELECT injection_points_detach('invalidate-catalog-snapshot-end');
SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
@@ -106,7 +103,6 @@ injection_points_wakeup
(1 row)
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s1_start_upsert: <... completed>
step s2_start_upsert: <... completed>
step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
deleted file mode 100644
index d453dd62617..00000000000
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
+++ /dev/null
@@ -1,116 +0,0 @@
-Parsed test spec with 5 sessions
-
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
-step s1_attach_invalidate_catalog_snapshot:
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
- <waiting ...>
-step s4_wakeup_s1_setup:
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-
-case
-----
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot: <... completed>
-injection_points_attach
------------------------
-
-(1 row)
-
-step s5_noop:
- <waiting ...>
-step s3_start_create_index:
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
- <waiting ...>
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_noop: <... completed>
-step s4_wakeup_define_index_before_set_valid:
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_wakeup_s1_from_invalidate_catalog_snapshot:
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert.out b/src/test/modules/injection_points/expected/index-concurrently-upsert.out
index 97386a35bed..7bcf8d34053 100644
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert.out
+++ b/src/test/modules/injection_points/expected/index-concurrently-upsert.out
@@ -1,6 +1,6 @@
Parsed test spec with 5 sessions
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
injection_points_attach
-----------------------
@@ -27,8 +27,8 @@ injection_points_attach
step s4_wakeup_s1_setup:
SELECT CASE WHEN
(SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
+ WHERE wait_event_type = 'InjectionPoint' AND
+ wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
END;
@@ -37,16 +37,12 @@ case
(1 row)
-step s5_noop:
- <waiting ...>
step s3_start_create_index:
CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
<waiting ...>
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+ INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
<waiting ...>
-step s5_noop: <... completed>
step s4_wakeup_define_index_before_set_valid:
SELECT injection_points_detach('define-index-before-set-valid');
SELECT injection_points_wakeup('define-index-before-set-valid');
@@ -65,6 +61,7 @@ step s2_start_upsert:
INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
<waiting ...>
step s5_wakeup_s1_from_invalidate_catalog_snapshot:
+ CALL test.wait_for_injection_point();
SELECT injection_points_detach('invalidate-catalog-snapshot-end');
SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
@@ -106,7 +103,6 @@ injection_points_wakeup
(1 row)
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
step s1_start_upsert: <... completed>
step s2_start_upsert: <... completed>
step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out b/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
deleted file mode 100644
index 4bd51b2b511..00000000000
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
+++ /dev/null
@@ -1,116 +0,0 @@
-Parsed test spec with 5 sessions
-
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s5_noop s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
-step s1_attach_invalidate_catalog_snapshot:
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
- <waiting ...>
-step s4_wakeup_s1_setup:
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-
-case
-----
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot: <... completed>
-injection_points_attach
------------------------
-
-(1 row)
-
-step s5_noop:
- <waiting ...>
-step s3_start_create_index:
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
- <waiting ...>
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_noop: <... completed>
-step s4_wakeup_define_index_before_set_valid:
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_wakeup_s1_from_invalidate_catalog_snapshot:
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-s1: NOTICE: notice triggered for injection point pre-invalidate-catalog-snapshot-end
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec b/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
index 1cbcaa6963f..34016b93bb0 100644
--- a/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
+++ b/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
@@ -27,7 +27,6 @@ setup
{
SELECT injection_points_set_local();
SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
- SELECT injection_points_attach('pre-invalidate-catalog-snapshot-end', 'notice');
}
step s1_attach_invalidate_catalog_snapshot
{
@@ -91,19 +90,43 @@ step s4_wakeup_define_index_before_set_valid
}
session s5
-step s5_noop
-{
+setup {
+ CREATE OR REPLACE PROCEDURE test.wait_for_injection_point()
+ LANGUAGE plpgsql
+ AS $$
+ DECLARE
+ v_waiting_pid INTEGER;
+ BEGIN
+ SELECT pid INTO v_waiting_pid
+ FROM pg_stat_activity
+ WHERE wait_event_type = 'InjectionPoint'
+ AND wait_event = 'invalidate-catalog-snapshot-end'
+ LIMIT 1;
+
+ IF v_waiting_pid IS NOT NULL THEN
+ RETURN;
+ END IF;
+
+ PERFORM pg_sleep(100);
+
+ CALL test.wait_for_injection_point();
+ END;
+ $$;
}
step s5_wakeup_s1_from_invalidate_catalog_snapshot
{
+ CALL test.wait_for_injection_point();
SELECT injection_points_detach('invalidate-catalog-snapshot-end');
SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
}
+teardown
+{
+ DROP PROCEDURE test.wait_for_injection_point;
+}
permutation
s1_attach_invalidate_catalog_snapshot
s4_wakeup_s1_setup
- s5_noop(s1_start_upsert notices 1)
s3_start_create_index(s1_start_upsert, s2_start_upsert)
s1_start_upsert
s4_wakeup_define_index_before_set_valid
diff --git a/src/test/modules/injection_points/specs/index-concurrently-upsert.spec b/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
index 2a6d888dcea..92fc0933e91 100644
--- a/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
+++ b/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
@@ -26,7 +26,6 @@ setup
{
SELECT injection_points_set_local();
SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
- SELECT injection_points_attach('pre-invalidate-catalog-snapshot-end', 'notice');
}
step s1_attach_invalidate_catalog_snapshot
{
@@ -34,7 +33,7 @@ step s1_attach_invalidate_catalog_snapshot
}
step s1_start_upsert
{
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+ INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
}
session s2
@@ -68,8 +67,8 @@ step s4_wakeup_s1_setup
{
SELECT CASE WHEN
(SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
+ WHERE wait_event_type = 'InjectionPoint' AND
+ wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
END;
}
@@ -90,19 +89,44 @@ step s4_wakeup_define_index_before_set_valid
}
session s5
-step s5_noop
+setup
{
+ CREATE OR REPLACE PROCEDURE test.wait_for_injection_point()
+ LANGUAGE plpgsql
+ AS $$
+ DECLARE
+ v_waiting_pid INTEGER;
+ BEGIN
+ SELECT pid INTO v_waiting_pid
+ FROM pg_stat_activity
+ WHERE wait_event_type = 'InjectionPoint'
+ AND wait_event = 'invalidate-catalog-snapshot-end'
+ LIMIT 1;
+
+ IF v_waiting_pid IS NOT NULL THEN
+ RETURN;
+ END IF;
+
+ PERFORM pg_sleep(100);
+
+ CALL test.wait_for_injection_point();
+ END;
+ $$;
}
step s5_wakeup_s1_from_invalidate_catalog_snapshot
{
+ CALL test.wait_for_injection_point();
SELECT injection_points_detach('invalidate-catalog-snapshot-end');
SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
}
+teardown
+{
+ DROP PROCEDURE test.wait_for_injection_point;
+}
permutation
s1_attach_invalidate_catalog_snapshot
s4_wakeup_s1_setup
- s5_noop(s1_start_upsert notices 1)
s3_start_create_index(s1_start_upsert, s2_start_upsert)
s1_start_upsert
s4_wakeup_define_index_before_set_valid
--
2.43.0
v16-0001-Modify-the-ExecInitPartitionInfo-function-to-con.patchtext/plain; charset=US-ASCII; name=v16-0001-Modify-the-ExecInitPartitionInfo-function-to-con.patchDownload
From af5e27d4150dd53d313122c02da7ce4d3c07f332 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sun, 30 Nov 2025 16:48:55 +0100
Subject: [PATCH v16 1/3] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY
as arbiters as well.
This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
---
src/backend/executor/execPartition.c | 127 +++++++++-
src/test/modules/injection_points/Makefile | 3 +-
...eindex-concurrently-upsert-partitioned.out | 232 ++++++++++++++++++
src/test/modules/injection_points/meson.build | 1 +
...index-concurrently-upsert-partitioned.spec | 96 ++++++++
5 files changed, 446 insertions(+), 13 deletions(-)
create mode 100644 src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
create mode 100644 src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index 0dcce181f09..bd3527393ec 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -15,6 +15,7 @@
#include "access/table.h"
#include "access/tableam.h"
+#include "catalog/index.h"
#include "catalog/partition.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -490,6 +491,62 @@ ExecFindPartition(ModifyTableState *mtstate,
return rri;
}
+/*
+ * IsIndexCompatibleAsArbiter
+ * Checks if the indexes are identical in terms of being used
+ * as arbiters for the INSERT ON CONFLICT operation by comparing
+ * them to the provided arbiter index.
+ *
+ * Only indexes of the same relation are supported.
+ *
+ * Returns the true if indexes are compatible.
+ */
+static bool
+IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation,
+ IndexInfo *arbiterIndexInfo,
+ Relation indexRelation,
+ IndexInfo *indexInfo)
+{
+ int i;
+
+ Assert(arbiterIndexRelation->rd_index->indrelid == indexRelation->rd_index->indrelid);
+
+ if (arbiterIndexInfo->ii_Unique != indexInfo->ii_Unique)
+ return false;
+
+ if (arbiterIndexInfo->ii_NullsNotDistinct != indexInfo->ii_NullsNotDistinct)
+ return false;
+
+ /* and same number of key attributes */
+ if (arbiterIndexInfo->ii_NumIndexKeyAttrs != indexInfo->ii_NumIndexKeyAttrs)
+ return false;
+
+ /* No support currently for comparing exclusion indexes. */
+ if (arbiterIndexInfo->ii_ExclusionOps != NULL || indexInfo->ii_ExclusionOps != NULL)
+ return false;
+
+ for (i = 0; i < arbiterIndexInfo->ii_NumIndexKeyAttrs; i++)
+ {
+ if (arbiterIndexRelation->rd_indcollation[i] != indexRelation->rd_indcollation[i])
+ return false;
+
+ if (arbiterIndexRelation->rd_opfamily[i] != indexRelation->rd_opfamily[i])
+ return false;
+
+ if (arbiterIndexRelation->rd_index->indkey.values[i] != indexRelation->rd_index->indkey.values[i])
+ return false;
+ }
+
+ if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation),
+ RelationGetIndexExpressions(indexRelation)) != NIL)
+ return false;
+
+ if (list_difference(RelationGetIndexPredicate(arbiterIndexRelation),
+ RelationGetIndexPredicate(indexRelation)) != NIL)
+ return false;
+ return true;
+}
+
/*
* ExecInitPartitionInfo
* Lock the partition and initialize ResultRelInfo. Also setup other
@@ -690,7 +747,9 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
TupleDesc partrelDesc = RelationGetDescr(partrel);
ExprContext *econtext = mtstate->ps.ps_ExprContext;
ListCell *lc;
- List *arbiterIndexes = NIL;
+ List *arbiterIndexes = NIL,
+ *arbiterIndexesOffset = NIL;
+ int additional_arbiters = 0;
/*
* If there is a list of arbiter indexes, map it to a list of indexes
@@ -698,36 +757,80 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
* list and searching for ancestry relationships to each index in the
* ancestor table.
*/
- if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL)
- {
- List *childIdxs;
+ if (rootResultRelInfo->ri_onConflictArbiterIndexes != NIL) {
+ List *childIdxs,
+ *nonAncestorIdxOffset = NIL;
childIdxs = RelationGetIndexList(leaf_part_rri->ri_RelationDesc);
foreach(lc, childIdxs)
{
Oid childIdx = lfirst_oid(lc);
+ int i = foreach_current_index(lc);
List *ancestors;
- ListCell *lc2;
ancestors = get_partition_ancestors(childIdx);
- foreach(lc2, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ if (ancestors)
{
- if (list_member_oid(ancestors, lfirst_oid(lc2)))
- arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ foreach_oid(oid, rootResultRelInfo->ri_onConflictArbiterIndexes)
+ {
+ if (list_member_oid(ancestors, oid))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes, childIdx);
+ arbiterIndexesOffset = lappend_int(arbiterIndexesOffset, i);
+ }
+ }
}
+ else /* No ancestor was found for that index. Save it for rechecking later. */
+ nonAncestorIdxOffset = lappend_int(nonAncestorIdxOffset, i);
+
list_free(ancestors);
}
+
+ /*
+ * If any non-ancestor indexes are found, we need to compare them with other
+ * indexes of the relation that will be used as arbiters. This is necessary
+ * when a partitioned index is processed by REINDEX CONCURRENTLY. Both indexes
+ * must be considered as arbiters to ensure that all concurrent transactions
+ * use the same set of arbiters.
+ */
+ if (nonAncestorIdxOffset && arbiterIndexesOffset)
+ {
+ foreach_int(childIdxOffset, nonAncestorIdxOffset)
+ {
+ Relation nonAncestorIndexRelation = leaf_part_rri->ri_IndexRelationDescs[childIdxOffset];
+ IndexInfo *nonAncestorIndexInfo = leaf_part_rri->ri_IndexRelationInfo[childIdxOffset];
+ Assert(!list_member_oid(arbiterIndexes, nonAncestorIndexRelation->rd_index->indexrelid));
+
+ /* It is too early to use non-ready indexes as arbiters */
+ if (!nonAncestorIndexInfo->ii_ReadyForInserts)
+ continue;
+
+ foreach_int(arbiterIdxOffset, arbiterIndexesOffset)
+ {
+ Relation arbiterIndexRelation = leaf_part_rri->ri_IndexRelationDescs[arbiterIdxOffset];
+ IndexInfo *arbiterIndexInfo = leaf_part_rri->ri_IndexRelationInfo[arbiterIdxOffset];
+
+ /* If non-ancestor index are compatible to arbiter - use it as arbiter too. */
+ if (IsIndexCompatibleAsArbiter(arbiterIndexRelation, arbiterIndexInfo,
+ nonAncestorIndexRelation, nonAncestorIndexInfo))
+ {
+ arbiterIndexes = lappend_oid(arbiterIndexes,
+ nonAncestorIndexRelation->rd_index->indexrelid);
+ additional_arbiters++;
+ }
+ }
+ }
+ }
+ list_free(nonAncestorIdxOffset);
}
/*
* If the resulting lists are of inequal length, something is wrong.
- * XXX This may happen because we don't match the lists correctly when
- * a partitioned index is being processed by REINDEX CONCURRENTLY.
- * FIXME later.
+ * But we need to consider additional arbiter indexes also.
*/
if (list_length(rootResultRelInfo->ri_onConflictArbiterIndexes) !=
- list_length(arbiterIndexes))
+ list_length(arbiterIndexes) - additional_arbiters)
elog(ERROR, "invalid arbiter index list");
leaf_part_rri->ri_onConflictArbiterIndexes = arbiterIndexes;
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index 7b3c0c4b716..e10475d2665 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -19,7 +19,8 @@ ISOLATION = basic \
syscache-update-pruned \
index-concurrently-upsert \
reindex-concurrently-upsert \
- index-concurrently-upsert-predicate
+ index-concurrently-upsert-predicate \
+ reindex-concurrently-upsert-partitioned
TAP_TESTS = 1
diff --git a/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out b/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
new file mode 100644
index 00000000000..588ffbd57a5
--- /dev/null
+++ b/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
@@ -0,0 +1,232 @@
+Parsed test spec with 4 sessions
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_swap:
+ SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_to_swap:
+ SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
+
+starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_set_local
+--------------------------
+
+(1 row)
+
+step s3_setup_wait_before_set_dead:
+ SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_reindex: REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+ <waiting ...>
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s4_wakeup_to_set_dead:
+ SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 485b483e3ca..3e06220fe31 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -51,6 +51,7 @@ tests += {
'index-concurrently-upsert',
'reindex-concurrently-upsert',
'index-concurrently-upsert-predicate',
+ 'reindex-concurrently-upsert-partitioned'
],
'runningcheck': false, # see syscache-update-pruned
# Some tests wait for all snapshots, so avoid parallel execution
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
new file mode 100644
index 00000000000..c8a8eb9cca5
--- /dev/null
+++ b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
@@ -0,0 +1,96 @@
+# Test race conditions involving:
+# - s1: UPSERT a tuple
+# - s2: UPSERT the same tuple
+# - s3: REINDEX concurrent primary key index
+# - s4: operations with injection points
+
+setup
+{
+ CREATE EXTENSION injection_points;
+ CREATE SCHEMA test;
+ CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+ CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+}
+
+teardown
+{
+ DROP SCHEMA test CASCADE;
+ DROP EXTENSION injection_points;
+}
+
+session s1
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+}
+step s1_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s2
+setup {
+ SELECT injection_points_set_local();
+ SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+}
+step s2_start_upsert {
+ INSERT INTO test.tbl VALUES(13,now()) on conflict(i) do update set updated_at = now();
+}
+
+session s3
+setup {
+ SELECT injection_points_set_local();
+}
+step s3_setup_wait_before_set_dead {
+ SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+}
+step s3_setup_wait_before_swap {
+ SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+}
+step s3_start_reindex { REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey; }
+
+session s4
+step s4_wakeup_to_swap {
+ SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
+}
+step s4_wakeup_s1 {
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+}
+step s4_wakeup_s2 {
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+}
+step s4_wakeup_to_set_dead {
+ SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
+ SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
+}
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_set_dead
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_s2
+
+permutation
+ s3_setup_wait_before_swap
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s4_wakeup_to_swap
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s2
+ s4_wakeup_s1
+
+permutation
+ s3_setup_wait_before_set_dead
+ s3_start_reindex(s1_start_upsert, s2_start_upsert)
+ s1_start_upsert
+ s2_start_upsert(s1_start_upsert)
+ s4_wakeup_s1
+ s4_wakeup_to_set_dead
+ s4_wakeup_s2
\ No newline at end of file
--
2.43.0
On 2025-Dec-01, Mihail Nikalayeu wrote:
From af5e27d4150dd53d313122c02da7ce4d3c07f332 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sun, 30 Nov 2025 16:48:55 +0100
Subject: [PATCH v16 1/3] Modify the ExecInitPartitionInfo function to consider
partitioned indexes that are potentially processed by REINDEX CONCURRENTLY
as arbiters as well.This is necessary to ensure that all concurrent transactions use the same set of arbiter indexes.
Thanks, pushed this one after some more editorialization. I rewrote
most comments and renamed variables, but I also changed the code
somewhat. For instance I made it use the ResultRelInfo's array of
indexes instead of doing RelationGetIndexList, because it seemed to
match better with the usage of the list index to match the indexes in
the array again later. Maybe now would be a good time to dust off your
stress tests and verify that everything is still working as intended.
In doing these changes, I realized that there are several places in the
code (this one, but also others) that are using
get_partition_ancestors(), which I think might be a somewhat expensive
routine. It obtains ancestors by recursing up the hierarchy, and at
each step does an indexscan on pg_inherits. I bet this is not very nice
for performance. I bet we can make this visible in a profile with an
inheritance hierarchy not terribly deep and a few hundred partitions.
We currently don't have a syscache on pg_inherits, as far as I
understand because back then we didn't think it was going to give us any
advantages, but maybe it would be useful for this and other operations.
If not a syscache, then maybe a different way to cache ancestors for a
relation, perhaps straight in the relcache entry.
Thanks for working on this,
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Before you were born your parents weren't as boring as they are now. They
got that way paying your bills, cleaning up your room and listening to you
tell them how idealistic you are." -- Charles J. Sykes' advice to teenagers
On 2025-Dec-01, Mihail Nikalayeu wrote:
I think I have implemented a better solution - without any additional NOTICE.
It just actually waits for the other backend to hang on the injection point.
Pushed. I changed it to be a loop in a DO block rather than a procedure
recursively calling itself, though, which seemed strangely complicated.
Should be pretty much the same, I hope.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Puedes vivir sólo una vez, pero si lo haces bien, una vez es suficiente"
From 5dc3e4eb50e445a291a13663fc9ce93d0db96b1c Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sun, 30 Nov 2025 16:49:20 +0100
Subject: [PATCH v16 2/3] Revert "Doc: cover index CONCURRENTLY causing errors
in INSERT ... ON CONFLICT."This reverts commit 8b18ed6dfbb8b3e4483801b513fea6b429140569.
Pushed this one also, and marked the commitfest item as committed.
Thanks!
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
Hello, Álvaro!
Maybe now would be a good time to dust off your
stress tests and verify that everything is still working as intended.
Done, passing without any issues.
I bet we can make this visible in a profile with an
inheritance hierarchy not terribly deep and a few hundred partitions.
I'll put it on the TODO list for some day.
Pushed this one also, and marked the commitfest item as committed.
Thanks for pushing\reviewing\rewriting it all!
Best regards,
Mikhail.
=?utf-8?Q?=C3=81lvaro?= Herrera <alvherre@kurilemu.de> writes:
Pushed this one also, and marked the commitfest item as committed.
BF member prion has been failing since 5dee7a603 went in. Apparently
that solution doesn't work under -DRELCACHE_FORCE_RELEASE and/or
-DCATCACHE_FORCE_RELEASE (so I'm expecting the CLOBBER_CACHE_ALWAYS
animals to fail too, whenever they next report).
regards, tom lane
Hi, Tom!
On Wed, Dec 3, 2025 at 5:43 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
BF member prion has been failing since 5dee7a603 went in. Apparently
that solution doesn't work under -DRELCACHE_FORCE_RELEASE and/or
-DCATCACHE_FORCE_RELEASE (so I'm expecting the CLOBBER_CACHE_ALWAYS
animals to fail too, whenever they next report).
Oh, my bad, sorry.
Attached patch with output variant for
RELCACHE_FORCE_RELEASE\CATCACHE_FORCE_RELEASE case.
But for CLOBBER_CACHE_ALWAYS - it is just a mess, too many different
kinds of wakeup loops need to be done.
So, my proposals:
* either drop that test - currently it looks like a test that will
fail 99% of time due to relcache changes instead of actual issue
* either add some kind of mechanics to skip that test in case of
particular build arguments
* either replace it by some kind of small stress test, without any
injection_points
Best regards,
Mikhail.
Attachments:
Test_output_variant_for_CATCACHE_FORCE_RELEASE.patchtext/x-patch; charset=US-ASCII; name=Test_output_variant_for_CATCACHE_FORCE_RELEASE.patchDownload
Subject: [PATCH] Test output variant for CATCACHE_FORCE_RELEASE
---
Index: src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
new file mode 100644
--- /dev/null (revision faa03762de3e3f0ce3cbeea8d0e0c8f148e449eb)
+++ b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out (revision faa03762de3e3f0ce3cbeea8d0e0c8f148e449eb)
@@ -0,0 +1,124 @@
+Parsed test spec with 5 sessions
+
+starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s1_attach_invalidate_catalog_snapshot:
+ SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+ <waiting ...>
+step s4_wakeup_s1_setup:
+ SELECT CASE WHEN
+ (SELECT pid FROM pg_stat_activity
+ WHERE wait_event_type = 'InjectionPoint' AND
+ wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
+ THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
+ END;
+
+case
+----
+
+(1 row)
+
+step s1_attach_invalidate_catalog_snapshot: <... completed>
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index:
+ CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
+ <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define-index-before-set-valid');
+ SELECT injection_points_wakeup('define-index-before-set-valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s5_wakeup_s1_from_invalidate_catalog_snapshot:
+ DO $$
+ DECLARE
+ v_waiting_pid INTEGER;
+ BEGIN
+ LOOP
+ SELECT pid INTO v_waiting_pid
+ FROM pg_stat_activity
+ WHERE wait_event_type = 'InjectionPoint'
+ AND wait_event = 'invalidate-catalog-snapshot-end'
+ LIMIT 1;
+ EXIT WHEN v_waiting_pid IS NOT NULL;
+ PERFORM pg_sleep(100);
+ END LOOP;
+ END
+ $$;
+
+ SELECT injection_points_detach('invalidate-catalog-snapshot-end');
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
Index: src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out b/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
new file mode 100644
--- /dev/null (revision faa03762de3e3f0ce3cbeea8d0e0c8f148e449eb)
+++ b/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out (revision faa03762de3e3f0ce3cbeea8d0e0c8f148e449eb)
@@ -0,0 +1,124 @@
+Parsed test spec with 5 sessions
+
+starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s1_attach_invalidate_catalog_snapshot:
+ SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+ <waiting ...>
+step s4_wakeup_s1_setup:
+ SELECT CASE WHEN
+ (SELECT pid FROM pg_stat_activity
+ WHERE wait_event_type = 'InjectionPoint' AND
+ wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
+ THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
+ END;
+
+case
+----
+
+(1 row)
+
+step s1_attach_invalidate_catalog_snapshot: <... completed>
+injection_points_attach
+-----------------------
+
+(1 row)
+
+step s3_start_create_index:
+ CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
+ <waiting ...>
+step s1_start_upsert:
+ INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s4_wakeup_define_index_before_set_valid:
+ SELECT injection_points_detach('define-index-before-set-valid');
+ SELECT injection_points_wakeup('define-index-before-set-valid');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s2_start_upsert:
+ INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+ <waiting ...>
+step s5_wakeup_s1_from_invalidate_catalog_snapshot:
+ DO $$
+ DECLARE
+ v_waiting_pid INTEGER;
+ BEGIN
+ LOOP
+ SELECT pid INTO v_waiting_pid
+ FROM pg_stat_activity
+ WHERE wait_event_type = 'InjectionPoint'
+ AND wait_event = 'invalidate-catalog-snapshot-end'
+ LIMIT 1;
+ EXIT WHEN v_waiting_pid IS NOT NULL;
+ PERFORM pg_sleep(100);
+ END LOOP;
+ END
+ $$;
+
+ SELECT injection_points_detach('invalidate-catalog-snapshot-end');
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s2:
+ SELECT injection_points_detach('exec-insert-before-insert-speculative');
+ SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s4_wakeup_s1:
+ SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
+ SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
+
+injection_points_detach
+-----------------------
+
+(1 row)
+
+injection_points_wakeup
+-----------------------
+
+(1 row)
+
+step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_create_index: <... completed>
On 2025-Dec-03, Mihail Nikalayeu wrote:
Oh, my bad, sorry.
Attached patch with output variant for
RELCACHE_FORCE_RELEASE\CATCACHE_FORCE_RELEASE case.
Thanks.
But for CLOBBER_CACHE_ALWAYS - it is just a mess, too many different
kinds of wakeup loops need to be done.
Hmm, maybe we can do "SET debug_discard_caches=0" in the setup block of
the sessions that need it.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"La victoria es para quien se atreve a estar solo"
On 2025-Dec-03, Mihail Nikalayeu wrote:
Oh, my bad, sorry.
Attached patch with output variant for
RELCACHE_FORCE_RELEASE\CATCACHE_FORCE_RELEASE case.
I have pushed this, thanks.
But for CLOBBER_CACHE_ALWAYS - it is just a mess, too many different
kinds of wakeup loops need to be done.
I ran the tests under CLOBBER_CACHE_ALWAYS, and as far as I can tell,
they succeed. We'll see what the CLOBBER_CACHE_ALWAYS members say ...
assuming the other problems [1]/messages/by-id/baf1ae02-83bd-4f5d-872a-1d04f11a9073@vondra.me can be fixed.
[1]: /messages/by-id/baf1ae02-83bd-4f5d-872a-1d04f11a9073@vondra.me
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"If you want to have good ideas, you must have many ideas. Most of them
will be wrong, and what you have to learn is which ones to throw away."
(Linus Pauling)
The "partitioned" test failed recently also on flaviventris and adder:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=flaviventris&dt=2025-12-04%2017%3A28%3A20
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=adder&dt=2025-12-03%2009%3A23%3A08
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=adder&dt=2025-12-02%2008%3A48%3A24
diff -U3 /home/bf/bf-build/flaviventris/HEAD/pgsql/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out /home/bf/bf-build/flaviventris/HEAD/pgsql.build/testrun/injection_points/isolation/results/reindex-concurrently-upsert-partitioned.out
--- /home/bf/bf-build/flaviventris/HEAD/pgsql/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out 2025-12-02 14:28:16.233200773 +0100
+++ /home/bf/bf-build/flaviventris/HEAD/pgsql.build/testrun/injection_points/isolation/results/reindex-concurrently-upsert-partitioned.out 2025-12-04 18:31:29.340033507 +0100
@@ -61,7 +61,6 @@
(1 row)
-step s1_start_upsert: <... completed>
step s4_wakeup_s2:
SELECT injection_points_detach('exec-insert-before-insert-speculative');
SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
@@ -76,6 +75,7 @@
(1 row)
+step s1_start_upsert: <... completed>
step s2_start_upsert: <... completed>
step s3_start_reindex: <... completed>
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
[…] indem ich in meinem Leben oft an euch gedacht, euch glücklich zu machen. Seyd es!
A menudo he pensado en vosotros, en haceros felices. ¡Sedlo, pues!
Heiligenstädter Testament, L. v. Beethoven, 1802
https://de.wikisource.org/wiki/Heiligenstädter_Testament
Hello!
Seems like it may be easily fixed (see attached patch).
Bwt, is it possible to somehow run the whole buildfarm over some branch?
Such way it will be possible to fix such issues much earlier (some of
them catched by github CI, but not all).
Best regards,
Mikhail.
Attachments:
more_test_stabilization.patchtext/x-patch; charset=US-ASCII; name=more_test_stabilization.patchDownload
Subject: [PATCH] more test stabilization
---
Index: src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec
--- a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec (revision 6bd469d26aca6ea413b35bfcb611dfa3a8f5ea45)
+++ b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec (date 1764870936317)
@@ -85,7 +85,7 @@
permutation
s3_setup_wait_before_set_dead
s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
+ s1_start_upsert(s4_wakeup_s2)
s4_wakeup_to_set_dead
s2_start_upsert(s1_start_upsert)
s4_wakeup_s1
@@ -94,7 +94,7 @@
permutation
s3_setup_wait_before_swap
s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
+ s1_start_upsert(s4_wakeup_s2)
s4_wakeup_to_swap
s2_start_upsert(s1_start_upsert)
s4_wakeup_s2
@@ -103,7 +103,7 @@
permutation
s3_setup_wait_before_set_dead
s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
+ s1_start_upsert(s4_wakeup_s2)
s2_start_upsert(s1_start_upsert)
s4_wakeup_s1
s4_wakeup_to_set_dead
Index: src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec
--- a/src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec (revision 6bd469d26aca6ea413b35bfcb611dfa3a8f5ea45)
+++ b/src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec (date 1764870872510)
@@ -86,7 +86,7 @@
permutation
s3_setup_wait_before_set_dead
s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
+ s1_start_upsert(s4_wakeup_s2)
s4_wakeup_to_set_dead
s2_start_upsert(s1_start_upsert)
s4_wakeup_s1
@@ -95,7 +95,7 @@
permutation
s3_setup_wait_before_swap
s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
+ s1_start_upsert(s4_wakeup_s2)
s4_wakeup_to_swap
s2_start_upsert(s1_start_upsert)
s4_wakeup_s2
@@ -104,7 +104,7 @@
permutation
s3_setup_wait_before_set_dead
s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
+ s1_start_upsert(s4_wakeup_s2)
s2_start_upsert(s1_start_upsert)
s4_wakeup_s1
s4_wakeup_to_set_dead
Index: src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
--- a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec (revision 6bd469d26aca6ea413b35bfcb611dfa3a8f5ea45)
+++ b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec (date 1764870936340)
@@ -88,7 +88,7 @@
permutation
s3_setup_wait_before_set_dead
s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
+ s1_start_upsert(s4_wakeup_s2)
s4_wakeup_to_set_dead
s2_start_upsert(s1_start_upsert)
s4_wakeup_s1
@@ -97,7 +97,7 @@
permutation
s3_setup_wait_before_swap
s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
+ s1_start_upsert(s4_wakeup_s2)
s4_wakeup_to_swap
s2_start_upsert(s1_start_upsert)
s4_wakeup_s2
@@ -106,7 +106,7 @@
permutation
s3_setup_wait_before_set_dead
s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
+ s1_start_upsert(s4_wakeup_s2)
s2_start_upsert(s1_start_upsert)
s4_wakeup_s1
s4_wakeup_to_set_dead
On 2025-Dec-04, Mihail Nikalayeu wrote:
Hello!
Seems like it may be easily fixed (see attached patch).
Makes sense -- thanks, pushed.
Bwt, is it possible to somehow run the whole buildfarm over some branch?
Such way it will be possible to fix such issues much earlier (some of
them catched by github CI, but not all).
Nope.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
Hello Álvaro and Mihail,
02.12.2025 15:07, Álvaro Herrera wrote:
Thanks, pushed this one after some more editorialization.
I've discovered that despite removing FIXME (in 90eae926a), the error
"invalid arbiter index list" can still be triggered with:
CREATE TABLE pt (a int PRIMARY KEY) PARTITION BY RANGE (a);
CREATE TABLE p1 PARTITION OF pt FOR VALUES FROM (1) to (2) PARTITION BY RANGE (a);
CREATE TABLE p1_1 PARTITION OF p1 FOR VALUES FROM (1) TO (2);
CREATE UNIQUE INDEX ON ONLY p1 (a);
INSERT INTO p1 VALUES (1) ON CONFLICT (a) DO NOTHING;
ERROR: XX000: invalid arbiter index list
LOCATION: ExecInitPartitionInfo, execPartition.c:863
The first commit it produced on with this script is bc32a12e0.
Best regards,
Alexander
Hello, Alexander!
On Sat, Dec 6, 2025 at 7:00 AM Alexander Lakhin <exclusion@gmail.com> wrote:
I've discovered that despite removing FIXME (in 90eae926a), the error
"invalid arbiter index list" can still be triggered with:
Wow, thanks for finding this.
The first commit it produced on with this script is bc32a12e0.
Such commands add an indisready, but not indisvalid index on pt, which
is added to to the list of potential arbiters.
It happens because of [0]https://github.com/postgres/postgres/blob/bc32a12e0db2df203a9cb2315461578e08568b9c/src/backend/commands/indexcmds.c#L1225-L1235.
From my perspective, the correct solution - is to just remove the
error message, because it looks obsolete now. Or somehow calculate
compensation offset differently - but I am not sure it is a good idea.
Let's wait for Álvaro decision - I'll prepare the fix and regress test then.
Best regards,
Mikhail.
On 2025-Dec-06, Mihail Nikalayeu wrote:
Such commands add an indisready, but not indisvalid index on pt, which
is added to to the list of potential arbiters.
It happens because of [0].From my perspective, the correct solution - is to just remove the
error message, because it looks obsolete now. Or somehow calculate
compensation offset differently - but I am not sure it is a good idea.
Hmm, as I recall it's quite intentional that the index on a partitioned
table is marked !indisvalid; such indexes are only supposed to be marked
valid once indexes on all partitions have been attached. As I recall,
if you remove that prohibition, some pg_dump scenarios fail.
I'd rather consider the idea of avoiding indexes marked !indisvalid on
partitioned tables as arbiter lists ... but then we need to verify the
scenario where there is one, and INSERT ON CONFLICT runs concurrently
with ALTER INDEX ATTACH PARTITION for the last partition lacking the
index (which is the point where the index is marked indisvalid on the
partitioned table). There may not be a problem with that, because we
grab AccessExclusiveLock on the index partition, so no query can be
running concurrently ... unless the INSERT is targeting a partition
other than the one where the index is being attached. (On the
partitioned table and index, we only have ShareUpdateExclusiveLock).
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Crear es tan difícil como ser libre" (Elsa Triolet)
Hello!
On Sun, Dec 7, 2025 at 10:07 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
I'd rather consider the idea of avoiding indexes marked !indisvalid on
partitioned tables as arbiter lists ... but then we need to verify the
scenario where there is one, and INSERT ON CONFLICT runs concurrently
with ALTER INDEX ATTACH PARTITION for the last partition lacking the
index (which is the point where the index is marked indisvalid on the
partitioned table). There may not be a problem with that, because we
grab AccessExclusiveLock on the index partition, so no query can be
running concurrently ... unless the INSERT is targeting a partition
other than the one where the index is being attached. (On the
partitioned table and index, we only have ShareUpdateExclusiveLock).
For my taste it feels too complicated for such a case.
What is about changing the logic of this check to the next:
For each valid index used as arbiter in a partitioned table we need to
have a valid in particular partition (but it is okay to also have an
"ready"-only as additional).
If some of the arbiters is invalid in the partitioned table (but we
have valid compatible in any case) - it is okay. We just have to have
an appropriate "companion" for every valid arbiter.
Such a check looks correct to me, at least at the very end of the weekend.
Thanks,
Mikhail.
On Sun, Dec 07, 2025 at 10:07:45PM +0100, Alvaro Herrera wrote:
Hmm, as I recall it's quite intentional that the index on a partitioned
table is marked !indisvalid; such indexes are only supposed to be marked
valid once indexes on all partitions have been attached. As I recall,
if you remove that prohibition, some pg_dump scenarios fail.
Right. If indisvalid is true on a partitioned table, then we are sure
that all its partitions have valid indexes. If indisvalid is false,
some of its partitions may have an invalid index. In the false case,
things can be a bit lossy as well. For example, an ALTER TABLE ONLY
could switch a partition's indisvalid to be true, without switching to
true the indisvalid of its partitioned table.
--
Michael
On 2025-Dec-06, Mihail Nikalayeu wrote:
From my perspective, the correct solution - is to just remove the
error message, because it looks obsolete now.
Rereading this -- did you mean to propose that a possible fix was to
remove the "invalid arbiter index list" error? I had understood
something different.
Your idea downthread of changing the way that check works (so that we
don't throw an error in this case, but we continue to double-check that
the arbiter list is sensible) sounds reasonable to me. Do you want to
propose a specific check for it?
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
<inflex> really, I see PHP as like a strange amalgamation of C, Perl, Shell
<crab> inflex: you know that "amalgam" means "mixture with mercury",
more or less, right?
<crab> i.e., "deadly poison"
Hello, Álvaro!
On Mon, Dec 8, 2025 at 10:58 AM Álvaro Herrera <alvherre@kurilemu.de> wrote:
Rereading this -- did you mean to propose that a possible fix was to
remove the "invalid arbiter index list" error? I had understood
something different.
Yes, it was the initial idea.
Your idea downthread of changing the way that check works (so that we
don't throw an error in this case, but we continue to double-check that
the arbiter list is sensible) sounds reasonable to me. Do you want to
propose a specific check for it?
I think the next logic is correct:
* for each IS indisvalid arbiter in the parent table we should have AT
LEAST ONE compatible indisvalid pair in the partition (we may have
multiple or a few more ready-only)
* for each NOT indisvalid arbiter in parent - nothing is expected
from partition
I'll try to create a patch with such later.
Best regards,
Mikhail.
Hello!
After some investigation I ended up with a much simpler fix.
It is self-explanatory in code and a commit message.
Best regards,
Mikhail.
Attachments:
v1-0001-Fix-infer_arbiter_index-for-partitioned-tables.patchtext/x-patch; charset=UTF-8; name=v1-0001-Fix-infer_arbiter_index-for-partitioned-tables.patchDownload
From de4fcbcbd35912cea70f0f00f81878ea8b94416c Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <nkey@toloka.ai>
Date: Tue, 9 Dec 2025 19:39:06 +0100
Subject: [PATCH v1] Fix infer_arbiter_index for partitioned tables
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
The fix for concurrent index operations in bc32a12e0db started considering indexes that are not yet marked indisvalid. However, for partitioned tables this is incorrect because partitioned tables don't store data themselves; the actual arbiter indexes are inferred later at the leaf partition level. Considering non-valid indexes on partitioned tables (such as those created with CREATE INDEX ... ON ONLY) led to incorrect arbiter selection and resulted in an error.
Skip non-valid indexes when processing partitioned tables, restoring the previous behavior for them while preserving the concurrent index operation fix for regular tables.
Author: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/17622f79-117a-4a44-aa8e-0374e53faaf0%40gmail.com
---
src/backend/optimizer/util/plancat.c | 7 ++++++-
src/test/regress/expected/partition_info.out | 7 +++++++
src/test/regress/sql/partition_info.sql | 8 ++++++++
3 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index ed0dac37f51..4775669f868 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -995,8 +995,13 @@ infer_arbiter_indexes(PlannerInfo *root)
* wouldn't detect possible duplicate rows. In order to prevent false
* negatives, we require that we include in the set of inferred
* indexes at least one index that is marked valid.
+ *
+ * Also, in the case of partitioned table - only valid indexes are
+ * considered because real arbiters inferred later.
*/
- if (!idxForm->indisready)
+ if (!idxForm->indisready ||
+ (relation->rd_rel->relkind == RELKIND_PARTITIONED_TABLE &&
+ !idxForm->indisvalid))
continue;
/*
diff --git a/src/test/regress/expected/partition_info.out b/src/test/regress/expected/partition_info.out
index 42b6bc77cad..4defa66e5b3 100644
--- a/src/test/regress/expected/partition_info.out
+++ b/src/test/regress/expected/partition_info.out
@@ -349,3 +349,10 @@ SELECT pg_partition_root('ptif_li_child');
DROP VIEW ptif_test_view;
DROP MATERIALIZED VIEW ptif_test_matview;
DROP TABLE ptif_li_parent, ptif_li_child;
+-- Test about selection of arbiter indexes for partitioned tables with
+-- non-valid index on the parent table
+CREATE TABLE pt (a int PRIMARY KEY) PARTITION BY RANGE (a);
+CREATE TABLE p1 PARTITION OF pt FOR VALUES FROM (1) to (2) PARTITION BY RANGE (a);
+CREATE TABLE p1_1 PARTITION OF p1 FOR VALUES FROM (1) TO (2);
+CREATE UNIQUE INDEX ON ONLY p1 (a);
+INSERT INTO p1 VALUES (1) ON CONFLICT (a) DO NOTHING;
diff --git a/src/test/regress/sql/partition_info.sql b/src/test/regress/sql/partition_info.sql
index b5060bec7f0..2ef65292bab 100644
--- a/src/test/regress/sql/partition_info.sql
+++ b/src/test/regress/sql/partition_info.sql
@@ -127,3 +127,11 @@ SELECT pg_partition_root('ptif_li_child');
DROP VIEW ptif_test_view;
DROP MATERIALIZED VIEW ptif_test_matview;
DROP TABLE ptif_li_parent, ptif_li_child;
+
+-- Test about selection of arbiter indexes for partitioned tables with
+-- non-valid index on the parent table
+CREATE TABLE pt (a int PRIMARY KEY) PARTITION BY RANGE (a);
+CREATE TABLE p1 PARTITION OF pt FOR VALUES FROM (1) to (2) PARTITION BY RANGE (a);
+CREATE TABLE p1_1 PARTITION OF p1 FOR VALUES FROM (1) TO (2);
+CREATE UNIQUE INDEX ON ONLY p1 (a);
+INSERT INTO p1 VALUES (1) ON CONFLICT (a) DO NOTHING;
--
2.52.0
On 2025-Dec-09, Mihail Nikalayeu wrote:
Hello!
After some investigation I ended up with a much simpler fix.
It is self-explanatory in code and a commit message.
Yeah, that makes sense. It's what I was trying to say in
/messages/by-id/202512072050.hcyysny65ugj@alvherre.pgsql
Pushed your patch, thanks.
I still wonder if it's possible to break things by doing something like
CREATE TABLE pt (a int) PARTITION BY LISt (a);
CREATE TABLE p1 PARTITION OF pt FOR VALUES IN (1);
CREATE TABLE p2 PARTITION OF pt FOR VALUES IN (2);
CREATE UNIQUE INDEX pti ON ONLY pt (a);
CREATE UNIQUE INDEX p1i ON p1 (a);
CREATE UNIQUE INDEX p2i ON p2 (a);
ALTER INDEX pti ATTACH PARTITION p1i;
and then do the
INSERT INTO pt VALUES (1) ON CONFLICT (a) DO NOTHING;
dance concurrently with
ALTER INDEX pti ATTACH PARTITION p2i;
This would be a much smaller problem than the already fixed ones though,
I think.
Thanks!
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Learn about compilers. Then everything looks like either a compiler or
a database, and now you have two problems but one of them is fun."
https://twitter.com/thingskatedid/status/1456027786158776329
Hi,
I just saw a failure in CI for an unrelated patch,
diff -U3 /home/postgres/postgres/src/test/modules/injection_points/expected/reindex-concurrently-upsert.out /home/postgres/postgres/build/testrun/injection_points/isolation/results/reindex-concurrently-upsert.out
--- /home/postgres/postgres/src/test/modules/injection_points/expected/reindex-concurrently-upsert.out 2025-12-11 12:41:23.982167232 +0000
+++ /home/postgres/postgres/build/testrun/injection_points/isolation/results/reindex-concurrently-upsert.out 2025-12-11 12:45:24.038078804 +0000
@@ -62,22 +62,13 @@
(1 row)
step s1_start_upsert: <... completed>
+step s2_start_upsert: <... completed>
+step s3_start_reindex: <... completed>
step s4_wakeup_s2:
SELECT injection_points_detach('exec-insert-before-insert-speculative');
SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
+ERROR: could not detach injection point "exec-insert-before-insert-speculative"
starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
injection_points_attach
I haven't seen anywhere else. I don't believe this to be a problem in
the patch, because the patch is about partitions and this test case does
not involve partitioned tables -- so I'm inclined to think it's a timing
issue in the test itself. If so, this can probably be fixed with
additional constraints on on of the steps in the permutation ...
probably maybe s2_start_upsert depend on s4_wakeup_s2? But since I
can't easily reproduce this, I have no idea if this is a valid
solution.
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
Hello, Álvaro!
On Thu, Dec 11, 2025 at 10:36 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
I just saw a failure in CI for an unrelated patch
I'll try to dive deeper tomorrow to find a fix, but it feels like we
are doing something wrong here.
The tests were good to prove the issue and demonstrate it was fixed
after some changes.
But currently we are just trying (not the first time already) to make
sure OUTPUT of the test is EXACTLY equal to some variant.
At the same time I think a more correct approach here - is to test
something like "output does not contain `duplicate key value violates
unique constraint` message". Or even better real case - pgbench of
concurrent REINDEX + INSERT (takes seconds to reproduce, but CPU is
high).
It is a way to test something essential what we want to be not broken,
not exact output of concurrent commands.... But current
isolationtester does not support anything like that.
I am afraid amount of time needed to stabilize such test (in its
output, not the sense) is not cover potential value of it.
Also, I imaging someone changing something unrelated (catalog snapshot
invalidation, for example) and test starts to fail on some rear animal
once a week.... Ughn.
Maybe I am inclined by my main programming experience (Java, backends,
distributed systems, etc.) and databases need to be much more accurate
and strict even if it pains...
What do you think about it?
Best regards,
Mikhail.
On 2025-Dec-12, Mihail Nikalayeu wrote:
Hello, Álvaro!
On Thu, Dec 11, 2025 at 10:36 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
I just saw a failure in CI for an unrelated patch
I'll try to dive deeper tomorrow to find a fix, but it feels like we
are doing something wrong here.
Hmm, this is a good point.
But currently we are just trying (not the first time already) to make
sure OUTPUT of the test is EXACTLY equal to some variant.
A low-cost option might be to add alternative expected file(s), which
matches other variant(s). I think trying to make isolationtester "smart
match" the output might be more complicated than is warranted.
I am afraid amount of time needed to stabilize such test (in its
output, not the sense) is not cover potential value of it.
Yeah, could be.
Also, I imaging someone changing something unrelated (catalog snapshot
invalidation, for example) and test starts to fail on some rear animal
once a week.... Ughn.
Another idea might be to rewrite these tests using BackgroundPsql under
the TAP infrastructure. That's quite a bit more tedious to write, but
we can be more precise on detecting whether some particular error
message was thrown or not.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"How amazing is that? I call it a night and come back to find that a bug has
been identified and patched while I sleep." (Robert Davidson)
http://archives.postgresql.org/pgsql-sql/2006-03/msg00378.php
Hello, Álvaro!
On Fri, Dec 12, 2025 at 11:17 AM Álvaro Herrera <alvherre@kurilemu.de> wrote:
Another idea might be to rewrite these tests using BackgroundPsql under
the TAP infrastructure. That's quite a bit more tedious to write, but
we can be more precise on detecting whether some particular error
message was thrown or not.
I think I understood the race, currently thinking about two possible approaches:
1) extract LOOP waiting for injection point into some function like
"injection_points_await_waiter" and add it almost between each steps
to ensure it all executed by guardrails (effectively reducing
concurrency instead of making isolationtester to report data the same
way)
2) rewrite using TAP infra
What do you think about this? Which one do you prefer?
Best regards,
Mikhail.
Hi,
On 2025-12-11 22:36:55 +0100, �lvaro Herrera wrote:
I just saw a failure in CI for an unrelated patch,
There are a *lot* of these right now. CI on master currently has a failure
rate of 14 failures / 50 runs on the postgres/postgres repo. All of the
failures are due to this issue from what I can tell. That means there are a
lot of false failures for cfbot.
Could we perhaps disable this test until it's been rewritten to be stable?
Greetings,
Andres Freund
Hello, Andres!
On Sun, Dec 14, 2025 at 4:19 PM Andres Freund <andres@anarazel.de> wrote:
There are a *lot* of these right now. CI on master currently has a failure
rate of 14 failures / 50 runs on the postgres/postgres repo. All of the
failures are due to this issue from what I can tell. That means there are a
lot of false failures for cfbot.
Oh, that's sad :(
Could we perhaps disable this test until it's been rewritten to be stable?
Of course, if you can do it yourself, feel free to do so for all test
from the set:
index-concurrently-upsert
index-concurrently-upsert-predicate
reindex-concurrently-upsert
reindex-concurrently-upsert-on-constraint
reindex-concurrently-upsert-partitioned
You may even remove them for now - I plan to rewrite them using TAP
infrastructure, because "endless fight" from [0]/messages/by-id/CADzfLwUhvaEsPbYaT3Z5cMO779JruDqBbE5nijeaBcXiNPoCYw@mail.gmail.com seems to be true now.
[0]: /messages/by-id/CADzfLwUhvaEsPbYaT3Z5cMO779JruDqBbE5nijeaBcXiNPoCYw@mail.gmail.com
On 2025-Dec-14, Mihail Nikalayeu wrote:
Of course, if you can do it yourself, feel free to do so for all test
from the set:index-concurrently-upsert
index-concurrently-upsert-predicate
reindex-concurrently-upsert
reindex-concurrently-upsert-on-constraint
reindex-concurrently-upsert-partitioned
I've commented them out from the meson/makefile for now.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"I dream about dreams about dreams", sang the nightingale
under the pale moon (Sandman)
Hello!
On Mon, Dec 15, 2025 at 12:31 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:
I've commented them out from the meson/makefile for now.
Thanks!
First version of TAP-based test in attachment. I think we should not
hurry with push - let's make sure it really stable now.
Best regards,
Mikhail.
Attachments:
nocfbot-0001-Preserve-visibility-information-of-the-conc.patchtext/plain; charset=US-ASCII; name=nocfbot-0001-Preserve-visibility-information-of-the-conc.patchDownload
From a26caa0ea6a1c9e7eb325fb7e9bd00c43bab4476 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Sat, 13 Dec 2025 19:42:52 +0100
Subject: [PATCH vnocfbot] Preserve visibility information of the concurrent
data changes.
As explained in the commit message of the preceding patch of the series, the
data changes done by applications while REPACK CONCURRENTLY is copying the
table contents to a new file are decoded from WAL and eventually also applied
to the new file. To reduce the complexity a little bit, the preceding patch
uses the current transaction (i.e. transaction opened by the REPACK command)
to execute those INSERT, UPDATE and DELETE commands.
However, REPACK is not expected to change visibility of tuples. Therefore,
this patch fixes the handling of the "concurrent data changes". It ensures
that tuples written into the new table have the same XID and command ID (CID)
as they had in the old table.
To "replay" an UPDATE or DELETE command on the new table, we use SnapshotSelf to find the last alive version of tuple and update with stamp with xid of original transaction. It is safe because:
* all transactions we replaying are committed
* apply worker working without any concurrent modifiers of the table
As long as we preserve the tuple visibility information (which includes XID),
it's important to avoid logical decoding of the WAL generated by DMLs on the
new table: the logical decoding subsystem probably does not expect that the
incoming WAL records contain XIDs of an already decoded transactions. (And of
course, repeated decoding would be wasted effort.)
Author: Antonin Houska <ah@cybertec.at> with changes from Mikhail Nikalayeu <mihailnikalayeu@gmail.com
---
contrib/amcheck/meson.build | 1 +
contrib/amcheck/t/007_repack_concurrently.pl | 2 +-
contrib/amcheck/t/008_repack_concurrently.pl | 2 +-
.../amcheck/t/009_repack_concurrently_mvcc.pl | 111 ++++++++++++++++++
doc/src/sgml/mvcc.sgml | 12 +-
doc/src/sgml/ref/repack.sgml | 9 --
src/backend/access/common/toast_internals.c | 3 +-
src/backend/access/heap/heapam.c | 29 +++--
src/backend/access/heap/heapam_handler.c | 24 ++--
src/backend/commands/cluster.c | 107 ++++++++++++-----
.../pgoutput_repack/pgoutput_repack.c | 16 ++-
src/include/access/heapam.h | 6 +-
.../injection_points/specs/repack.spec | 4 -
13 files changed, 243 insertions(+), 83 deletions(-)
create mode 100644 contrib/amcheck/t/009_repack_concurrently_mvcc.pl
diff --git a/contrib/amcheck/meson.build b/contrib/amcheck/meson.build
index f7c70735989..6946c684259 100644
--- a/contrib/amcheck/meson.build
+++ b/contrib/amcheck/meson.build
@@ -52,6 +52,7 @@ tests += {
't/006_verify_gin.pl',
't/007_repack_concurrently.pl',
't/008_repack_concurrently.pl',
+ 't/009_repack_concurrently_mvcc.pl',
],
},
}
diff --git a/contrib/amcheck/t/007_repack_concurrently.pl b/contrib/amcheck/t/007_repack_concurrently.pl
index a47cebb347b..57769097af9 100644
--- a/contrib/amcheck/t/007_repack_concurrently.pl
+++ b/contrib/amcheck/t/007_repack_concurrently.pl
@@ -91,7 +91,7 @@ $node->pgbench(
COMMIT;
BEGIN
- --TRANSACTION ISOLATION LEVEL REPEATABLE READ
+ TRANSACTION ISOLATION LEVEL REPEATABLE READ
;
SELECT 1;
\\sleep 1 ms
diff --git a/contrib/amcheck/t/008_repack_concurrently.pl b/contrib/amcheck/t/008_repack_concurrently.pl
index 220524d41b3..14db7425ae4 100644
--- a/contrib/amcheck/t/008_repack_concurrently.pl
+++ b/contrib/amcheck/t/008_repack_concurrently.pl
@@ -82,7 +82,7 @@ $node->pgbench(
\\sleep 1 ms
BEGIN
- --TRANSACTION ISOLATION LEVEL REPEATABLE READ
+ TRANSACTION ISOLATION LEVEL REPEATABLE READ
;
SELECT 1;
\\sleep 1 ms
diff --git a/contrib/amcheck/t/009_repack_concurrently_mvcc.pl b/contrib/amcheck/t/009_repack_concurrently_mvcc.pl
new file mode 100644
index 00000000000..c0f8a63cf3d
--- /dev/null
+++ b/contrib/amcheck/t/009_repack_concurrently_mvcc.pl
@@ -0,0 +1,111 @@
+
+# Copyright (c) 2021-2025, PostgreSQL Global Development Group
+
+# Test REPACK CONCURRENTLY with concurrent modifications
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my $node;
+
+#
+# Test set-up
+#
+$node = PostgreSQL::Test::Cluster->new('CIC_test');
+$node->init;
+$node->append_conf('postgresql.conf',
+ 'lock_timeout = ' . (1000 * $PostgreSQL::Test::Utils::timeout_default));
+$node->append_conf(
+ 'postgresql.conf', qq(
+wal_level = logical
+));
+$node->start;
+$node->safe_psql('postgres', q(CREATE TABLE tbl1(i int PRIMARY KEY, j int)));
+$node->safe_psql('postgres', q(CREATE TABLE tbl2(i int PRIMARY KEY, j int)));
+
+
+# Insert 100 rows into tbl1
+$node->safe_psql('postgres', q(
+ INSERT INTO tbl1 SELECT i, i % 100 FROM generate_series(1,100) i
+));
+
+# Insert 100 rows into tbl2
+$node->safe_psql('postgres', q(
+ INSERT INTO tbl2 SELECT i, i % 100 FROM generate_series(1,100) i
+));
+
+$node->safe_psql('postgres', q(
+ CREATE OR REPLACE FUNCTION log_raise(i int, j1 int, j2 int) RETURNS VOID AS $$
+ BEGIN
+ RAISE NOTICE 'ERROR i=% j1=% j2=%', i, j1, j2;
+ END;$$ LANGUAGE plpgsql;
+));
+
+$node->safe_psql('postgres', q(CREATE UNLOGGED SEQUENCE in_row_rebuild START 1 INCREMENT 1;));
+$node->safe_psql('postgres', q(SELECT nextval('in_row_rebuild');));
+
+
+$node->pgbench(
+'--no-vacuum --client=10 --jobs=4 --exit-on-abort --transactions=2500',
+0,
+[qr{actually processed}],
+[qr{^$}],
+'concurrent operations with REINDEX/CREATE INDEX CONCURRENTLY',
+{
+ 'concurrent_ops' => q(
+ SELECT pg_try_advisory_lock(42)::integer AS gotlock \gset
+ \if :gotlock
+ SELECT nextval('in_row_rebuild') AS last_value \gset
+ \if :last_value = 2
+ REPACK (CONCURRENTLY) tbl1 USING INDEX tbl1_pkey;
+ \sleep 10 ms
+ REPACK (CONCURRENTLY) tbl2 USING INDEX tbl2_pkey;
+ \sleep 10 ms
+ \endif
+ SELECT pg_advisory_unlock(42);
+ \else
+ \set num random(1, 100)
+ BEGIN;
+ UPDATE tbl1 SET j = j + 1 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 2 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 3 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl1 SET j = j + 4 WHERE i = :num;
+ \sleep 1 ms
+
+ UPDATE tbl2 SET j = j + 1 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 2 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 3 WHERE i = :num;
+ \sleep 1 ms
+ UPDATE tbl2 SET j = j + 4 WHERE i = :num;
+
+ COMMIT;
+ SELECT setval('in_row_rebuild', 1);
+
+ BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+ SELECT COALESCE(SUM(j), 0) AS t1 FROM tbl1 WHERE i = :num \gset p_
+ \sleep 10 ms
+ SELECT COALESCE(SUM(j), 0) AS t2 FROM tbl2 WHERE i = :num \gset p_
+ \if :p_t1 != :p_t2
+ COMMIT;
+ SELECT log_raise(tbl1.i, tbl1.j, tbl2.j) FROM tbl1 LEFT OUTER JOIN tbl2 ON tbl1.i = tbl2.i WHERE tbl1.j != tbl2.j;
+ \sleep 10 ms
+ SELECT log_raise(tbl1.i, tbl1.j, tbl2.j) FROM tbl1 LEFT OUTER JOIN tbl2 ON tbl1.i = tbl2.i WHERE tbl1.j != tbl2.j;
+ SELECT (:p_t1 + :p_t2) / 0;
+ \endif
+
+ COMMIT;
+ \endif
+ )
+});
+
+$node->stop;
+done_testing();
diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml
index 0f5c34af542..049ee75a4ba 100644
--- a/doc/src/sgml/mvcc.sgml
+++ b/doc/src/sgml/mvcc.sgml
@@ -1833,17 +1833,15 @@ SELECT pg_advisory_lock(q.id) FROM
<title>Caveats</title>
<para>
- Some commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link>, the
- table-rewriting forms of <link linkend="sql-altertable"><command>ALTER
- TABLE</command></link> and <command>REPACK</command> with
- the <literal>CONCURRENTLY</literal> option, are not
+ Some DDL commands, currently only <link linkend="sql-truncate"><command>TRUNCATE</command></link> and the
+ table-rewriting forms of <link linkend="sql-altertable"><command>ALTER TABLE</command></link>, are not
MVCC-safe. This means that after the truncation or rewrite commits, the
table will appear empty to concurrent transactions, if they are using a
- snapshot taken before the command committed. This will only be an
+ snapshot taken before the DDL command committed. This will only be an
issue for a transaction that did not access the table in question
- before the command started — any transaction that has done so
+ before the DDL command started — any transaction that has done so
would hold at least an <literal>ACCESS SHARE</literal> table lock,
- which would block the truncating or rewriting command until that transaction completes.
+ which would block the DDL command until that transaction completes.
So these commands will not cause any apparent inconsistency in the
table contents for successive queries on the target table, but they
could cause visible inconsistency between the contents of the target
diff --git a/doc/src/sgml/ref/repack.sgml b/doc/src/sgml/ref/repack.sgml
index 30c43c49069..9796a923597 100644
--- a/doc/src/sgml/ref/repack.sgml
+++ b/doc/src/sgml/ref/repack.sgml
@@ -308,15 +308,6 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] USING
</listitem>
</itemizedlist>
</para>
-
- <warning>
- <para>
- <command>REPACK</command> with the <literal>CONCURRENTLY</literal>
- option is not MVCC-safe, see <xref linkend="mvcc-caveats"/> for
- details.
- </para>
- </warning>
-
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 63b848473f8..91119da5cd5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -311,7 +311,8 @@ toast_save_datum(Relation rel, Datum value,
toasttup = heap_form_tuple(toasttupDesc, t_values, t_isnull);
- heap_insert(toastrel, toasttup, mycid, options, NULL);
+ heap_insert(toastrel, toasttup, GetCurrentTransactionId(), mycid,
+ options, NULL);
/*
* Create the index entry. We cheat a little here by not using
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index e11833f01b4..94ca07e4b55 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2085,7 +2085,7 @@ ReleaseBulkInsertStatePin(BulkInsertState bistate)
/*
* heap_insert - insert tuple into a heap
*
- * The new tuple is stamped with current transaction ID and the specified
+ * The new tuple is stamped with specified transaction ID and the specified
* command ID.
*
* See table_tuple_insert for comments about most of the input flags, except
@@ -2101,15 +2101,16 @@ ReleaseBulkInsertStatePin(BulkInsertState bistate)
* reflected into *tup.
*/
void
-heap_insert(Relation relation, HeapTuple tup, CommandId cid,
- int options, BulkInsertState bistate)
+heap_insert(Relation relation, HeapTuple tup, TransactionId xid,
+ CommandId cid, int options, BulkInsertState bistate)
{
- TransactionId xid = GetCurrentTransactionId();
HeapTuple heaptup;
Buffer buffer;
Buffer vmbuffer = InvalidBuffer;
bool all_visible_cleared = false;
+ Assert(TransactionIdIsValid(xid));
+
/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(tup->t_data) <=
RelationGetNumberOfAttributes(relation));
@@ -2375,7 +2376,6 @@ void
heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
CommandId cid, int options, BulkInsertState bistate)
{
- TransactionId xid = GetCurrentTransactionId();
HeapTuple *heaptuples;
int i;
int ndone;
@@ -2408,7 +2408,7 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
tuple = ExecFetchSlotHeapTuple(slots[i], true, NULL);
slots[i]->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slots[i]->tts_tableOid;
- heaptuples[i] = heap_prepare_insert(relation, tuple, xid, cid,
+ heaptuples[i] = heap_prepare_insert(relation, tuple, GetCurrentTransactionId(), cid,
options);
}
@@ -2746,7 +2746,8 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
void
simple_heap_insert(Relation relation, HeapTuple tup)
{
- heap_insert(relation, tup, GetCurrentCommandId(true), 0, NULL);
+ heap_insert(relation, tup, GetCurrentTransactionId(),
+ GetCurrentCommandId(true), 0, NULL);
}
/*
@@ -2803,11 +2804,10 @@ xmax_infomask_changed(uint16 new_infomask, uint16 old_infomask)
*/
TM_Result
heap_delete(Relation relation, const ItemPointerData *tid,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, bool changingPart, bool walLogical)
{
TM_Result result;
- TransactionId xid = GetCurrentTransactionId();
ItemId lp;
HeapTupleData tp;
Page page;
@@ -2824,6 +2824,7 @@ heap_delete(Relation relation, const ItemPointerData *tid,
bool old_key_copied = false;
Assert(ItemPointerIsValid(tid));
+ Assert(TransactionIdIsValid(xid));
AssertHasSnapshotForToast(relation);
@@ -3240,7 +3241,7 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
TM_Result result;
TM_FailureData tmfd;
- result = heap_delete(relation, tid,
+ result = heap_delete(relation, tid, GetCurrentTransactionId(),
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, false, /* changingPart */
@@ -3283,12 +3284,11 @@ simple_heap_delete(Relation relation, const ItemPointerData *tid)
*/
TM_Result
heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, LockTupleMode *lockmode,
TU_UpdateIndexes *update_indexes, bool walLogical)
{
TM_Result result;
- TransactionId xid = GetCurrentTransactionId();
Bitmapset *hot_attrs;
Bitmapset *sum_attrs;
Bitmapset *key_attrs;
@@ -3328,6 +3328,7 @@ heap_update(Relation relation, const ItemPointerData *otid, HeapTuple newtup,
infomask2_new_tuple;
Assert(ItemPointerIsValid(otid));
+ Assert(TransactionIdIsValid(xid));
/* Cheap, simplistic check that the tuple matches the rel's rowtype. */
Assert(HeapTupleHeaderGetNatts(newtup->t_data) <=
@@ -4534,7 +4535,7 @@ simple_heap_update(Relation relation, const ItemPointerData *otid, HeapTuple tup
TM_FailureData tmfd;
LockTupleMode lockmode;
- result = heap_update(relation, otid, tup,
+ result = heap_update(relation, otid, tup, GetCurrentTransactionId(),
GetCurrentCommandId(true), InvalidSnapshot,
true /* wait for commit */ ,
&tmfd, &lockmode, update_indexes,
@@ -5373,8 +5374,6 @@ compute_new_xmax_infomask(TransactionId xmax, uint16 old_infomask,
uint16 new_infomask,
new_infomask2;
- Assert(TransactionIdIsCurrentTransactionId(add_to_xmax));
-
l5:
new_infomask = 0;
new_infomask2 = 0;
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index e6d630fa2f7..b49f9add5bb 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -252,7 +252,8 @@ heapam_tuple_insert(Relation relation, TupleTableSlot *slot, CommandId cid,
tuple->t_tableOid = slot->tts_tableOid;
/* Perform the insertion, and copy the resulting ItemPointer */
- heap_insert(relation, tuple, cid, options, bistate);
+ heap_insert(relation, tuple, GetCurrentTransactionId(), cid, options,
+ bistate);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
if (shouldFree)
@@ -275,7 +276,8 @@ heapam_tuple_insert_speculative(Relation relation, TupleTableSlot *slot,
options |= HEAP_INSERT_SPECULATIVE;
/* Perform the insertion, and copy the resulting ItemPointer */
- heap_insert(relation, tuple, cid, options, bistate);
+ heap_insert(relation, tuple, GetCurrentTransactionId(), cid, options,
+ bistate);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
if (shouldFree)
@@ -309,8 +311,8 @@ heapam_tuple_delete(Relation relation, ItemPointer tid, CommandId cid,
* the storage itself is cleaning the dead tuples by itself, it is the
* time to call the index tuple deletion also.
*/
- return heap_delete(relation, tid, cid, crosscheck, wait, tmfd, changingPart,
- true);
+ return heap_delete(relation, tid, GetCurrentTransactionId(), cid,
+ crosscheck, wait, tmfd, changingPart, true);
}
@@ -328,7 +330,8 @@ heapam_tuple_update(Relation relation, ItemPointer otid, TupleTableSlot *slot,
slot->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slot->tts_tableOid;
- result = heap_update(relation, otid, tuple, cid, crosscheck, wait,
+ result = heap_update(relation, otid, tuple, GetCurrentTransactionId(),
+ cid, crosscheck, wait,
tmfd, lockmode, update_indexes, true);
ItemPointerCopy(&tuple->t_self, &slot->tts_tid);
@@ -2441,9 +2444,16 @@ reform_and_rewrite_tuple(HeapTuple tuple,
* flag to skip logical decoding: as soon as REPACK CONCURRENTLY swaps
* the relation files, it drops this relation, so no logical
* replication subscription should need the data.
+ *
+ * It is also crucial to stamp the new record with the exact same xid
+ * and cid, because the tuple must be visible to the snapshots of the
+ * concurrent transactions later.
*/
- heap_insert(NewHeap, copiedTuple, GetCurrentCommandId(true),
- HEAP_INSERT_NO_LOGICAL, NULL);
+ // TODO: looks like cid is not required
+ CommandId cid = HeapTupleHeaderGetRawCommandId(tuple->t_data);
+ TransactionId xid = HeapTupleHeaderGetXmin(tuple->t_data);
+
+ heap_insert(NewHeap, copiedTuple, xid, cid, HEAP_INSERT_NO_LOGICAL, NULL);
}
heap_freetuple(copiedTuple);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index f2a2ec6d3e5..0b79175f7f4 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -58,6 +58,7 @@
#include "storage/ipc.h"
#include "storage/lmgr.h"
#include "storage/predicate.h"
+#include "storage/procarray.h"
#include "storage/procsignal.h"
#include "tcop/tcopprot.h"
#include "utils/acl.h"
@@ -249,15 +250,20 @@ static bool decode_concurrent_changes(LogicalDecodingContext *ctx,
DecodingWorkerShared *shared);
static void apply_concurrent_changes(BufFile *file, ChangeDest *dest);
static void apply_concurrent_insert(Relation rel, HeapTuple tup,
+ TransactionId xid,
IndexInsertState *iistate,
TupleTableSlot *index_slot);
static void apply_concurrent_update(Relation rel, HeapTuple tup,
HeapTuple tup_target,
+ TransactionId xid,
IndexInsertState *iistate,
TupleTableSlot *index_slot);
-static void apply_concurrent_delete(Relation rel, HeapTuple tup_target);
+static void apply_concurrent_delete(Relation rel,
+ TransactionId xid,
+ HeapTuple tup_target);
static HeapTuple find_target_tuple(Relation rel, ChangeDest *dest,
HeapTuple tup_key,
+ Snapshot snapshot,
TupleTableSlot *ident_slot);
static void process_concurrent_changes(XLogRecPtr end_of_wal,
ChangeDest *dest,
@@ -1091,7 +1097,14 @@ rebuild_relation(Relation OldHeap, Relation index, bool verbose, bool concurrent
/* The historic snapshot won't be needed anymore. */
if (snapshot)
+ {
+ TransactionId xmin = snapshot->xmin;
PopActiveSnapshot();
+ Assert(concurrent);
+ // TODO: seems like it not required: need to check SnapBuildInitialSnapshotForRepack
+ WaitForOlderSnapshots(xmin, false);
+ }
+
if (concurrent)
{
@@ -1382,30 +1395,35 @@ copy_table_data(Relation NewHeap, Relation OldHeap, Relation OldIndex,
* not to be aggressive about this.
*/
memset(¶ms, 0, sizeof(VacuumParams));
- vacuum_get_cutoffs(OldHeap, params, &cutoffs);
-
- /*
- * FreezeXid will become the table's new relfrozenxid, and that mustn't go
- * backwards, so take the max.
- */
+ if (!concurrent)
{
TransactionId relfrozenxid = OldHeap->rd_rel->relfrozenxid;
+ MultiXactId relminmxid = OldHeap->rd_rel->relminmxid;
+ vacuum_get_cutoffs(OldHeap, params, &cutoffs);
+ /*
+ * FreezeXid will become the table's new relfrozenxid, and that mustn't go
+ * backwards, so take the max.
+ */
if (TransactionIdIsValid(relfrozenxid) &&
TransactionIdPrecedes(cutoffs.FreezeLimit, relfrozenxid))
cutoffs.FreezeLimit = relfrozenxid;
- }
-
- /*
- * MultiXactCutoff, similarly, shouldn't go backwards either.
- */
- {
- MultiXactId relminmxid = OldHeap->rd_rel->relminmxid;
-
+ /*
+ * MultiXactCutoff, similarly, shouldn't go backwards either.
+ */
if (MultiXactIdIsValid(relminmxid) &&
MultiXactIdPrecedes(cutoffs.MultiXactCutoff, relminmxid))
cutoffs.MultiXactCutoff = relminmxid;
}
+ else
+ {
+ /*
+ * In concurrent mode we reuse all the xmin/xmax,
+ * so just use current values for simplicity.
+ */
+ cutoffs.FreezeLimit = OldHeap->rd_rel->relfrozenxid;
+ cutoffs.MultiXactCutoff = OldHeap->rd_rel->relminmxid;
+ }
/*
* Decide whether to use an indexscan or seqscan-and-optional-sort to scan
@@ -2745,6 +2763,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
size_t nread;
HeapTuple tup,
tup_exist;
+ TransactionId xid;
CHECK_FOR_INTERRUPTS();
@@ -2761,6 +2780,17 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
tup->t_len = t_len;
ItemPointerSetInvalid(&tup->t_self);
tup->t_tableOid = RelationGetRelid(dest->rel);
+ BufFileReadExact(file, &xid, sizeof(TransactionId));
+
+ if (TransactionIdIsValid(xid) && TransactionIdIsInProgress(xid))
+ {
+ /* xmin is committed for sure because we got that update from reorderbuffer.
+ * but there is a possibility procarray is not yet updated and current backend still see it as
+ * in-progress. Let's wait for procarray to be updated. */
+ XactLockTableWait(xid, NULL, NULL, XLTW_None);
+ Assert(!TransactionIdIsInProgress(xid));
+ Assert(TransactionIdDidCommit(xid));
+ }
if (kind == CHANGE_UPDATE_OLD)
{
@@ -2771,7 +2801,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
{
Assert(tup_old == NULL);
- apply_concurrent_insert(rel, tup, dest->iistate, index_slot);
+ apply_concurrent_insert(rel, tup, xid, dest->iistate, index_slot);
pfree(tup);
}
@@ -2790,17 +2820,21 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
}
/*
- * Find the tuple to be updated or deleted.
+ * Find the tuple to be updated or deleted using SnapshotSelf.
+ * That way we receive the last alive version in case of HOT chain.
+ * It is guaranteed there is no any non-yet committed, but updated version
+ * because we here replaying all-committed transactions without any concurrency
+ * involved.
*/
- tup_exist = find_target_tuple(rel, dest, tup_key, ident_slot);
+ tup_exist = find_target_tuple(rel, dest, tup_key, SnapshotSelf, ident_slot);
if (tup_exist == NULL)
elog(ERROR, "failed to find target tuple");
if (kind == CHANGE_UPDATE_NEW)
- apply_concurrent_update(rel, tup, tup_exist, dest->iistate,
+ apply_concurrent_update(rel, tup, tup_exist, xid, dest->iistate,
index_slot);
else
- apply_concurrent_delete(rel, tup_exist);
+ apply_concurrent_delete(rel, xid, tup_exist);
if (tup_old != NULL)
{
@@ -2819,6 +2853,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
*/
if (kind != CHANGE_UPDATE_OLD)
{
+ // TODO: not sure it is required at all: we are replaying committed transactions stamping them with committed XID
CommandCounterIncrement();
UpdateActiveSnapshotCommandId();
}
@@ -2830,7 +2865,7 @@ apply_concurrent_changes(BufFile *file, ChangeDest *dest)
}
static void
-apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
+apply_concurrent_insert(Relation rel, HeapTuple tup, TransactionId xid, IndexInsertState *iistate,
TupleTableSlot *index_slot)
{
List *recheck;
@@ -2840,9 +2875,12 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
* Like simple_heap_insert(), but make sure that the INSERT is not
* logically decoded - see reform_and_rewrite_tuple() for more
* information.
+ *
+ * Use already committed xid to stamp the tuple.
*/
- heap_insert(rel, tup, GetCurrentCommandId(true), HEAP_INSERT_NO_LOGICAL,
- NULL);
+ Assert(TransactionIdIsValid(xid));
+ heap_insert(rel, tup, xid, GetCurrentCommandId(true),
+ HEAP_INSERT_NO_LOGICAL, NULL);
/*
* Update indexes.
@@ -2850,6 +2888,7 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
* In case functions in the index need the active snapshot and caller
* hasn't set one.
*/
+ PushActiveSnapshot(GetLatestSnapshot());
ExecStoreHeapTuple(tup, index_slot, false);
recheck = ExecInsertIndexTuples(iistate->rri,
index_slot,
@@ -2860,6 +2899,7 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
NIL, /* arbiterIndexes */
false /* onlySummarizing */
);
+ PopActiveSnapshot();
/*
* If recheck is required, it must have been preformed on the source
@@ -2873,6 +2913,7 @@ apply_concurrent_insert(Relation rel, HeapTuple tup, IndexInsertState *iistate,
static void
apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
+ TransactionId xid,
IndexInsertState *iistate, TupleTableSlot *index_slot)
{
LockTupleMode lockmode;
@@ -2887,9 +2928,12 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
*
* Do it like in simple_heap_update(), except for 'wal_logical' (and
* except for 'wait').
+ *
+ * Use already committed xid to stamp the tuple.
*/
+ Assert(TransactionIdIsValid(xid));
res = heap_update(rel, &tup_target->t_self, tup,
- GetCurrentCommandId(true),
+ xid, GetCurrentCommandId(true),
InvalidSnapshot,
false, /* no wait - only we are doing changes */
&tmfd, &lockmode, &update_indexes,
@@ -2901,6 +2945,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
if (update_indexes != TU_None)
{
+ PushActiveSnapshot(GetLatestSnapshot());
recheck = ExecInsertIndexTuples(iistate->rri,
index_slot,
iistate->estate,
@@ -2910,6 +2955,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
NIL, /* arbiterIndexes */
/* onlySummarizing */
update_indexes == TU_Summarizing);
+ PopActiveSnapshot();
list_free(recheck);
}
@@ -2917,7 +2963,7 @@ apply_concurrent_update(Relation rel, HeapTuple tup, HeapTuple tup_target,
}
static void
-apply_concurrent_delete(Relation rel, HeapTuple tup_target)
+apply_concurrent_delete(Relation rel, TransactionId xid, HeapTuple tup_target)
{
TM_Result res;
TM_FailureData tmfd;
@@ -2927,9 +2973,12 @@ apply_concurrent_delete(Relation rel, HeapTuple tup_target)
*
* Do it like in simple_heap_delete(), except for 'wal_logical' (and
* except for 'wait').
+ *
+ * Use already committed xid to stamp the tuple.
*/
- res = heap_delete(rel, &tup_target->t_self, GetCurrentCommandId(true),
- InvalidSnapshot, false,
+ Assert(TransactionIdIsValid(xid));
+ res = heap_delete(rel, &tup_target->t_self, xid,
+ GetCurrentCommandId(true), InvalidSnapshot, false,
&tmfd,
false, /* no wait - only we are doing changes */
false /* wal_logical */ );
@@ -2950,7 +2999,7 @@ apply_concurrent_delete(Relation rel, HeapTuple tup_target)
*/
static HeapTuple
find_target_tuple(Relation rel, ChangeDest *dest, HeapTuple tup_key,
- TupleTableSlot *ident_slot)
+ Snapshot snapshot, TupleTableSlot *ident_slot)
{
Relation ident_index = dest->ident_index;
IndexScanDesc scan;
@@ -2959,7 +3008,7 @@ find_target_tuple(Relation rel, ChangeDest *dest, HeapTuple tup_key,
HeapTuple result = NULL;
/* XXX no instrumentation for now */
- scan = index_beginscan(rel, ident_index, GetActiveSnapshot(),
+ scan = index_beginscan(rel, ident_index, snapshot,
NULL, dest->ident_key_nentries, 0);
/*
diff --git a/src/backend/replication/pgoutput_repack/pgoutput_repack.c b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
index fb9956d392d..8d796e0a684 100644
--- a/src/backend/replication/pgoutput_repack/pgoutput_repack.c
+++ b/src/backend/replication/pgoutput_repack/pgoutput_repack.c
@@ -29,7 +29,8 @@ static void plugin_commit_txn(LogicalDecodingContext *ctx,
static void plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Relation rel, ReorderBufferChange *change);
static void store_change(LogicalDecodingContext *ctx,
- ConcurrentChangeKind kind, HeapTuple tuple);
+ ConcurrentChangeKind kind, HeapTuple tuple,
+ TransactionId xid);
void
_PG_output_plugin_init(OutputPluginCallbacks *cb)
@@ -120,7 +121,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (newtuple == NULL)
elog(ERROR, "Incomplete insert info.");
- store_change(ctx, CHANGE_INSERT, newtuple);
+ store_change(ctx, CHANGE_INSERT, newtuple, change->txn->xid);
}
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -137,9 +138,11 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
elog(ERROR, "Incomplete update info.");
if (oldtuple != NULL)
- store_change(ctx, CHANGE_UPDATE_OLD, oldtuple);
+ store_change(ctx, CHANGE_UPDATE_OLD, oldtuple,
+ change->txn->xid);
- store_change(ctx, CHANGE_UPDATE_NEW, newtuple);
+ store_change(ctx, CHANGE_UPDATE_NEW, newtuple,
+ change->txn->xid);
}
break;
case REORDER_BUFFER_CHANGE_DELETE:
@@ -152,7 +155,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (oldtuple == NULL)
elog(ERROR, "Incomplete delete info.");
- store_change(ctx, CHANGE_DELETE, oldtuple);
+ store_change(ctx, CHANGE_DELETE, oldtuple, change->txn->xid);
}
break;
default:
@@ -165,7 +168,7 @@ plugin_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
/* Store concurrent data change. */
static void
store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
- HeapTuple tuple)
+ HeapTuple tuple, TransactionId xid)
{
RepackDecodingState *dstate;
char kind_byte = (char) kind;
@@ -195,6 +198,7 @@ store_change(LogicalDecodingContext *ctx, ConcurrentChangeKind kind,
BufFileWrite(dstate->file, &tuple->t_len, sizeof(tuple->t_len));
/* ... and the tuple itself. */
BufFileWrite(dstate->file, tuple->t_data, tuple->t_len);
+ BufFileWrite(dstate->file, &xid, sizeof(TransactionId));
/* Free the flat copy if created above. */
if (flattened)
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index b7cd25896f6..d9776f61a0d 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -354,20 +354,20 @@ extern BulkInsertState GetBulkInsertState(void);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
-extern void heap_insert(Relation relation, HeapTuple tup, CommandId cid,
+extern void heap_insert(Relation relation, HeapTuple tup, TransactionId xid, CommandId cid,
int options, BulkInsertState bistate);
extern void heap_multi_insert(Relation relation, TupleTableSlot **slots,
int ntuples, CommandId cid, int options,
BulkInsertState bistate);
extern TM_Result heap_delete(Relation relation, const ItemPointerData *tid,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, bool changingPart,
bool wal_logical);
extern void heap_finish_speculative(Relation relation, const ItemPointerData *tid);
extern void heap_abort_speculative(Relation relation, const ItemPointerData *tid);
extern TM_Result heap_update(Relation relation, const ItemPointerData *otid,
HeapTuple newtup,
- CommandId cid, Snapshot crosscheck, bool wait,
+ TransactionId xid, CommandId cid, Snapshot crosscheck, bool wait,
TM_FailureData *tmfd, LockTupleMode *lockmode,
TU_UpdateIndexes *update_indexes, bool wal_logical);
extern TM_Result heap_lock_tuple(Relation relation, HeapTuple tuple,
diff --git a/src/test/modules/injection_points/specs/repack.spec b/src/test/modules/injection_points/specs/repack.spec
index d727a9b056b..accd42d78aa 100644
--- a/src/test/modules/injection_points/specs/repack.spec
+++ b/src/test/modules/injection_points/specs/repack.spec
@@ -85,9 +85,6 @@ step change_new
# When applying concurrent data changes, we should see the effects of an
# in-progress subtransaction.
#
-# XXX Not sure this test is useful now - it was designed for the patch that
-# preserves tuple visibility and which therefore modifies
-# TransactionIdIsCurrentTransactionId().
step change_subxact1
{
BEGIN;
@@ -102,7 +99,6 @@ step change_subxact1
# When applying concurrent data changes, we should not see the effects of a
# rolled back subtransaction.
#
-# XXX Is this test useful? See above.
step change_subxact2
{
BEGIN;
--
2.43.0
Hello!
On Tue, Dec 16, 2025 at 3:48 AM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:
First version of TAP-based test in attachment. I think we should not
hurry with push - let's make sure it really stable now.
Second version - some issues fixed, added some log and diagnostic for
the next fail.
Mikhail.
Attachments:
nocfbot_v2-0001-Replace-flaky-isolation-tests-with-a-TAP-te.patchtext/x-patch; charset=US-ASCII; name=nocfbot_v2-0001-Replace-flaky-isolation-tests-with-a-TAP-te.patchDownload
From f532028260b8da0697772d0f5c0c16c58fc15f02 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Mon, 15 Dec 2025 12:03:04 +0100
Subject: [PATCH vnocfbo2] Replace flaky isolation tests with a TAP test for
concurrent upserts
The isolation tests verifying INSERT ON CONFLICT behavior during
concurrent index creation/reindexing were prone to flakiness. The
timing dependencies on injection points were difficult to guarantee
in the isolation tester.
Remove the disabled isolation tests and their specs/expected output.
Add TAP test '010_index_concurrently_upsert.pl' to
cover these scenarios more reliably. The TAP test now includes better
session management and more robust injection point
waiting logic.
---
src/test/modules/injection_points/Makefile | 8 -
.../index-concurrently-upsert-predicate.out | 123 ---
.../index-concurrently-upsert-predicate_1.out | 124 ---
.../expected/index-concurrently-upsert.out | 123 ---
.../expected/index-concurrently-upsert_1.out | 124 ---
...ndex-concurrently-upsert-on-constraint.out | 238 -----
...eindex-concurrently-upsert-partitioned.out | 238 -----
.../expected/reindex-concurrently-upsert.out | 238 -----
src/test/modules/injection_points/meson.build | 6 -
.../index-concurrently-upsert-predicate.spec | 124 ---
.../specs/index-concurrently-upsert.spec | 123 ---
...dex-concurrently-upsert-on-constraint.spec | 110 ---
...index-concurrently-upsert-partitioned.spec | 113 ---
.../specs/reindex-concurrently-upsert.spec | 111 ---
src/test/modules/test_misc/Makefile | 3 +
src/test/modules/test_misc/meson.build | 3 +
.../t/010_index_concurrently_upsert.pl | 894 ++++++++++++++++++
17 files changed, 900 insertions(+), 1803 deletions(-)
delete mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
delete mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
delete mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert.out
delete mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
delete mode 100644 src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out
delete mode 100644 src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
delete mode 100644 src/test/modules/injection_points/expected/reindex-concurrently-upsert.out
delete mode 100644 src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
delete mode 100644 src/test/modules/injection_points/specs/index-concurrently-upsert.spec
delete mode 100644 src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec
delete mode 100644 src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
delete mode 100644 src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec
create mode 100644 src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index bfdb3f53377..3cb50d13e52 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -16,14 +16,6 @@ ISOLATION = basic \
inplace \
syscache-update-pruned
-# Temporarily disabled because of flakiness
-#ISOLATION =+
-# index-concurrently-upsert \
-# index-concurrently-upsert-predicate \
-# reindex-concurrently-upsert \
-# reindex-concurrently-upsert-on-constraint \
-# reindex-concurrently-upsert-partitioned
-
# The injection points are cluster-wide, so disable installcheck
NO_INSTALLCHECK = 1
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
deleted file mode 100644
index 77e7d1a7815..00000000000
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
+++ /dev/null
@@ -1,123 +0,0 @@
-Parsed test spec with 5 sessions
-
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot:
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s4_wakeup_s1_setup:
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-
-case
-----
-
-(1 row)
-
-step s3_start_create_index:
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_define_index_before_set_valid:
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_wakeup_s1_from_invalidate_catalog_snapshot:
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
deleted file mode 100644
index e72848d6a78..00000000000
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
+++ /dev/null
@@ -1,124 +0,0 @@
-Parsed test spec with 5 sessions
-
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot:
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
- <waiting ...>
-step s4_wakeup_s1_setup:
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-
-case
-----
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot: <... completed>
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_create_index:
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_define_index_before_set_valid:
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_wakeup_s1_from_invalidate_catalog_snapshot:
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert.out b/src/test/modules/injection_points/expected/index-concurrently-upsert.out
deleted file mode 100644
index a2ef122625c..00000000000
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert.out
+++ /dev/null
@@ -1,123 +0,0 @@
-Parsed test spec with 5 sessions
-
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot:
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s4_wakeup_s1_setup:
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-
-case
-----
-
-(1 row)
-
-step s3_start_create_index:
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_define_index_before_set_valid:
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_wakeup_s1_from_invalidate_catalog_snapshot:
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out b/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
deleted file mode 100644
index ee3b6641b90..00000000000
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
+++ /dev/null
@@ -1,124 +0,0 @@
-Parsed test spec with 5 sessions
-
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot:
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
- <waiting ...>
-step s4_wakeup_s1_setup:
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-
-case
-----
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot: <... completed>
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_create_index:
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_define_index_before_set_valid:
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_wakeup_s1_from_invalidate_catalog_snapshot:
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out b/src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out
deleted file mode 100644
index c1ac1f77c61..00000000000
--- a/src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out
+++ /dev/null
@@ -1,238 +0,0 @@
-Parsed test spec with 4 sessions
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_swap:
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_swap:
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out b/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
deleted file mode 100644
index 4c79a43d986..00000000000
--- a/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
+++ /dev/null
@@ -1,238 +0,0 @@
-Parsed test spec with 4 sessions
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_swap:
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_swap:
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex-concurrently-upsert.out b/src/test/modules/injection_points/expected/reindex-concurrently-upsert.out
deleted file mode 100644
index c9cc9989d02..00000000000
--- a/src/test/modules/injection_points/expected/reindex-concurrently-upsert.out
+++ /dev/null
@@ -1,238 +0,0 @@
-Parsed test spec with 4 sessions
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_swap:
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_swap:
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 493e11053dc..2c6cf54a33e 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -46,12 +46,6 @@ tests += {
'basic',
'inplace',
'syscache-update-pruned',
- # temporarily disabled because of flakiness
- # 'index-concurrently-upsert',
- # 'index-concurrently-upsert-predicate',
- # 'reindex-concurrently-upsert',
- # 'reindex-concurrently-upsert-on-constraint',
- # 'reindex-concurrently-upsert-partitioned',
],
'runningcheck': false, # see syscache-update-pruned
# Some tests wait for all snapshots, so avoid parallel execution
diff --git a/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec b/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
deleted file mode 100644
index d9b8d27fd1f..00000000000
--- a/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
+++ /dev/null
@@ -1,124 +0,0 @@
-# This test verifies INSERT ON CONFLICT DO UPDATE behavior concurrent with
-# CREATE INDEX CONCURRENTLY a partial index.
-#
-# - s1: UPSERT a tuple
-# - s2: UPSERT the same tuple
-# - s3: CREATE UNIQUE INDEX CONCURRENTLY (with a predicate)
-#
-# - s4 and s5: control concurrency via injection points
-
-setup
-{
- CREATE EXTENSION injection_points;
- CREATE SCHEMA test;
- CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
- CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
- ALTER TABLE test.tbl SET (parallel_workers=0);
-}
-
-teardown
-{
- DROP SCHEMA test CASCADE;
- DROP EXTENSION injection_points;
-}
-
-session s1
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
-}
-step s1_attach_invalidate_catalog_snapshot
-{
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
-}
-step s1_start_upsert
-{
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
-}
-
-session s2
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
-}
-step s2_start_upsert
-{
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
-}
-
-session s3
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('define-index-before-set-valid', 'wait');
-}
-step s3_start_create_index
-{
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
-}
-
-session s4
-# Step s1_attach_invalidate_catalog_snapshot sleeps or not depending on
-# build conditions (CATCACHE_FORCE_RELEASE). Here we send a wakeup signal if
-# it's sleeping or do nothing otherwise, and print a null value in either
-# case.
-step s4_wakeup_s1_setup
-{
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-}
-step s4_wakeup_s1
-{
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-}
-step s4_wakeup_s2
-{
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-}
-step s4_wakeup_define_index_before_set_valid
-{
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-}
-
-session s5
-step s5_wakeup_s1_from_invalidate_catalog_snapshot
-{
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-}
-
-permutation
- s1_attach_invalidate_catalog_snapshot
- s4_wakeup_s1_setup
- s3_start_create_index(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
- s4_wakeup_define_index_before_set_valid
- s2_start_upsert(s1_start_upsert)
- s5_wakeup_s1_from_invalidate_catalog_snapshot
- s4_wakeup_s2
- s4_wakeup_s1
diff --git a/src/test/modules/injection_points/specs/index-concurrently-upsert.spec b/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
deleted file mode 100644
index 6e08af74a93..00000000000
--- a/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
+++ /dev/null
@@ -1,123 +0,0 @@
-# This test verifies INSERT ON CONFLICT DO UPDATE behavior concurrent with
-# CREATE INDEX CONCURRENTLY.
-#
-# - s1: UPSERT a tuple
-# - s2: UPSERT the same tuple
-# - s3: CREATE UNIQUE INDEX CONCURRENTLY
-#
-# - s4: Control concurrency using injection points
-
-setup
-{
- CREATE EXTENSION injection_points;
- CREATE SCHEMA test;
- CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
- ALTER TABLE test.tbl SET (parallel_workers=0);
-}
-
-teardown
-{
- DROP SCHEMA test CASCADE;
- DROP EXTENSION injection_points;
-}
-
-session s1
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
-}
-step s1_attach_invalidate_catalog_snapshot
-{
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
-}
-step s1_start_upsert
-{
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s2
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
-}
-step s2_start_upsert
-{
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s3
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('define-index-before-set-valid', 'wait');
-}
-step s3_start_create_index
-{
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
-}
-
-session s4
-# Step s1_attach_invalidate_catalog_snapshot sleeps or not depending on
-# build conditions (CATCACHE_FORCE_RELEASE). Here we send a wakeup signal if
-# it's sleeping or do nothing otherwise, and print a null value in either
-# case.
-step s4_wakeup_s1_setup
-{
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-}
-step s4_wakeup_s1
-{
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-}
-step s4_wakeup_s2
-{
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-}
-step s4_wakeup_define_index_before_set_valid
-{
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-}
-
-session s5
-step s5_wakeup_s1_from_invalidate_catalog_snapshot
-{
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-}
-
-permutation
- s1_attach_invalidate_catalog_snapshot
- s4_wakeup_s1_setup
- s3_start_create_index(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
- s4_wakeup_define_index_before_set_valid
- s2_start_upsert(s1_start_upsert)
- s5_wakeup_s1_from_invalidate_catalog_snapshot
- s4_wakeup_s2
- s4_wakeup_s1
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec
deleted file mode 100644
index 4bbdda3cf04..00000000000
--- a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec
+++ /dev/null
@@ -1,110 +0,0 @@
-# Test race conditions involving:
-#
-# - s1: UPSERT a tuple
-# - s2: UPSERT the same tuple
-# - s3: concurrently REINDEX the primary key
-#
-# - s4: operations with injection points
-
-setup
-{
- CREATE EXTENSION injection_points;
- CREATE SCHEMA test;
- CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
- ALTER TABLE test.tbl SET (parallel_workers=0);
-}
-
-teardown
-{
- DROP SCHEMA test CASCADE;
- DROP EXTENSION injection_points;
-}
-
-session s1
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
-}
-step s1_start_upsert
-{
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
-}
-
-session s2
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
-}
-step s2_start_upsert
-{
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
-}
-
-session s3
-setup
-{
- SELECT injection_points_set_local();
-}
-step s3_setup_wait_before_set_dead
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-}
-step s3_setup_wait_before_swap
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-}
-step s3_start_reindex
-{
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
-}
-
-session s4
-step s4_wakeup_to_swap
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-}
-step s4_wakeup_s1
-{
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-}
-step s4_wakeup_s2
-{
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-}
-step s4_wakeup_to_set_dead
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-}
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_set_dead
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_s2
-
-permutation
- s3_setup_wait_before_swap
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_swap
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s2
- s4_wakeup_s1
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_to_set_dead
- s4_wakeup_s2
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
deleted file mode 100644
index c3504b9ef38..00000000000
--- a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
+++ /dev/null
@@ -1,113 +0,0 @@
-# This test verifies INSERT ON CONFLICT DO UPDATE behavior on partitioned
-# tables concurrent with REINDEX CONCURRENTLY.
-#
-# - s1: UPSERT a tuple
-# - s2: UPSERT the same tuple
-# - s3: concurrently REINDEX the primary key index
-#
-# - s4: controls concurrency via injection points
-
-setup
-{
- CREATE EXTENSION injection_points;
- CREATE SCHEMA test;
- CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
- CREATE TABLE test.tbl_partition PARTITION OF test.tbl
- FOR VALUES FROM (0) TO (10000)
- WITH (parallel_workers = 0);
-}
-
-teardown
-{
- DROP SCHEMA test CASCADE;
- DROP EXTENSION injection_points;
-}
-
-session s1
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
-}
-step s1_start_upsert
-{
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s2
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
-}
-step s2_start_upsert
-{
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s3
-setup
-{
- SELECT injection_points_set_local();
-}
-step s3_setup_wait_before_set_dead
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-}
-step s3_setup_wait_before_swap
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-}
-step s3_start_reindex
-{
- REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
-}
-
-session s4
-step s4_wakeup_to_swap
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-}
-step s4_wakeup_s1
-{
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-}
-step s4_wakeup_s2
-{
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-}
-step s4_wakeup_to_set_dead
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-}
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_set_dead
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_s2
-
-permutation
- s3_setup_wait_before_swap
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_swap
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s2
- s4_wakeup_s1
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_to_set_dead
- s4_wakeup_s2
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec
deleted file mode 100644
index 1b043a48ff4..00000000000
--- a/src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec
+++ /dev/null
@@ -1,111 +0,0 @@
-# This test verifies INSERT ON CONFLICT DO UPDATE behavior concurrent with
-# REINDEX CONCURRENTLY.
-#
-# - s1: UPSERT a tuple
-# - s2: UPSERT the same tuple
-# - s3: REINDEX concurrent primary key index
-#
-# - s4: controls concurrency via injection points
-
-setup
-{
- CREATE EXTENSION injection_points;
- CREATE SCHEMA test;
- CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
- ALTER TABLE test.tbl SET (parallel_workers=0);
-}
-
-teardown
-{
- DROP SCHEMA test CASCADE;
- DROP EXTENSION injection_points;
-}
-
-session s1
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
-}
-step s1_start_upsert
-{
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s2
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
-}
-step s2_start_upsert
-{
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s3
-setup
-{
- SELECT injection_points_set_local();
-}
-step s3_setup_wait_before_set_dead
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-}
-step s3_setup_wait_before_swap
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-}
-step s3_start_reindex
-{
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
-}
-
-session s4
-step s4_wakeup_to_swap
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-}
-step s4_wakeup_s1
-{
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-}
-step s4_wakeup_s2
-{
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-}
-step s4_wakeup_to_set_dead
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-}
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_set_dead
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_s2
-
-permutation
- s3_setup_wait_before_swap
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_swap
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s2
- s4_wakeup_s1
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_to_set_dead
- s4_wakeup_s2
diff --git a/src/test/modules/test_misc/Makefile b/src/test/modules/test_misc/Makefile
index 399b9094a38..fedbef071ef 100644
--- a/src/test/modules/test_misc/Makefile
+++ b/src/test/modules/test_misc/Makefile
@@ -5,6 +5,9 @@ TAP_TESTS = 1
EXTRA_INSTALL=src/test/modules/injection_points \
contrib/test_decoding
+# The injection points are cluster-wide, so disable installcheck
+NO_INSTALLCHECK = 1
+
export enable_injection_points
ifdef USE_PGXS
diff --git a/src/test/modules/test_misc/meson.build b/src/test/modules/test_misc/meson.build
index f258bf1ccd9..129c4ae587a 100644
--- a/src/test/modules/test_misc/meson.build
+++ b/src/test/modules/test_misc/meson.build
@@ -18,6 +18,9 @@ tests += {
't/007_catcache_inval.pl',
't/008_replslot_single_user.pl',
't/009_log_temp_files.pl',
+ 't/010_index_concurrently_upsert.pl',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
},
}
diff --git a/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl b/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
new file mode 100644
index 00000000000..55ea384edb3
--- /dev/null
+++ b/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
@@ -0,0 +1,894 @@
+
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+# Test INSERT ON CONFLICT DO UPDATE behavior concurrent with
+# CREATE INDEX CONCURRENTLY and REINDEX CONCURRENTLY.
+#
+# These tests verify the fix for "duplicate key value violates unique constraint"
+# errors that occurred when infer_arbiter_indexes() only considered indisvalid
+# indexes, causing different transactions to use different arbiter indexes.
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+if ($ENV{enable_injection_points} ne 'yes')
+{
+ plan skip_all => 'Injection points not supported by this build';
+}
+
+# Node initialization
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init();
+$node->start;
+
+# Check if the extension injection_points is available
+if (!$node->check_extension('injection_points'))
+{
+ plan skip_all => 'Extension injection_points not installed';
+}
+
+$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;');
+
+# Helper: Wait for a session to hit an injection point.
+# Optional second argument is timeout in seconds.
+# Returns true if found, false if timeout.
+# On timeout, logs diagnostic information about all active queries.
+sub wait_for_injection_point
+{
+ my ($node, $point_name, $timeout) = @_;
+ $timeout //= 120;
+
+ for (my $elapsed = 0; $elapsed < $timeout; $elapsed++)
+ {
+ my $pid = $node->safe_psql('postgres', qq[
+ SELECT pid FROM pg_stat_activity
+ WHERE wait_event_type = 'InjectionPoint'
+ AND wait_event = '$point_name'
+ LIMIT 1;
+ ]);
+ return 1 if $pid ne '';
+ sleep(1);
+ }
+
+ # Timeout - report diagnostic information
+ my $activity = $node->safe_psql('postgres', q[
+ SELECT format('pid=%s, state=%s, wait_event_type=%s, wait_event=%s, backend_xmin=%s, backend_xid=%s, query=%s',
+ pid, state, wait_event_type, wait_event, backend_xmin, backend_xid, left(query, 100))
+ FROM pg_stat_activity
+ ORDER BY pid;
+ ]);
+ diag("wait_for_injection_point timeout waiting for: $point_name\n" .
+ "Current queries in pg_stat_activity:\n$activity");
+
+ return 0;
+}
+
+# Helper: Wait for a specific backend to become idle.
+# Returns true if idle, false if timeout.
+sub wait_for_idle
+{
+ my ($node, $pid, $timeout) = @_;
+ $timeout //= 15;
+
+ for (my $elapsed = 0; $elapsed < $timeout; $elapsed++)
+ {
+ my $state = $node->safe_psql('postgres', qq[
+ SELECT state FROM pg_stat_activity WHERE pid = $pid;
+ ]);
+ return 1 if $state eq 'idle';
+ sleep(1);
+ }
+ return 0;
+}
+
+# Helper: Detach and wakeup an injection point
+sub wakeup_injection_point
+{
+ my ($node, $point_name) = @_;
+ $node->safe_psql(
+ 'postgres', qq[
+SELECT injection_points_detach('$point_name');
+SELECT injection_points_wakeup('$point_name');
+]);
+}
+
+# Wait for any pending query to complete, capture stderr, and close the session.
+# Returns the stderr output (excluding internal markers).
+sub safe_quit
+{
+ my ($session) = @_;
+
+ # Send a marker and wait for it to ensure any pending query completes
+ my $banner = "safe_quit_marker";
+ my $banner_match = qr/(^|\n)$banner\r?\n/;
+
+ $session->{stdin} .= "\\echo $banner\n\\warn $banner\n";
+
+ pump_until($session->{run}, $session->{timeout},
+ \$session->{stdout}, $banner_match);
+ pump_until($session->{run}, $session->{timeout},
+ \$session->{stderr}, $banner_match);
+
+ # Capture stderr (excluding the banner)
+ my $stderr = $session->{stderr};
+ $stderr =~ s/$banner_match//;
+
+ # Close the session
+ $session->quit;
+
+ return $stderr;
+}
+
+###############################################################################
+# Test 1: REINDEX CONCURRENTLY + UPSERT (wakeup at set-dead phase)
+# Based on reindex-concurrently-upsert.spec
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+# Create sessions with on_error_stop => 0 so psql doesn't exit on SQL errors.
+# This allows us to collect stderr and detect errors after the test completes.
+my $s1 = $node->background_psql('postgres', on_error_stop => 0);
+my $s2 = $node->background_psql('postgres', on_error_stop => 0);
+my $s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+# Setup injection points for each session
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+# s3 starts REINDEX (will block on reindex-relation-concurrently-before-set-dead)
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+# Wait for s3 to hit injection point
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+# s1 starts UPSERT (will block on check-exclusion-or-unique-constraint-no-conflict)
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+# Wait for s1 to hit injection point
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+# Wakeup s3 to continue (reindex-relation-concurrently-before-set-dead)
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+
+# s2 starts UPSERT (will block on exec-insert-before-insert-speculative)
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+# Wait for s2 to hit injection point
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+# Wakeup s1 (check-exclusion-or-unique-constraint-no-conflict)
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+# Wakeup s2 (exec-insert-before-insert-speculative)
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 1 (REINDEX swap): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 1 (REINDEX swap): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 1 (REINDEX swap): session s3 quit successfully');
+
+# Cleanup test 1
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 2: REINDEX CONCURRENTLY + UPSERT (wakeup at swap phase)
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-swap'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-swap');
+
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+is(safe_quit($s1), '', 'Test 2 (REINDEX set-dead): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 2 (REINDEX set-dead): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 2 (REINDEX set-dead): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 2b: REINDEX CONCURRENTLY + UPSERT (permutation 3: s1 wakes before reindex)
+# Different timing: s2 starts, then s1 wakes, then reindex wakes, then s2 wakes
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+# Start s2 BEFORE waking reindex (key difference from permutation 1)
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+# Wake s1 first, then reindex, then s2
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 2b (REINDEX perm3): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 2b (REINDEX perm3): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 2b (REINDEX perm3): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 3: REINDEX + UPSERT ON CONSTRAINT (set-dead phase)
+# Based on reindex-concurrently-upsert-on-constraint.spec
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 3 (ON CONSTRAINT set-dead): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 3 (ON CONSTRAINT set-dead): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 3 (ON CONSTRAINT set-dead): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 4: REINDEX + UPSERT ON CONSTRAINT (swap phase)
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-swap'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-swap');
+
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+is(safe_quit($s1), '', 'Test 4 (ON CONSTRAINT swap): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 4 (ON CONSTRAINT swap): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 4 (ON CONSTRAINT swap): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 4b: REINDEX + UPSERT ON CONSTRAINT (permutation 3: s1 wakes before reindex)
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+# Start s2 BEFORE waking reindex
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+# Wake s1 first, then reindex, then s2
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 4b (ON CONSTRAINT perm3): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 4b (ON CONSTRAINT perm3): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 4b (ON CONSTRAINT perm3): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 5: REINDEX on partitioned table (set-dead phase)
+# Based on reindex-concurrently-upsert-partitioned.spec
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 5 (partitioned set-dead): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 5 (partitioned set-dead): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 5 (partitioned set-dead): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 6: REINDEX on partitioned table (swap phase)
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-swap'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-swap');
+
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+is(safe_quit($s1), '', 'Test 6 (partitioned swap): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 6 (partitioned swap): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 6 (partitioned swap): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 6b: REINDEX on partitioned table (permutation 3: s1 wakes before reindex)
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+# Start s2 BEFORE waking reindex
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+# Wake s1 first, then reindex, then s2
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 6b (partitioned perm3): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 6b (partitioned perm3): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 6b (partitioned perm3): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 7: CREATE INDEX CONCURRENTLY + UPSERT
+# Based on index-concurrently-upsert.spec
+# Uses invalidate-catalog-snapshot-end to test catalog invalidation during UPSERT
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+my $s1_pid = $s1->query_safe('SELECT pg_backend_pid()');
+
+# s1 attaches BOTH injection points - the unique constraint check AND catalog snapshot
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s1->query_until(qr/attaching_injection_point/, q[
+\echo attaching_injection_point
+SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+]);
+# In case of CLOBBER_CACHE_ALWAYS - s1 may hit the injection point during attach.
+# Wait for s1 to become idle (attach completed) or wakeup if stuck on injection point.
+if (!wait_for_idle($node, $s1_pid))
+{
+ ok(wait_for_injection_point($node, 'invalidate-catalog-snapshot-end'),
+ 'Test 7: s1 hit injection point during attach (CLOBBER_CACHE_ALWAYS)');
+ $node->safe_psql('postgres', q[
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+ ]);
+}
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('define-index-before-set-valid', 'wait');
+]);
+
+# s3: Start CREATE INDEX CONCURRENTLY (blocks on define-index-before-set-valid)
+$s3->query_until(qr/starting_create_index/, q[
+\echo starting_create_index
+CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
+]);
+
+ok(wait_for_injection_point($node, 'define-index-before-set-valid'));
+
+# s1: Start UPSERT (blocks on invalidate-catalog-snapshot-end)
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'invalidate-catalog-snapshot-end'));
+
+# Wakeup s3 (CREATE INDEX continues, triggers catalog invalidation)
+wakeup_injection_point($node, 'define-index-before-set-valid');
+
+# s2: Start UPSERT (blocks on exec-insert-before-insert-speculative)
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'invalidate-catalog-snapshot-end');
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+is(safe_quit($s1), '', 'Test 7 (CREATE INDEX): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 7 (CREATE INDEX): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 7 (CREATE INDEX): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 8: CREATE INDEX CONCURRENTLY on partial index + UPSERT
+# Based on index-concurrently-upsert-predicate.spec
+# Uses invalidate-catalog-snapshot-end to test catalog invalidation during UPSERT
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1_pid = $s1->query_safe('SELECT pg_backend_pid()');
+
+# s1 attaches BOTH injection points - the unique constraint check AND catalog snapshot
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s1->query_until(qr/attaching_injection_point/, q[
+\echo attaching_injection_point
+SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+]);
+# In case of CLOBBER_CACHE_ALWAYS - s1 may hit the injection point during attach.
+# Wait for s1 to become idle (attach completed) or wakeup if stuck on injection point.
+if (!wait_for_idle($node, $s1_pid))
+{
+ ok(wait_for_injection_point($node, 'invalidate-catalog-snapshot-end'),
+ 'Test 8: s1 hit injection point during attach (CLOBBER_CACHE_ALWAYS)');
+ $node->safe_psql('postgres', q[
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+ ]);
+}
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('define-index-before-set-valid', 'wait');
+]);
+
+# s3: Start CREATE INDEX CONCURRENTLY (blocks on define-index-before-set-valid)
+$s3->query_until(qr/starting_create_index/, q[
+\echo starting_create_index
+CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
+]);
+
+ok(wait_for_injection_point($node, 'define-index-before-set-valid'));
+
+# s1: Start UPSERT (blocks on invalidate-catalog-snapshot-end)
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'invalidate-catalog-snapshot-end'));
+
+# Wakeup s3 (CREATE INDEX continues, triggers catalog invalidation)
+wakeup_injection_point($node, 'define-index-before-set-valid');
+
+# s2: Start UPSERT (blocks on exec-insert-before-insert-speculative)
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'invalidate-catalog-snapshot-end');
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+is(safe_quit($s1), '', 'Test 7 (CREATE INDEX): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 7 (CREATE INDEX): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 7 (CREATE INDEX): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+done_testing();
--
2.52.0
Hello, Álvaro and others!
Attached version feels stable enough so far - 20 builds in a row on
all CI variants (including 3 BSD) - no failures so far.
I updated the commit message to include reference to previous commits.
Also, tests designed in a way to fail fast if something is going wrong
+ log some debug information in that case (active queries with its
states).
Special tricks to handle forced-cache release builds included too.
Also, there is a test which "breaks" all the fixes - to ensure the
test actually catches them, not intended to be committed of course.
Best regards,
Mikhail.
Attachments:
v2-0001-Replace-flaky-CIC-RI-isolation-tests-with-stable-.patchapplication/x-patch; name=v2-0001-Replace-flaky-CIC-RI-isolation-tests-with-stable-.patchDownload
From 113f1ad1040c3646f6a990a14f5304db903bfe15 Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <mihailnikalayeu@gmail.com>
Date: Mon, 15 Dec 2025 12:03:04 +0100
Subject: [PATCH v2 1/2] Replace flaky CIC/RI isolation tests with stable TAP
tests
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
The isolation tests for INSERT ON CONFLICT behavior during CREATE INDEX CONCURRENTLY and REINDEX CONCURRENTLY were disabled in 77038d6d0b4 due to persistent CI flakiness.
The isolation tester based solution was struggling to reliably ensure exact the same test output. The tests were commented out pending a complete rewrite.
This commit removes the disabled isolation tests and their spec files, originally added in bc32a12e0db, 2bc7e886fc1 and 90eae926abb and replaces them with a TAP test in test_misc module (010_index_concurrently_upsert.pl) that covers the same scenarios.
These tests verify the fixes from referenced commits, and 81f72115cf1 remain effective, preventing "duplicate key value violates unique constraint" errors when concurrent transactions select different arbiter indexes during index state transitions.
Author: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Reported-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/ccssrhafzbp3a3beju3ptyc56a7gbfimj4vwkbokoldofckrc7@bso37rxskjtf
Discussion: https://postgr.es/m/CANtu0ogv+6wqRzPK241jik4U95s1pW3MCZ3rX5ZqbFdUysz7Qw@mail.gmail.com
Discussion: https://postgr.es/m/202512112014.icpomgc37zx4@alvherre.pgsql
---
src/test/modules/injection_points/Makefile | 8 -
.../index-concurrently-upsert-predicate.out | 123 ---
.../index-concurrently-upsert-predicate_1.out | 124 ---
.../expected/index-concurrently-upsert.out | 123 ---
.../expected/index-concurrently-upsert_1.out | 124 ---
...ndex-concurrently-upsert-on-constraint.out | 238 -----
...eindex-concurrently-upsert-partitioned.out | 238 -----
.../expected/reindex-concurrently-upsert.out | 238 -----
src/test/modules/injection_points/meson.build | 6 -
.../index-concurrently-upsert-predicate.spec | 124 ---
.../specs/index-concurrently-upsert.spec | 123 ---
...dex-concurrently-upsert-on-constraint.spec | 110 ---
...index-concurrently-upsert-partitioned.spec | 113 ---
.../specs/reindex-concurrently-upsert.spec | 111 ---
src/test/modules/test_misc/Makefile | 3 +
src/test/modules/test_misc/meson.build | 3 +
.../t/010_index_concurrently_upsert.pl | 894 ++++++++++++++++++
17 files changed, 900 insertions(+), 1803 deletions(-)
delete mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
delete mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
delete mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert.out
delete mode 100644 src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
delete mode 100644 src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out
delete mode 100644 src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
delete mode 100644 src/test/modules/injection_points/expected/reindex-concurrently-upsert.out
delete mode 100644 src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
delete mode 100644 src/test/modules/injection_points/specs/index-concurrently-upsert.spec
delete mode 100644 src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec
delete mode 100644 src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
delete mode 100644 src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec
create mode 100644 src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index bfdb3f53377..3cb50d13e52 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -16,14 +16,6 @@ ISOLATION = basic \
inplace \
syscache-update-pruned
-# Temporarily disabled because of flakiness
-#ISOLATION =+
-# index-concurrently-upsert \
-# index-concurrently-upsert-predicate \
-# reindex-concurrently-upsert \
-# reindex-concurrently-upsert-on-constraint \
-# reindex-concurrently-upsert-partitioned
-
# The injection points are cluster-wide, so disable installcheck
NO_INSTALLCHECK = 1
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
deleted file mode 100644
index 77e7d1a7815..00000000000
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate.out
+++ /dev/null
@@ -1,123 +0,0 @@
-Parsed test spec with 5 sessions
-
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot:
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s4_wakeup_s1_setup:
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-
-case
-----
-
-(1 row)
-
-step s3_start_create_index:
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_define_index_before_set_valid:
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_wakeup_s1_from_invalidate_catalog_snapshot:
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out b/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
deleted file mode 100644
index e72848d6a78..00000000000
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert-predicate_1.out
+++ /dev/null
@@ -1,124 +0,0 @@
-Parsed test spec with 5 sessions
-
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot:
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
- <waiting ...>
-step s4_wakeup_s1_setup:
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-
-case
-----
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot: <... completed>
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_create_index:
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_define_index_before_set_valid:
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_wakeup_s1_from_invalidate_catalog_snapshot:
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert.out b/src/test/modules/injection_points/expected/index-concurrently-upsert.out
deleted file mode 100644
index a2ef122625c..00000000000
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert.out
+++ /dev/null
@@ -1,123 +0,0 @@
-Parsed test spec with 5 sessions
-
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot:
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s4_wakeup_s1_setup:
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-
-case
-----
-
-(1 row)
-
-step s3_start_create_index:
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_define_index_before_set_valid:
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_wakeup_s1_from_invalidate_catalog_snapshot:
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out b/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
deleted file mode 100644
index ee3b6641b90..00000000000
--- a/src/test/modules/injection_points/expected/index-concurrently-upsert_1.out
+++ /dev/null
@@ -1,124 +0,0 @@
-Parsed test spec with 5 sessions
-
-starting permutation: s1_attach_invalidate_catalog_snapshot s4_wakeup_s1_setup s3_start_create_index s1_start_upsert s4_wakeup_define_index_before_set_valid s2_start_upsert s5_wakeup_s1_from_invalidate_catalog_snapshot s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot:
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
- <waiting ...>
-step s4_wakeup_s1_setup:
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-
-case
-----
-
-(1 row)
-
-step s1_attach_invalidate_catalog_snapshot: <... completed>
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_create_index:
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_define_index_before_set_valid:
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s5_wakeup_s1_from_invalidate_catalog_snapshot:
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_create_index: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out b/src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out
deleted file mode 100644
index c1ac1f77c61..00000000000
--- a/src/test/modules/injection_points/expected/reindex-concurrently-upsert-on-constraint.out
+++ /dev/null
@@ -1,238 +0,0 @@
-Parsed test spec with 4 sessions
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_swap:
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_swap:
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out b/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
deleted file mode 100644
index 4c79a43d986..00000000000
--- a/src/test/modules/injection_points/expected/reindex-concurrently-upsert-partitioned.out
+++ /dev/null
@@ -1,238 +0,0 @@
-Parsed test spec with 4 sessions
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_swap:
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_swap:
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/expected/reindex-concurrently-upsert.out b/src/test/modules/injection_points/expected/reindex-concurrently-upsert.out
deleted file mode 100644
index c9cc9989d02..00000000000
--- a/src/test/modules/injection_points/expected/reindex-concurrently-upsert.out
+++ /dev/null
@@ -1,238 +0,0 @@
-Parsed test spec with 4 sessions
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s4_wakeup_to_set_dead s2_start_upsert s4_wakeup_s1 s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_swap s3_start_reindex s1_start_upsert s4_wakeup_to_swap s2_start_upsert s4_wakeup_s2 s4_wakeup_s1
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_swap:
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_to_swap:
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
-
-starting permutation: s3_setup_wait_before_set_dead s3_start_reindex s1_start_upsert s2_start_upsert s4_wakeup_s1 s4_wakeup_to_set_dead s4_wakeup_s2
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_attach
------------------------
-
-(1 row)
-
-injection_points_set_local
---------------------------
-
-(1 row)
-
-step s3_setup_wait_before_set_dead:
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-
-injection_points_attach
------------------------
-
-(1 row)
-
-step s3_start_reindex:
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
- <waiting ...>
-step s1_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s2_start_upsert:
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
- <waiting ...>
-step s4_wakeup_s1:
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s1_start_upsert: <... completed>
-step s4_wakeup_to_set_dead:
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s4_wakeup_s2:
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-
-injection_points_detach
------------------------
-
-(1 row)
-
-injection_points_wakeup
------------------------
-
-(1 row)
-
-step s2_start_upsert: <... completed>
-step s3_start_reindex: <... completed>
diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build
index 493e11053dc..2c6cf54a33e 100644
--- a/src/test/modules/injection_points/meson.build
+++ b/src/test/modules/injection_points/meson.build
@@ -46,12 +46,6 @@ tests += {
'basic',
'inplace',
'syscache-update-pruned',
- # temporarily disabled because of flakiness
- # 'index-concurrently-upsert',
- # 'index-concurrently-upsert-predicate',
- # 'reindex-concurrently-upsert',
- # 'reindex-concurrently-upsert-on-constraint',
- # 'reindex-concurrently-upsert-partitioned',
],
'runningcheck': false, # see syscache-update-pruned
# Some tests wait for all snapshots, so avoid parallel execution
diff --git a/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec b/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
deleted file mode 100644
index d9b8d27fd1f..00000000000
--- a/src/test/modules/injection_points/specs/index-concurrently-upsert-predicate.spec
+++ /dev/null
@@ -1,124 +0,0 @@
-# This test verifies INSERT ON CONFLICT DO UPDATE behavior concurrent with
-# CREATE INDEX CONCURRENTLY a partial index.
-#
-# - s1: UPSERT a tuple
-# - s2: UPSERT the same tuple
-# - s3: CREATE UNIQUE INDEX CONCURRENTLY (with a predicate)
-#
-# - s4 and s5: control concurrency via injection points
-
-setup
-{
- CREATE EXTENSION injection_points;
- CREATE SCHEMA test;
- CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
- CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
- ALTER TABLE test.tbl SET (parallel_workers=0);
-}
-
-teardown
-{
- DROP SCHEMA test CASCADE;
- DROP EXTENSION injection_points;
-}
-
-session s1
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
-}
-step s1_attach_invalidate_catalog_snapshot
-{
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
-}
-step s1_start_upsert
-{
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
-}
-
-session s2
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
-}
-step s2_start_upsert
-{
- INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
-}
-
-session s3
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('define-index-before-set-valid', 'wait');
-}
-step s3_start_create_index
-{
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
-}
-
-session s4
-# Step s1_attach_invalidate_catalog_snapshot sleeps or not depending on
-# build conditions (CATCACHE_FORCE_RELEASE). Here we send a wakeup signal if
-# it's sleeping or do nothing otherwise, and print a null value in either
-# case.
-step s4_wakeup_s1_setup
-{
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-}
-step s4_wakeup_s1
-{
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-}
-step s4_wakeup_s2
-{
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-}
-step s4_wakeup_define_index_before_set_valid
-{
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-}
-
-session s5
-step s5_wakeup_s1_from_invalidate_catalog_snapshot
-{
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-}
-
-permutation
- s1_attach_invalidate_catalog_snapshot
- s4_wakeup_s1_setup
- s3_start_create_index(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
- s4_wakeup_define_index_before_set_valid
- s2_start_upsert(s1_start_upsert)
- s5_wakeup_s1_from_invalidate_catalog_snapshot
- s4_wakeup_s2
- s4_wakeup_s1
diff --git a/src/test/modules/injection_points/specs/index-concurrently-upsert.spec b/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
deleted file mode 100644
index 6e08af74a93..00000000000
--- a/src/test/modules/injection_points/specs/index-concurrently-upsert.spec
+++ /dev/null
@@ -1,123 +0,0 @@
-# This test verifies INSERT ON CONFLICT DO UPDATE behavior concurrent with
-# CREATE INDEX CONCURRENTLY.
-#
-# - s1: UPSERT a tuple
-# - s2: UPSERT the same tuple
-# - s3: CREATE UNIQUE INDEX CONCURRENTLY
-#
-# - s4: Control concurrency using injection points
-
-setup
-{
- CREATE EXTENSION injection_points;
- CREATE SCHEMA test;
- CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
- ALTER TABLE test.tbl SET (parallel_workers=0);
-}
-
-teardown
-{
- DROP SCHEMA test CASCADE;
- DROP EXTENSION injection_points;
-}
-
-session s1
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
-}
-step s1_attach_invalidate_catalog_snapshot
-{
- SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
-}
-step s1_start_upsert
-{
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s2
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
-}
-step s2_start_upsert
-{
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s3
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('define-index-before-set-valid', 'wait');
-}
-step s3_start_create_index
-{
- CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
-}
-
-session s4
-# Step s1_attach_invalidate_catalog_snapshot sleeps or not depending on
-# build conditions (CATCACHE_FORCE_RELEASE). Here we send a wakeup signal if
-# it's sleeping or do nothing otherwise, and print a null value in either
-# case.
-step s4_wakeup_s1_setup
-{
- SELECT CASE WHEN
- (SELECT pid FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint' AND
- wait_event = 'invalidate-catalog-snapshot-end') IS NOT NULL
- THEN injection_points_wakeup('invalidate-catalog-snapshot-end')
- END;
-}
-step s4_wakeup_s1
-{
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-}
-step s4_wakeup_s2
-{
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-}
-step s4_wakeup_define_index_before_set_valid
-{
- SELECT injection_points_detach('define-index-before-set-valid');
- SELECT injection_points_wakeup('define-index-before-set-valid');
-}
-
-session s5
-step s5_wakeup_s1_from_invalidate_catalog_snapshot
-{
- DO $$
- DECLARE
- v_waiting_pid INTEGER;
- BEGIN
- LOOP
- SELECT pid INTO v_waiting_pid
- FROM pg_stat_activity
- WHERE wait_event_type = 'InjectionPoint'
- AND wait_event = 'invalidate-catalog-snapshot-end'
- LIMIT 1;
- EXIT WHEN v_waiting_pid IS NOT NULL;
- PERFORM pg_sleep(100);
- END LOOP;
- END
- $$;
-
- SELECT injection_points_detach('invalidate-catalog-snapshot-end');
- SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
-}
-
-permutation
- s1_attach_invalidate_catalog_snapshot
- s4_wakeup_s1_setup
- s3_start_create_index(s1_start_upsert, s2_start_upsert)
- s1_start_upsert
- s4_wakeup_define_index_before_set_valid
- s2_start_upsert(s1_start_upsert)
- s5_wakeup_s1_from_invalidate_catalog_snapshot
- s4_wakeup_s2
- s4_wakeup_s1
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec
deleted file mode 100644
index 4bbdda3cf04..00000000000
--- a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-on-constraint.spec
+++ /dev/null
@@ -1,110 +0,0 @@
-# Test race conditions involving:
-#
-# - s1: UPSERT a tuple
-# - s2: UPSERT the same tuple
-# - s3: concurrently REINDEX the primary key
-#
-# - s4: operations with injection points
-
-setup
-{
- CREATE EXTENSION injection_points;
- CREATE SCHEMA test;
- CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
- ALTER TABLE test.tbl SET (parallel_workers=0);
-}
-
-teardown
-{
- DROP SCHEMA test CASCADE;
- DROP EXTENSION injection_points;
-}
-
-session s1
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
-}
-step s1_start_upsert
-{
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
-}
-
-session s2
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
-}
-step s2_start_upsert
-{
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
-}
-
-session s3
-setup
-{
- SELECT injection_points_set_local();
-}
-step s3_setup_wait_before_set_dead
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-}
-step s3_setup_wait_before_swap
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-}
-step s3_start_reindex
-{
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
-}
-
-session s4
-step s4_wakeup_to_swap
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-}
-step s4_wakeup_s1
-{
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-}
-step s4_wakeup_s2
-{
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-}
-step s4_wakeup_to_set_dead
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-}
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_set_dead
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_s2
-
-permutation
- s3_setup_wait_before_swap
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_swap
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s2
- s4_wakeup_s1
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_to_set_dead
- s4_wakeup_s2
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
deleted file mode 100644
index c3504b9ef38..00000000000
--- a/src/test/modules/injection_points/specs/reindex-concurrently-upsert-partitioned.spec
+++ /dev/null
@@ -1,113 +0,0 @@
-# This test verifies INSERT ON CONFLICT DO UPDATE behavior on partitioned
-# tables concurrent with REINDEX CONCURRENTLY.
-#
-# - s1: UPSERT a tuple
-# - s2: UPSERT the same tuple
-# - s3: concurrently REINDEX the primary key index
-#
-# - s4: controls concurrency via injection points
-
-setup
-{
- CREATE EXTENSION injection_points;
- CREATE SCHEMA test;
- CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
- CREATE TABLE test.tbl_partition PARTITION OF test.tbl
- FOR VALUES FROM (0) TO (10000)
- WITH (parallel_workers = 0);
-}
-
-teardown
-{
- DROP SCHEMA test CASCADE;
- DROP EXTENSION injection_points;
-}
-
-session s1
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
-}
-step s1_start_upsert
-{
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s2
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
-}
-step s2_start_upsert
-{
- INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s3
-setup
-{
- SELECT injection_points_set_local();
-}
-step s3_setup_wait_before_set_dead
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-}
-step s3_setup_wait_before_swap
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-}
-step s3_start_reindex
-{
- REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
-}
-
-session s4
-step s4_wakeup_to_swap
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-}
-step s4_wakeup_s1
-{
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-}
-step s4_wakeup_s2
-{
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-}
-step s4_wakeup_to_set_dead
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-}
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_set_dead
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_s2
-
-permutation
- s3_setup_wait_before_swap
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_swap
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s2
- s4_wakeup_s1
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_to_set_dead
- s4_wakeup_s2
diff --git a/src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec b/src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec
deleted file mode 100644
index 1b043a48ff4..00000000000
--- a/src/test/modules/injection_points/specs/reindex-concurrently-upsert.spec
+++ /dev/null
@@ -1,111 +0,0 @@
-# This test verifies INSERT ON CONFLICT DO UPDATE behavior concurrent with
-# REINDEX CONCURRENTLY.
-#
-# - s1: UPSERT a tuple
-# - s2: UPSERT the same tuple
-# - s3: REINDEX concurrent primary key index
-#
-# - s4: controls concurrency via injection points
-
-setup
-{
- CREATE EXTENSION injection_points;
- CREATE SCHEMA test;
- CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
- ALTER TABLE test.tbl SET (parallel_workers=0);
-}
-
-teardown
-{
- DROP SCHEMA test CASCADE;
- DROP EXTENSION injection_points;
-}
-
-session s1
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
-}
-step s1_start_upsert
-{
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s2
-setup
-{
- SELECT injection_points_set_local();
- SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
-}
-step s2_start_upsert
-{
- INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
-}
-
-session s3
-setup
-{
- SELECT injection_points_set_local();
-}
-step s3_setup_wait_before_set_dead
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
-}
-step s3_setup_wait_before_swap
-{
- SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
-}
-step s3_start_reindex
-{
- REINDEX INDEX CONCURRENTLY test.tbl_pkey;
-}
-
-session s4
-step s4_wakeup_to_swap
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-swap');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-swap');
-}
-step s4_wakeup_s1
-{
- SELECT injection_points_detach('check-exclusion-or-unique-constraint-no-conflict');
- SELECT injection_points_wakeup('check-exclusion-or-unique-constraint-no-conflict');
-}
-step s4_wakeup_s2
-{
- SELECT injection_points_detach('exec-insert-before-insert-speculative');
- SELECT injection_points_wakeup('exec-insert-before-insert-speculative');
-}
-step s4_wakeup_to_set_dead
-{
- SELECT injection_points_detach('reindex-relation-concurrently-before-set-dead');
- SELECT injection_points_wakeup('reindex-relation-concurrently-before-set-dead');
-}
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_set_dead
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_s2
-
-permutation
- s3_setup_wait_before_swap
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s4_wakeup_to_swap
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s2
- s4_wakeup_s1
-
-permutation
- s3_setup_wait_before_set_dead
- s3_start_reindex(s1_start_upsert, s2_start_upsert)
- s1_start_upsert(s4_wakeup_s2)
- s2_start_upsert(s1_start_upsert)
- s4_wakeup_s1
- s4_wakeup_to_set_dead
- s4_wakeup_s2
diff --git a/src/test/modules/test_misc/Makefile b/src/test/modules/test_misc/Makefile
index 399b9094a38..fedbef071ef 100644
--- a/src/test/modules/test_misc/Makefile
+++ b/src/test/modules/test_misc/Makefile
@@ -5,6 +5,9 @@ TAP_TESTS = 1
EXTRA_INSTALL=src/test/modules/injection_points \
contrib/test_decoding
+# The injection points are cluster-wide, so disable installcheck
+NO_INSTALLCHECK = 1
+
export enable_injection_points
ifdef USE_PGXS
diff --git a/src/test/modules/test_misc/meson.build b/src/test/modules/test_misc/meson.build
index f258bf1ccd9..129c4ae587a 100644
--- a/src/test/modules/test_misc/meson.build
+++ b/src/test/modules/test_misc/meson.build
@@ -18,6 +18,9 @@ tests += {
't/007_catcache_inval.pl',
't/008_replslot_single_user.pl',
't/009_log_temp_files.pl',
+ 't/010_index_concurrently_upsert.pl',
],
+ # The injection points are cluster-wide, so disable installcheck
+ 'runningcheck': false,
},
}
diff --git a/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl b/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
new file mode 100644
index 00000000000..e3feb6f2861
--- /dev/null
+++ b/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
@@ -0,0 +1,894 @@
+
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+# Test INSERT ON CONFLICT DO UPDATE behavior concurrent with
+# CREATE INDEX CONCURRENTLY and REINDEX CONCURRENTLY.
+#
+# These tests verify the fix for "duplicate key value violates unique constraint"
+# errors that occurred when infer_arbiter_indexes() only considered indisvalid
+# indexes, causing different transactions to use different arbiter indexes.
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+if ($ENV{enable_injection_points} ne 'yes')
+{
+ plan skip_all => 'Injection points not supported by this build';
+}
+
+# Node initialization
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init();
+$node->start;
+
+# Check if the extension injection_points is available
+if (!$node->check_extension('injection_points'))
+{
+ plan skip_all => 'Extension injection_points not installed';
+}
+
+$node->safe_psql('postgres', 'CREATE EXTENSION injection_points;');
+
+# Helper: Wait for a session to hit an injection point.
+# Optional second argument is timeout in seconds.
+# Returns true if found, false if timeout.
+# On timeout, logs diagnostic information about all active queries.
+sub wait_for_injection_point
+{
+ my ($node, $point_name, $timeout) = @_;
+ $timeout //= 120;
+
+ for (my $elapsed = 0; $elapsed < $timeout; $elapsed++)
+ {
+ my $pid = $node->safe_psql('postgres', qq[
+ SELECT pid FROM pg_stat_activity
+ WHERE wait_event_type = 'InjectionPoint'
+ AND wait_event = '$point_name'
+ LIMIT 1;
+ ]);
+ return 1 if $pid ne '';
+ sleep(1);
+ }
+
+ # Timeout - report diagnostic information
+ my $activity = $node->safe_psql('postgres', q[
+ SELECT format('pid=%s, state=%s, wait_event_type=%s, wait_event=%s, backend_xmin=%s, backend_xid=%s, query=%s',
+ pid, state, wait_event_type, wait_event, backend_xmin, backend_xid, left(query, 100))
+ FROM pg_stat_activity
+ ORDER BY pid;
+ ]);
+ diag("wait_for_injection_point timeout waiting for: $point_name\n" .
+ "Current queries in pg_stat_activity:\n$activity");
+
+ return 0;
+}
+
+# Helper: Wait for a specific backend to become idle.
+# Returns true if idle, false if timeout.
+sub wait_for_idle
+{
+ my ($node, $pid, $timeout) = @_;
+ $timeout //= 15;
+
+ for (my $elapsed = 0; $elapsed < $timeout; $elapsed++)
+ {
+ my $state = $node->safe_psql('postgres', qq[
+ SELECT state FROM pg_stat_activity WHERE pid = $pid;
+ ]);
+ return 1 if $state eq 'idle';
+ sleep(1);
+ }
+ return 0;
+}
+
+# Helper: Detach and wakeup an injection point
+sub wakeup_injection_point
+{
+ my ($node, $point_name) = @_;
+ $node->safe_psql(
+ 'postgres', qq[
+SELECT injection_points_detach('$point_name');
+SELECT injection_points_wakeup('$point_name');
+]);
+}
+
+# Wait for any pending query to complete, capture stderr, and close the session.
+# Returns the stderr output (excluding internal markers).
+sub safe_quit
+{
+ my ($session) = @_;
+
+ # Send a marker and wait for it to ensure any pending query completes
+ my $banner = "safe_quit_marker";
+ my $banner_match = qr/(^|\n)$banner\r?\n/;
+
+ $session->{stdin} .= "\\echo $banner\n\\warn $banner\n";
+
+ pump_until($session->{run}, $session->{timeout},
+ \$session->{stdout}, $banner_match);
+ pump_until($session->{run}, $session->{timeout},
+ \$session->{stderr}, $banner_match);
+
+ # Capture stderr (excluding the banner)
+ my $stderr = $session->{stderr};
+ $stderr =~ s/$banner_match//;
+
+ # Close the session
+ $session->quit;
+
+ return $stderr;
+}
+
+###############################################################################
+# Test 1: REINDEX CONCURRENTLY + UPSERT (wakeup at set-dead phase)
+# Based on reindex-concurrently-upsert.spec
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+# Create sessions with on_error_stop => 0 so psql doesn't exit on SQL errors.
+# This allows us to collect stderr and detect errors after the test completes.
+my $s1 = $node->background_psql('postgres', on_error_stop => 0);
+my $s2 = $node->background_psql('postgres', on_error_stop => 0);
+my $s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+# Setup injection points for each session
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+# s3 starts REINDEX (will block on reindex-relation-concurrently-before-set-dead)
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+# Wait for s3 to hit injection point
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+# s1 starts UPSERT (will block on check-exclusion-or-unique-constraint-no-conflict)
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+# Wait for s1 to hit injection point
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+# Wakeup s3 to continue (reindex-relation-concurrently-before-set-dead)
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+
+# s2 starts UPSERT (will block on exec-insert-before-insert-speculative)
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+# Wait for s2 to hit injection point
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+# Wakeup s1 (check-exclusion-or-unique-constraint-no-conflict)
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+# Wakeup s2 (exec-insert-before-insert-speculative)
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 1 (REINDEX set-dead): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 1 (REINDEX set-dead): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 1 (REINDEX set-dead): session s3 quit successfully');
+
+# Cleanup test 1
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 2: REINDEX CONCURRENTLY + UPSERT (wakeup at swap phase)
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-swap'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-swap');
+
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+is(safe_quit($s1), '', 'Test 2 (REINDEX swap): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 2 (REINDEX swap): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 2 (REINDEX swap): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 2b: REINDEX CONCURRENTLY + UPSERT (permutation 3: s1 wakes before reindex)
+# Different timing: s2 starts, then s1 wakes, then reindex wakes, then s2 wakes
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+# Start s2 BEFORE waking reindex (key difference from permutation 1)
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+# Wake s1 first, then reindex, then s2
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 2b (REINDEX perm3): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 2b (REINDEX perm3): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 2b (REINDEX perm3): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 3: REINDEX + UPSERT ON CONSTRAINT (set-dead phase)
+# Based on reindex-concurrently-upsert-on-constraint.spec
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 3 (ON CONSTRAINT set-dead): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 3 (ON CONSTRAINT set-dead): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 3 (ON CONSTRAINT set-dead): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 4: REINDEX + UPSERT ON CONSTRAINT (swap phase)
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-swap'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-swap');
+
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+is(safe_quit($s1), '', 'Test 4 (ON CONSTRAINT swap): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 4 (ON CONSTRAINT swap): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 4 (ON CONSTRAINT swap): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 4b: REINDEX + UPSERT ON CONSTRAINT (permutation 3: s1 wakes before reindex)
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl (i int PRIMARY KEY, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+# Start s2 BEFORE waking reindex
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT ON CONSTRAINT tbl_pkey DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+# Wake s1 first, then reindex, then s2
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 4b (ON CONSTRAINT perm3): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 4b (ON CONSTRAINT perm3): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 4b (ON CONSTRAINT perm3): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 5: REINDEX on partitioned table (set-dead phase)
+# Based on reindex-concurrently-upsert-partitioned.spec
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 5 (partitioned set-dead): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 5 (partitioned set-dead): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 5 (partitioned set-dead): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 6: REINDEX on partitioned table (swap phase)
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-swap', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-swap'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-swap');
+
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+is(safe_quit($s1), '', 'Test 6 (partitioned swap): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 6 (partitioned swap): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 6 (partitioned swap): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 6b: REINDEX on partitioned table (permutation 3: s1 wakes before reindex)
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE TABLE test.tbl(i int primary key, updated_at timestamp) PARTITION BY RANGE (i);
+CREATE TABLE test.tbl_partition PARTITION OF test.tbl
+ FOR VALUES FROM (0) TO (10000)
+ WITH (parallel_workers = 0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('reindex-relation-concurrently-before-set-dead', 'wait');
+]);
+
+$s3->query_until(qr/starting_reindex/, q[
+\echo starting_reindex
+REINDEX INDEX CONCURRENTLY test.tbl_partition_pkey;
+]);
+
+ok(wait_for_injection_point($node, 'reindex-relation-concurrently-before-set-dead'));
+
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+# Start s2 BEFORE waking reindex
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13, now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+# Wake s1 first, then reindex, then s2
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+wakeup_injection_point($node, 'reindex-relation-concurrently-before-set-dead');
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+is(safe_quit($s1), '', 'Test 6b (partitioned perm3): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 6b (partitioned perm3): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 6b (partitioned perm3): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 7: CREATE INDEX CONCURRENTLY + UPSERT
+# Based on index-concurrently-upsert.spec
+# Uses invalidate-catalog-snapshot-end to test catalog invalidation during UPSERT
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl(i int primary key, updated_at timestamp);
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+my $s1_pid = $s1->query_safe('SELECT pg_backend_pid()');
+
+# s1 attaches BOTH injection points - the unique constraint check AND catalog snapshot
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s1->query_until(qr/attaching_injection_point/, q[
+\echo attaching_injection_point
+SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+]);
+# In case of CLOBBER_CACHE_ALWAYS - s1 may hit the injection point during attach.
+# Wait for s1 to become idle (attach completed) or wakeup if stuck on injection point.
+if (!wait_for_idle($node, $s1_pid))
+{
+ ok(wait_for_injection_point($node, 'invalidate-catalog-snapshot-end'),
+ 'Test 7: s1 hit injection point during attach (CLOBBER_CACHE_ALWAYS)');
+ $node->safe_psql('postgres', q[
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+ ]);
+}
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('define-index-before-set-valid', 'wait');
+]);
+
+# s3: Start CREATE INDEX CONCURRENTLY (blocks on define-index-before-set-valid)
+$s3->query_until(qr/starting_create_index/, q[
+\echo starting_create_index
+CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_duplicate ON test.tbl(i);
+]);
+
+ok(wait_for_injection_point($node, 'define-index-before-set-valid'));
+
+# s1: Start UPSERT (blocks on invalidate-catalog-snapshot-end)
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'invalidate-catalog-snapshot-end'));
+
+# Wakeup s3 (CREATE INDEX continues, triggers catalog invalidation)
+wakeup_injection_point($node, 'define-index-before-set-valid');
+
+# s2: Start UPSERT (blocks on exec-insert-before-insert-speculative)
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES (13,now()) ON CONFLICT (i) DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'invalidate-catalog-snapshot-end');
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+is(safe_quit($s1), '', 'Test 7 (CREATE INDEX): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 7 (CREATE INDEX): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 7 (CREATE INDEX): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+###############################################################################
+# Test 8: CREATE INDEX CONCURRENTLY on partial index + UPSERT
+# Based on index-concurrently-upsert-predicate.spec
+# Uses invalidate-catalog-snapshot-end to test catalog invalidation during UPSERT
+###############################################################################
+
+$node->safe_psql(
+ 'postgres', q[
+CREATE SCHEMA test;
+CREATE UNLOGGED TABLE test.tbl(i int, updated_at timestamp);
+CREATE UNIQUE INDEX tbl_pkey_special ON test.tbl(abs(i)) WHERE i < 1000;
+ALTER TABLE test.tbl SET (parallel_workers=0);
+]);
+
+$s1 = $node->background_psql('postgres', on_error_stop => 0);
+$s2 = $node->background_psql('postgres', on_error_stop => 0);
+$s3 = $node->background_psql('postgres', on_error_stop => 0);
+
+$s1_pid = $s1->query_safe('SELECT pg_backend_pid()');
+
+# s1 attaches BOTH injection points - the unique constraint check AND catalog snapshot
+$s1->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('check-exclusion-or-unique-constraint-no-conflict', 'wait');
+]);
+
+$s1->query_until(qr/attaching_injection_point/, q[
+\echo attaching_injection_point
+SELECT injection_points_attach('invalidate-catalog-snapshot-end', 'wait');
+]);
+# In case of CLOBBER_CACHE_ALWAYS - s1 may hit the injection point during attach.
+# Wait for s1 to become idle (attach completed) or wakeup if stuck on injection point.
+if (!wait_for_idle($node, $s1_pid))
+{
+ ok(wait_for_injection_point($node, 'invalidate-catalog-snapshot-end'),
+ 'Test 8: s1 hit injection point during attach (CLOBBER_CACHE_ALWAYS)');
+ $node->safe_psql('postgres', q[
+ SELECT injection_points_wakeup('invalidate-catalog-snapshot-end');
+ ]);
+}
+
+$s2->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('exec-insert-before-insert-speculative', 'wait');
+]);
+
+$s3->query_safe(q[
+SELECT injection_points_set_local();
+SELECT injection_points_attach('define-index-before-set-valid', 'wait');
+]);
+
+# s3: Start CREATE INDEX CONCURRENTLY (blocks on define-index-before-set-valid)
+$s3->query_until(qr/starting_create_index/, q[
+\echo starting_create_index
+CREATE UNIQUE INDEX CONCURRENTLY tbl_pkey_special_duplicate ON test.tbl(abs(i)) WHERE i < 10000;
+]);
+
+ok(wait_for_injection_point($node, 'define-index-before-set-valid'));
+
+# s1: Start UPSERT (blocks on invalidate-catalog-snapshot-end)
+$s1->query_until(qr/starting_upsert_s1/, q[
+\echo starting_upsert_s1
+INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'invalidate-catalog-snapshot-end'));
+
+# Wakeup s3 (CREATE INDEX continues, triggers catalog invalidation)
+wakeup_injection_point($node, 'define-index-before-set-valid');
+
+# s2: Start UPSERT (blocks on exec-insert-before-insert-speculative)
+$s2->query_until(qr/starting_upsert_s2/, q[
+\echo starting_upsert_s2
+INSERT INTO test.tbl VALUES(13,now()) ON CONFLICT (abs(i)) WHERE i < 100 DO UPDATE SET updated_at = now();
+]);
+
+ok(wait_for_injection_point($node, 'exec-insert-before-insert-speculative'));
+
+wakeup_injection_point($node, 'invalidate-catalog-snapshot-end');
+
+ok(wait_for_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict'));
+
+wakeup_injection_point($node, 'exec-insert-before-insert-speculative');
+
+wakeup_injection_point($node, 'check-exclusion-or-unique-constraint-no-conflict');
+
+is(safe_quit($s1), '', 'Test 8 (CREATE INDEX predicate): session s1 quit successfully');
+is(safe_quit($s2), '', 'Test 8 (CREATE INDEX predicate): session s2 quit successfully');
+is(safe_quit($s3), '', 'Test 8 (CREATE INDEX predicate): session s3 quit successfully');
+
+$node->safe_psql('postgres', 'DROP SCHEMA test CASCADE;');
+
+done_testing();
--
2.52.0
nocfbot-0002-DO-NOT-PUSH-IT.patchapplication/x-patch; name=nocfbot-0002-DO-NOT-PUSH-IT.patchDownload
From 8ee39bbf4804eb653b3db5d1527f5f6cceb2d72d Mon Sep 17 00:00:00 2001
From: Mikhail Nikalayeu <nkey@toloka.ai>
Date: Thu, 18 Dec 2025 12:15:03 +0100
Subject: [PATCH v2 2/2] !!DO NOT PUSH IT!!
intentionally breaks PG to ensure the test actually covering possible issues
---
src/backend/executor/execPartition.c | 2 +-
src/backend/optimizer/util/plancat.c | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
index e30db12113b..fe18fac371b 100644
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -807,7 +807,7 @@ ExecInitPartitionInfo(ModifyTableState *mtstate, EState *estate,
* same set as arbiters during REINDEX CONCURRENTLY, to avoid
* spurious "duplicate key" errors.
*/
- if (unparented_idxs && arbiterIndexes)
+ if (unparented_idxs && arbiterIndexes && false)
{
foreach_int(unparented_i, unparented_idxs)
{
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index bf45c355b77..a888d230411 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -996,7 +996,7 @@ infer_arbiter_indexes(PlannerInfo *root)
* negatives, we require that we include in the set of inferred
* indexes at least one index that is marked valid.
*/
- if (!idxForm->indisready)
+ if (!idxForm->indisvalid)
continue;
/*
@@ -1029,7 +1029,7 @@ infer_arbiter_indexes(PlannerInfo *root)
/* Consider this one a match already */
results = lappend_oid(results, idxForm->indexrelid);
foundValid |= idxForm->indisvalid;
- continue;
+ break;
}
else if (indexOidFromConstraint != InvalidOid)
{
--
2.52.0
On 2025-Dec-19, Mihail Nikalayeu wrote:
Hello, Álvaro and others!
Attached version feels stable enough so far - 20 builds in a row on
all CI variants (including 3 BSD) - no failures so far.
Hello Mihail, thanks for the time spent on this new test. I have pushed
it now after some editorializing, just to make the code a bit easier to
read. I did lose the test numbering, but added some note() lines for
each subtest so that somebody reading through the output would be able
to match a failure to each subtest.
I also changed to create a bunch of tables at the start of the file
which are reused for each subtest (via truncate), instead of dropping
and recreating them all the time. This reduces the test runtime (not
insignificantly, some 25% I think, though I didn't spend time measuring
it very scientifically).
I'm not sure what the implications are about marking this no longer used
in installcheck. I think it means some buildfarm animals may no longer
run it, if they're configured to only do installcheck. It's a bit of a
bummer because potential loss of runs for the other files in this test
module. Still, I'm not crying about it.
Incidentally, I think it would be great to use Test::More subtests with
these tests (because then the failures would be more clearly attributed
to a specific part of the test file), and I experimented with that, but
it turns out that Meson doesn't like the subtest output (see [1]/messages/by-id/20220224150033.5lql2lkiv7y5kkme@alap3.anarazel.de for
more on this topic). I tried to make it accept TAP version 14 (which is
supposed to make it understand subtests) by adding things like "TAP
version 14" at the start of the output, which is supposed to be the way
to achieve this, but it seems to go completely ignored and I didn't want
to waste time trying to figure out why. The note()s should achieve more
or less the same, in not so elegant a manner.
[1]: /messages/by-id/20220224150033.5lql2lkiv7y5kkme@alap3.anarazel.de
Also, tests designed in a way to fail fast if something is going wrong
+ log some debug information in that case (active queries with its
states).
Nice. I forgot to actually verify that this works as intended before
pushing.
Special tricks to handle forced-cache release builds included too.
I ran the test in this mode, and it passes fine.
Also, there is a test which "breaks" all the fixes - to ensure the
test actually catches them, not intended to be committed of course.
I played with this; each change makes some of the subtests fail in a
pretty obvious way, and the failures are easy to track down in the
output log file.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"But static content is just dynamic content that isn't moving!"
http://smylers.hates-software.com/2007/08/15/fe244d0c.html
Special tricks to handle forced-cache release builds included too.
I ran the test in this mode, and it passes fine.
Hmm, it timed out in prion, which runs this "special tricks" code. (I did change the hardcoded timeouts you had to the TAP framework one, but I also made it sleep 0.1s rather than 1s). It is kinda slow on my laptop, though it doesn't time out. But this failure doesn't surprise me. I suspect we need to make the timeout higher.
--
Álvaro Herrera
Hello, Álvaro!
Thanks for looking into this and pushing.
Hmm, it timed out in prion, which runs this "special tricks" code.
(I did change the hardcoded timeouts you had to the TAP framework one
I think you misunderstood the wait_for_idle role. Its role is to
understand if we are stuck in an injection point during the execution
of commands. If timeout set for a TAP framework timeout and we stuck -
we will wait until test framework will be terminate us due to timeout.
So, "special trick" is not working with such timeout. It should be
relatively small.
The same also for wait_for_injection_point - timeout should be less
then TAP timeout to have some time to output debug information.
But I agree - hardcoded timeouts looks weird.
I'm not sure what the implications are about marking this no longer used
in installcheck. I think it means some buildfarm animals may no longer
run it, if they're configured to only do installcheck. It's a bit of a
bummer because potential loss of runs for the other files in this test
module. Still, I'm not crying about it.
Do you know any way to detect if we are running installcheck in the
test? Or we may add some (env variable like
'enable_injection_points')?
I that case we may skip just the single test without any regression here.
I'll try to prepare next set of fixes today/tomorrow:
* In wait_for_idle exit early if we see backend stuck into injection
point (no sense to wait more, my bad - should be added from the
start), also reduce timeout to 1/2 of TAP
* In wait_for_injection_point use 1/2 of TAP timeout.
* Try to detect install_check in tests and skip instead of whole pack
* Change year to 2026 :)
Best regards,
Mikhail.
Hello, Álvaro!
On Thu, Jan 8, 2026 at 11:08 AM Mihail Nikalayeu
<mihailnikalayeu@gmail.com> wrote:
I'll try to prepare next set of fixes today/tomorrow:
* In wait_for_idle exit early if we see backend stuck into injection
point (no sense to wait more, my bad - should be added from the
start), also reduce timeout to 1/2 of TAP
* In wait_for_injection_point use 1/2 of TAP timeout.
* Try to detect install_check in tests and skip instead of whole pack
* Change year to 2026 :)
Fixes for the fixes are in attachment.
I am not very confident about the second commit, because I am not an
expert in meson/Makefiles.
Some additional explanations are available in commit messages.
Tested with -DCLOBBER_CACHE_ALWAYS=1
I hope this is the last one.
Best regards,
Mikhail.
Attachments:
v1-0001-Fix-wait_for_idle-to-properly-handle-CLOBBER_CACH.patchtext/x-patch; charset=UTF-8; name=v1-0001-Fix-wait_for_idle-to-properly-handle-CLOBBER_CACH.patchDownload
From 538a20ec9abab8642e0770cd52a3adeb458c7d42 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 8 Jan 2026 20:25:41 +0300
Subject: [PATCH v1 1/2] Fix wait_for_idle() to properly handle
CLOBBER_CACHE_ALWAYS builds
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
In commit e1c971945d62, the test 010_index_concurrently_upsert.pl was added with logic to handle CLOBBER_CACHE_ALWAYS builds, where a backend can hit an injection point while attaching it due to catalog cache invalidation. The wait_for_idle() function detects whether the attachment completed or the backend became stuck waiting on an injection point.
However, using the full TAP framework timeout causes the test to wait indefinitely when a backend is stuck, until the test harness terminates it. This prevents the recovery logic from executing.
Fix by reducing the timeout to half the TAP default, and adding early exit when the backend is detected waiting on an injection point. This allows the test to promptly detect stuck backends and wake them up.
Also reduce wait_for_injection_point() timeout to half the TAP default, ensuring time remains for diagnostic output if the test gets stuck.
Author: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/CADzfLwWOVyJygX6BFuyuhTKkJ7uw2e8OcVCDnf6iqnOFhMPE%2BA%40mail.gmail.com
---
.../test_misc/t/010_index_concurrently_upsert.pl | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl b/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
index 5953af11e9e..1f3dcd80886 100644
--- a/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
+++ b/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
@@ -1,5 +1,5 @@
-# Copyright (c) 2025, PostgreSQL Global Development Group
+# Copyright (c) 2026, PostgreSQL Global Development Group
# Test INSERT ON CONFLICT DO UPDATE behavior concurrent with
# CREATE INDEX CONCURRENTLY and REINDEX CONCURRENTLY.
@@ -793,7 +793,7 @@ done_testing();
sub wait_for_injection_point
{
my ($node, $point_name, $timeout) = @_;
- $timeout //= $PostgreSQL::Test::Utils::timeout_default;
+ $timeout //= $PostgreSQL::Test::Utils::timeout_default / 2;
for (my $elapsed = 0; $elapsed < $timeout * 10; $elapsed++)
{
@@ -833,19 +833,23 @@ sub ok_injection_point
}
# Helper: Wait for a specific backend to become idle.
-# Returns true if idle, false if timeout.
+# Returns true if idle, false if waiting for injection point or timeout.
sub wait_for_idle
{
my ($node, $pid, $timeout) = @_;
- $timeout //= $PostgreSQL::Test::Utils::timeout_default;
+ $timeout //= $PostgreSQL::Test::Utils::timeout_default / 2;
for (my $elapsed = 0; $elapsed < $timeout * 10; $elapsed++)
{
- my $state = $node->safe_psql(
+ my $result = $node->safe_psql(
'postgres', qq[
- SELECT state FROM pg_stat_activity WHERE pid = $pid;
+ SELECT state, wait_event_type FROM pg_stat_activity WHERE pid = $pid;
]);
+ my ($state, $wait_event_type) = split(/\|/, $result, 2);
+ $state //= '';
+ $wait_event_type //= '';
return 1 if $state eq 'idle';
+ return 0 if $wait_event_type eq 'InjectionPoint';
sleep(0.1);
}
return 0;
--
2.43.0
v1-0002-Allow-test_misc-TAP-tests-to-run-during-installch.patchtext/x-patch; charset=UTF-8; name=v1-0002-Allow-test_misc-TAP-tests-to-run-during-installch.patchDownload
From 7f0aa3ae4b6be42b0797f757faedf2d43a02ca72 Mon Sep 17 00:00:00 2001
From: nkey <mihailnikalayeu@gmail.com>
Date: Thu, 8 Jan 2026 21:09:07 +0300
Subject: [PATCH v1 2/2] Allow test_misc TAP tests to run during installcheck
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Commit e1c971945d62 disabled installcheck for the entire test_misc module because injection points are cluster-wide and incompatible with a concurrently running test execution. However, this also prevents other tests in the module (001-009) from running during installcheck.
Instead of disabling the whole module, introduce an 'is_installcheck' environment variable set by both Meson and Make build systems.
Individual tests can check this variable and skip themselves when running under installcheck. The injection point test now skips itself when is_installcheck='yes', while other tests in test_misc continue to run normally.
Author: Mihail Nikalayeu <mihailnikalayeu@gmail.com>
Reviewed-by: Álvaro Herrera <alvherre@kurilemu.de>
Discussion: https://postgr.es/m/CADzfLwWOVyJygX6BFuyuhTKkJ7uw2e8OcVCDnf6iqnOFhMPE%2BA%40mail.gmail.com
---
meson.build | 6 ++++--
src/Makefile.global.in | 7 +++++++
src/test/modules/test_misc/Makefile | 3 ---
src/test/modules/test_misc/meson.build | 2 --
.../modules/test_misc/t/010_index_concurrently_upsert.pl | 5 +++++
5 files changed, 16 insertions(+), 7 deletions(-)
diff --git a/meson.build b/meson.build
index 2064d1b0a8d..91d6bed7d46 100644
--- a/meson.build
+++ b/meson.build
@@ -3811,9 +3811,11 @@ endforeach # directories with tests
# repeat condition so meson realizes version dependency
add_test_setup('tmp_install',
is_default: true,
- exclude_suites: running_suites)
+ exclude_suites: running_suites,
+ env: {'is_installcheck': 'no'})
add_test_setup('running',
- exclude_suites: ['setup'] + install_suites)
+ exclude_suites: ['setup'] + install_suites,
+ env: {'is_installcheck': 'yes'})
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 371cd7eba2c..2879310a92e 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -469,6 +469,7 @@ cd $(srcdir) && \
TESTLOGDIR='$(CURDIR)/tmp_check/log' \
TESTDATADIR='$(CURDIR)/tmp_check' \
PATH="$(bindir):$(CURDIR):$$PATH" \
+ is_installcheck='yes' \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
share_contrib_dir='$(DESTDIR)$(datadir)/$(datamoduledir)' \
@@ -483,6 +484,7 @@ cd $(srcdir) && \
TESTLOGDIR='$(CURDIR)/tmp_check/log' \
TESTDATADIR='$(CURDIR)/tmp_check' \
PATH="$(bindir):$(CURDIR):$$PATH" \
+ is_installcheck='yes' \
PGPORT='6$(DEF_PGPORT)' \
PG_REGRESS='$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
@@ -497,6 +499,7 @@ cd $(srcdir) && \
TESTLOGDIR='$(CURDIR)/tmp_check/log' \
TESTDATADIR='$(CURDIR)/tmp_check' \
$(with_temp_install) \
+ is_installcheck='no' \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
share_contrib_dir='$(abs_top_builddir)/tmp_install$(datadir)/$(datamoduledir)' \
@@ -705,6 +708,7 @@ pg_regress_clean_files = results/ regression.diffs regression.out tmp_check/ tmp
pg_regress_check = \
echo "\# +++ regress check in $(subdir) +++" && \
$(with_temp_install) \
+ is_installcheck='no' \
$(top_builddir)/src/test/regress/pg_regress \
--temp-instance=./tmp_check \
--inputdir=$(srcdir) \
@@ -713,6 +717,7 @@ pg_regress_check = \
$(pg_regress_locale_flags) $(EXTRA_REGRESS_OPTS)
pg_regress_installcheck = \
echo "\# +++ regress install-check in $(subdir) +++" && \
+ is_installcheck='yes' \
$(top_builddir)/src/test/regress/pg_regress \
--inputdir=$(srcdir) \
--bindir='$(bindir)' \
@@ -721,6 +726,7 @@ pg_regress_installcheck = \
pg_isolation_regress_check = \
echo "\# +++ isolation check in $(subdir) +++" && \
$(with_temp_install) \
+ is_installcheck='no' \
$(top_builddir)/src/test/isolation/pg_isolation_regress \
--temp-instance=./tmp_check_iso \
--inputdir=$(srcdir) --outputdir=output_iso \
@@ -729,6 +735,7 @@ pg_isolation_regress_check = \
$(pg_regress_locale_flags) $(EXTRA_REGRESS_OPTS)
pg_isolation_regress_installcheck = \
echo "\# +++ isolation install-check in $(subdir) +++" && \
+ is_installcheck='yes' \
$(top_builddir)/src/test/isolation/pg_isolation_regress \
--inputdir=$(srcdir) --outputdir=output_iso \
--bindir='$(bindir)' \
diff --git a/src/test/modules/test_misc/Makefile b/src/test/modules/test_misc/Makefile
index fedbef071ef..399b9094a38 100644
--- a/src/test/modules/test_misc/Makefile
+++ b/src/test/modules/test_misc/Makefile
@@ -5,9 +5,6 @@ TAP_TESTS = 1
EXTRA_INSTALL=src/test/modules/injection_points \
contrib/test_decoding
-# The injection points are cluster-wide, so disable installcheck
-NO_INSTALLCHECK = 1
-
export enable_injection_points
ifdef USE_PGXS
diff --git a/src/test/modules/test_misc/meson.build b/src/test/modules/test_misc/meson.build
index 6e8db1621a7..423f66d8bdc 100644
--- a/src/test/modules/test_misc/meson.build
+++ b/src/test/modules/test_misc/meson.build
@@ -20,7 +20,5 @@ tests += {
't/009_log_temp_files.pl',
't/010_index_concurrently_upsert.pl',
],
- # The injection points are cluster-wide, so disable installcheck
- 'runningcheck': false,
},
}
diff --git a/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl b/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
index 1f3dcd80886..dbf04c6e345 100644
--- a/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
+++ b/src/test/modules/test_misc/t/010_index_concurrently_upsert.pl
@@ -19,6 +19,11 @@ use Test::More;
plan skip_all => 'Injection points not supported by this build'
unless $ENV{enable_injection_points} eq 'yes';
+plan skip_all => 'The injection points are cluster-wide, so skip for installcheck'
+ unless ($ENV{is_installcheck} eq 'no');
+
+is($ENV{is_installcheck}, 'no', 'should not be executed during installcheck');
+
# Node initialization
my $node = PostgreSQL::Test::Cluster->new('node');
$node->init();
--
2.43.0