REINDEX (CONCURRENTLY) TABLE handles DEFERRED constraints as IMMEDIATE while processing

Started by Dag Lem3 months ago4 messagesbugs
Jump to latest
#1Dag Lem
dag@nimrod.no

Hi,

The following bug was detected on PostgreSQL 16.14, and I was able to
reproduce it on

PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 16.1.1
20260501 (Red Hat 16.1.1-1), 64-bit

We have a database table with some 150 million rows, with several
indexes and a couple of constraints on the form CONSTRAINT
uq_constraint_name UNIQUE (...) DEFERRABLE INITIALLY DEFERRED.

While processing, "REINDEX (CONCURRENTLY) TABLE table_name" temporarily
treats the DEFERRED constraints as IMMEDIATE, causing transactions to
fail with errors on the form 'ERROR: duplicate key value violates
unique constraint "uq_constraint_name"'.

To see what could cause this issue, I repeatedly did the SELECT below
while REINDEX (CONCURRENTLY) was working, while also doing transactions
which depend on constraints being DEFERRED.

SELECT i.indexrelid, c.relname, i.indisunique, i.indimmediate,
i.indisvalid, i.indisready
FROM pg_class c
JOIN pg_index i ON i.indexrelid = c.oid
WHERE c.relkind = 'i' AND c.relname LIKE '%_ccnew'
ORDER BY i.indexrelid;

This yields output on the form below.

indexrelid | relname | indisunique |
indimmediate | indisvalid | indisready
------------+-------------------------------------------+-------------+--------------+------------+------------
124315 | ix_non_unique_index_1_ccnew | f |
t | f | t
124316 | ix_non_unique_index_2_ccnew | f |
t | f | t
...
124342 | pk_primary_key_ccnew | t |
t | f | t
124343 | uq_unique_deferred_constraint_1_ccnew | t |
t | f | f
124344 | uq_unique_deferred_constraint_2_ccnew | t |
t | f | f
124345 | pg_toast_71419_index_ccnew | t |
t | f | f

Note how the new indexes backing the DEFERRED constraints are marked
with indimmediate = true(!), and that the progress can be followed by
looking at the indisready column.

As soon as construction of the first index backing a UNIQUE DEFERRED is
completed (indisready = true), transactions depending on DEFERRED
constraints will start to fail, and will continue to do so until REINDEX
(CONCURRENTLY) has completed.

Could this be an oversight wrt. how indexes and (deferred) constraints
are related? Note how it is currently not possible to safely add a
UNIQUE DEFERRED constraint following the example in
https://www.postgresql.org/docs/18/sql-altertable.html

CREATE UNIQUE INDEX CONCURRENTLY dist_id_temp_idx ON distributors
(dist_id);
ALTER TABLE distributors DROP CONSTRAINT distributors_pkey,
ADD CONSTRAINT distributors_pkey PRIMARY KEY USING INDEX
dist_id_temp_idx;

For this to work safely with UNIQUE DEFERRED constraints, I assume it
would be necessary to add an option to CREATE INDEX to make an index
DEFERRED.

Best regards,
Dag Lem

#2Dag Lem
dag@nimrod.no
In reply to: Dag Lem (#1)
Re: REINDEX (CONCURRENTLY) TABLE handles DEFERRED constraints as IMMEDIATE while processing

On 2026-06-05 13:25, Dag Lem wrote:

While processing, "REINDEX (CONCURRENTLY) TABLE table_name" temporarily
treats the DEFERRED constraints as IMMEDIATE, causing transactions to
fail with errors on the form 'ERROR: duplicate key value violates
unique constraint "uq_constraint_name"'.

Apologies, this should be "constraint_name_ccnew" - the new index
backing the constraint.

Best regards,
Dag Lem

#3Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Dag Lem (#1)
Re: REINDEX (CONCURRENTLY) TABLE handles DEFERRED constraints as IMMEDIATE while processing

On 2026-Jun-05, Dag Lem wrote:

While processing, "REINDEX (CONCURRENTLY) TABLE table_name" temporarily
treats the DEFERRED constraints as IMMEDIATE, causing transactions to fail
with errors on the form 'ERROR: duplicate key value violates unique
constraint "uq_constraint_name"'.

I would say that this is clearly an oversight.

Note how it is currently not possible to safely add a UNIQUE
DEFERRED constraint following the example in
https://www.postgresql.org/docs/18/sql-altertable.html

CREATE UNIQUE INDEX CONCURRENTLY dist_id_temp_idx ON distributors (dist_id);
ALTER TABLE distributors DROP CONSTRAINT distributors_pkey,
ADD CONSTRAINT distributors_pkey PRIMARY KEY USING INDEX
dist_id_temp_idx;

For this to work safely with UNIQUE DEFERRED constraints, I assume it would
be necessary to add an option to CREATE INDEX to make an index DEFERRED.

I think this closely related problem is different. We'd probably want
to fix the above in a backpatchable manner, but this one sounds like a
new feature.

--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Linux transformó mi computadora, de una `máquina para hacer cosas',
en un aparato realmente entretenido, sobre el cual cada día aprendo
algo nuevo" (Jaime Salinas)

#4Nitin Motiani
nitinmotiani@google.com
In reply to: Alvaro Herrera (#3)
Re: REINDEX (CONCURRENTLY) TABLE handles DEFERRED constraints as IMMEDIATE while processing

On Mon, Jul 20, 2026 at 9:40 PM Álvaro Herrera <alvherre@kurilemu.de> wrote:

On 2026-Jun-05, Dag Lem wrote:

While processing, "REINDEX (CONCURRENTLY) TABLE table_name" temporarily
treats the DEFERRED constraints as IMMEDIATE, causing transactions to fail
with errors on the form 'ERROR: duplicate key value violates unique
constraint "uq_constraint_name"'.

I would say that this is clearly an oversight.

Hi,

I have proposed a patch for this on pgsql-hackers. Please take a look.
It can be found at
/messages/by-id/CAH5HC97JmjPpgiQOqW9xm8qXhNiu7zZ1Qh+FfhEESJuDv69kuQ@mail.gmail.com.

Thanks & Regards,
Nitin Motiani
Google