SERIALIZABLE can commit write skew after SSI summarization

Started by Jacob Brazealabout 2 months ago2 messageshackers
Beta feature

Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.

needs rebasesuccessCI history

You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:

docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t253279
psql -h localhost -U postgres

Built from patchset v2 (message #2), September 09, 2026 at 01:15 PM.

Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:

git clone --branch t253279_2 https://github.com/hackorum-dev/postgres.git

In a checkout you already have, add the fork once:

git remote add hackorum https://github.com/hackorum-dev/postgres.git

then, for this patchset and every later one:

git fetch hackorum t253279_2 && git checkout t253279_2

Patchset v2 (message #2) is on t253279_2

Jump to latest
#1Jacob Brazeal
jacob.brazeal@gmail.com

Hi,

I found a reproducible case where SERIALIZABLE commits a non-serializable
write-skew schedule after SSI summarizes a committed transaction's SIREAD
locks into OldCommittedSxact.

I reproduced this on PostgreSQL 17.10, 18.4, and current master, although
it appears to date back to 9.1.

Setup:

CREATE TABLE t(id int PRIMARY KEY, v int);
INSERT INTO t VALUES (1, 0), (2, 0);

Let's say our application rule is that a transaction may set its row to 1
only if no row is already 1:

UPDATE t
SET v = 1
WHERE id = <n>
AND (SELECT count(*) FROM t WHERE v = 1) = 0;

In either serial order, the second transaction sees one row already set and
does not update, so the final count is 1.

Session 1:

BEGIN ISOLATION LEVEL SERIALIZABLE;
SELECT count(*) FROM t WHERE v = 1;

Leave this transaction open.

Session 2:

BEGIN ISOLATION LEVEL SERIALIZABLE;
SELECT count(*) FROM t WHERE v = 1;

UPDATE t
SET v = 1
WHERE id = 1
AND (SELECT count(*) FROM t WHERE v = 1) = 0;

COMMIT;

At this point session 2 is committed, but its SIREAD locks are still
retained because session 1 remains open.

Now run enough short serializable transactions to exhaust the
SERIALIZABLEXACT array and force summarization:

python3 - <<'PY' > /tmp/churn.sql
for i in range(1500):
print("BEGIN ISOLATION LEVEL SERIALIZABLE; SELECT 1; COMMIT;")
PY

psql -d test -f /tmp/churn.sql

The exact threshold depends on MaxBackends. On my stock max_connections =
100 cluster it was between 1400 and 1420 transactions.
MaxBackends = max_connections + autovacuum_worker_slots
+ max_worker_processes + max_wal_senders + 2
(NUM_SPECIAL_WORKER_PROCS)
max_serializable_xacts = (MaxBackends + max_prepared_transactions) * 10

The transfer of session 2's SIREAD locks to the summarization dummy is
visible in pg_locks:

SELECT pid, virtualtransaction, locktype, relation::regclass
FROM pg_locks
WHERE mode = 'SIReadLock'
AND relation = 't'::regclass;

Before summarization, the relevant locks have session 2's pid. Afterwards
they appear with:

pid | virtualtransaction
-----+-------------------
NULL | -1/0

Back in session 1:

UPDATE t
SET v = 1
WHERE id = 2
AND (SELECT count(*) FROM t WHERE v = 1) = 0;

COMMIT;

SELECT count(*) FROM t WHERE v = 1;

Observed after summarization:

count
-------
2

Both transactions commit with no serialization failure.

Controls:

- with no churn, session 1's UPDATE raises 40001;
- with sub-threshold churn, it also raises 40001;
- both serial orders produce a final count of 1;
- once session 2's SIREAD locks move to virtualtransaction = '-1/0',
session 1 commits and the final count is 2.

The control error is:

ERROR: could not serialize access due to read/write dependencies among
transactions
DETAIL: Reason code: Canceled on identification as a pivot, during write.
HINT: The transaction might succeed if retried.

Note - this appears to be a live variant of the DTEST_SUMMARIZE_SERIAL case
referenced here [0]/messages/by-id/787936C5-4155-4CF9-939D-39DC0EC1C892@yandex-team.ru. Starting a new thread since that was not that primary
point of the thread, I think.

[0]: /messages/by-id/787936C5-4155-4CF9-939D-39DC0EC1C892@yandex-team.ru
/messages/by-id/787936C5-4155-4CF9-939D-39DC0EC1C892@yandex-team.ru

Regards,
Jacob

#2Andrey Borodin
amborodin@acm.org
In reply to: Jacob Brazeal (#1)
Re: SERIALIZABLE can commit write skew after SSI summarization

Hi Jacob,

once session 2's SIREAD locks move to virtualtransaction = '-1/0',
session 1 commits and the final count is 2.

I reproduced this. The overlap test in CheckTargetForConflictsIn() uses
finishedBefore for a committed transaction. OldCommittedSxact has no
valid finishedBefore, so the test ignores its predicate locks.

Each summarized predicate lock already stores the latest commitSeqNo
among the transactions folded into it. PFA a tentative patch set which
compares that with lastCommitBeforeSnapshot. A later commit means that
at least one reader represented by the lock overlapped the writer.

The first patch adds a deterministic injection-point test, and the
second adds the fix.

I have two buildfarm animals available for unusual configurations. One
currently runs with WAL consistency checking, and the other checks WAL
compatibility with a zero release version. I could enable
TEST_SUMMARIZE_SERIAL on one of them. I had seen a related false
negative while trying different modes around the Jepsen report, but did
not pursue it then. Your report made me return to it. I am not sure
whether the existing check-world schedules would have exposed this bug.
Would permanently testing this mode be useful, and is there any reason
not to do that?

Thank you!

Best regards, Andrey Borodin.

Attachments:

t253279_2
v1-0001-Test-SSI-conflict-detection-after-summarization.patchapplication/octet-stream; name=v1-0001-Test-SSI-conflict-detection-after-summarization.patch; x-unix-mode=0644Download+178-1
v1-0002-Detect-SSI-conflicts-with-summarized-predicate-lo.patchapplication/octet-stream; name=v1-0002-Detect-SSI-conflicts-with-summarized-predicate-lo.patch; x-unix-mode=0644Download+29-7