Recovery conflict resolution misses backends that import snapshots

Started by Scott Rayabout 8 hours ago1 messageshackers
Jump to latest
#1Scott Ray
scott@scottray.io

Hello,

There is a silent bug in standby conflict resolution that can cause
incorrect results. In ResolveRecoveryConflictWithSnapshot, the standby
calls GetConflictingVirtualXIDs once, then waits for each returned VXID
to disappear. However, after GetConflictingVirtualXIDs releases the
ProcArrayLock, a backend may import a conflicting snapshot from one of
the backends GetConflictingVirtualXIDs identified as conflicting. The
resulting VXID conflicts with recovery, but
ResolveRecoveryConflictWithSnapshot doesn't know it has to wait for the
new VXID to disappear before proceeding, so the standby removes tuple
versions or index entries that the importing backend needs.

This problem has existed since PG 10 (6c2003f8a1b), which introduced
standby snapshot export and broke ResolveRecoveryConflictWithSnapshot's
assumption that after GetConflictingVirtualXIDs returns, no new
conflicting VXIDs can be created.

The first attached patch adds a TAP test that fails deterministically
on unpatched master. The second makes
ResolveRecoveryConflictWithSnapshot call GetConflictingVirtualXIDs
until the result is empty, at which point importing a conflicting
snapshot is impossible: acquiring a conflicting xmin requires a live,
conflicting VXID as a source.

--
Scott Ray

Attachments:

v1-0002-Fix-recovery-conflict-resolution-to-account-for-i.patchapplication/octet-stream; filename=v1-0002-Fix-recovery-conflict-resolution-to-account-for-i.patch; name=v1-0002-Fix-recovery-conflict-resolution-to-account-for-i.patchDownload+41-10
v1-0001-Add-TAP-test-for-recovery-conflicts-from-imported.patchapplication/octet-stream; filename=v1-0001-Add-TAP-test-for-recovery-conflicts-from-imported.patch; name=v1-0001-Add-TAP-test-for-recovery-conflicts-from-imported.patchDownload+152-1