pg_upgrade stores an invalid datminmxid/relminmxid when the old cluster's newest multixact is 0xFFFFFFFF

Started by Manuabout 1 month ago3 messageshackers
Jump to latest
#1Manu
manuelreyesbravo@gmail.com

Hi,

While testing pg_upgrade's multixact conversion around commit 2323f03,
I ran into an adjacent, pre-existing issue.

When the old cluster's newest multixact is 0xFFFFFFFF, its checkpoint
has NextMultiXactId = 0. copy_xact_xlog_xid() maps that to
FirstMultiXactId for pg_resetwal, but set_frozenxids() writes the raw 0
into pg_database.datminmxid and pg_class.relminmxid (the code dates to
a61daa14d56, 2014).

The effect:
- on assert-enabled builds, VACUUM fails Assert(MultiXactIdIsValid(...));
- otherwise template0 keeps datminmxid = 0 (mxid_age 2147483647) and
multixact truncation is skipped ("cannot truncate up to MultiXact 0
... skipping truncation") until autovacuum vacuums template0 -- which
it does within ~60 s by default, but never with autovacuum = off.

19's new oldestMultiXid consistency check does not catch it: it uses
modular arithmetic, so 0 "follows" 0xFFFFFFFF and passes. It
reproduces with pg_upgrade 18 and master.

The attached patch applies the same adjustment in set_frozenxids(), not
in controldata.c: the raw nextMulti = 0 is still needed so the multixact
reader sees that 0xFFFFFFFF is the newest multixid -- the case 2323f03
fixed. It also adds a case to t/007_multixact_conversion.pl.

Measured on cassert builds from 14..18 across 8 offset/multixid
wraparound boundaries (40 upgrades, plus --link from 18): with the fix,
no invalid datminmxid/relminmxid and no other datminmxid change (only
template0's 0 -> 1 in the 8 affected upgrades); without it, VACUUM traps
and the two new test checks fail. check-world passes.

So the severity is low with the default autovacuum, persistent with
autovacuum off, and a hard failure on assert-enabled builds.

Regards,
Manu

Attachments:

t254249_1
v2-0001-pg_upgrade-Don-t-store-an-invalid-datminmxid-relm.patchtext/x-patchDownload+73-3
#2Zsolt Parragi
zsolt.parragi@percona.com
In reply to: Manu (#1)
Re: pg_upgrade stores an invalid datminmxid/relminmxid when the old cluster's newest multixact is 0xFFFFFFFF

Hello

I agree that this seems to be another corner case, but about the commit message:

servers before v19 skip 0 only when the next multixid is assigned.

It doesn't seem to be 19 specific to me?

#3Manu
manuelreyesbravo@gmail.com
In reply to: Zsolt Parragi (#2)
Re: pg_upgrade stores an invalid datminmxid/relminmxid when the old cluster's newest multixact is 0xFFFFFFFF

Hi Zsolt,

servers before v19 skip 0 only when the next multixid is assigned.

It doesn't seem to be 19 specific to me?

You're right, it isn't 19-specific, and that line in the commit message was
misleading. The bug is in set_frozenxids() and dates back to 2014; what
changed is only whether you can reach it:

- pg_upgrade to 14 still fails earlier, in pg_resetwal ("multitransaction ID
(-m) must not be 0"), because 8747b969fd8 was not backpatched to 14, so 14
is not affected.
- pg_upgrade to 15..18 has been able to store the 0 since 8747b969fd8 made
pg_resetwal accept -m 0.
- 19 and master reach the same code through the multixact conversion path.

The only part that is really about 19 is the trigger: a pre-19 server leaves
NextMultiXactId = 0 in the control file after assigning 0xFFFFFFFF, while
since 87a350e1f28 a 19 server never stores 0. I checked by recompiling:
14..18 clusters record 0, 19 and master record 1, so only a pre-19 old
cluster reaches it. And the 0 is not only template0's datminmxid -- it is the
relminmxid of the initdb-created catalogs in every database.

Attached is v3 for master, with the commit message rewritten and
Backpatch-through: 15, plus per-branch versions for 19 down to 15. The
back-branch files are attached as .txt so the cfbot does not try to apply
them to master. 007_multixact_conversion.pl only exists in 19+, so the
15..18 versions add a small same-version test. On every branch the test
fails without the fix and passes with it, and every old-version ->
target-version combination is clean with it.

Regards,
Manu

Attachments:

t254249_3
v3-0001-pg_upgrade-Don-t-store-an-invalid-datminmxid-relm.patchtext/x-patchDownload+72-3
v3-0001-pg_upgrade-Don-t-store-an-invalid-datminmxid-relm-REL_19.txttext/plainDownload+72-3
v3-0001-pg_upgrade-Don-t-store-an-invalid-datminmxid-relm-REL_18.txttext/plainDownload+106-3
v3-0001-pg_upgrade-Don-t-store-an-invalid-datminmxid-relm-REL_17.txttext/plainDownload+106-3
v3-0001-pg_upgrade-Don-t-store-an-invalid-datminmxid-relm-REL_16.txttext/plainDownload+106-3
v3-0001-pg_upgrade-Don-t-store-an-invalid-datminmxid-relm-REL_15.txttext/plainDownload+105-3