[PATCH] Harden recovery/t/051_effective_wal_level against WAL recycling
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t253357psql -h localhost -U postgresBuilt from patchset v1 (message #1), August 24, 2026 at 01:46 PM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t253357_1 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t253357_1 && git checkout t253357_1Patchset v1 (message #1) is on t253357_1
Greetings,
In recovery/t/051_effective_wal_level.pl, standby3, standby4, and standby5
are all init_from_backup()'d from 'my_backup', taken near the top of the
test, and started much later, after the primary has produced and recycled a
lot of WAL. Nothing keeps the WAL they need to reach consistency: no slot
covering it, no wal_keep_size, no archive. (standby5 has phys_slot, but a
physical slot only pins WAL from its own creation, not the older WAL needed
to replay from 'my_backup'.) So each one starts only because the primary
happens not to have recycled that WAL yet.
That holds with the default WAL page layout but is fragile: anything that
consumes WAL address space a bit faster can push the needed segment out of
the retention window, and the standby then fails to start with "requested
WAL segment ... has already been removed". These standbys are scaffolding
for the promotion, logical-decoding, and slot-synchronization tests, not a
test of WAL recycling.
The fix takes a fresh backup immediately before each of them, so the start
point is recent and within retained WAL regardless of what the earlier part
of the test produced.
It depends on retention timing, so there's no on-demand reproduction; the
change is justified by the structure (late standbys restoring from an early,
unpinned backup). 051 passes with injection points enabled, so all three
standby blocks run.
--
Bryan Green
EDB: https://www.enterprisedb.com
On 8/8/26 22:44, Bryan Green wrote:
Greetings,
In recovery/t/051_effective_wal_level.pl, standby3, standby4, and standby5
are all init_from_backup()'d from 'my_backup', taken near the top of the
test, and started much later, after the primary has produced and recycled a
lot of WAL. Nothing keeps the WAL they need to reach consistency: no slot
covering it, no wal_keep_size, no archive. (standby5 has phys_slot, but a
physical slot only pins WAL from its own creation, not the older WAL needed
to replay from 'my_backup'.) So each one starts only because the primary
happens not to have recycled that WAL yet.That holds with the default WAL page layout but is fragile: anything that
consumes WAL address space a bit faster can push the needed segment out of
the retention window, and the standby then fails to start with "requested
WAL segment ... has already been removed". These standbys are scaffolding
for the promotion, logical-decoding, and slot-synchronization tests, not a
test of WAL recycling.The fix takes a fresh backup immediately before each of them, so the start
point is recent and within retained WAL regardless of what the earlier part
of the test produced.It depends on retention timing, so there's no on-demand reproduction; the
change is justified by the structure (late standbys restoring from an early,
unpinned backup). 051 passes with injection points enabled, so all three
standby blocks run.--
Bryan Green
EDB: https://www.enterprisedb.com
I have a use case that is not stock PG that this is related to. I'm
withdrawing this.
--
Bryan Green
EDB: https://www.enterprisedb.com
Hello!
Bryan Green <dbryan.green@gmail.com> writes:
...
The fix takes a fresh backup immediately before each of them, so the start
point is recent and within retained WAL regardless of what the earlier part
of the test produced.It depends on retention timing, so there's no on-demand reproduction; the
change is justified by the structure (late standbys restoring from an early,
unpinned backup). 051 passes with injection points enabled, so all three
standby blocks run.
I've been in this situation and by experience, reproducing this it's
really hard, but it's easier if the test force the situation, so what
I've done is add to test, the following lines:
$standby2->stop;
$cascade->stop;
+for (1 .. 10)
+{
+ $primary->safe_psql('postgres', 'SELECT pg_switch_wal()');
+}
+$primary->safe_psql('postgres', 'CHECKPOINT');
+# Initialize standby3 node and start it. Take a fresh backup rather than
This was useful to create some WAL files and helped to reproduce the
issue, probably is worthy to have this in the test to avoid replicating
the same issue in the future?
I wonder if there's any other test with a similar issue, but that's not
related to this patch at all.
Regards,
--
Jonathan Gonzalez V.
EDB
https://www.enterprisedb.com
Hello Bryan,
On 2026-Aug-10, Bryan Green wrote:
On 8/8/26 22:44, Bryan Green wrote:
In recovery/t/051_effective_wal_level.pl, standby3, standby4, and standby5
are all init_from_backup()'d from 'my_backup', taken near the top of the
test, and started much later, after the primary has produced and recycled a
lot of WAL. Nothing keeps the WAL they need to reach consistency: no slot
covering it, no wal_keep_size, no archive.I have a use case that is not stock PG that this is related to. I'm
withdrawing this.
You have a use case for ... this test remaining borderline broken? I'm
wondering if you wanted to reply to some other thread.
Thanks
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"La vida es para el que se aventura"
On 8/12/26 06:26, Álvaro Herrera wrote:
Hello Bryan,
On 2026-Aug-10, Bryan Green wrote:
On 8/8/26 22:44, Bryan Green wrote:
In recovery/t/051_effective_wal_level.pl, standby3, standby4, and standby5
are all init_from_backup()'d from 'my_backup', taken near the top of the
test, and started much later, after the primary has produced and recycled a
lot of WAL. Nothing keeps the WAL they need to reach consistency: no slot
covering it, no wal_keep_size, no archive.I have a use case that is not stock PG that this is related to. I'm
withdrawing this.You have a use case for ... this test remaining borderline broken? I'm
wondering if you wanted to reply to some other thread.Thanks
Ah, you are correct. This was meant for another thread. Not withdrawn.
Thanks
--
Bryan Green
https://www.enterprisedb.com
On Sat, Aug 8, 2026 at 11:45 PM Bryan Green <dbryan.green@gmail.com> wrote:
The fix takes a fresh backup immediately before each of them, so the start
point is recent and within retained WAL regardless of what the earlier part
of the test produced.
A new backup seems a little expensive, could we just create a slot?
Cheers,
Greg