Let physical WAL senders restore missing segments from the archive
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t254335psql -h localhost -U postgresBuilt from patchset v1 (message #1), October 23, 2026 at 04:58 PM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t254335_1 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t254335_1 && git checkout t254335_1Patchset v1 (message #1) is on t254335_1
Hi hackers,
A standby can fall behind the WAL retained by its upstream while the
missing segments are still available in the archive. We would like to
keep that history in the archive, not retain all of it locally for a
replication slot. Giving the receiver its own restore_command is not
always an option either: we may be able to grant replication access
without giving it access to the archive. Could the sender provide the
missing WAL over the existing replication connection instead?
Fujii proposed this in 2010 [0]/messages/by-id/AANLkTinN=xsPOoaXzVFSp1OkfMDAB1f_d-F91xjEZDV8@mail.gmail.com, and Japin posted a patch in 2025 [1]/messages/by-id/ME0P300MB0445C80039D6534452A87539B657A@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM.
I would like to revisit it with the archive access restriction above as
the use case, and with private copies instead of restoring into the
normal WAL namespace.
Attached is a prototype. With wal_sender_restore = on, a physical sender
that cannot open a segment in pg_wal tries restore_command before
failing. This works on a primary or a [cascading] standby.
The temporary storage and cleanup questions came up when discussing
reviving lost replication slots [2]/messages/by-id/CALj2ACVUgCYVgKc42GDCFZo_=xjYyDtHXBhzyhT3s4UqAzLK4Q@mail.gmail.com. Here each sender keeps at most one
restored segment in its own pg_wal/xlogtemp.*.walsender directory.
Alexander pointed out the cost of repeated failed restores on the
primary [3]/messages/by-id/CAFh8B=nH=41scx3F_EAh2H5O1w-nj8b7uCMp_0z4p4wsv2tFDA@mail.gmail.com. A request arriving within one second of a failed restore
waits for the remainder of that interval.
The existing timeline and flush/replay limits still apply. This does not
enable logical decoding from the archive or revive an invalidated slot.
It can serve a receiver without a slot, or with a usable physical slot.
Joao's proposal to revalidate physical slots [4]https://commitfest.postgresql.org/patch/6865/ addresses that separate
part of the problem.
The prototype runs restore_command synchronously. A slow fetch prevents
keepalives and feedback processing, can cause receiver timeouts, and can
delay shutdown. An interruptible implementation would need to manage the
child command while servicing the replication connection.
What do you think?
Best regards, Andrey Borodin.
[0]: /messages/by-id/AANLkTinN=xsPOoaXzVFSp1OkfMDAB1f_d-F91xjEZDV8@mail.gmail.com
[1]: /messages/by-id/ME0P300MB0445C80039D6534452A87539B657A@ME0P300MB0445.AUSP300.PROD.OUTLOOK.COM
[2]: /messages/by-id/CALj2ACVUgCYVgKc42GDCFZo_=xjYyDtHXBhzyhT3s4UqAzLK4Q@mail.gmail.com
[3]: /messages/by-id/CAFh8B=nH=41scx3F_EAh2H5O1w-nj8b7uCMp_0z4p4wsv2tFDA@mail.gmail.com
[4]: https://commitfest.postgresql.org/patch/6865/