Avoid streaming zero-filled WAL switch padding

On Thu, Aug 6, 2026 at 3:49 AM Andrey Borodin <x4mmm@yandex-team.ru> wrote:

The attached patch adds a compact replication message for zero-filled WAL
padding.

This is neat. It's almost a custom compression handler for a common
situation.

I tried this atop 798bdcae89d. It applies cleanly and builds without
warnings. The suite passes and the numbers match yours for the
walsender traffic.

The reasoning behind the zero check looks sound to me. Messages start at
a record or page boundary, both of which begin with nonzero bytes, and
CopyXLogRecordToWAL zeroes the padding pages including their headers, so
an all-zero payload can only be switch padding. The sentPtr /
WalSndCaughtUp handling in XLogSendPhysical also looks right.

Also thought a bit about how this would interact with wire compression as
well. I think it'd still be useful together as even if the network byte
savings are less, you're still avoiding the disk writes.

A few things I noticed:

1. The new test is not added to src/test/recovery/meson.build, so meson
never runs it (I added it locally to test it out).

2. There is no negotiation. Every physical START_REPLICATION client gets
the 'z' message, so third-party physical stream consumers and older
pg_receivewal fail with "unrecognized streaming header". I think this
needs an opt-in from the client. Something like a START_REPLICATION
option or at least a protocol guard.

3. The sparse tail defeats wal_init_zero on the standby. Those segments
get recycled with holes, so later writes into them allocate blocks.
That would show up after a promotion. Perhaps only take the truncate
path when wal_init_zero is off, and pwrite the zeros otherwise? The
network saving is the same either way.

--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -59,6 +59,7 @@
   #include "access/xlogwait.h"
   #include "catalog/pg_authid.h"
+#include "common/file_utils.h"
   #include "funcapi.h"
@@ -1048,7 +1049,21 @@ XLogWalRcvWriteZeros(Size nbytes, XLogRecPtr
recptr, TimeLineID tli)
                                 errmsg_internal("zero WAL range
crosses a segment boundary")));

-     if (recvFileZeroedFrom < 0 || recvFileZeroedFrom > startoff)
+     if (wal_init_zero)
+     {
+             /*
+              * XLogFileInit allocated the whole segment so that
writes into it,
+              * including after it is recycled, never have to allocate blocks.
+              */
+             pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
+             if (pg_pwrite_zeros(recvFile, nbytes, (pgoff_t) startoff) < 0)
+                     ereport(PANIC,
+                                     (errcode_for_file_access(),
+                                      errmsg("could not write
zero-filled WAL tail: %m")));
+             pgstat_report_wait_end();
+     }
+     else if (recvFileZeroedFrom < 0 || recvFileZeroedFrom > startoff)

4. If the wal_init_zero case is changed to write the zero range with
pg_pwrite_zeros(), it should use the same pgstat_...() accounting as
XLogWalRcvWrite(). Otherwise those writes will be missing stats.

5. A crash between the two ftruncate calls leaves a short segment. I
think replay is fine since the padding is never read, but anything
that expects to read whole segments would have a problem.

6. The walsender scan can use pg_memory_is_all_zeros() instead of the
byte loop. Early exit case would be the same and faster on long
zero runs.

Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/

Hi Sehrope,

Thank you for the detailed review!

On Sat, Sep 5, 2026, Sehrope Sarkuni wrote:

I think this needs an opt-in from the client. Something like a
START_REPLICATION option or at least a protocol guard.

I agree. v2 adds a SKIP_WAL_PADDING option to physical
START_REPLICATION. Existing clients continue to receive ordinary WALData
messages, and built-in clients request the new format only from PG20 servers.

The receiver no longer truncates and extends the segment. A segment newly
created by walreceiver is already zero-filled according to wal_init_zero. For
an existing or recycled segment, walreceiver writes the zeros locally with
the normal wait-event and pg_stat_io accounting. This also removes the
short-file crash window. New uncompressed pg_receivewal output is already
pre-padded. Resumed, compressed, and tar output generate the zeros locally.

I also added the missing Meson registration and used
pg_memory_is_all_zeros(). As a compatibility check, I connected a pre-patch
PG20 pg_receivewal to the patched server and compared the resulting switched
segment byte for byte.

PFA v2. Thank you!

Best regards, Andrey Borodin.

On Sun, Sep 6, 2026 at 8:49 AM Andrey Borodin <x4mmm@yandex-team.ru> wrote:

On Sat, Sep 5, 2026, Sehrope Sarkuni wrote:

I think this needs an opt-in from the client. Something like a
START_REPLICATION option or at least a protocol guard.

I agree. v2 adds a SKIP_WAL_PADDING option to physical
START_REPLICATION. Existing clients continue to receive ordinary WALData
messages, and built-in clients request the new format only from PG20 servers.

The receiver no longer truncates and extends the segment. A segment newly
created by walreceiver is already zero-filled according to wal_init_zero. For
an existing or recycled segment, walreceiver writes the zeros locally with
the normal wait-event and pg_stat_io accounting. This also removes the
short-file crash window. New uncompressed pg_receivewal output is already
pre-padded. Resumed, compressed, and tar output generate the zeros locally.

I also added the missing Meson registration and used
pg_memory_is_all_zeros(). As a compatibility check, I connected a pre-patch
PG20 pg_receivewal to the patched server and compared the resulting switched
segment byte for byte.

Looks good. v2 addresses everything from my first pass and it all looks
right to me.

Tested on 798bdcae89d. Builds clean, recovery suite passes, and a
pre-patch pg_receivewal against the patched server gets plain WALData
and byte-identical segments. With the patch on a standby the wire
traffic drops from 16 MiB to ~125 kB.

I'm not particularly familiar with the replication command grammar,
so other eyes on the START_REPLICATION option parsing would be good,
but it looks fine and behaves correctly in all the tests.

A few small things:

1. The test no longer proves the 'z' path was taken. An old client
produces the same bytes, so the compare passes either way. The
attached patch resets pg_stat_io on the standby before the switch
and checks the walreceiver wrote less than wal_segment_size: 248 kB
here vs exactly 16 MB unpatched. It restarts the standby first to
flush the counters, since the walreceiver only reports them on its
status timer. Byte counts also avoid the Windows skip a sparse-file
check would need.

2. The pg_stat_activity check runs right after $standby->start, before
the walreceiver has necessarily connected. That should use
poll_query_until(...).

3. recvFileIsNew is reset in XLogWalRcvClose but not at the
end-of-timeline close in WalReceiverMain.

See attached for a patch atop your v2 to address 1-3. The 0001 is your
v2 patch rebased and the additions are in 0002. Nothing special for the
rebase either (the patch is identical).

CI run for the modified tests shows it ran on Windows and passed there
as well:

https://github.com/sehrope/postgres/actions/runs/34376945257/job/102552798366#step:14:94

Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/

Hi Sehrope,

On 9 Sep 2026, Sehrope Sarkuni wrote:

See attached for a patch atop your v2 to address 1-3. The 0001 is your
v2 patch rebased and the additions are in 0002.

Thank you for the review and fixes! Your changes look good to me.
The byte-count check is a useful addition: matching files alone did not
prove that we actually skipped the padding. Let's consider your v3 as the
current version.

Best regards, Andrey Borodin.