[PATCH] Speed up pg_waldump TAP test and fix some GitHub CI Windows flakiness
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t253599psql -h localhost -U postgresBuilt from patchset v1 (message #1), August 30, 2026 at 03:58 AM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t253599_1 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t253599_1 && git checkout t253599_1Patchset v1 (message #1) is on t253599_1
Hi hackers,
Was testing a rebased Windows patch using GitHub CI and had some flaky slow
test timeout errors for an unrelated pg_waldump test. On Windows (and only
Windows) it was running for 500+ seconds and timing out. After a bit of
research, I figured it's because of the notoriously slow spoofed IPC that
perl does for the forked pg_waldump invocations (that then gets grepped by
perl).
001_basic.pl dumps the full WAL range, segments and records, a multiple
times per scenario across multiple scenarios, and splits the ~9 MB of
output in Perl each time.
The attached decodes the full range once against the plain data directory,
so every rmgr's desc output is still processed at least once. The
fall-off-the-end checks start at the contrecord LSN, a few records from the
end. Everything else gets --limit, which counts after filtering, so the
--rmgr, --fullpage, --fork, --relation and --block runs stop at the first
few matches. This cuts down the IPC considerably.
While it was written to speed up Windows, it's faster on Linux too. There's
not much to save though as it was already only a couple seconds. On a quiet
Linux box of master vs this patch, meson test pg_waldump/001_basic, 10 runs
each: 4.43s (4.18-4.65) -> 2.88s (2.49-3.13)
Windows should gain a lot more. I only have a few GitHub CI numbers to
compare against, but I'm seeing 300-500s for master down to ~40-90s for the
patch.
Here's a recent 538s run on master:
https://github.com/postgres/postgres/actions/runs/33207297796/job/98971778299#step:14:262
Here's the patched version (72s):
https://github.com/sehrope/postgres/actions/runs/33228440369/job/99036914098#step:14:199
Note that the bare --stats run has --limit option in front of it (rather
than as the last option) on purpose as the reverse failed on Windows.
That's due to a bug in src/port/getopt_long.c handling for optional
arguments. I'm putting together a separate patch for that.
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/