[PG19]pg_verifybackup never finishes on a gzip-compressed tar backup
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
This thread has been committed, so CI has stopped here. Anything below is the last result it produced.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t254076psql -h localhost -U postgresBuilt from patchset v1 (message #1), October 08, 2026 at 01:32 AM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t254076_1 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t254076_1 && git checkout t254076_1Patchset v1 (message #1) is on t254076_1
Hi hackers,
I used Opus to analyze the new features in PG 19.
It found that pg_verifybackup never finishes on a tar-format
backup taken with pg_basebackup -Ft -z.
I had manually proved that by hand.
On REL_19_STABLE:
pg_basebackup -D /tmp/verify_targz.66809/bk -c fast -Ft -z
pg_verifybackup /tmp/verify_targz.66809/bk
# still running after 20 s, killing pg_waldump
pg_verifybackup: error: WAL parsing failed for timeline 1
pg_verifybackup -n /tmp/verify_targz.66809/bk
backup successfully verified
rewrite pg_wal.tar.gz as a single gzip member
pg_verifybackup /tmp/verify_targz.66809/bk
backup successfully verified
With WAL streaming, pg_basebackup ends the gzip stream before and
after every tar header, so pg_wal.tar.gz is a chain of gzip members.
astreamer_gzip_decompressor_content() loops until avail_in is zero.
After the first member inflate() keeps returning Z_STREAM_END and
takes no more input, so the loop never ends. The loop runs in
pg_waldump, and pg_verifybackup waits for it.
Patches attached. 0001 calls inflateReset() when inflate() returns
Z_STREAM_END, so the next member is read. 0002 adds a --gzip case to
the tar test in 007_wal.pl and is optional.
Thanks,
Shihao
Attachments:
t254076_1v1-0001-Handle-concatenated-gzip-members-in-astreamer-gzi.patchapplication/octet-stream; name=v1-0001-Handle-concatenated-gzip-members-in-astreamer-gzi.patchDownload+8-1
v1-0002-Add-test-for-pg_verifybackup-on-a-gzip-compressed.patchapplication/octet-stream; name=v1-0002-Add-test-for-pg_verifybackup-on-a-gzip-compressed.patchDownload+22-1
On Oct 6, 2026, at 09:15, shihao zhong <zhong950419@gmail.com> wrote:
Hi hackers,
I used Opus to analyze the new features in PG 19.
It found that pg_verifybackup never finishes on a tar-format
backup taken with pg_basebackup -Ft -z.I had manually proved that by hand.
On REL_19_STABLE:
pg_basebackup -D /tmp/verify_targz.66809/bk -c fast -Ft -z
pg_verifybackup /tmp/verify_targz.66809/bk
# still running after 20 s, killing pg_waldump
pg_verifybackup: error: WAL parsing failed for timeline 1
pg_verifybackup -n /tmp/verify_targz.66809/bk
backup successfully verified
rewrite pg_wal.tar.gz as a single gzip member
pg_verifybackup /tmp/verify_targz.66809/bk
backup successfully verifiedWith WAL streaming, pg_basebackup ends the gzip stream before and
after every tar header, so pg_wal.tar.gz is a chain of gzip members.
astreamer_gzip_decompressor_content() loops until avail_in is zero.
After the first member inflate() keeps returning Z_STREAM_END and
takes no more input, so the loop never ends. The loop runs in
pg_waldump, and pg_verifybackup waits for it.Patches attached. 0001 calls inflateReset() when inflate() returns
Z_STREAM_END, so the next member is read. 0002 adds a --gzip case to
the tar test in 007_wal.pl and is optional.Thanks,
Shihao<v1-0001-Handle-concatenated-gzip-members-in-astreamer-gzi.patch><v1-0002-Add-test-for-pg_verifybackup-on-a-gzip-compressed.patch>
Look like a dup of /messages/by-id/5962B878-C43D-4EBC-9E95-1F945CE5E586@gmail.com.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
Hi Chao,
I took a look at the code and I think we are solving different problems.
This is not the truncated backup problem in [1]/messages/by-id/5962B878-C43D-4EBC-9E95-1F945CE5E586@gmail.com. Head with v9-0001 still
hangs.
[1]: /messages/by-id/5962B878-C43D-4EBC-9E95-1F945CE5E586@gmail.com
/messages/by-id/5962B878-C43D-4EBC-9E95-1F945CE5E586@gmail.com
Thanks,
Shihao
On 6 Oct 2026, at 03:48, shihao zhong <zhong950419@gmail.com> wrote:
I took a look at the code and I think we are solving different problems.
I concur that this is a separate issue. Today I returned to looking at the
truncation patchset and I am including this fix in that review effort.
--
Daniel Gustafsson
On Wed, Oct 7, 2026 at 7:44 AM Daniel Gustafsson <daniel@yesql.se> wrote:
On 6 Oct 2026, at 03:48, shihao zhong <zhong950419@gmail.com> wrote:
I took a look at the code and I think we are solving different problems.
I concur that this is a separate issue. Today I returned to looking at the
truncation patchset and I am including this fix in that review effort.
Should this have a 19 open item?
- Melanie
On 7 Oct 2026, at 17:44, Melanie Plageman <melanieplageman@gmail.com> wrote:
On Wed, Oct 7, 2026 at 7:44 AM Daniel Gustafsson <daniel@yesql.se> wrote:
I concur that this is a separate issue. Today I returned to looking at the
truncation patchset and I am including this fix in that review effort.Should this have a 19 open item?
That's probably a good idea.
--
Daniel Gustafsson
On Oct 7, 2026, at 19:44, Daniel Gustafsson <daniel@yesql.se> wrote:
On 6 Oct 2026, at 03:48, shihao zhong <zhong950419@gmail.com> wrote:
I took a look at the code and I think we are solving different problems.
I concur that this is a separate issue. Today I returned to looking at the
truncation patchset and I am including this fix in that review effort.--
Daniel Gustafsson
Sorry for the confusion. I was on vacation and didn’t look into the patch closely.
This seems to be an old bug that couldn’t be triggered before PG19 (b15c1513984 added tar support to pg_waldump). So, I agree with Melanie that this should be an 19 open item.
The patch calls inflateReset() after each member finishes, allowing decompression to continue with the next member, which looks like a correct fix to me.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
On 2026-10-07 We 8:48 PM, Chao Li wrote:
On Oct 7, 2026, at 19:44, Daniel Gustafsson <daniel@yesql.se> wrote:
On 6 Oct 2026, at 03:48, shihao zhong <zhong950419@gmail.com> wrote:
I took a look at the code and I think we are solving different problems.I concur that this is a separate issue. Today I returned to looking at the
truncation patchset and I am including this fix in that review effort.--
Daniel GustafssonSorry for the confusion. I was on vacation and didn’t look into the patch closely.
This seems to be an old bug that couldn’t be triggered before PG19 (b15c1513984 added tar support to pg_waldump). So, I agree with Melanie that this should be an 19 open item.
The patch calls inflateReset() after each member finishes, allowing decompression to continue with the next member, which looks like a correct fix to me.
Yes, looks good to me. I will commit later today.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On 2026-10-08 Th 10:14 AM, Andrew Dunstan wrote:
On 2026-10-07 We 8:48 PM, Chao Li wrote:
On Oct 7, 2026, at 19:44, Daniel Gustafsson <daniel@yesql.se> wrote:
On 6 Oct 2026, at 03:48, shihao zhong <zhong950419@gmail.com> wrote:
I took a look at the code and I think we are solving different
problems.I concur that this is a separate issue. Today I returned to looking
at the
truncation patchset and I am including this fix in that review effort.--
Daniel GustafssonSorry for the confusion. I was on vacation and didn’t look into the
patch closely.This seems to be an old bug that couldn’t be triggered before PG19
(b15c1513984 added tar support to pg_waldump). So, I agree with
Melanie that this should be an 19 open item.The patch calls inflateReset() after each member finishes, allowing
decompression to continue with the next member, which looks like a
correct fix to me.Yes, looks good to me. I will commit later today.
On re-reading I see Daniel is going to address it.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On 8 Oct 2026, at 19:28, Andrew Dunstan <andrew@dunslane.net> wrote:
On 2026-10-08 Th 10:14 AM, Andrew Dunstan wrote:
Yes, looks good to me. I will commit later today.
On re-reading I see Daniel is going to address it.
I had this patched staged along with a few more compression related fixes but
as I found some issues to fix in those I broke this one out and pushed
separately to ensure it gets into 19 in time for RC1. Pushed and backpatched.
--
Daniel Gustafsson
On 2026-10-08 Th 4:03 PM, Daniel Gustafsson wrote:
On 8 Oct 2026, at 19:28, Andrew Dunstan <andrew@dunslane.net> wrote:
On 2026-10-08 Th 10:14 AM, Andrew Dunstan wrote:Yes, looks good to me. I will commit later today.
On re-reading I see Daniel is going to address it.
I had this patched staged along with a few more compression related fixes but
as I found some issues to fix in those I broke this one out and pushed
separately to ensure it gets into 19 in time for RC1. Pushed and backpatched.
Thanks, Daniel!
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com