Fix pg_restore gzip stream validation (bug #19754)

On Oct 14, 2026, at 15:49, Chao Li <li.evan.chao@gmail.com> wrote:

Hi,

Bug #19754 [1] reported an issue with zlib 1.3.2 where gzread() and gzclose() may return incorrect results. The current implementation of pg_restore relies on these two functions, so pg_restore may reject valid dump files or fail to detect truncated gzip streams in some cases. Please read [1] for details if you are interested.

We already have astreamer_gzip, which directly calls zlib's low-level inflate() function to decompress data. So I initially thought about directly reusing astreamer_gzip in compress_gzip.c. However, after some experiments, I realized that its push interface does not fit pg_restore's read interface well. It also reports errors through pg_fatal(), whereas a shared helper should let callers handle errors themselves.

So, I extracted the core decompression logic into a separate gzip.c and used it in both astreamer_gzip and compress_gzip.c.

To reproduce the issue from the bug report, I created a new TAP test, 007_pg_dump_gzip.pl. When built and run with zlib 1.3.2, the test should fail without this patch:
```
# +++ tap check in src/bin/pg_dump +++
t/007_pg_dump_gzip.pl .. 1/?
# Failed test 'restore the intact gzip file whose sizes are on both buffer boundaries'
# at t/007_pg_dump_gzip.pl line 102.
# ---------- command failed ----------
# pg_restore --file /Users/chaol/Documents/code/postgresql/src/bin/pg_dump/tmp_check/tmp_test_5nHr/gzip_boundary_restore.sql /Users/chaol/Documents/code/postgresql/src/bin/pg_dump/tmp_check/tmp_test_5nHr/gzip_boundary_dump
# -------------- stderr --------------
# pg_restore: error: could not close data file "/Users/chaol/Documents/code/postgresql/src/bin/pg_dump/tmp_check/tmp_test_5nHr/gzip_boundary_dump/3875.dat": Undefined error: 0
# ------------------------------------

# Failed test 'fail to restore the truncated gzip data file'
# at t/007_pg_dump_gzip.pl line 120.
```

See the attached patch for details:
* 0001 - Only a TAP test that reproduce bug #19754
* 0002 - The fix plus more test coverage

@Stefan, would you please help verify the fix on your end?

[1] /messages/by-id/19754-b335a019b76fd89b@postgresql.org

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

<v1-0001-Add-a-TAP-test-for-pg_restore-bug-19754.patch><v1-0002-Fix-gzip-stream-validation-in-pg_restore.patch>

PFA v2 - fix the CI failures.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

On Oct 15, 2026, Chao Li <li.evan.chao@gmail.com> wrote:

@Stefan, would you please help verify the fix on your end?

PFA v2 - fix the CI failures.

Hi Chao,

Thanks a lot for the quick fix. I tested v2 on master (a999971) on Debian 13
and on Windows 11, both built from source (on Windows with MSVC 19.51 and
meson). zlib 1.3.2 was the default. For the end-to-end runs I also swapped
in 1.3.1 and the current zlib develop branch: via LD_LIBRARY_PATH on Linux,
and on Windows by putting zlib1.dll next to pg_restore.exe, as in the EDB
packages.

TAP: with only 0001 applied, 007_pg_dump_gzip fails as expected on both
systems (tests 5 and 7). With 0001 and 0002 it passes on both, and so does
the rest of the pg_dump suite on Linux.

End-to-end, restoring into a database (--section=data), a table of 20001
rows whose uncompressed data is a multiple of 128 kB, with the same results
on both systems:

- intact file, csize % 8192 == 0: unpatched master fails; with v2 all
20001 rows are restored with zlib 1.3.2, 1.3.1 and develop
- one more gzip header byte: 20001 rows with and without v2
- data file cut in half: v2 reports "compressed stream is incomplete"
with all three zlib versions; without the patch, see below

On the intact file, unpatched master fails with "could not close data file
...: Success" on Linux and "...: No error" on Windows, the message we saw in
production. With zlib 1.3.1 next to it, unpatched master restores the same
file without complaint.

Without the patch, what happens to the truncated file depends on where the
cut lands. In these runs the server rejected the incomplete last line; in an
earlier run with 18.6 the restore loaded 10073 of 20001 rows and exited 0.

I also restored a whole dump with -j 4 that contains the boundary file, an
empty table (25-byte data file), and an empty and a non-empty large object
(the empty one is a 20-byte member). With v2 everything is restored on both
systems: 20001 rows, 0 rows, and large objects of 0 and 50000 bytes.
Unpatched master fails with the same "could not close data file" error,
followed by "a worker process died unexpectedly". So the empty-member case
that caused trouble in the zstd/LZ4 change looks fine here. The stricter
parsing (zero-byte files and trailing bytes after the last member are now
errors) is fine from our side, because pg_dump always writes at least an
empty member.

I think this would be worth back-patching. With zlib 1.3.2, pg_restore
rejects intact dumps, and the Windows binaries of 14 to 18 ship that version
(in 18.6 it is linked statically into pg_dump and pg_restore).
The November 12 release is also the last one for 14. I realize that 14 and
15 do not have compress_gzip.c yet, so they would need their own version of
the fix. In 14 to 18, pg_restore reads in 4096-byte chunks; the test's 128
kB alignment is a multiple of that, so the test should catch the problem
there as well.

Regards,
Stefan Graßl

On Oct 16, 2026, at 15:27, stefangrassl@gmx.net wrote:

On Oct 15, 2026, Chao Li <li.evan.chao@gmail.com> wrote:

@Stefan, would you please help verify the fix on your end?

PFA v2 - fix the CI failures.

Hi Chao,

Thanks a lot for the quick fix. I tested v2 on master (a999971) on Debian 13
and on Windows 11, both built from source (on Windows with MSVC 19.51 and
meson). zlib 1.3.2 was the default. For the end-to-end runs I also swapped
in 1.3.1 and the current zlib develop branch: via LD_LIBRARY_PATH on Linux,
and on Windows by putting zlib1.dll next to pg_restore.exe, as in the EDB
packages.

TAP: with only 0001 applied, 007_pg_dump_gzip fails as expected on both
systems (tests 5 and 7). With 0001 and 0002 it passes on both, and so does
the rest of the pg_dump suite on Linux.

End-to-end, restoring into a database (--section=data), a table of 20001
rows whose uncompressed data is a multiple of 128 kB, with the same results
on both systems:

- intact file, csize % 8192 == 0: unpatched master fails; with v2 all
20001 rows are restored with zlib 1.3.2, 1.3.1 and develop
- one more gzip header byte: 20001 rows with and without v2
- data file cut in half: v2 reports "compressed stream is incomplete"
with all three zlib versions; without the patch, see below

On the intact file, unpatched master fails with "could not close data file
...: Success" on Linux and "...: No error" on Windows, the message we saw in
production. With zlib 1.3.1 next to it, unpatched master restores the same
file without complaint.

Without the patch, what happens to the truncated file depends on where the
cut lands. In these runs the server rejected the incomplete last line; in an
earlier run with 18.6 the restore loaded 10073 of 20001 rows and exited 0.

I also restored a whole dump with -j 4 that contains the boundary file, an
empty table (25-byte data file), and an empty and a non-empty large object
(the empty one is a 20-byte member). With v2 everything is restored on both
systems: 20001 rows, 0 rows, and large objects of 0 and 50000 bytes.
Unpatched master fails with the same "could not close data file" error,
followed by "a worker process died unexpectedly". So the empty-member case
that caused trouble in the zstd/LZ4 change looks fine here. The stricter
parsing (zero-byte files and trailing bytes after the last member are now
errors) is fine from our side, because pg_dump always writes at least an
empty member.

I think this would be worth back-patching. With zlib 1.3.2, pg_restore
rejects intact dumps, and the Windows binaries of 14 to 18 ship that version
(in 18.6 it is linked statically into pg_dump and pg_restore).
The November 12 release is also the last one for 14. I realize that 14 and
15 do not have compress_gzip.c yet, so they would need their own version of
the fix. In 14 to 18, pg_restore reads in 4096-byte chunks; the test's 128
kB alignment is a multiple of that, so the test should catch the problem
there as well.

Regards,
Stefan Graßl

Hi Stefan,

Thanks for the detailed testing on both platforms.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

On 16 Oct 2026, at 10:36, Chao Li <li.evan.chao@gmail.com> wrote:

Thanks for the detailed testing on both platforms.

Thanks both of you for the report, patch and testing. I have this on my TODO
for the next few days.

--
Daniel Gustafsson