Fix resource leaks in pg_receivewal on LZ4 close failures
Hi,
I'd like to propose the attached patch to fix file descriptor and
memory leaks when LZ4 compression finalization or the final write
fails. Without this fix, repeated failures can cause these resources
to accumulate.
Thoughts?
Regards,
--
Fujii Masao
On Nov 11, 2026, at 17:48, Fujii Masao <masao.fujii@gmail.com> wrote:
Hi,
I'd like to propose the attached patch to fix file descriptor and
memory leaks when LZ4 compression finalization or the final write
fails. Without this fix, repeated failures can cause these resources
to accumulate.Thoughts?
Regards,
--
Fujii Masao
<v1-0001-pg_receivewal-Fix-resource-leaks-and-error-report.patch>
I have a question about the fix in dir_close(). In the current code, r = close(df->fd);, then if r == 0, it will do some operations based on method, such as unlink tmppath. But this patch does:
```
(void) close(df->fd);
goto cleanup;
```
Which bypasses the operations for successful close(df->fd), is that intentional?
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
On Wed, Nov 11, 2026 at 7:52 PM Chao Li <li.evan.chao@gmail.com> wrote:
Which bypasses the operations for successful close(df->fd), is that intentional?
Yes. Previously, if compression finalization or the final write failed,
the function returned -1 immediately, skipping rename, unlink, and
sync. So the patch just preserves that behavior while releasing
the resources.
Regards,
--
Fujii Masao
On Wed, Nov 11, 2026 at 06:48:51PM +0900, Fujii Masao wrote:
I'd like to propose the attached patch to fix file descriptor and
memory leaks when LZ4 compression finalization or the final write
fails. Without this fix, repeated failures can cause these resources
to accumulate.Thoughts?
That's worth a backpatch to me all the way down to me, where this has
been introduced. This stuff could be run in a loop in a service-like
fashion, bloating the fds on repeated failures.
--
Michael
On Nov 11, 2026, at 23:55, Fujii Masao <masao.fujii@gmail.com> wrote:
On Wed, Nov 11, 2026 at 7:52 PM Chao Li <li.evan.chao@gmail.com> wrote:
Which bypasses the operations for successful close(df->fd), is that intentional?
Yes. Previously, if compression finalization or the final write failed,
the function returned -1 immediately, skipping rename, unlink, and
sync. So the patch just preserves that behavior while releasing
the resources.Regards,
--
Fujii Masao
It’s true that the previous code returned -1 immediately on LZ4F_isError, causing the fd leak that this patch fixes. But I think that also cause a temporary file leak. In writeTimeLineHistoryFile(), there is a call:
```
/*
* If we fail to make the file, delete it to release disk space
*/
stream->walmethod->ops->close(f, CLOSE_UNLINK);
```
This call is intended to delete a temporary file, so I don’t think an LZ4F error should prevent that deletion. Maybe fix this leak as well?
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
On Wed, Nov 11, 2026 at 11:56 PM Fujii Masao <masao.fujii@gmail.com> wrote:
On Wed, Nov 11, 2026 at 7:52 PM Chao Li <li.evan.chao@gmail.com> wrote:
Which bypasses the operations for successful close(df->fd), is that intentional?
Yes. Previously, if compression finalization or the final write failed,
the function returned -1 immediately, skipping rename, unlink, and
sync. So the patch just preserves that behavior while releasing
the resources.
The fix LGTM. I am also a bit curious why unlink is also skipped
before though it is unrelated to the cur patch.
--
Regards,
Xuneng Zhou
HighGo Software Co., Ltd.
On Thu, Nov 12, 2026 at 9:36 AM Chao Li <li.evan.chao@gmail.com> wrote:
This call is intended to delete a temporary file, so I don’t think an LZ4F error should prevent that deletion. Maybe fix this leak as well?
Maybe yes, seems we can attempt to unlink the file on error when
CLOSE_UNLINK is specified. This should also cover close() and gzclose()
failures, not just LZ4 finalization or final write failures? If so, I think
a separate patch would be better, since this changes cleanup behavior
beyond the LZ4 resource leak fix.
Regards,
--
Fujii Masao
On Nov 19, 2026, at 15:19, Fujii Masao <masao.fujii@gmail.com> wrote:
On Thu, Nov 12, 2026 at 9:36 AM Chao Li <li.evan.chao@gmail.com> wrote:
This call is intended to delete a temporary file, so I don’t think an LZ4F error should prevent that deletion. Maybe fix this leak as well?
Maybe yes, seems we can attempt to unlink the file on error when
CLOSE_UNLINK is specified. This should also cover close() and gzclose()
failures, not just LZ4 finalization or final write failures? If so, I think
a separate patch would be better, since this changes cleanup behavior
beyond the LZ4 resource leak fix.
Sounds fair.
Would you prepare for a follow up patch? Or if you want me to do so?
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
On Thu, Nov 19, 2026 at 4:42 PM Chao Li <li.evan.chao@gmail.com> wrote:
Would you prepare for a follow up patch? Or if you want me to do so?
Please feel free to work on that :)
Regards,
--
Fujii Masao
On Nov 19, 2026, at 20:37, Fujii Masao <masao.fujii@gmail.com> wrote:
On Thu, Nov 19, 2026 at 4:42 PM Chao Li <li.evan.chao@gmail.com> wrote:
Would you prepare for a follow up patch? Or if you want me to do so?
Please feel free to work on that :)
Sure, will do.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
On Thu, Nov 19, 2026 at 9:45 PM Chao Li <li.evan.chao@gmail.com> wrote:
Sure, will do.
Thanks!
I think we can commit my proposed patch independently of the file unlink
cleanup patch, so I've pushed it first.
Regards,
--
Fujii Masao
On Nov 20, 2026, at 19:49, Fujii Masao <masao.fujii@gmail.com> wrote:
On Thu, Nov 19, 2026 at 9:45 PM Chao Li <li.evan.chao@gmail.com> wrote:
Sure, will do.
Thanks!
I think we can commit my proposed patch independently of the file unlink
cleanup patch, so I've pushed it first.Regards,
--
Fujii Masao
Yea, I’m waiting for this patch pushed. I’ll work on a follow up patch next week.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/