Add malloc attribute to memory allocation functions
Given that we now have a tree that compiles fine against
-Werror=mismatched-dealloc, we need to make sure that we don't regress.
By adding the malloc attribute[0]https://gcc.gnu.org/onlinedocs/gcc/Common-Attributes.html#index-malloc, we can protect against regressions,
enable more accurate code coverage with -fanalyzer, and allow the
compiler to do some optimizations.
Let me know if I missed any allocators. Additionally, does it make sense
to add the attribute to the various memory context allocator functions
like AllocSetAlloc(), or only on the higher level functions.
[0]: https://gcc.gnu.org/onlinedocs/gcc/Common-Attributes.html#index-malloc
--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)
Hi Tristan,
I tested this patch on the current master branch.
Before applying the patch, I checked that functions like palloc(),
MemoryContextAlloc(), and pstrdup() were not annotated with a malloc
attribute.
After applying the patch, I verified that pg_attribute_malloc is added
in src/include/c.h and that the relevant backend and frontend memory
allocation functions are annotated appropriately. I also confirmed
that functions intended to be freed with pfree() use
pg_attribute_malloc(pfree), while the frontend allocation functions
use pg_attribute_malloc(pg_free).
The patch applied cleanly, PostgreSQL built successfully, and make
check completed successfully with all 245 tests passing. I didn't
notice any regressions during testing.
Thanks for working on this patch.
Regards,
Solai
"Tristan Partin" <tristan@partin.io> writes:
Given that we now have a tree that compiles fine against
-Werror=mismatched-dealloc, we need to make sure that we don't regress.
By adding the malloc attribute[0], we can protect against regressions,
enable more accurate code coverage with -fanalyzer, and allow the
compiler to do some optimizations.
I'm skeptical that this is going to lead to anything but grief.
In particular, since gcc has never heard of memory contexts,
I don't see how we are not going to get buried in bogus
-Wanalyzer-malloc-leak warnings. It doesn't really help
to add compiler annotations that only sort-of match our semantics.
(This opinion is based on years of dismissing useless Coverity
warnings of this kind.)
regards, tom lane
On Mon Jul 6, 2026 at 4:26 AM UTC, Tom Lane wrote:
"Tristan Partin" <tristan@partin.io> writes:
Given that we now have a tree that compiles fine against
-Werror=mismatched-dealloc, we need to make sure that we don't regress.
By adding the malloc attribute[0], we can protect against regressions,
enable more accurate code coverage with -fanalyzer, and allow the
compiler to do some optimizations.I'm skeptical that this is going to lead to anything but grief.
In particular, since gcc has never heard of memory contexts,
I don't see how we are not going to get buried in bogus
-Wanalyzer-malloc-leak warnings. It doesn't really help
to add compiler annotations that only sort-of match our semantics.(This opinion is based on years of dismissing useless Coverity
warnings of this kind.)
This is a fair criticism. On master, the number of
-Wanalyzer-malloc-leak warnings is 62. With this patch applied, it
balloons to 598.
--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)
On 06.07.26 18:34, Tristan Partin wrote:
On Mon Jul 6, 2026 at 4:26 AM UTC, Tom Lane wrote:
"Tristan Partin" <tristan@partin.io> writes:
Given that we now have a tree that compiles fine against
-Werror=mismatched-dealloc, we need to make sure that we don't regress.
By adding the malloc attribute[0], we can protect against regressions,
enable more accurate code coverage with -fanalyzer, and allow the
compiler to do some optimizations.I'm skeptical that this is going to lead to anything but grief.
In particular, since gcc has never heard of memory contexts,
I don't see how we are not going to get buried in bogus
-Wanalyzer-malloc-leak warnings. It doesn't really help
to add compiler annotations that only sort-of match our semantics.(This opinion is based on years of dismissing useless Coverity
warnings of this kind.)This is a fair criticism. On master, the number of
-Wanalyzer-malloc-leak warnings is 62. With this patch applied, it
balloons to 598.
But this can also check for a lot more, such as
- mismatching deallocator
- double free
- use after free
- free of things that are not an allocation
If we could tell it, check for all these things but don't worry about
the leaks, that could be useful.
Also, for frontend tools, libpq, etc. that don't use memory contexts.
Peter Eisentraut <peter@eisentraut.org> writes:
On 06.07.26 18:34, Tristan Partin wrote:
On Mon Jul 6, 2026 at 4:26 AM UTC, Tom Lane wrote:
I'm skeptical that this is going to lead to anything but grief.
In particular, since gcc has never heard of memory contexts,
I don't see how we are not going to get buried in bogus
-Wanalyzer-malloc-leak warnings. It doesn't really help
to add compiler annotations that only sort-of match our semantics.
But this can also check for a lot more, such as
- mismatching deallocator
- double free
- use after free
- free of things that are not an allocation
If we could tell it, check for all these things but don't worry about
the leaks, that could be useful.
Also, for frontend tools, libpq, etc. that don't use memory contexts.
Yeah, I was thinking about that last point. The frontend environment
is a lot closer to the semantics these markers expect, so we could
try doing this in frontend only and see how well that works.
There are still places that I'd expect to be trouble. For example,
Coverity has never understood the pattern we use in pg_dump's data
collection subroutines, ie, malloc a big array of structs, fill
the individual structs and insert pointers to them into the hash
tables, done. It always thinks we leaked the array, and I suspect
tools like this will too. In Coverity's case there's enough
infrastructure to dismiss individual false-positive complaints,
and then it won't bug you about them (until somebody changes the
relevant code enough that the dismissal doesn't match :-(). Unless
there's some similar way to silence individual reports, I don't
foresee tools like this to be usable. We're not going to change
coding patterns like that one just because some static analyzer
doesn't understand them.
regards, tom lane
On Tue Jul 7, 2026 at 10:34 AM CDT, Peter Eisentraut wrote:
On 06.07.26 18:34, Tristan Partin wrote:
On Mon Jul 6, 2026 at 4:26 AM UTC, Tom Lane wrote:
"Tristan Partin" <tristan@partin.io> writes:
Given that we now have a tree that compiles fine against
-Werror=mismatched-dealloc, we need to make sure that we don't regress.
By adding the malloc attribute[0], we can protect against regressions,
enable more accurate code coverage with -fanalyzer, and allow the
compiler to do some optimizations.I'm skeptical that this is going to lead to anything but grief.
In particular, since gcc has never heard of memory contexts,
I don't see how we are not going to get buried in bogus
-Wanalyzer-malloc-leak warnings. It doesn't really help
to add compiler annotations that only sort-of match our semantics.(This opinion is based on years of dismissing useless Coverity
warnings of this kind.)This is a fair criticism. On master, the number of
-Wanalyzer-malloc-leak warnings is 62. With this patch applied, it
balloons to 598.But this can also check for a lot more, such as
- mismatching deallocator
- double free
- use after free
- free of things that are not an allocationIf we could tell it, check for all these things but don't worry about
the leaks, that could be useful.Also, for frontend tools, libpq, etc. that don't use memory contexts.
We could add -Wno-analyzer-malloc-leak to backend code.
--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)
On Tue, Jul 07, 2026 at 11:53:36AM -0400, Tom Lane wrote:
There are still places that I'd expect to be trouble. For example,
Coverity has never understood the pattern we use in pg_dump's data
collection subroutines, ie, malloc a big array of structs, fill
the individual structs and insert pointers to them into the hash
tables, done. It always thinks we leaked the array, and I suspect
tools like this will too. In Coverity's case there's enough
infrastructure to dismiss individual false-positive complaints,
and then it won't bug you about them (until somebody changes the
relevant code enough that the dismissal doesn't match :-(). Unless
there's some similar way to silence individual reports, I don't
foresee tools like this to be usable. We're not going to change
coding patterns like that one just because some static analyzer
doesn't understand them.
Additional question. Does this help with requirements like the one
listed in fe-exec.c for PQfreemem() under WIN32? If the answer to
this question is yes, then it would sound like a win for me, we'd had
our share of issues in the past where we would use a free() call that
interacts with an allocation done in a completely different context
library-wise. That's something WIN32 cares a lot about, to mention
one place.
--
Michael
Michael Paquier <michael@paquier.xyz> writes:
Additional question. Does this help with requirements like the one
listed in fe-exec.c for PQfreemem() under WIN32?
Yeah, I was wondering about that point too. It would be really nice
to have automated checks for that. Maybe the right thing to do here
(to start anyway) is to put in targeted annotations that address
specific pain points like that one.
One issue that'd have to be dealt with is that that would involve
putting annotations into the public header file libpq-fe.h. We'd need
to be sure we do not break things for applications using compilers
other than what we built libpq with. That seems reasonably easy to do
with some macro trickery, but it's a point to keep in mind.
regards, tom lane
On Tue Jul 7, 2026 at 6:54 PM CDT, Tom Lane wrote:
Michael Paquier <michael@paquier.xyz> writes:
Additional question. Does this help with requirements like the one
listed in fe-exec.c for PQfreemem() under WIN32?Yeah, I was wondering about that point too. It would be really nice
to have automated checks for that. Maybe the right thing to do here
(to start anyway) is to put in targeted annotations that address
specific pain points like that one.One issue that'd have to be dealt with is that that would involve
putting annotations into the public header file libpq-fe.h. We'd need
to be sure we do not break things for applications using compilers
other than what we built libpq with. That seems reasonably easy to do
with some macro trickery, but it's a point to keep in mind.
Ok, I think the path forward is two-fold:
1. Add the annotations to specific pain points as Tom suggested
(PGfreemem(), frontend memory allocators)
2. Add the annotations to the backend allocators knowing that the patch
may not be committed
--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)
On Tue Jul 7, 2026 at 11:57 PM UTC, Tristan Partin wrote:
On Tue Jul 7, 2026 at 6:54 PM CDT, Tom Lane wrote:
Michael Paquier <michael@paquier.xyz> writes:
Additional question. Does this help with requirements like the one
listed in fe-exec.c for PQfreemem() under WIN32?Yeah, I was wondering about that point too. It would be really nice
to have automated checks for that. Maybe the right thing to do here
(to start anyway) is to put in targeted annotations that address
specific pain points like that one.One issue that'd have to be dealt with is that that would involve
putting annotations into the public header file libpq-fe.h. We'd need
to be sure we do not break things for applications using compilers
other than what we built libpq with. That seems reasonably easy to do
with some macro trickery, but it's a point to keep in mind.Ok, I think the path forward is two-fold:
1. Add the annotations to specific pain points as Tom suggested
(PGfreemem(), frontend memory allocators)
2. Add the annotations to the backend allocators knowing that the patch
may not be committed
After talking to Peter off-list, I've come up with a new strategy by
splitting up __attribute__((malloc)) and
__attribute__((malloc(deallocator, arg))).
For those that are unaware, these two attributes, though named the same,
have two separate meanings:
malloc:
Attribute malloc indicates that a function is malloc-like, i.e., that
the pointer P returned by the function cannot alias any other pointer
valid when the function returns, and moreover no pointers to valid
objects occur in any storage addressed by P. In addition, GCC predicts
that a function with the attribute returns non-null in most cases.
malloc(deallocator, arg):
Associating a function with a deallocator helps detect calls to
mismatched allocation and deallocation functions and diagnose them
under the control of options such as -Wmismatched-dealloc. It also
makes it possible to diagnose attempts to deallocate objects that were
not allocated dynamically, by -Wfree-nonheap-object. To indicate that
an allocation function both satisfies the nonaliasing property and has
a deallocator associated with it, both the plain form of the attribute
and the one with the deallocator argument must be used. The same
function can be both an allocator and a deallocator. Since inlining
one of the associated functions but not the other could result in
apparent mismatches, this form of attribute malloc is not accepted on
inline functions. For the same reason, using the attribute prevents
both the allocation and deallocation functions from being expanded
inline.
To summarize the second a bit, getting the extra analysis is not free.
There are performance implications with the potential lack of inlining.
I think adding the deallocator is important, but probably only in debug
builds.
Attached are patches which add pg_attribute_malloc and apply it to
various memory allocating functions.
I have the patches prepared for what I am calling
pg_attribute_dealloc(...), which is defined to be
__attribute__((malloc(deallocator, arg))). I'll hold off on inundating
the list with those until this plain version is reviewed and committed.
--
Tristan Partin
PostgreSQL Contributors Team
AWS (https://aws.amazon.com)