BuildTupleFromCStrings Memory Documentation?
Within the core codebase, BuildTupleFromCStrings is often called within a temporary memory context cleared after the call. In dblink.c, this is justified as being needed to “[clean up] not only the data we have direct access to, but any cruft the I/O functions might leak”.
I wrote a pretty minimal case to call BuildTupleFromCStrings in a loop (attached) and found that I was using 40GB of RAM in a few minutes, though I was not allocating any memory myself and immediately freed the tuple it returned.
Is the need to wrap this call in a protective context documented anywhere? Portions of the documentation use BuildTupleFromCStrings in examples without mentioning this precaution. Is it just well-known, or did I miss a README or comment somewhere?
--
Jason Petersen
Software Engineer | Citus Data
303.736.9255
jason@citusdata.com
Attachments:
Jason Petersen <jason@citusdata.com> writes:
Within the core codebase, BuildTupleFromCStrings is often called within a temporary memory context cleared after the call. In dblink.c, this is justified as being needed to “[clean up] not only the data we have direct access to, but any cruft the I/O functions might leak”.
I wrote a pretty minimal case to call BuildTupleFromCStrings in a loop (attached) and found that I was using 40GB of RAM in a few minutes, though I was not allocating any memory myself and immediately freed the tuple it returned.
Is the need to wrap this call in a protective context documented anywhere? Portions of the documentation use BuildTupleFromCStrings in examples without mentioning this precaution. Is it just well-known, or did I miss a README or comment somewhere?
Most uses of BuildTupleFromCStrings only do it once per invocation of the
calling function, so that an outer-level reset of the calling function's
evaluation context is what takes care of any memory leaked by the I/O
functions BuildTupleFromCStrings invokes. If you intend to call it many
times within the same function call then you should use a context you can
reset between calls. This risk is hardly unique to BuildTupleFromCStrings,
which is why the documentation doesn't make a big point of it.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers