PL/Python - lifetime of variables?
Hello,
I noticed that variables in PL/Python are not released at the end of procedure.
Does it expected behavior?
See this example below:
https://github.com/heterodb/pg-strom/blob/master/test/input/arrow_python.source#L53
This PL/Python function maps a GPU buffer as cupy.ndarray object by
cupy_strom.ipc_import()
at the L59. It shall be stored in X. I have expected that X shall be
released at end of the
procedure invocation, but not happen.
The object X internally hold IpcMemory class,
https://github.com/heterodb/pg-strom/blob/master/python/cupy_strom.pyx
And, it has destructor routine that unmaps the above GPU buffer using CUDA API.
https://github.com/heterodb/pg-strom/blob/master/python/cupy_ipcmem.c#L242
Because of the restriction by CUDA API, we cannot map a certain GPU buffer twice
on the same process space. So, I noticed that the second invocation of
the PL/Python
procedure on the same session failed.
The L103 explicitly reset X, by X=0, to invoke the destructor manually.
I wonder whether it is an expected behavior, or oversight of something.
Best regards,
--
HeteroDB, Inc / The PG-Strom Project
KaiGai Kohei <kaigai@heterodb.com>
On Wed, 19 Feb 2020 at 09:12, Kohei KaiGai <kaigai@heterodb.com> wrote:
I noticed that variables in PL/Python are not released at the end of procedure.
Does it expected behavior?
PL/Python vars are freed when the interpreter instance is freed and
their refcounts reach zero.
I believe we use one subinterpreter for the lifetime of the backend
session. It might be worth checking whether we do an eager refcount
check and sweep when a procedure finishes.
But in general, I suggest that relying on object
finalizers/destructors to accomplish side effects visible outside the
procedure is bad development practice. Instead, use a "with" block, or
a try/finally block, and do explicit cleanup for external resources.
--
Craig Ringer http://www.2ndQuadrant.com/
2ndQuadrant - PostgreSQL Solutions for the Enterprise