Persistent data across SETOF calls

Started by Ed Behn24 days ago2 messageshackers
Jump to latest
#1Ed Behn
ed@behn.us

Good day-
I maintain the PL/Haskell language handler. Typically, the handler
compiles function code and stores the compiled data in the fn_extra field
of FmgrInfo. The fn_extra field persists from one call to the next within a
query meaning the expensive compilation only needs to happen once per
query.

However, when calling a set-returning function multiple times in a
query, there appears to be no way to carry data from one set to another.
This is because the fn_extra field stores the FuncCallContext structure.
While the user_fctx field of the FuncCallContext structure persists from
one call to the next, it is NULLed out at the end of a set.

For example, if there is a table, t, with a column, i, and a
set-returning function, func, the following query is very inefficient:

SELECT *
FROM t, func(i)

This is because the function is recompiled for each row of t.

Is there a way around this? Ideally, there would be a field that
persists from one set to another within a query.

Thank you for any help you can provide.
-Ed

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Ed Behn (#1)
Re: Persistent data across SETOF calls

Ed Behn <ed@behn.us> writes:

I maintain the PL/Haskell language handler. Typically, the handler
compiles function code and stores the compiled data in the fn_extra field
of FmgrInfo. The fn_extra field persists from one call to the next within a
query meaning the expensive compilation only needs to happen once per
query.

However, when calling a set-returning function multiple times in a
query, there appears to be no way to carry data from one set to another.
This is because the fn_extra field stores the FuncCallContext structure.
While the user_fctx field of the FuncCallContext structure persists from
one call to the next, it is NULLed out at the end of a set.

For example, if there is a table, t, with a column, i, and a
set-returning function, func, the following query is very inefficient:

SELECT *
FROM t, func(i)

This is because the function is recompiled for each row of t.

Hmm, I don't believe any of the built-in PLs have that issue,
but they mostly don't use the funcapi.h SRF infrastructure.
Maybe you should look at the way that the SQL-language function
executor (executor/functions.c) deals with SRFs. I wouldn't have
recommended it as a model before v18, but the new implementation
is reasonably good I think, and it does manage to cache compiled
functions for the life of the session.

regards, tom lane