[PG19] Bogus errors for polymorphic plpython functions

Started by Tom Lane23 days ago9 messageshackers
Jump to latest
#1Tom Lane
tgl@sss.pgh.pa.us

PL/Python doesn't make any effort to support polymorphic arguments
or results. (It should, but that's a task for some other day.)
However, I happened to notice that while previous releases throw
a useful error:

regression=# CREATE FUNCTION python_poly(x anyelement) RETURNS anyelement AS $$
return x
$$ LANGUAGE plpython3u;
ERROR: PL/Python functions cannot return type anyelement

in v19/master you get something pretty off-point:

regression=# CREATE FUNCTION python_poly(x anyelement) RETURNS anyelement AS $$
return x
$$ LANGUAGE plpython3u;
ERROR: could not determine actual result type for function "python_poly" declared to return type anyelement

The reason for this is that commit 9bcb8a694 restructured things
to let plpython use the funccache.c infrastructure. That means
that the argument-type checks done in PLy_procedure_create are
now inside a callback function that funccache.c calls after it's
discovered it lacks a cache entry for the function. And in order
to lookup a cache entry, it has to resolve polymorphic types to
real types, and the fake fcinfo that plpython3_validator makes
doesn't contain enough data to support that.

I don't want to resolve that conundrum right now --- for one thing,
I've not made up my mind whether it's more plpython's fault or
more funccache.c's fault. What I think we should do about this
is to move the no-pseudotype-argument checks from where they are
into the validator's startup section, where they will get enforced
whether check_function_bodies is on or not. (It's really a bug that
right now you don't hear about this if !check_function_bodies.)
In that way, they'll happen before funccache.c is asked to do
anything, so we don't encounter the underlying problem. Even when
we do resolve the problem on the way to fixing plpython to support
polymorphics, this is a better code structure: it reports errors
sooner and takes some catalog lookups out of the normal function
call path.

Comments?

regards, tom lane

Attachments:

t254485_1
fix-plpython-polymorphic-check.patchtext/x-diff; charset=us-ascii; name=fix-plpython-polymorphic-check.patchDownload+73-46
#2shihao zhong
zhong950419@gmail.com
In reply to: Tom Lane (#1)
Re: [PG19] Bogus errors for polymorphic plpython functions

Hi Tom,

Looks right to me. plpython now checks the types before looking at
check_function_bodies, like plpgsql does.

One thing. The patch rejects every pseudotype for OUT and TABLE
arguments, including record. The old code skipped OUT arguments, and
plpgsql does allow record there. Is that intended?

+            if (argmodes &&
+                (argmodes[i] == PROARGMODE_OUT ||
+                 argmodes[i] == PROARGMODE_TABLE))
+                ereport(ERROR,
+                        (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+                         errmsg("PL/Python functions cannot return type
%s",

Also the new tests only run with check_function_bodies on. Maybe move
to early sections before resetting the guc?

Thanks,
Shihao

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: shihao zhong (#2)
Re: [PG19] Bogus errors for polymorphic plpython functions

shihao zhong <zhong950419@gmail.com> writes:

One thing. The patch rejects every pseudotype for OUT and TABLE
arguments, including record. The old code skipped OUT arguments, and
plpgsql does allow record there. Is that intended?

It is tighter than before. At least for polymorphic types, I don't
think the difference matters, because the core code will enforce that
you can't have a polymorphic output without a polymorphic input, so
we'd reject the function anyway. But maybe we should allow a RECORD
output; did that work before? (I'd bet not, but haven't tested)

regards, tom lane

#4shihao zhong
zhong950419@gmail.com
In reply to: Tom Lane (#3)
Re: [PG19] Bogus errors for polymorphic plpython functions

Hi Tom,

Did a quick test, I think it worked on master

SQL:

CREATE FUNCTION r3(OUT r record)
AS $$ return (1, 2) $$ LANGUAGE plpython3u;

SELECT * FROM r3() AS (a int, b int);
SELECT * FROM r3();

master:
CREATE FUNCTION
SELECT * FROM r3() AS (a int, b int);
a | b
---+---
1 | 2
(1 row)

SELECT * FROM r3();
ERROR: a column definition list is required for functions returning
"record"

with patch:
CREATE FUNCTION r3(OUT r record)
ERROR: PL/Python functions cannot return type record
SELECT * FROM r3() AS (a int, b int);
ERROR: function r3() does not exist

Thanks,
Shihao

#5Tom Lane
tgl@sss.pgh.pa.us
In reply to: shihao zhong (#4)
Re: [PG19] Bogus errors for polymorphic plpython functions

shihao zhong <zhong950419@gmail.com> writes:

Did a quick test, I think it worked on master

CREATE FUNCTION r3(OUT r record)
AS $$ return (1, 2) $$ LANGUAGE plpython3u;
SELECT * FROM r3() AS (a int, b int);
SELECT * FROM r3();

Ah, good point. I was thinking about the general case,
which doesn't really work:

regression=# CREATE FUNCTION multiout_record(INOUT n integer, OUT record) AS $$
return (n, (n * 2, n * 3))
$$ LANGUAGE plpython3u;
CREATE FUNCTION
regression=# SELECT * FROM multiout_record(2);
ERROR: record type has not been registered
CONTEXT: while creating return value
PL/Python function "multiout_record"

But with just one "OUT record" parameter, that's equivalent to
"RETURNS record", so it should work. This is showing us another
omission in plpython's regression tests :-(

After sleeping on it, I've decided that while moving these checks into
the validator is a good idea, it's not the most appropriate way to fix
this in v19. That is, I now think that this is really funccache.c's
bug not plpython's: it should be on funccache.c to not fail in this
situation. Yesterday I'd been dithering over whether external users
of funccache.c might want the existing behavior, but I can't see how
it'd be useful to them. Validators subject to this issue would have
to construct fake fcinfos that are much more complete than what we've
expected so far.

So here's a v2 patch series. 0001 fixes the immediate problem by
teaching funccache.c to not fail, and I envision back-patching that
to v19 or maybe even v18. Then 0002 rearranges plpython's validator,
but I now think that should go to master only. It does cause a user-
visible behavioral change (tighter checking with check_function_bodies
off), and we should refrain from unnecessary changes of that kind
post-rc1.

regards, tom lane

Attachments:

t254485_5
v2-0001-Fix-failure-in-function-cache-entry-hashing-durin.patchtext/x-diff; charset=us-ascii; name*0=v2-0001-Fix-failure-in-function-cache-entry-hashing-durin.p; name*1=atchDownload+32-4
v2-0002-Move-plpython-s-rejection-of-pseudotype-arguments.patchtext/x-diff; charset=us-ascii; name*0=v2-0002-Move-plpython-s-rejection-of-pseudotype-arguments.p; name*1=atchDownload+123-47
#6shihao zhong
zhong950419@gmail.com
In reply to: Tom Lane (#5)
Re: [PG19] Bogus errors for polymorphic plpython functions

Hi Tom,

Thanks for v2.

One small thing in 0002. A single OUT void parameter worked before
and is now rejected. RETURNS TABLE(v void) is the same.

This happens with check_function_bodies off too. And pg_dump auto
set that guc to off, so the backup is not able to restore...

The attached nocfbot patch goes on top of v2 and fixes it.

Thanks,
Shihao

Attachments:

nocfbot-plpython-check-only-input-args.patch.txttext/plain; charset=US-ASCII; name=nocfbot-plpython-check-only-input-args.patch.txtDownload+30-28
#7Tom Lane
tgl@sss.pgh.pa.us
In reply to: shihao zhong (#6)
Re: [PG19] Bogus errors for polymorphic plpython functions

shihao zhong <zhong950419@gmail.com> writes:

One small thing in 0002. A single OUT void parameter worked before
and is now rejected. RETURNS TABLE(v void) is the same.

I'm pretty skeptical that anyone ever wrote this, or that we should
consider it a supported thing to do, so I'm inclined to leave the
patch as-is. RETURNS VOID is supposed to mean "no interesting
return value", but the two syntaxes you sketch treat it as a regular
data type, which is not a thing we should encourage.

regards, tom lane

#8shihao zhong
zhong950419@gmail.com
In reply to: Tom Lane (#7)
Re: [PG19] Bogus errors for polymorphic plpython functions

Hi Tom,

Fair enough. I have not seen anyone write that either, and using void as a
regular data type is not something we should encourage. I'm fine with
leaving 0002 as is. Both patches look good to me

Thanks,

Shihao

#9Tom Lane
tgl@sss.pgh.pa.us
In reply to: shihao zhong (#8)
Re: [PG19] Bogus errors for polymorphic plpython functions

shihao zhong <zhong950419@gmail.com> writes:

Fair enough. I have not seen anyone write that either, and using void as a
regular data type is not something we should encourage. I'm fine with
leaving 0002 as is. Both patches look good to me

Pushed then, thanks for reviewing!

regards, tom lane