Why does plpython delay composite type resolution?

On 12/21/2016 04:14 AM, Jim Nasby wrote:

Why do functions that accept composite types delay type resolution until
execution? I have a naive patch that speeds up plpy.execute() by 8% by
caching interred python strings for the dictionary key names (which are
repeated over and over). The next step is to just pre-allocate those
strings as appropriate for the calling context, but it's not clear how
to handle that for input arguments.

Does your patch handle "ALTER TYPE name ADD ATTRIBUTE ..."? My immediate
guess would be that it could be a cache invalidation thing.

Andreas

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/21/16 1:55 AM, Andreas Karlsson wrote:

On 12/21/2016 04:14 AM, Jim Nasby wrote:

Why do functions that accept composite types delay type resolution until
execution? I have a naive patch that speeds up plpy.execute() by 8% by
caching interred python strings for the dictionary key names (which are
repeated over and over). The next step is to just pre-allocate those
strings as appropriate for the calling context, but it's not clear how
to handle that for input arguments.

Does your patch handle "ALTER TYPE name ADD ATTRIBUTE ..."? My immediate
guess would be that it could be a cache invalidation thing.

Won't that only happen at end of transaction?

After reading the tuple queue code I'm wondering if part of the issue is
anonymous records, though that doesn't make much sense since plpython
doesn't support those.

Given the lackluster support for arrays and composites in plpython, I
suspect this is just a wart that hasn't been removed yet...
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Jim Nasby <Jim.Nasby@BlueTreble.com> writes:

On 12/21/16 1:55 AM, Andreas Karlsson wrote:

Does your patch handle "ALTER TYPE name ADD ATTRIBUTE ..."? My immediate
guess would be that it could be a cache invalidation thing.

Won't that only happen at end of transaction?

No.

BEGIN;
SELECT plpython_function();
ALTER TYPE ...;
SELECT plpython_function();
COMMIT;

For that matter, the plpython function could execute the ALTER itself
through SPI, or call another function that does so.

(I'm not claiming that the existing code, either in plpython or other
PLs, necessarily handles such all scenarios nicely. But we shouldn't
make it worse.)

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 12/21/16 8:39 AM, Tom Lane wrote:

Jim Nasby <Jim.Nasby@BlueTreble.com> writes:

On 12/21/16 1:55 AM, Andreas Karlsson wrote:

Does your patch handle "ALTER TYPE name ADD ATTRIBUTE ..."? My immediate
guess would be that it could be a cache invalidation thing.

Won't that only happen at end of transaction?

No.

BEGIN;
SELECT plpython_function();
ALTER TYPE ...;
SELECT plpython_function();
COMMIT;

For that matter, the plpython function could execute the ALTER itself
through SPI, or call another function that does so.

(I'm not claiming that the existing code, either in plpython or other
PLs, necessarily handles such all scenarios nicely. But we shouldn't
make it worse.)

Hmm... so I guess the only way we could safely handle this is if any
caching of type info happened via fcinfo->flinfo->fn_extra? Would it
also work if we verified pg_type.(tid,xmin) hadn't changed? (That's what
plpython currently does to verify a row in pg_procedure hasn't changed.)
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers