PL/Python: Add cursor and execute methods to plan object

Started by Peter Eisentrautalmost 9 years ago7 messages
#1Peter Eisentraut
peter.eisentraut@2ndquadrant.com
1 attachment(s)

Something that has been bothering me in PL/Python for a long time is the
non-object-oriented way in which plans are prepared and executed:

plan = plpy.prepare(...)
res = plpy.execute(plan, ...)

where plpy.execute() takes either a plan or a query string.

I think a better style would be

plan = plpy.prepare(...)
res = plan.execute(...)

so that the "plan" is more like a statement handle that one finds in
other APIs.

This ended up being very easy to implement, so I'm proposing to allow
this new syntax as an alternative.

I came across this again as I was developing the background sessions API
for PL/Python. So I'm also wondering here which style people prefer so
I can implement it there.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachments:

0001-PL-Python-Add-cursor-and-execute-methods-to-plan-obj.patchtext/x-patch; name=0001-PL-Python-Add-cursor-and-execute-methods-to-plan-obj.patchDownload
From 9dccf70110d9d5818318c651c2662f2b8f86b2bc Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter_e@gmx.net>
Date: Sat, 25 Feb 2017 08:42:25 -0500
Subject: [PATCH] PL/Python: Add cursor and execute methods to plan object

Instead of

    plan = plpy.prepare(...)
    res = plpy.execute(plan, ...)

you can now write

    plan = plpy.prepare(...)
    res = plan.execute(...)

or even

    res = plpy.prepare(...).execute(...)

and similarly for the cursor() method.

This is more in object oriented style, and makes the hybrid nature of
the existing execute() function less confusing.
---
 doc/src/sgml/plpython.sgml                | 14 ++++++++++++--
 src/pl/plpython/expected/plpython_spi.out | 19 ++++++++++++++++---
 src/pl/plpython/plpy_cursorobject.c       |  3 +--
 src/pl/plpython/plpy_cursorobject.h       |  1 +
 src/pl/plpython/plpy_planobject.c         | 31 +++++++++++++++++++++++++++++++
 src/pl/plpython/plpy_spi.c                |  3 +--
 src/pl/plpython/plpy_spi.h                |  1 +
 src/pl/plpython/sql/plpython_spi.sql      | 18 ++++++++++++++++--
 8 files changed, 79 insertions(+), 11 deletions(-)

diff --git a/doc/src/sgml/plpython.sgml b/doc/src/sgml/plpython.sgml
index 46397781be..6888ce1ae3 100644
--- a/doc/src/sgml/plpython.sgml
+++ b/doc/src/sgml/plpython.sgml
@@ -1048,6 +1048,14 @@ <title>Database Access Functions</title>
      </para>
 
      <para>
+      Alternatively, you can call the <function>execute</function> method on
+      the plan object:
+<programlisting>
+rv = plan.execute(["name"], 5)
+</programlisting>
+     </para>
+
+     <para>
       Query parameters and result row fields are converted between PostgreSQL
       and Python data types as described in <xref linkend="plpython-data">.
      </para>
@@ -1082,7 +1090,9 @@ <title>Database Access Functions</title>
       as <literal>plpy.execute</literal> (except for the row limit) and returns
       a cursor object, which allows you to process large result sets in smaller
       chunks.  As with <literal>plpy.execute</literal>, either a query string
-      or a plan object along with a list of arguments can be used.
+      or a plan object along with a list of arguments can be used, or
+      the <function>cursor</function> function can be called as a method of
+      the plan object.
      </para>
 
      <para>
@@ -1126,7 +1136,7 @@ <title>Database Access Functions</title>
 CREATE FUNCTION count_odd_prepared() RETURNS integer AS $$
 odd = 0
 plan = plpy.prepare("select num from largetable where num % $1 &lt;&gt; 0", ["integer"])
-rows = list(plpy.cursor(plan, [2]))
+rows = list(plpy.cursor(plan, [2]))  # or: = list(plan.cursor([2]))
 
 return len(rows)
 $$ LANGUAGE plpythonu;
diff --git a/src/pl/plpython/expected/plpython_spi.out b/src/pl/plpython/expected/plpython_spi.out
index 0d78ca1de4..e54dca9e2e 100644
--- a/src/pl/plpython/expected/plpython_spi.out
+++ b/src/pl/plpython/expected/plpython_spi.out
@@ -31,6 +31,19 @@ except Exception, ex:
 return None
 '
 	LANGUAGE plpythonu;
+CREATE FUNCTION spi_prepared_plan_test_two(a text) RETURNS text
+	AS
+'if "myplan" not in SD:
+	q = "SELECT count(*) FROM users WHERE lname = $1"
+	SD["myplan"] = plpy.prepare(q, [ "text" ])
+try:
+	rv = SD["myplan"].execute([a])
+	return "there are " + str(rv[0]["count"]) + " " + str(a) + "s"
+except Exception, ex:
+	plpy.error(str(ex))
+return None
+'
+	LANGUAGE plpythonu;
 CREATE FUNCTION spi_prepared_plan_test_nested(a text) RETURNS text
 	AS
 'if "myplan" not in SD:
@@ -80,8 +93,8 @@ select spi_prepared_plan_test_one('doe');
  there are 3 does
 (1 row)
 
-select spi_prepared_plan_test_one('smith');
- spi_prepared_plan_test_one 
+select spi_prepared_plan_test_two('smith');
+ spi_prepared_plan_test_two 
 ----------------------------
  there are 1 smiths
 (1 row)
@@ -372,7 +385,7 @@ plan = plpy.prepare(
     ["text"])
 for row in plpy.cursor(plan, ["w"]):
     yield row['fname']
-for row in plpy.cursor(plan, ["j"]):
+for row in plan.cursor(["j"]):
     yield row['fname']
 $$ LANGUAGE plpythonu;
 CREATE FUNCTION cursor_plan_wrong_args() RETURNS SETOF text AS $$
diff --git a/src/pl/plpython/plpy_cursorobject.c b/src/pl/plpython/plpy_cursorobject.c
index 7bb8992148..18e689f141 100644
--- a/src/pl/plpython/plpy_cursorobject.c
+++ b/src/pl/plpython/plpy_cursorobject.c
@@ -25,7 +25,6 @@
 
 
 static PyObject *PLy_cursor_query(const char *query);
-static PyObject *PLy_cursor_plan(PyObject *ob, PyObject *args);
 static void PLy_cursor_dealloc(PyObject *arg);
 static PyObject *PLy_cursor_iternext(PyObject *self);
 static PyObject *PLy_cursor_fetch(PyObject *self, PyObject *args);
@@ -160,7 +159,7 @@ PLy_cursor_query(const char *query)
 	return (PyObject *) cursor;
 }
 
-static PyObject *
+PyObject *
 PLy_cursor_plan(PyObject *ob, PyObject *args)
 {
 	PLyCursorObject *cursor;
diff --git a/src/pl/plpython/plpy_cursorobject.h b/src/pl/plpython/plpy_cursorobject.h
index c73033c486..ef23865dd2 100644
--- a/src/pl/plpython/plpy_cursorobject.h
+++ b/src/pl/plpython/plpy_cursorobject.h
@@ -19,5 +19,6 @@ typedef struct PLyCursorObject
 
 extern void PLy_cursor_init_type(void);
 extern PyObject *PLy_cursor(PyObject *self, PyObject *args);
+extern PyObject *PLy_cursor_plan(PyObject *ob, PyObject *args);
 
 #endif   /* PLPY_CURSOROBJECT_H */
diff --git a/src/pl/plpython/plpy_planobject.c b/src/pl/plpython/plpy_planobject.c
index 16c39a05dd..390b4e90d4 100644
--- a/src/pl/plpython/plpy_planobject.c
+++ b/src/pl/plpython/plpy_planobject.c
@@ -10,11 +10,15 @@
 
 #include "plpy_planobject.h"
 
+#include "plpy_cursorobject.h"
 #include "plpy_elog.h"
+#include "plpy_spi.h"
 #include "utils/memutils.h"
 
 
 static void PLy_plan_dealloc(PyObject *arg);
+static PyObject *PLy_plan_cursor(PyObject *self, PyObject *args);
+static PyObject *PLy_plan_execute(PyObject *self, PyObject *args);
 static PyObject *PLy_plan_status(PyObject *self, PyObject *args);
 
 static char PLy_plan_doc[] = {
@@ -22,6 +26,8 @@ static char PLy_plan_doc[] = {
 };
 
 static PyMethodDef PLy_plan_methods[] = {
+	{"cursor", PLy_plan_cursor, METH_VARARGS, NULL},
+	{"execute", PLy_plan_execute, METH_VARARGS, NULL},
 	{"status", PLy_plan_status, METH_VARARGS, NULL},
 	{NULL, NULL, 0, NULL}
 };
@@ -112,6 +118,31 @@ PLy_plan_dealloc(PyObject *arg)
 
 
 static PyObject *
+PLy_plan_cursor(PyObject *self, PyObject *args)
+{
+	PyObject   *planargs = NULL;
+
+	if (!PyArg_ParseTuple(args, "|O", &planargs))
+		return NULL;
+
+	return PLy_cursor_plan(self, planargs);
+}
+
+
+static PyObject *
+PLy_plan_execute(PyObject *self, PyObject *args)
+{
+	PyObject   *list = NULL;
+	long		limit = 0;
+
+	if (!PyArg_ParseTuple(args, "|Ol", &list, &limit))
+		return NULL;
+
+	return PLy_spi_execute_plan(self, list, limit);
+}
+
+
+static PyObject *
 PLy_plan_status(PyObject *self, PyObject *args)
 {
 	if (PyArg_ParseTuple(args, ":status"))
diff --git a/src/pl/plpython/plpy_spi.c b/src/pl/plpython/plpy_spi.c
index 07ab6a087e..c6856ccbac 100644
--- a/src/pl/plpython/plpy_spi.c
+++ b/src/pl/plpython/plpy_spi.c
@@ -30,7 +30,6 @@
 
 
 static PyObject *PLy_spi_execute_query(char *query, long limit);
-static PyObject *PLy_spi_execute_plan(PyObject *ob, PyObject *list, long limit);
 static PyObject *PLy_spi_execute_fetch_result(SPITupleTable *tuptable,
 							 uint64 rows, int status);
 static void PLy_spi_exception_set(PyObject *excclass, ErrorData *edata);
@@ -193,7 +192,7 @@ PLy_spi_execute(PyObject *self, PyObject *args)
 	return NULL;
 }
 
-static PyObject *
+PyObject *
 PLy_spi_execute_plan(PyObject *ob, PyObject *list, long limit)
 {
 	volatile int nargs;
diff --git a/src/pl/plpython/plpy_spi.h b/src/pl/plpython/plpy_spi.h
index b0427947ef..817a7584e7 100644
--- a/src/pl/plpython/plpy_spi.h
+++ b/src/pl/plpython/plpy_spi.h
@@ -10,6 +10,7 @@
 
 extern PyObject *PLy_spi_prepare(PyObject *self, PyObject *args);
 extern PyObject *PLy_spi_execute(PyObject *self, PyObject *args);
+extern PyObject *PLy_spi_execute_plan(PyObject *ob, PyObject *list, long limit);
 
 typedef struct PLyExceptionEntry
 {
diff --git a/src/pl/plpython/sql/plpython_spi.sql b/src/pl/plpython/sql/plpython_spi.sql
index 7427de824b..fcf049cb66 100644
--- a/src/pl/plpython/sql/plpython_spi.sql
+++ b/src/pl/plpython/sql/plpython_spi.sql
@@ -37,6 +37,20 @@ CREATE FUNCTION spi_prepared_plan_test_one(a text) RETURNS text
 '
 	LANGUAGE plpythonu;
 
+CREATE FUNCTION spi_prepared_plan_test_two(a text) RETURNS text
+	AS
+'if "myplan" not in SD:
+	q = "SELECT count(*) FROM users WHERE lname = $1"
+	SD["myplan"] = plpy.prepare(q, [ "text" ])
+try:
+	rv = SD["myplan"].execute([a])
+	return "there are " + str(rv[0]["count"]) + " " + str(a) + "s"
+except Exception, ex:
+	plpy.error(str(ex))
+return None
+'
+	LANGUAGE plpythonu;
+
 CREATE FUNCTION spi_prepared_plan_test_nested(a text) RETURNS text
 	AS
 'if "myplan" not in SD:
@@ -79,7 +93,7 @@ CREATE FUNCTION spi_recursive_sum(a int) RETURNS int
 --
 select nested_call_one('pass this along');
 select spi_prepared_plan_test_one('doe');
-select spi_prepared_plan_test_one('smith');
+select spi_prepared_plan_test_two('smith');
 select spi_prepared_plan_test_nested('smith');
 
 SELECT join_sequences(sequences) FROM sequences;
@@ -275,7 +289,7 @@ CREATE FUNCTION cursor_plan() RETURNS SETOF text AS $$
     ["text"])
 for row in plpy.cursor(plan, ["w"]):
     yield row['fname']
-for row in plpy.cursor(plan, ["j"]):
+for row in plan.cursor(["j"]):
     yield row['fname']
 $$ LANGUAGE plpythonu;
 
-- 
2.11.1

#2David Steele
david@pgmasters.net
In reply to: Peter Eisentraut (#1)
Re: PL/Python: Add cursor and execute methods to plan object

On 2/25/17 1:27 PM, Peter Eisentraut wrote:

Something that has been bothering me in PL/Python for a long time is the
non-object-oriented way in which plans are prepared and executed:

plan = plpy.prepare(...)
res = plpy.execute(plan, ...)

where plpy.execute() takes either a plan or a query string.

I think a better style would be

plan = plpy.prepare(...)
res = plan.execute(...)

so that the "plan" is more like a statement handle that one finds in
other APIs.

This ended up being very easy to implement, so I'm proposing to allow
this new syntax as an alternative.

I came across this again as I was developing the background sessions API
for PL/Python. So I'm also wondering here which style people prefer so
I can implement it there.

This patch applies cleanly at cccbdde.

Any Python folks out there who would like to take a crack at reviewing this?

--
-David
david@pgmasters.net

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Andrew Dunstan
andrew.dunstan@2ndquadrant.com
In reply to: David Steele (#2)
Re: PL/Python: Add cursor and execute methods to plan object

On 03/16/2017 05:32 PM, David Steele wrote:

On 2/25/17 1:27 PM, Peter Eisentraut wrote:

Something that has been bothering me in PL/Python for a long time is the
non-object-oriented way in which plans are prepared and executed:

plan = plpy.prepare(...)
res = plpy.execute(plan, ...)

where plpy.execute() takes either a plan or a query string.

I think a better style would be

plan = plpy.prepare(...)
res = plan.execute(...)

so that the "plan" is more like a statement handle that one finds in
other APIs.

This ended up being very easy to implement, so I'm proposing to allow
this new syntax as an alternative.

I came across this again as I was developing the background sessions API
for PL/Python. So I'm also wondering here which style people prefer so
I can implement it there.

This patch applies cleanly at cccbdde.

Any Python folks out there who would like to take a crack at reviewing this?

I'm not particularly a Python folk, but I've done enough over the years
with PLs, including PLPython, that I think I can review this :-)

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Andrew Dunstan
andrew.dunstan@2ndquadrant.com
In reply to: Andrew Dunstan (#3)
Re: PL/Python: Add cursor and execute methods to plan object

On 03/21/2017 06:27 PM, Andrew Dunstan wrote:

On 03/16/2017 05:32 PM, David Steele wrote:

On 2/25/17 1:27 PM, Peter Eisentraut wrote:

Something that has been bothering me in PL/Python for a long time is the
non-object-oriented way in which plans are prepared and executed:

plan = plpy.prepare(...)
res = plpy.execute(plan, ...)

where plpy.execute() takes either a plan or a query string.

I think a better style would be

plan = plpy.prepare(...)
res = plan.execute(...)

so that the "plan" is more like a statement handle that one finds in
other APIs.

This ended up being very easy to implement, so I'm proposing to allow
this new syntax as an alternative.

I came across this again as I was developing the background sessions API
for PL/Python. So I'm also wondering here which style people prefer so
I can implement it there.

This patch applies cleanly at cccbdde.

Any Python folks out there who would like to take a crack at reviewing this?

I'm not particularly a Python folk, but I've done enough over the years
with PLs, including PLPython, that I think I can review this :-)

This is a very simple patch that does what it advertises. It applies
cleanly and provides tests for both the new methods (plan.cursor and
plan.execute).

Marking Ready For Committer.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Jim Nasby
jim@nasby.net
In reply to: Peter Eisentraut (#1)
Re: PL/Python: Add cursor and execute methods to plan object

On 2/25/17 10:27 AM, Peter Eisentraut wrote:

So I'm also wondering here which style people prefer so
I can implement it there.

I think the more OO style is definitely better. I expect it would
simplify the code as well.
--
Jim C. Nasby, Data Architect jim@nasby.net
512.569.9461 (cell) http://jim.nasby.net

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#6Noname
ilmari@ilmari.org
In reply to: Jim Nasby (#5)
Re: PL/Python: Add cursor and execute methods to plan object

Jim Nasby <jim@nasby.net> writes:

On 2/25/17 10:27 AM, Peter Eisentraut wrote:

So I'm also wondering here which style people prefer so
I can implement it there.

I think the more OO style is definitely better. I expect it would
simplify the code as well.

I'm not a Python person, but I'd argue that the "more OO" style should
be the primary style documented, and the old style should just be
mentioned for reference.

- ilmari

--
- Twitter seems more influential [than blogs] in the 'gets reported in
the mainstream press' sense at least. - Matt McLeod
- That'd be because the content of a tweet is easier to condense down
to a mainstream media article. - Calle Dybedahl

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Peter Eisentraut
peter.eisentraut@2ndquadrant.com
In reply to: Andrew Dunstan (#4)
Re: PL/Python: Add cursor and execute methods to plan object

On 3/22/17 11:46, Andrew Dunstan wrote:

This is a very simple patch that does what it advertises. It applies
cleanly and provides tests for both the new methods (plan.cursor and
plan.execute).

Marking Ready For Committer.

committed

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers