pl/python refactoring

Started by Jan Urbańskiabout 15 years ago24 messages
#1Jan Urbański
wulczer@wulczer.org
1 attachment(s)

Here's the base refactoring patch mentioned in
http://archives.postgresql.org/pgsql-hackers/2010-12/msg01991.php

Git branch for this patch: https://github.com/wulczer/postgres/tree/refactor

It does some architectural changes to PL/Python that make it easier to
implement other features, like for instance a validator function. The
full list of changes in the patch is:

* Use HTABs keyed with OIDs for procedure and trigger caches. Using
Python dictionaries for these caches seemes like using the wrong tool,
and then you get better error reporting and memory management with PG HTABs.

* Get rid of global error state. It was wrong, 'nuff said.

* Fix an error when the iterator was left half-open after an error.

* Factor out PLy_procedure_{input,output}_conversion functions.

* Use palloc in TopMemoryContext instead of malloc. As discussed in
http://archives.postgresql.org/pgsql-hackers/2010-11/msg01857.php

* Use PyObject_New instead of PyObject_NEW (which is undocumented).

* Reset exception state after catching an error from SPI calls. Doing
SPI calls in subtransactions will remove the need to do that altogether,
so this little wart will go away soon.

* Correctly put the exceptions in the plpy module. Previous coding
failed for Python 3 (the exceptions were not available on Python 3 from
the plpy module, see the new test in plpython_test.sql and also
http://archives.postgresql.org/message-id/4D0CFE5C.5060704@wulczer.org).

* Define all fields of the PyModuleDef structure for Python 3 (see
http://docs.python.org/py3k/c-api/module.html#PyModuleDef). Not sure why
this worked at all, I guess by some lucky coincidence the undefined
fields were falling in an accessible memory region filled with zeroes...

* Raise normal exceptions from plpy.error() instead of relying on the
global error state.

* Do not prefix error messages with "PL/Python: ". It's redundant given
the error context.

All other PL/Python patches rely on these refactorings.

Cheers,
Jan

Attachments:

plpython-refactor.difftext/x-patch; name=plpython-refactor.diffDownload
diff --git a/src/pl/plpython/expected/plpython_do.out b/src/pl/plpython/expected/plpython_do.out
index 9de261a..a21b088 100644
*** a/src/pl/plpython/expected/plpython_do.out
--- b/src/pl/plpython/expected/plpython_do.out
*************** DO $$ plpy.notice("This is plpythonu.")
*** 2,6 ****
  NOTICE:  This is plpythonu.
  CONTEXT:  PL/Python anonymous code block
  DO $$ nonsense $$ LANGUAGE plpythonu;
! ERROR:  PL/Python: NameError: global name 'nonsense' is not defined
  CONTEXT:  PL/Python anonymous code block
--- 2,6 ----
  NOTICE:  This is plpythonu.
  CONTEXT:  PL/Python anonymous code block
  DO $$ nonsense $$ LANGUAGE plpythonu;
! ERROR:  NameError: global name 'nonsense' is not defined
  CONTEXT:  PL/Python anonymous code block
diff --git a/src/pl/plpython/expected/plpython_error.out b/src/pl/plpython/expected/plpython_error.out
index 1f24c13..70890a8 100644
*** a/src/pl/plpython/expected/plpython_error.out
--- b/src/pl/plpython/expected/plpython_error.out
*************** CREATE FUNCTION sql_syntax_error() RETUR
*** 8,19 ****
  'plpy.execute("syntax error")'
          LANGUAGE plpythonu;
  SELECT sql_syntax_error();
! WARNING:  PL/Python: plpy.SPIError: unrecognized error in PLy_spi_execute_query
  CONTEXT:  PL/Python function "sql_syntax_error"
! ERROR:  syntax error at or near "syntax"
! LINE 1: syntax error
!         ^
! QUERY:  syntax error
  CONTEXT:  PL/Python function "sql_syntax_error"
  /* check the handling of uncaught python exceptions
   */
--- 8,16 ----
  'plpy.execute("syntax error")'
          LANGUAGE plpythonu;
  SELECT sql_syntax_error();
! WARNING:  plpy.SPIError: unrecognized error in PLy_spi_execute_query
  CONTEXT:  PL/Python function "sql_syntax_error"
! ERROR:  plpy.SPIError: syntax error at or near "syntax"
  CONTEXT:  PL/Python function "sql_syntax_error"
  /* check the handling of uncaught python exceptions
   */
*************** CREATE FUNCTION exception_index_invalid(
*** 22,28 ****
  'return args[1]'
  	LANGUAGE plpythonu;
  SELECT exception_index_invalid('test');
! ERROR:  PL/Python: IndexError: list index out of range
  CONTEXT:  PL/Python function "exception_index_invalid"
  /* check handling of nested exceptions
   */
--- 19,25 ----
  'return args[1]'
  	LANGUAGE plpythonu;
  SELECT exception_index_invalid('test');
! ERROR:  IndexError: list index out of range
  CONTEXT:  PL/Python function "exception_index_invalid"
  /* check handling of nested exceptions
   */
*************** CREATE FUNCTION exception_index_invalid_
*** 32,44 ****
  return rv[0]'
  	LANGUAGE plpythonu;
  SELECT exception_index_invalid_nested();
! WARNING:  PL/Python: plpy.SPIError: unrecognized error in PLy_spi_execute_query
  CONTEXT:  PL/Python function "exception_index_invalid_nested"
! ERROR:  function test5(unknown) does not exist
! LINE 1: SELECT test5('foo')
!                ^
! HINT:  No function matches the given name and argument types. You might need to add explicit type casts.
! QUERY:  SELECT test5('foo')
  CONTEXT:  PL/Python function "exception_index_invalid_nested"
  /* a typo
   */
--- 29,37 ----
  return rv[0]'
  	LANGUAGE plpythonu;
  SELECT exception_index_invalid_nested();
! WARNING:  plpy.SPIError: unrecognized error in PLy_spi_execute_query
  CONTEXT:  PL/Python function "exception_index_invalid_nested"
! ERROR:  plpy.SPIError: function test5(unknown) does not exist
  CONTEXT:  PL/Python function "exception_index_invalid_nested"
  /* a typo
   */
*************** return None
*** 54,62 ****
  '
  	LANGUAGE plpythonu;
  SELECT invalid_type_uncaught('rick');
! WARNING:  PL/Python: plpy.SPIError: unrecognized error in PLy_spi_prepare
  CONTEXT:  PL/Python function "invalid_type_uncaught"
! ERROR:  type "test" does not exist
  CONTEXT:  PL/Python function "invalid_type_uncaught"
  /* for what it's worth catch the exception generated by
   * the typo, and return None
--- 47,55 ----
  '
  	LANGUAGE plpythonu;
  SELECT invalid_type_uncaught('rick');
! WARNING:  plpy.SPIError: unrecognized error in PLy_spi_prepare
  CONTEXT:  PL/Python function "invalid_type_uncaught"
! ERROR:  plpy.SPIError: type "test" does not exist
  CONTEXT:  PL/Python function "invalid_type_uncaught"
  /* for what it's worth catch the exception generated by
   * the typo, and return None
*************** return None
*** 77,86 ****
  '
  	LANGUAGE plpythonu;
  SELECT invalid_type_caught('rick');
! WARNING:  PL/Python: plpy.SPIError: unrecognized error in PLy_spi_prepare
  CONTEXT:  PL/Python function "invalid_type_caught"
! ERROR:  type "test" does not exist
  CONTEXT:  PL/Python function "invalid_type_caught"
  /* for what it's worth catch the exception generated by
   * the typo, and reraise it as a plain error
   */
--- 70,84 ----
  '
  	LANGUAGE plpythonu;
  SELECT invalid_type_caught('rick');
! WARNING:  plpy.SPIError: unrecognized error in PLy_spi_prepare
  CONTEXT:  PL/Python function "invalid_type_caught"
! NOTICE:  type "test" does not exist
  CONTEXT:  PL/Python function "invalid_type_caught"
+  invalid_type_caught 
+ ---------------------
+  
+ (1 row)
+ 
  /* for what it's worth catch the exception generated by
   * the typo, and reraise it as a plain error
   */
*************** return None
*** 99,107 ****
  '
  	LANGUAGE plpythonu;
  SELECT invalid_type_reraised('rick');
! WARNING:  PL/Python: plpy.SPIError: unrecognized error in PLy_spi_prepare
  CONTEXT:  PL/Python function "invalid_type_reraised"
! ERROR:  type "test" does not exist
  CONTEXT:  PL/Python function "invalid_type_reraised"
  /* no typo no messing about
   */
--- 97,105 ----
  '
  	LANGUAGE plpythonu;
  SELECT invalid_type_reraised('rick');
! WARNING:  plpy.SPIError: unrecognized error in PLy_spi_prepare
  CONTEXT:  PL/Python function "invalid_type_reraised"
! ERROR:  plpy.Error: type "test" does not exist
  CONTEXT:  PL/Python function "invalid_type_reraised"
  /* no typo no messing about
   */
diff --git a/src/pl/plpython/expected/plpython_test.out b/src/pl/plpython/expected/plpython_test.out
index a229b18..d92c987 100644
*** a/src/pl/plpython/expected/plpython_test.out
--- b/src/pl/plpython/expected/plpython_test.out
*************** select argument_test_one(users, fname, l
*** 35,40 ****
--- 35,53 ----
   willem doe => {fname: willem, lname: doe, userid: 3, username: w_doe}
  (3 rows)
  
+ -- check module contents
+ CREATE FUNCTION module_contents() RETURNS text AS
+ $$
+ contents = list(filter(lambda x: not x.startswith("__"), dir(plpy)))
+ contents.sort()
+ return ", ".join(contents)
+ $$ LANGUAGE plpythonu;
+ select module_contents();
+                                       module_contents                                      
+ -------------------------------------------------------------------------------------------
+  Error, Fatal, SPIError, debug, error, execute, fatal, info, log, notice, prepare, warning
+ (1 row)
+ 
  CREATE FUNCTION elog_test() RETURNS void
  AS $$
  plpy.debug('debug')
*************** NOTICE:  notice
*** 60,64 ****
  CONTEXT:  PL/Python function "elog_test"
  WARNING:  warning
  CONTEXT:  PL/Python function "elog_test"
! ERROR:  error
  CONTEXT:  PL/Python function "elog_test"
--- 73,77 ----
  CONTEXT:  PL/Python function "elog_test"
  WARNING:  warning
  CONTEXT:  PL/Python function "elog_test"
! ERROR:  plpy.Error: error
  CONTEXT:  PL/Python function "elog_test"
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index a165936..982005b 100644
*** a/src/pl/plpython/expected/plpython_types.out
--- b/src/pl/plpython/expected/plpython_types.out
*************** CREATE FUNCTION test_type_conversion_arr
*** 596,602 ****
  return 5
  $$ LANGUAGE plpythonu;
  SELECT * FROM test_type_conversion_array_error();
! ERROR:  PL/Python: return value of function with array return type is not a Python sequence
  CONTEXT:  while creating return value
  PL/Python function "test_type_conversion_array_error"
  --
--- 596,602 ----
  return 5
  $$ LANGUAGE plpythonu;
  SELECT * FROM test_type_conversion_array_error();
! ERROR:  return value of function with array return type is not a Python sequence
  CONTEXT:  while creating return value
  PL/Python function "test_type_conversion_array_error"
  --
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
index 38ddf02..eeb23b7 100644
*** a/src/pl/plpython/expected/plpython_types_3.out
--- b/src/pl/plpython/expected/plpython_types_3.out
*************** CREATE FUNCTION test_type_conversion_arr
*** 596,602 ****
  return 5
  $$ LANGUAGE plpython3u;
  SELECT * FROM test_type_conversion_array_error();
! ERROR:  PL/Python: return value of function with array return type is not a Python sequence
  CONTEXT:  while creating return value
  PL/Python function "test_type_conversion_array_error"
  --
--- 596,602 ----
  return 5
  $$ LANGUAGE plpython3u;
  SELECT * FROM test_type_conversion_array_error();
! ERROR:  return value of function with array return type is not a Python sequence
  CONTEXT:  while creating return value
  PL/Python function "test_type_conversion_array_error"
  --
diff --git a/src/pl/plpython/plpython.c b/src/pl/plpython/plpython.c
index 1ae1239..67eb0f3 100644
*** a/src/pl/plpython/plpython.c
--- b/src/pl/plpython/plpython.c
*************** typedef struct PLyProcedure
*** 214,223 ****
  	PyObject   *code;			/* compiled procedure code */
  	PyObject   *statics;		/* data saved across calls, local scope */
  	PyObject   *globals;		/* data saved across calls, global scope */
- 	PyObject   *me;				/* PyCObject containing pointer to this
- 								 * PLyProcedure */
  } PLyProcedure;
  
  
  /* Python objects */
  typedef struct PLyPlanObject
--- 214,227 ----
  	PyObject   *code;			/* compiled procedure code */
  	PyObject   *statics;		/* data saved across calls, local scope */
  	PyObject   *globals;		/* data saved across calls, global scope */
  } PLyProcedure;
  
+ /* the procedure cache entry */
+ typedef struct PLyProcedureEntry
+ {
+ 	Oid				fn_oid;		/* hash key */
+ 	PLyProcedure	*proc;
+ } PLyProcedureEntry;
  
  /* Python objects */
  typedef struct PLyPlanObject
*************** static HeapTuple PLy_modify_tuple(PLyPro
*** 311,321 ****
  
  static PyObject *PLy_procedure_call(PLyProcedure *, char *, PyObject *);
  
! static PLyProcedure *PLy_procedure_get(FunctionCallInfo fcinfo,
! 				  Oid tgreloid);
  
! static PLyProcedure *PLy_procedure_create(HeapTuple procTup, Oid tgreloid,
! 					 char *key);
  
  static void PLy_procedure_compile(PLyProcedure *, const char *);
  static char *PLy_procedure_munge_source(const char *, const char *);
--- 315,324 ----
  
  static PyObject *PLy_procedure_call(PLyProcedure *, char *, PyObject *);
  
! static PLyProcedure *PLy_procedure_get(Oid fn_oid, bool is_trigger);
  
! static PLyProcedure *PLy_procedure_create(HeapTuple procTup,
! 										  Oid fn_oid, bool is_trigger);
  
  static void PLy_procedure_compile(PLyProcedure *, const char *);
  static char *PLy_procedure_munge_source(const char *, const char *);
*************** static HeapTuple PLyObject_ToTuple(PLyTy
*** 358,379 ****
   */
  static PLyProcedure *PLy_curr_procedure = NULL;
  
- /*
-  * When a callback from Python into PG incurs an error, we temporarily store
-  * the error information here, and return NULL to the Python interpreter.
-  * Any further callback attempts immediately fail, and when the Python
-  * interpreter returns to the calling function, we re-throw the error (even if
-  * Python thinks it trapped the error and doesn't return NULL).  Eventually
-  * this ought to be improved to let Python code really truly trap the error,
-  * but that's more of a change from the pre-8.0 semantics than I have time for
-  * now --- it will only be possible if the callback query is executed inside a
-  * subtransaction.
-  */
- static ErrorData *PLy_error_in_progress = NULL;
  
  static PyObject *PLy_interp_globals = NULL;
  static PyObject *PLy_interp_safe_globals = NULL;
! static PyObject *PLy_procedure_cache = NULL;
  
  /* Python exceptions */
  static PyObject *PLy_exc_error = NULL;
--- 361,371 ----
   */
  static PLyProcedure *PLy_curr_procedure = NULL;
  
  
  static PyObject *PLy_interp_globals = NULL;
  static PyObject *PLy_interp_safe_globals = NULL;
! static HTAB *PLy_procedure_cache = NULL;
! static HTAB *PLy_trigger_cache = NULL;
  
  /* Python exceptions */
  static PyObject *PLy_exc_error = NULL;
*************** plpython_call_handler(PG_FUNCTION_ARGS)
*** 444,450 ****
  {
  	Datum		retval;
  	PLyProcedure *save_curr_proc;
! 	PLyProcedure *volatile proc = NULL;
  	ErrorContextCallback plerrcontext;
  
  	if (SPI_connect() != SPI_OK_CONNECT)
--- 436,442 ----
  {
  	Datum		retval;
  	PLyProcedure *save_curr_proc;
! 	PLyProcedure *proc = NULL;
  	ErrorContextCallback plerrcontext;
  
  	if (SPI_connect() != SPI_OK_CONNECT)
*************** plpython_call_handler(PG_FUNCTION_ARGS)
*** 463,480 ****
  	{
  		if (CALLED_AS_TRIGGER(fcinfo))
  		{
- 			TriggerData *tdata = (TriggerData *) fcinfo->context;
  			HeapTuple	trv;
  
! 			proc = PLy_procedure_get(fcinfo,
! 									 RelationGetRelid(tdata->tg_relation));
  			PLy_curr_procedure = proc;
  			trv = PLy_trigger_handler(fcinfo, proc);
  			retval = PointerGetDatum(trv);
  		}
  		else
  		{
! 			proc = PLy_procedure_get(fcinfo, InvalidOid);
  			PLy_curr_procedure = proc;
  			retval = PLy_function_handler(fcinfo, proc);
  		}
--- 455,470 ----
  	{
  		if (CALLED_AS_TRIGGER(fcinfo))
  		{
  			HeapTuple	trv;
  
! 			proc = PLy_procedure_get(fcinfo->flinfo->fn_oid, true);
  			PLy_curr_procedure = proc;
  			trv = PLy_trigger_handler(fcinfo, proc);
  			retval = PointerGetDatum(trv);
  		}
  		else
  		{
! 			proc = PLy_procedure_get(fcinfo->flinfo->fn_oid, false);
  			PLy_curr_procedure = proc;
  			retval = PLy_function_handler(fcinfo, proc);
  		}
*************** plpython_call_handler(PG_FUNCTION_ARGS)
*** 482,492 ****
  	PG_CATCH();
  	{
  		PLy_curr_procedure = save_curr_proc;
- 		if (proc)
- 		{
- 			/* note: Py_DECREF needs braces around it, as of 2003/08 */
- 			Py_DECREF(proc->me);
- 		}
  		PyErr_Clear();
  		PG_RE_THROW();
  	}
--- 472,477 ----
*************** plpython_call_handler(PG_FUNCTION_ARGS)
*** 497,504 ****
  
  	PLy_curr_procedure = save_curr_proc;
  
- 	Py_DECREF(proc->me);
- 
  	return retval;
  }
  
--- 482,487 ----
*************** static HeapTuple
*** 573,580 ****
  PLy_trigger_handler(FunctionCallInfo fcinfo, PLyProcedure *proc)
  {
  	HeapTuple	rv = NULL;
! 	PyObject   *volatile plargs = NULL;
! 	PyObject   *volatile plrv = NULL;
  
  	PG_TRY();
  	{
--- 556,578 ----
  PLy_trigger_handler(FunctionCallInfo fcinfo, PLyProcedure *proc)
  {
  	HeapTuple	rv = NULL;
! 	PyObject	*volatile plargs = NULL;
! 	PyObject   	*volatile plrv = NULL;
! 	TriggerData	*tdata;
! 
! 	AssertMacro(CALLED_AS_TRIGGER(fcinfo));
! 	/*
! 	 * Input/output conversion for trigger tuples.	Use the result
! 	 * TypeInfo variable to store the tuple conversion info.  We do this
! 	 * over again on each call to cover the possibility that the
! 	 * relation's tupdesc changed since the trigger was last called.
! 	 * PLy_input_tuple_funcs and PLy_output_tuple_funcs are responsible
! 	 * for not doing repetitive work.
! 	 */
! 	tdata = (TriggerData *) fcinfo->context;
! 
! 	PLy_input_tuple_funcs(&(proc->result), tdata->tg_relation->rd_att);
! 	PLy_output_tuple_funcs(&(proc->result), tdata->tg_relation->rd_att);
  
  	PG_TRY();
  	{
*************** PLy_trigger_handler(FunctionCallInfo fci
*** 582,588 ****
  		plrv = PLy_procedure_call(proc, "TD", plargs);
  
  		Assert(plrv != NULL);
- 		Assert(!PLy_error_in_progress);
  
  		/*
  		 * Disconnect from SPI manager
--- 580,585 ----
*************** PLy_function_handler(FunctionCallInfo fc
*** 992,1005 ****
  			plargs = PLy_function_build_args(fcinfo, proc);
  			plrv = PLy_procedure_call(proc, "args", plargs);
  			if (!proc->is_setof)
! 
  				/*
  				 * SETOF function parameters will be deleted when last row is
  				 * returned
  				 */
  				PLy_function_delete_args(proc);
  			Assert(plrv != NULL);
- 			Assert(!PLy_error_in_progress);
  		}
  
  		/*
--- 989,1002 ----
  			plargs = PLy_function_build_args(fcinfo, proc);
  			plrv = PLy_procedure_call(proc, "args", plargs);
  			if (!proc->is_setof)
! 			{
  				/*
  				 * SETOF function parameters will be deleted when last row is
  				 * returned
  				 */
  				PLy_function_delete_args(proc);
+ 			}
  			Assert(plrv != NULL);
  		}
  
  		/*
*************** PLy_function_handler(FunctionCallInfo fc
*** 1149,1154 ****
--- 1146,1159 ----
  		Py_XDECREF(plargs);
  		Py_XDECREF(plrv);
  
+ 		/*
+ 		 * If there was an error the iterator might have not been exhausted
+ 		 * yet. Set it to NULL so the next invocation of the function will
+ 		 * start the iteration again.
+ 		 */
+ 		Py_XDECREF(proc->setof);
+ 		proc->setof = NULL;
+ 
  		PG_RE_THROW();
  	}
  	PG_END_TRY();
*************** PLy_procedure_call(PLyProcedure *proc, c
*** 1174,1192 ****
  	 * If there was an error in a PG callback, propagate that no matter what
  	 * Python claims about its success.
  	 */
- 	if (PLy_error_in_progress)
- 	{
- 		ErrorData  *edata = PLy_error_in_progress;
- 
- 		PLy_error_in_progress = NULL;
- 		ReThrowError(edata);
- 	}
  
! 	if (rv == NULL || PyErr_Occurred())
! 	{
! 		Py_XDECREF(rv);
  		PLy_elog(ERROR, NULL);
- 	}
  
  	return rv;
  }
--- 1179,1188 ----
  	 * If there was an error in a PG callback, propagate that no matter what
  	 * Python claims about its success.
  	 */
  
! 	/* if the Python code returned an error, propagate it */
! 	if (rv == NULL)
  		PLy_elog(ERROR, NULL);
  
  	return rv;
  }
*************** PLy_function_delete_args(PLyProcedure *p
*** 1286,1291 ****
--- 1282,1301 ----
  }
  
  
+ /* Decide if a cached PLyProcedure struct is still valid */
+ static bool
+ PLy_procedure_valid(PLyProcedure *proc, HeapTuple procTup)
+ {
+ 	Assert(proc != NULL);
+ 
+ 	/* If the pg_proc tuple has changed, it's not valid */
+ 	if (!(proc->fn_xmin == HeapTupleHeaderGetXmin(procTup->t_data) &&
+ 		  ItemPointerEquals(&proc->fn_tid, &procTup->t_self)))
+ 		return false;
+ 
+ 	return true;
+ }
+ 
  /*
   * PLyProcedure functions
   */
*************** PLy_function_delete_args(PLyProcedure *p
*** 1296,1368 ****
   * function calls.
   */
  static PLyProcedure *
! PLy_procedure_get(FunctionCallInfo fcinfo, Oid tgreloid)
  {
! 	Oid			fn_oid;
! 	HeapTuple	procTup;
! 	char		key[128];
! 	PyObject   *plproc;
! 	PLyProcedure *proc = NULL;
! 	int			rv;
  
- 	fn_oid = fcinfo->flinfo->fn_oid;
  	procTup = SearchSysCache1(PROCOID, ObjectIdGetDatum(fn_oid));
  	if (!HeapTupleIsValid(procTup))
  		elog(ERROR, "cache lookup failed for function %u", fn_oid);
  
! 	rv = snprintf(key, sizeof(key), "%u_%u", fn_oid, tgreloid);
! 	if (rv >= sizeof(key) || rv < 0)
! 		elog(ERROR, "key too long");
! 
! 	plproc = PyDict_GetItemString(PLy_procedure_cache, key);
  
! 	if (plproc != NULL)
  	{
! 		Py_INCREF(plproc);
! 		if (!PyCObject_Check(plproc))
! 			elog(FATAL, "expected a PyCObject, didn't get one");
! 
! 		proc = PyCObject_AsVoidPtr(plproc);
! 		if (!proc)
! 			PLy_elog(ERROR, "PyCObject_AsVoidPtr() failed");
! 		if (proc->me != plproc)
! 			elog(FATAL, "proc->me != plproc");
! 		/* did we find an up-to-date cache entry? */
! 		if (proc->fn_xmin != HeapTupleHeaderGetXmin(procTup->t_data) ||
! 			!ItemPointerEquals(&proc->fn_tid, &procTup->t_self))
  		{
! 			Py_DECREF(plproc);
! 			proc = NULL;
  		}
  	}
  
! 	if (proc == NULL)
! 		proc = PLy_procedure_create(procTup, tgreloid, key);
  
! 	if (OidIsValid(tgreloid))
  	{
  		/*
! 		 * Input/output conversion for trigger tuples.	Use the result
! 		 * TypeInfo variable to store the tuple conversion info.  We do this
! 		 * over again on each call to cover the possibility that the
! 		 * relation's tupdesc changed since the trigger was last called.
! 		 * PLy_input_tuple_funcs and PLy_output_tuple_funcs are responsible
! 		 * for not doing repetitive work.
  		 */
! 		TriggerData *tdata = (TriggerData *) fcinfo->context;
  
! 		Assert(CALLED_AS_TRIGGER(fcinfo));
! 		PLy_input_tuple_funcs(&(proc->result), tdata->tg_relation->rd_att);
! 		PLy_output_tuple_funcs(&(proc->result), tdata->tg_relation->rd_att);
  	}
  
! 	ReleaseSysCache(procTup);
  
! 	return proc;
  }
  
  static PLyProcedure *
! PLy_procedure_create(HeapTuple procTup, Oid tgreloid, char *key)
  {
  	char		procName[NAMEDATALEN + 256];
  	Form_pg_proc procStruct;
--- 1306,1492 ----
   * function calls.
   */
  static PLyProcedure *
! PLy_procedure_get(Oid fn_oid, bool is_trigger)
  {
! 	HeapTuple			procTup;
! 	PLyProcedureEntry	*entry;
! 	bool				found;
  
  	procTup = SearchSysCache1(PROCOID, ObjectIdGetDatum(fn_oid));
  	if (!HeapTupleIsValid(procTup))
  		elog(ERROR, "cache lookup failed for function %u", fn_oid);
  
! 	/* Look for the function in the corresponding cache */
! 	if (is_trigger)
! 		entry = hash_search(PLy_trigger_cache,
! 							&fn_oid, HASH_ENTER, &found);
! 	else
! 		entry = hash_search(PLy_procedure_cache,
! 							&fn_oid, HASH_ENTER, &found);
  
! 	PG_TRY();
  	{
! 		if (!found)
  		{
! 			/* Haven't found it, create a new cache entry */
! 			entry->proc = PLy_procedure_create(procTup, fn_oid, is_trigger);
  		}
+ 		else if (!PLy_procedure_valid(entry->proc, procTup))
+ 		{
+ 			/* Found it, but it's invalid, free and reuse the cache entry */
+ 			PLy_procedure_delete(entry->proc);
+ 			PLy_free(entry->proc);
+ 			entry->proc = PLy_procedure_create(procTup, fn_oid, is_trigger);
+ 		}
+ 		/* Found it and it's valid, it's fine to use it */
+ 	}
+ 	PG_CATCH();
+ 	{
+ 		/* Do not leave an uninitialised entry in the cache */
+ 		if (is_trigger)
+ 			hash_search(PLy_trigger_cache,
+ 						&fn_oid, HASH_REMOVE, NULL);
+ 		else
+ 			hash_search(PLy_procedure_cache,
+ 						&fn_oid, HASH_REMOVE, NULL);
+ 		PG_RE_THROW();
  	}
+ 	PG_END_TRY();
  
! 	ReleaseSysCache(procTup);
  
! 	return entry->proc;
! }
! 
! /* Set up output conversion functions for a procedure */
! static void
! PLy_procedure_output_conversion(PLyProcedure *proc, Form_pg_proc procStruct)
! {
! 	HeapTuple	rvTypeTup;
! 	Form_pg_type rvTypeStruct;
! 
! 	/* Get the return type */
! 	rvTypeTup = SearchSysCache1(TYPEOID,
! 								ObjectIdGetDatum(procStruct->prorettype));
! 	if (!HeapTupleIsValid(rvTypeTup))
! 		elog(ERROR, "cache lookup failed for type %u",
! 			 procStruct->prorettype);
! 	rvTypeStruct = (Form_pg_type) GETSTRUCT(rvTypeTup);
! 
! 	/* Disallow pseudotype result, except for void */
! 	if (rvTypeStruct->typtype == TYPTYPE_PSEUDO &&
! 		procStruct->prorettype != VOIDOID)
! 	{
! 		if (procStruct->prorettype == TRIGGEROID)
! 			ereport(ERROR,
! 					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 					 errmsg("trigger functions can only be called as triggers")));
! 		else
! 			ereport(ERROR,
! 					(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 					 errmsg("PL/Python functions cannot return type %s",
! 							format_type_be(procStruct->prorettype))));
! 	}
! 
! 	if (rvTypeStruct->typtype == TYPTYPE_COMPOSITE)
  	{
  		/*
! 		 * Tuple: set up later, during first call to
! 		 * PLy_function_handler
  		 */
! 		proc->result.out.d.typoid = procStruct->prorettype;
! 		proc->result.is_rowtype = 2;
! 	}
! 	else
! 	{
! 		/* Do the real work */
! 		PLy_output_datum_func(&proc->result, rvTypeTup);
! 	}
  
! 	ReleaseSysCache(rvTypeTup);
! }
! 
! /* Set up output conversion functions for a procedure */
! static void
! PLy_procedure_input_conversion(PLyProcedure *proc, HeapTuple procTup,
! 							   Form_pg_proc procStruct)
! {
! 	Oid		*types;
! 	char	**names,
! 			*modes;
! 	int		i,
! 			pos,
! 			total;
! 
! 	/* Extract argument type info from the pg_proc tuple */
! 	total = get_func_arg_info(procTup, &types, &names, &modes);
! 
! 	/* Count number of in+inout args into proc->nargs */
! 	if (modes == NULL)
! 		proc->nargs = total;
! 	else
! 	{
! 		/* proc->nargs was initialized to 0 above */
! 		for (i = 0; i < total; i++)
! 		{
! 			if (modes[i] != PROARGMODE_OUT &&
! 				modes[i] != PROARGMODE_TABLE)
! 				(proc->nargs)++;
! 		}
  	}
  
! 	proc->argnames = (char **) PLy_malloc0(sizeof(char *) * proc->nargs);
! 	for (i = pos = 0; i < total; i++)
! 	{
! 		HeapTuple	argTypeTup;
! 		Form_pg_type argTypeStruct;
  
! 		if (modes &&
! 			(modes[i] == PROARGMODE_OUT ||
! 			 modes[i] == PROARGMODE_TABLE))
! 			continue;	/* skip OUT arguments */
! 
! 		Assert(types[i] == procStruct->proargtypes.values[pos]);
! 
! 		argTypeTup = SearchSysCache1(TYPEOID,
! 									 ObjectIdGetDatum(types[i]));
! 		if (!HeapTupleIsValid(argTypeTup))
! 			elog(ERROR, "cache lookup failed for type %u", types[i]);
! 		argTypeStruct = (Form_pg_type) GETSTRUCT(argTypeTup);
! 
! 		/* Check argument type is OK, set up I/O function info */
! 		switch (argTypeStruct->typtype)
! 		{
! 			case TYPTYPE_PSEUDO:
! 				/* Disallow pseudotype argument */
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 						 errmsg("PL/Python functions cannot accept type %s",
! 								format_type_be(types[i]))));
! 				break;
! 			case TYPTYPE_COMPOSITE:
! 				/* We'll set IO funcs at first call */
! 				proc->args[pos].is_rowtype = 2;
! 				break;
! 			default:
! 				PLy_input_datum_func(&(proc->args[pos]),
! 									 types[i],
! 									 argTypeTup);
! 				break;
! 		}
! 
! 		/* Get argument name */
! 		proc->argnames[pos] = names ? PLy_strdup(names[i]) : NULL;
! 
! 		ReleaseSysCache(argTypeTup);
! 
! 		pos++;
! 	}
  }
  
+ /* Create a new PLyProcedure structure */
  static PLyProcedure *
! PLy_procedure_create(HeapTuple procTup, Oid fn_oid, bool is_trigger)
  {
  	char		procName[NAMEDATALEN + 256];
  	Form_pg_proc procStruct;
*************** PLy_procedure_create(HeapTuple procTup,
*** 1373,1395 ****
  	int			i,
  				rv;
  
! 	procStruct = (Form_pg_proc) GETSTRUCT(procTup);
  
! 	if (OidIsValid(tgreloid))
! 		rv = snprintf(procName, sizeof(procName),
! 					  "__plpython_procedure_%s_%u_trigger_%u",
! 					  NameStr(procStruct->proname),
! 					  HeapTupleGetOid(procTup),
! 					  tgreloid);
! 	else
! 		rv = snprintf(procName, sizeof(procName),
! 					  "__plpython_procedure_%s_%u",
! 					  NameStr(procStruct->proname),
! 					  HeapTupleGetOid(procTup));
  	if (rv >= sizeof(procName) || rv < 0)
  		elog(ERROR, "procedure name would overrun buffer");
  
- 	proc = PLy_malloc(sizeof(PLyProcedure));
  	proc->proname = PLy_strdup(NameStr(procStruct->proname));
  	proc->pyname = PLy_strdup(procName);
  	proc->fn_xmin = HeapTupleHeaderGetXmin(procTup->t_data);
--- 1497,1512 ----
  	int			i,
  				rv;
  
! 	proc = PLy_malloc(sizeof(PLyProcedure));
  
! 	procStruct = (Form_pg_proc) GETSTRUCT(procTup);
! 	rv = snprintf(procName, sizeof(procName),
! 				  "__plpython_procedure_%s_%u",
! 				  NameStr(procStruct->proname),
! 				  fn_oid);
  	if (rv >= sizeof(procName) || rv < 0)
  		elog(ERROR, "procedure name would overrun buffer");
  
  	proc->proname = PLy_strdup(NameStr(procStruct->proname));
  	proc->pyname = PLy_strdup(procName);
  	proc->fn_xmin = HeapTupleHeaderGetXmin(procTup->t_data);
*************** PLy_procedure_create(HeapTuple procTup,
*** 1402,1408 ****
  		PLy_typeinfo_init(&proc->args[i]);
  	proc->nargs = 0;
  	proc->code = proc->statics = NULL;
! 	proc->globals = proc->me = NULL;
  	proc->is_setof = procStruct->proretset;
  	proc->setof = NULL;
  	proc->argnames = NULL;
--- 1519,1525 ----
  		PLy_typeinfo_init(&proc->args[i]);
  	proc->nargs = 0;
  	proc->code = proc->statics = NULL;
! 	proc->globals = NULL;
  	proc->is_setof = procStruct->proretset;
  	proc->setof = NULL;
  	proc->argnames = NULL;
*************** PLy_procedure_create(HeapTuple procTup,
*** 1413,1459 ****
  		 * get information required for output conversion of the return value,
  		 * but only if this isn't a trigger.
  		 */
! 		if (!OidIsValid(tgreloid))
! 		{
! 			HeapTuple	rvTypeTup;
! 			Form_pg_type rvTypeStruct;
! 
! 			rvTypeTup = SearchSysCache1(TYPEOID,
! 								   ObjectIdGetDatum(procStruct->prorettype));
! 			if (!HeapTupleIsValid(rvTypeTup))
! 				elog(ERROR, "cache lookup failed for type %u",
! 					 procStruct->prorettype);
! 			rvTypeStruct = (Form_pg_type) GETSTRUCT(rvTypeTup);
! 
! 			/* Disallow pseudotype result, except for void */
! 			if (rvTypeStruct->typtype == TYPTYPE_PSEUDO &&
! 				procStruct->prorettype != VOIDOID)
! 			{
! 				if (procStruct->prorettype == TRIGGEROID)
! 					ereport(ERROR,
! 							(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 							 errmsg("trigger functions can only be called as triggers")));
! 				else
! 					ereport(ERROR,
! 							(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 						  errmsg("PL/Python functions cannot return type %s",
! 								 format_type_be(procStruct->prorettype))));
! 			}
! 
! 			if (rvTypeStruct->typtype == TYPTYPE_COMPOSITE)
! 			{
! 				/*
! 				 * Tuple: set up later, during first call to
! 				 * PLy_function_handler
! 				 */
! 				proc->result.out.d.typoid = procStruct->prorettype;
! 				proc->result.is_rowtype = 2;
! 			}
! 			else
! 				PLy_output_datum_func(&proc->result, rvTypeTup);
! 
! 			ReleaseSysCache(rvTypeTup);
! 		}
  
  		/*
  		 * Now get information required for input conversion of the
--- 1530,1537 ----
  		 * get information required for output conversion of the return value,
  		 * but only if this isn't a trigger.
  		 */
! 		if (!is_trigger)
! 			PLy_procedure_output_conversion(proc, procStruct);
  
  		/*
  		 * Now get information required for input conversion of the
*************** PLy_procedure_create(HeapTuple procTup,
*** 1463,1541 ****
  		 * arguments.
  		 */
  		if (procStruct->pronargs)
! 		{
! 			Oid		   *types;
! 			char	  **names,
! 					   *modes;
! 			int			i,
! 						pos,
! 						total;
! 
! 			/* extract argument type info from the pg_proc tuple */
! 			total = get_func_arg_info(procTup, &types, &names, &modes);
! 
! 			/* count number of in+inout args into proc->nargs */
! 			if (modes == NULL)
! 				proc->nargs = total;
! 			else
! 			{
! 				/* proc->nargs was initialized to 0 above */
! 				for (i = 0; i < total; i++)
! 				{
! 					if (modes[i] != PROARGMODE_OUT &&
! 						modes[i] != PROARGMODE_TABLE)
! 						(proc->nargs)++;
! 				}
! 			}
! 
! 			proc->argnames = (char **) PLy_malloc0(sizeof(char *) * proc->nargs);
! 			for (i = pos = 0; i < total; i++)
! 			{
! 				HeapTuple	argTypeTup;
! 				Form_pg_type argTypeStruct;
! 
! 				if (modes &&
! 					(modes[i] == PROARGMODE_OUT ||
! 					 modes[i] == PROARGMODE_TABLE))
! 					continue;	/* skip OUT arguments */
! 
! 				Assert(types[i] == procStruct->proargtypes.values[pos]);
! 
! 				argTypeTup = SearchSysCache1(TYPEOID,
! 											 ObjectIdGetDatum(types[i]));
! 				if (!HeapTupleIsValid(argTypeTup))
! 					elog(ERROR, "cache lookup failed for type %u", types[i]);
! 				argTypeStruct = (Form_pg_type) GETSTRUCT(argTypeTup);
! 
! 				/* check argument type is OK, set up I/O function info */
! 				switch (argTypeStruct->typtype)
! 				{
! 					case TYPTYPE_PSEUDO:
! 						/* Disallow pseudotype argument */
! 						ereport(ERROR,
! 								(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 						  errmsg("PL/Python functions cannot accept type %s",
! 								 format_type_be(types[i]))));
! 						break;
! 					case TYPTYPE_COMPOSITE:
! 						/* we'll set IO funcs at first call */
! 						proc->args[pos].is_rowtype = 2;
! 						break;
! 					default:
! 						PLy_input_datum_func(&(proc->args[pos]),
! 											 types[i],
! 											 argTypeTup);
! 						break;
! 				}
! 
! 				/* get argument name */
! 				proc->argnames[pos] = names ? PLy_strdup(names[i]) : NULL;
! 
! 				ReleaseSysCache(argTypeTup);
! 
! 				pos++;
! 			}
! 		}
  
  		/*
  		 * get the text of the function.
--- 1541,1547 ----
  		 * arguments.
  		 */
  		if (procStruct->pronargs)
! 			PLy_procedure_input_conversion(proc, procTup, procStruct);
  
  		/*
  		 * get the text of the function.
*************** PLy_procedure_create(HeapTuple procTup,
*** 1550,1560 ****
  
  		pfree(procSource);
  		procSource = NULL;
- 
- 		proc->me = PyCObject_FromVoidPtr(proc, NULL);
- 		if (!proc->me)
- 			PLy_elog(ERROR, "PyCObject_FromVoidPtr() failed");
- 		PyDict_SetItemString(PLy_procedure_cache, key, proc->me);
  	}
  	PG_CATCH();
  	{
--- 1556,1561 ----
*************** PLy_procedure_create(HeapTuple procTup,
*** 1569,1574 ****
--- 1570,1576 ----
  	return proc;
  }
  
+ /* Insert the procedure into the Python interpreter */
  static void
  PLy_procedure_compile(PLyProcedure *proc, const char *src)
  {
*************** PLy_procedure_compile(PLyProcedure *proc
*** 1589,1597 ****
  	 */
  	msrc = PLy_procedure_munge_source(proc->pyname, src);
  	crv = PyRun_String(msrc, Py_file_input, proc->globals, NULL);
! 	free(msrc);
  
! 	if (crv != NULL && (!PyErr_Occurred()))
  	{
  		int			clen;
  		char		call[NAMEDATALEN + 256];
--- 1591,1599 ----
  	 */
  	msrc = PLy_procedure_munge_source(proc->pyname, src);
  	crv = PyRun_String(msrc, Py_file_input, proc->globals, NULL);
! 	pfree(msrc);
  
! 	if (crv != NULL)
  	{
  		int			clen;
  		char		call[NAMEDATALEN + 256];
*************** PLy_procedure_compile(PLyProcedure *proc
*** 1605,1617 ****
  		if (clen < 0 || clen >= sizeof(call))
  			elog(ERROR, "string would overflow buffer");
  		proc->code = Py_CompileString(call, "<string>", Py_eval_input);
! 		if (proc->code != NULL && (!PyErr_Occurred()))
  			return;
  	}
- 	else
- 		Py_XDECREF(crv);
  
! 	PLy_elog(ERROR, "could not compile PL/Python function \"%s\"", proc->proname);
  }
  
  static char *
--- 1607,1621 ----
  		if (clen < 0 || clen >= sizeof(call))
  			elog(ERROR, "string would overflow buffer");
  		proc->code = Py_CompileString(call, "<string>", Py_eval_input);
! 		if (proc->code != NULL)
  			return;
  	}
  
! 	if (proc->proname)
! 		PLy_elog(ERROR, "could not compile PL/Python function \"%s\"",
! 				 proc->proname);
! 	else
! 		PLy_elog(ERROR, "could not compile anonymous PL/Python code block");
  }
  
  static char *
*************** PLy_procedure_munge_source(const char *n
*** 1628,1634 ****
  	 */
  	mlen = (strlen(src) * 2) + strlen(name) + 16;
  
! 	mrc = PLy_malloc(mlen);
  	plen = snprintf(mrc, mlen, "def %s():\n\t", name);
  	Assert(plen >= 0 && plen < mlen);
  
--- 1632,1638 ----
  	 */
  	mlen = (strlen(src) * 2) + strlen(name) + 16;
  
! 	mrc = palloc(mlen);
  	plen = snprintf(mrc, mlen, "def %s():\n\t", name);
  	Assert(plen >= 0 && plen < mlen);
  
*************** PLy_procedure_delete(PLyProcedure *proc)
*** 1667,1673 ****
  	Py_XDECREF(proc->code);
  	Py_XDECREF(proc->statics);
  	Py_XDECREF(proc->globals);
- 	Py_XDECREF(proc->me);
  	if (proc->proname)
  		PLy_free(proc->proname);
  	if (proc->pyname)
--- 1671,1676 ----
*************** PLy_procedure_delete(PLyProcedure *proc)
*** 1686,1692 ****
  	}
  	if (proc->argnames)
  		PLy_free(proc->argnames);
- 	PLy_free(proc);
  }
  
  /*
--- 1689,1694 ----
*************** PLyMapping_ToTuple(PLyTypeInfo *info, Py
*** 2289,2294 ****
--- 2291,2299 ----
  		PyObject   *volatile value;
  		PLyObToDatum *att;
  
+ 		if (desc->attrs[i]->attisdropped)
+ 			continue;
+ 
  		key = NameStr(desc->attrs[i]->attname);
  		value = NULL;
  		att = &info->out.r.atts[i];
*************** PLySequence_ToTuple(PLyTypeInfo *info, P
*** 2339,2355 ****
  	HeapTuple	tuple;
  	Datum	   *values;
  	bool	   *nulls;
  	volatile int i;
  
  	Assert(PySequence_Check(sequence));
- 
  	/*
  	 * Check that sequence length is exactly same as PG tuple's. We actually
  	 * can ignore exceeding items or assume missing ones as null but to avoid
  	 * plpython developer's errors we are strict here
  	 */
  	desc = lookup_rowtype_tupdesc(info->out.d.typoid, -1);
! 	if (PySequence_Length(sequence) != desc->natts)
  		ereport(ERROR,
  				(errcode(ERRCODE_DATATYPE_MISMATCH),
  				 errmsg("length of returned sequence did not match number of columns in row")));
--- 2344,2366 ----
  	HeapTuple	tuple;
  	Datum	   *values;
  	bool	   *nulls;
+ 	int			idx;
  	volatile int i;
  
  	Assert(PySequence_Check(sequence));
  	/*
  	 * Check that sequence length is exactly same as PG tuple's. We actually
  	 * can ignore exceeding items or assume missing ones as null but to avoid
  	 * plpython developer's errors we are strict here
  	 */
  	desc = lookup_rowtype_tupdesc(info->out.d.typoid, -1);
! 	idx = 0;
! 	for (i = 0; i < desc->natts; ++i)
! 	{
! 		if (!desc->attrs[i]->attisdropped)
! 			idx++;
! 	}
! 	if (PySequence_Length(sequence) != idx)
  		ereport(ERROR,
  				(errcode(ERRCODE_DATATYPE_MISMATCH),
  				 errmsg("length of returned sequence did not match number of columns in row")));
*************** PLySequence_ToTuple(PLyTypeInfo *info, P
*** 2361,2376 ****
  	/* Build tuple */
  	values = palloc(sizeof(Datum) * desc->natts);
  	nulls = palloc(sizeof(bool) * desc->natts);
  	for (i = 0; i < desc->natts; ++i)
  	{
  		PyObject   *volatile value;
  		PLyObToDatum *att;
  
  		value = NULL;
  		att = &info->out.r.atts[i];
  		PG_TRY();
  		{
! 			value = PySequence_GetItem(sequence, i);
  			Assert(value);
  			if (value == Py_None)
  			{
--- 2372,2391 ----
  	/* Build tuple */
  	values = palloc(sizeof(Datum) * desc->natts);
  	nulls = palloc(sizeof(bool) * desc->natts);
+ 	idx = 0;
  	for (i = 0; i < desc->natts; ++i)
  	{
  		PyObject   *volatile value;
  		PLyObToDatum *att;
  
+ 		if (desc->attrs[i]->attisdropped)
+ 			continue;
+ 
  		value = NULL;
  		att = &info->out.r.atts[i];
  		PG_TRY();
  		{
! 			value = PySequence_GetItem(sequence, idx);
  			Assert(value);
  			if (value == Py_None)
  			{
*************** PLySequence_ToTuple(PLyTypeInfo *info, P
*** 2392,2397 ****
--- 2407,2414 ----
  			PG_RE_THROW();
  		}
  		PG_END_TRY();
+ 
+ 		idx++;
  	}
  
  	tuple = heap_form_tuple(desc, values, nulls);
*************** PLyObject_ToTuple(PLyTypeInfo *info, PyO
*** 2426,2431 ****
--- 2443,2451 ----
  		PyObject   *volatile value;
  		PLyObToDatum *att;
  
+ 		if (desc->attrs[i]->attisdropped)
+ 			continue;
+ 
  		key = NameStr(desc->attrs[i]->attname);
  		value = NULL;
  		att = &info->out.r.atts[i];
*************** static PyModuleDef PLy_module = {
*** 2628,2633 ****
--- 2648,2657 ----
  	NULL,						/* m_doc */
  	-1,							/* m_size */
  	PLy_methods,				/* m_methods */
+ 	NULL,                       /* m_reload */
+ 	NULL,                       /* m_traverse */
+ 	NULL,                       /* m_clear */
+ 	NULL                        /* m_free */
  };
  #endif
  
*************** PLy_plan_new(void)
*** 2637,2648 ****
  {
  	PLyPlanObject *ob;
  
! 	if ((ob = PyObject_NEW(PLyPlanObject, &PLy_PlanType)) == NULL)
  		return NULL;
  
  	ob->plan = NULL;
  	ob->nargs = 0;
  	ob->types = NULL;
  	ob->args = NULL;
  
  	return (PyObject *) ob;
--- 2661,2673 ----
  {
  	PLyPlanObject *ob;
  
! 	if ((ob = PyObject_New(PLyPlanObject, &PLy_PlanType)) == NULL)
  		return NULL;
  
  	ob->plan = NULL;
  	ob->nargs = 0;
  	ob->types = NULL;
+ 	ob->values = NULL;
  	ob->args = NULL;
  
  	return (PyObject *) ob;
*************** PLy_plan_dealloc(PyObject *arg)
*** 2658,2663 ****
--- 2683,2690 ----
  		SPI_freeplan(ob->plan);
  	if (ob->types)
  		PLy_free(ob->types);
+ 	if (ob->values)
+ 		PLy_free(ob->values);
  	if (ob->args)
  	{
  		int			i;
*************** PLy_result_new(void)
*** 2693,2699 ****
  {
  	PLyResultObject *ob;
  
! 	if ((ob = PyObject_NEW(PLyResultObject, &PLy_ResultType)) == NULL)
  		return NULL;
  
  	/* ob->tuples = NULL; */
--- 2720,2726 ----
  {
  	PLyResultObject *ob;
  
! 	if ((ob = PyObject_New(PLyResultObject, &PLy_ResultType)) == NULL)
  		return NULL;
  
  	/* ob->tuples = NULL; */
*************** PLy_spi_prepare(PyObject *self, PyObject
*** 2799,2823 ****
  	PyObject   *volatile optr = NULL;
  	char	   *query;
  	void	   *tmpplan;
! 	volatile MemoryContext oldcontext;
  
! 	/* Can't execute more if we have an unhandled error */
! 	if (PLy_error_in_progress)
! 	{
! 		PLy_exception_set(PLy_exc_error, "transaction aborted");
! 		return NULL;
! 	}
  
  	if (!PyArg_ParseTuple(args, "s|O", &query, &list))
  	{
- 		PLy_exception_set(PLy_exc_spi_error,
- 						  "invalid arguments for plpy.prepare");
  		return NULL;
  	}
  
  	if (list && (!PySequence_Check(list)))
  	{
! 		PLy_exception_set(PLy_exc_spi_error,
  					   "second argument of plpy.prepare must be a sequence");
  		return NULL;
  	}
--- 2826,2843 ----
  	PyObject   *volatile optr = NULL;
  	char	   *query;
  	void	   *tmpplan;
! 	int			nargs;
  
! 	MemoryContext volatile oldcontext = CurrentMemoryContext;
  
  	if (!PyArg_ParseTuple(args, "s|O", &query, &list))
  	{
  		return NULL;
  	}
  
  	if (list && (!PySequence_Check(list)))
  	{
! 		PLy_exception_set(PyExc_ValueError,
  					   "second argument of plpy.prepare must be a sequence");
  		return NULL;
  	}
*************** PLy_spi_prepare(PyObject *self, PyObject
*** 2825,2904 ****
  	if ((plan = (PLyPlanObject *) PLy_plan_new()) == NULL)
  		return NULL;
  
! 	oldcontext = CurrentMemoryContext;
  	PG_TRY();
  	{
! 		if (list != NULL)
! 		{
! 			int			nargs,
! 						i;
! 
! 			nargs = PySequence_Length(list);
! 			if (nargs > 0)
! 			{
! 				plan->nargs = nargs;
! 				plan->types = PLy_malloc(sizeof(Oid) * nargs);
! 				plan->values = PLy_malloc(sizeof(Datum) * nargs);
! 				plan->args = PLy_malloc(sizeof(PLyTypeInfo) * nargs);
  
! 				/*
! 				 * the other loop might throw an exception, if PLyTypeInfo
! 				 * member isn't properly initialized the Py_DECREF(plan) will
! 				 * go boom
! 				 */
! 				for (i = 0; i < nargs; i++)
! 				{
! 					PLy_typeinfo_init(&plan->args[i]);
! 					plan->values[i] = PointerGetDatum(NULL);
! 				}
  
! 				for (i = 0; i < nargs; i++)
! 				{
! 					char	   *sptr;
! 					HeapTuple	typeTup;
! 					Oid			typeId;
! 					int32		typmod;
! 					Form_pg_type typeStruct;
  
! 					optr = PySequence_GetItem(list, i);
! 					if (PyString_Check(optr))
! 						sptr = PyString_AsString(optr);
! 					else if (PyUnicode_Check(optr))
! 						sptr = PLyUnicode_AsString(optr);
! 					else
! 					{
! 						ereport(ERROR,
! 								(errmsg("plpy.prepare: type name at ordinal position %d is not a string", i)));
! 						sptr = NULL;	/* keep compiler quiet */
! 					}
  
! 					/********************************************************
! 					 * Resolve argument type names and then look them up by
! 					 * oid in the system cache, and remember the required
! 					 *information for input conversion.
! 					 ********************************************************/
  
! 					parseTypeString(sptr, &typeId, &typmod);
  
! 					typeTup = SearchSysCache1(TYPEOID,
! 											  ObjectIdGetDatum(typeId));
! 					if (!HeapTupleIsValid(typeTup))
! 						elog(ERROR, "cache lookup failed for type %u", typeId);
  
! 					Py_DECREF(optr);
! 					optr = NULL;	/* this is important */
  
! 					plan->types[i] = typeId;
! 					typeStruct = (Form_pg_type) GETSTRUCT(typeTup);
! 					if (typeStruct->typtype != TYPTYPE_COMPOSITE)
! 						PLy_output_datum_func(&plan->args[i], typeTup);
! 					else
! 						ereport(ERROR,
! 								(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 								 errmsg("plpy.prepare does not support composite types")));
! 					ReleaseSysCache(typeTup);
! 				}
! 			}
  		}
  
  		pg_verifymbstr(query, strlen(query), false);
--- 2845,2921 ----
  	if ((plan = (PLyPlanObject *) PLy_plan_new()) == NULL)
  		return NULL;
  
! 	nargs = (list == NULL ? 0 : PySequence_Length(list));
! 
! 	plan->nargs = nargs;
! 	plan->types = PLy_malloc(sizeof(Oid) * nargs);
! 	plan->values = PLy_malloc(sizeof(Datum) * nargs);
! 	plan->args = PLy_malloc(sizeof(PLyTypeInfo) * nargs);
! 
  	PG_TRY();
  	{
! 		int	i;
  
! 		/*
! 		 * the other loop might throw an exception, if PLyTypeInfo
! 		 * member isn't properly initialized the Py_DECREF(plan) will
! 		 * go boom
! 		 */
! 		for (i = 0; i < nargs; i++)
! 		{
! 			PLy_typeinfo_init(&plan->args[i]);
! 			plan->values[i] = PointerGetDatum(NULL);
! 		}
  
! 		for (i = 0; i < nargs; i++)
! 		{
! 			char	   *sptr;
! 			HeapTuple	typeTup;
! 			Oid			typeId;
! 			int32		typmod;
! 			Form_pg_type typeStruct;
  
! 			optr = PySequence_GetItem(list, i);
! 			if (PyString_Check(optr))
! 				sptr = PyString_AsString(optr);
! 			else if (PyUnicode_Check(optr))
! 				sptr = PLyUnicode_AsString(optr);
! 			else
! 			{
! 				ereport(ERROR,
! 						(errmsg("plpy.prepare: type name at ordinal position %d is not a string", i)));
! 				sptr = NULL;	/* keep compiler quiet */
! 			}
  
! 			/********************************************************
! 			 * Resolve argument type names and then look them up by
! 			 * oid in the system cache, and remember the required
! 			 *information for input conversion.
! 			 ********************************************************/
  
! 			parseTypeString(sptr, &typeId, &typmod);
  
! 			typeTup = SearchSysCache1(TYPEOID,
! 									  ObjectIdGetDatum(typeId));
! 			if (!HeapTupleIsValid(typeTup))
! 				elog(ERROR, "cache lookup failed for type %u", typeId);
  
! 			Py_DECREF(optr);
! 			/*
! 			 * set optr to NULL, so we won't try to unref it again in
! 			 * case of an error
! 			 */
! 			optr = NULL;
  
! 			plan->types[i] = typeId;
! 			typeStruct = (Form_pg_type) GETSTRUCT(typeTup);
! 			if (typeStruct->typtype != TYPTYPE_COMPOSITE)
! 				PLy_output_datum_func(&plan->args[i], typeTup);
! 			else
! 				ereport(ERROR,
! 						(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
! 						 errmsg("plpy.prepare does not support composite types")));
! 			ReleaseSysCache(typeTup);
  		}
  
  		pg_verifymbstr(query, strlen(query), false);
*************** PLy_spi_prepare(PyObject *self, PyObject
*** 2917,2935 ****
  	}
  	PG_CATCH();
  	{
  		MemoryContextSwitchTo(oldcontext);
! 		PLy_error_in_progress = CopyErrorData();
! 		FlushErrorState();
  		Py_DECREF(plan);
  		Py_XDECREF(optr);
  		if (!PyErr_Occurred())
  			PLy_exception_set(PLy_exc_spi_error,
  							  "unrecognized error in PLy_spi_prepare");
  		PLy_elog(WARNING, NULL);
  		return NULL;
  	}
  	PG_END_TRY();
  
  	return (PyObject *) plan;
  }
  
--- 2934,2955 ----
  	}
  	PG_CATCH();
  	{
+ 		ErrorData	*edata;
+ 
  		MemoryContextSwitchTo(oldcontext);
! 		edata = CopyErrorData();
  		Py_DECREF(plan);
  		Py_XDECREF(optr);
  		if (!PyErr_Occurred())
  			PLy_exception_set(PLy_exc_spi_error,
  							  "unrecognized error in PLy_spi_prepare");
  		PLy_elog(WARNING, NULL);
+ 		PLy_exception_set(PLy_exc_spi_error, edata->message);
  		return NULL;
  	}
  	PG_END_TRY();
  
+ 	Assert(plan->plan != NULL);
  	return (PyObject *) plan;
  }
  
*************** PLy_spi_execute(PyObject *self, PyObject
*** 2944,2956 ****
  	PyObject   *list = NULL;
  	long		limit = 0;
  
- 	/* Can't execute more if we have an unhandled error */
- 	if (PLy_error_in_progress)
- 	{
- 		PLy_exception_set(PLy_exc_error, "transaction aborted");
- 		return NULL;
- 	}
- 
  	if (PyArg_ParseTuple(args, "s|l", &query, &limit))
  		return PLy_spi_execute_query(query, limit);
  
--- 2964,2969 ----
*************** PLy_spi_execute(PyObject *self, PyObject
*** 2967,2983 ****
  static PyObject *
  PLy_spi_execute_plan(PyObject *ob, PyObject *list, long limit)
  {
! 	volatile int nargs;
! 	int			i,
! 				rv;
! 	PLyPlanObject *plan;
! 	volatile MemoryContext oldcontext;
  
  	if (list != NULL)
  	{
  		if (!PySequence_Check(list) || PyString_Check(list) || PyUnicode_Check(list))
  		{
! 			PLy_exception_set(PLy_exc_spi_error, "plpy.execute takes a sequence as its second argument");
  			return NULL;
  		}
  		nargs = PySequence_Length(list);
--- 2980,2996 ----
  static PyObject *
  PLy_spi_execute_plan(PyObject *ob, PyObject *list, long limit)
  {
! 	int 			volatile nargs;
! 	int				i, rv;
! 	PLyPlanObject	*plan;
! 	PyObject		*ret;
! 	MemoryContext volatile oldcontext = CurrentMemoryContext;
  
  	if (list != NULL)
  	{
  		if (!PySequence_Check(list) || PyString_Check(list) || PyUnicode_Check(list))
  		{
! 			PLy_exception_set(PyExc_ValueError, "plpy.execute takes a sequence as its second argument");
  			return NULL;
  		}
  		nargs = PySequence_Length(list);
*************** PLy_spi_execute_plan(PyObject *ob, PyObj
*** 2992,3003 ****
  		char	   *sv;
  		PyObject   *so = PyObject_Str(list);
  
! 		if (!so)
  			PLy_elog(ERROR, "could not execute plan");
  		sv = PyString_AsString(so);
! 		PLy_exception_set_plural(PLy_exc_spi_error,
! 							  "Expected sequence of %d argument, got %d: %s",
! 							 "Expected sequence of %d arguments, got %d: %s",
  								 plan->nargs,
  								 plan->nargs, nargs, sv);
  		Py_DECREF(so);
--- 3005,3016 ----
  		char	   *sv;
  		PyObject   *so = PyObject_Str(list);
  
! 		if (so == NULL)
  			PLy_elog(ERROR, "could not execute plan");
  		sv = PyString_AsString(so);
! 		PLy_exception_set_plural(PyExc_ValueError,
! 								 "Expected sequence of %d argument, got %d: %s",
! 								 "Expected sequence of %d arguments, got %d: %s",
  								 plan->nargs,
  								 plan->nargs, nargs, sv);
  		Py_DECREF(so);
*************** PLy_spi_execute_plan(PyObject *ob, PyObj
*** 3005,3015 ****
  		return NULL;
  	}
  
- 	oldcontext = CurrentMemoryContext;
  	PG_TRY();
  	{
! 		char	   *nulls = palloc(nargs * sizeof(char));
! 		volatile int j;
  
  		for (j = 0; j < nargs; j++)
  		{
--- 3018,3032 ----
  		return NULL;
  	}
  
  	PG_TRY();
  	{
! 		char	*nulls;
! 		int		volatile j;
! 
! 		if (nargs > 0)
! 			nulls = palloc(nargs * sizeof(char));
! 		else
! 			nulls = NULL;
  
  		for (j = 0; j < nargs; j++)
  		{
*************** PLy_spi_execute_plan(PyObject *ob, PyObj
*** 3049,3082 ****
  
  		rv = SPI_execute_plan(plan->plan, plan->values, nulls,
  							  PLy_curr_procedure->fn_readonly, limit);
  
- 		pfree(nulls);
  	}
  	PG_CATCH();
  	{
! 		int			k;
  
  		MemoryContextSwitchTo(oldcontext);
! 		PLy_error_in_progress = CopyErrorData();
  		FlushErrorState();
  
  		/*
  		 * cleanup plan->values array
  		 */
! 		for (k = 0; k < nargs; k++)
  		{
! 			if (!plan->args[k].out.d.typbyval &&
! 				(plan->values[k] != PointerGetDatum(NULL)))
  			{
! 				pfree(DatumGetPointer(plan->values[k]));
! 				plan->values[k] = PointerGetDatum(NULL);
  			}
  		}
  
  		if (!PyErr_Occurred())
! 			PLy_exception_set(PLy_exc_error,
  							  "unrecognized error in PLy_spi_execute_plan");
  		PLy_elog(WARNING, NULL);
  		return NULL;
  	}
  	PG_END_TRY();
--- 3066,3104 ----
  
  		rv = SPI_execute_plan(plan->plan, plan->values, nulls,
  							  PLy_curr_procedure->fn_readonly, limit);
+ 		ret = PLy_spi_execute_fetch_result(SPI_tuptable, SPI_processed, rv);
+ 
+ 		if (nargs > 0)
+ 			pfree(nulls);
+ 
  
  	}
  	PG_CATCH();
  	{
! 		ErrorData	*edata;
  
  		MemoryContextSwitchTo(oldcontext);
! 		edata = CopyErrorData();
  		FlushErrorState();
  
  		/*
  		 * cleanup plan->values array
  		 */
! 		for (i = 0; i < nargs; i++)
  		{
! 			if (!plan->args[i].out.d.typbyval &&
! 				(plan->values[i] != PointerGetDatum(NULL)))
  			{
! 				pfree(DatumGetPointer(plan->values[i]));
! 				plan->values[i] = PointerGetDatum(NULL);
  			}
  		}
  
  		if (!PyErr_Occurred())
! 			PLy_exception_set(PLy_exc_spi_error,
  							  "unrecognized error in PLy_spi_execute_plan");
  		PLy_elog(WARNING, NULL);
+ 		PLy_exception_set(PLy_exc_spi_error, edata->message);
  		return NULL;
  	}
  	PG_END_TRY();
*************** PLy_spi_execute_plan(PyObject *ob, PyObj
*** 3091,3128 ****
  		}
  	}
  
! 	if (rv < 0)
! 	{
! 		PLy_exception_set(PLy_exc_spi_error,
! 						  "SPI_execute_plan failed: %s",
! 						  SPI_result_code_string(rv));
! 		return NULL;
! 	}
! 
! 	return PLy_spi_execute_fetch_result(SPI_tuptable, SPI_processed, rv);
  }
  
  static PyObject *
  PLy_spi_execute_query(char *query, long limit)
  {
  	int			rv;
! 	volatile MemoryContext oldcontext;
  
- 	oldcontext = CurrentMemoryContext;
  	PG_TRY();
  	{
  		pg_verifymbstr(query, strlen(query), false);
  		rv = SPI_execute(query, PLy_curr_procedure->fn_readonly, limit);
  	}
  	PG_CATCH();
  	{
  		MemoryContextSwitchTo(oldcontext);
! 		PLy_error_in_progress = CopyErrorData();
  		FlushErrorState();
  		if (!PyErr_Occurred())
  			PLy_exception_set(PLy_exc_spi_error,
  							  "unrecognized error in PLy_spi_execute_query");
  		PLy_elog(WARNING, NULL);
  		return NULL;
  	}
  	PG_END_TRY();
--- 3113,3146 ----
  		}
  	}
  
! 	return ret;
  }
  
  static PyObject *
  PLy_spi_execute_query(char *query, long limit)
  {
  	int			rv;
! 	PyObject	*ret;
! 	MemoryContext volatile oldcontext = CurrentMemoryContext;
  
  	PG_TRY();
  	{
  		pg_verifymbstr(query, strlen(query), false);
  		rv = SPI_execute(query, PLy_curr_procedure->fn_readonly, limit);
+ 		ret = PLy_spi_execute_fetch_result(SPI_tuptable, SPI_processed, rv);
  	}
  	PG_CATCH();
  	{
+ 		ErrorData	*edata;
+ 
  		MemoryContextSwitchTo(oldcontext);
! 		edata = CopyErrorData();
  		FlushErrorState();
  		if (!PyErr_Occurred())
  			PLy_exception_set(PLy_exc_spi_error,
  							  "unrecognized error in PLy_spi_execute_query");
  		PLy_elog(WARNING, NULL);
+ 		PLy_exception_set(PLy_exc_spi_error, edata->message);
  		return NULL;
  	}
  	PG_END_TRY();
*************** PLy_spi_execute_query(char *query, long
*** 3135,3141 ****
  		return NULL;
  	}
  
! 	return PLy_spi_execute_fetch_result(SPI_tuptable, SPI_processed, rv);
  }
  
  static PyObject *
--- 3153,3159 ----
  		return NULL;
  	}
  
! 	return ret;
  }
  
  static PyObject *
*************** PLy_spi_execute_fetch_result(SPITupleTab
*** 3183,3190 ****
  		PG_CATCH();
  		{
  			MemoryContextSwitchTo(oldcontext);
- 			PLy_error_in_progress = CopyErrorData();
- 			FlushErrorState();
  			if (!PyErr_Occurred())
  				PLy_exception_set(PLy_exc_error,
  					   "unrecognized error in PLy_spi_execute_fetch_result");
--- 3201,3206 ----
*************** PLy_spi_execute_fetch_result(SPITupleTab
*** 3202,3207 ****
--- 3218,3263 ----
  	return (PyObject *) result;
  }
  
+ /*
+  * Exception support
+  */
+ 
+ /*
+  * Add an exception to the module. The first parameter is either the actual
+  * module (for Python 3) or the module dictionary
+  */
+ static void PLy_add_exception(PyObject *mod, const char *name, PyObject *exc)
+ {
+ #if PY_MAJOR_VERSION >= 3
+ 	Py_INCREF(exc);
+ 	PyModule_AddObject(mod, name, exc);
+ #else
+ 	PyDict_SetItemString(mod, name, exc);
+ #endif
+ }
+ 
+ /* Add exceptions to the plpy module */
+ static void
+ PLy_initialize_exceptions(PyObject *plpy)
+ {
+ 	PyObject	*mod;
+ 
+ #if PY_MAJOR_VERSION < 3
+ 	/* For Python <3 we add the exceptions to the module dictionary */
+ 	mod = PyModule_GetDict(plpy);
+ #else
+ 	/* In Python 3 you add them directly into the module */
+ 	mod = plpy;
+ #endif
+ 
+ 	PLy_exc_error = PyErr_NewException("plpy.Error", NULL, NULL);
+ 	PLy_exc_fatal = PyErr_NewException("plpy.Fatal", NULL, NULL);
+ 	PLy_exc_spi_error = PyErr_NewException("plpy.SPIError", NULL, NULL);
+ 
+ 	PLy_add_exception(mod, "Error", PLy_exc_error);
+ 	PLy_add_exception(mod, "Fatal", PLy_exc_fatal);
+ 	PLy_add_exception(mod, "SPIError", PLy_exc_spi_error);
+ }
  
  /*
   * language handler and interpreter initialization
*************** PLy_spi_execute_fetch_result(SPITupleTab
*** 3211,3217 ****
  static PyMODINIT_FUNC
  PyInit_plpy(void)
  {
! 	return PyModule_Create(&PLy_module);
  }
  #endif
  
--- 3267,3281 ----
  static PyMODINIT_FUNC
  PyInit_plpy(void)
  {
! 	PyObject	*m;
! 
! 	m = PyModule_Create(&PLy_module);
! 	if (m == NULL)
! 		return NULL;
! 
! 	PLy_initialize_exceptions(m);
! 
! 	return m;
  }
  #endif
  
*************** _PG_init(void)
*** 3229,3234 ****
--- 3293,3300 ----
  	/* Be sure we do initialization only once (should be redundant now) */
  	static bool inited = false;
  	const int **version_ptr;
+ 	HASHCTL		hash_ctl;
+ 
  
  	if (inited)
  		return;
*************** _PG_init(void)
*** 3260,3268 ****
  	PLy_init_plpy();
  	if (PyErr_Occurred())
  		PLy_elog(FATAL, "untrapped error in initialization");
! 	PLy_procedure_cache = PyDict_New();
! 	if (PLy_procedure_cache == NULL)
! 		PLy_elog(ERROR, "could not create procedure cache");
  
  	inited = true;
  }
--- 3326,3346 ----
  	PLy_init_plpy();
  	if (PyErr_Occurred())
  		PLy_elog(FATAL, "untrapped error in initialization");
! 
! 	memset(&hash_ctl, 0, sizeof(hash_ctl));
! 	hash_ctl.keysize = sizeof(Oid);
! 	hash_ctl.entrysize = sizeof(PLyProcedureEntry);
! 	hash_ctl.hash = oid_hash;
! 	PLy_procedure_cache = hash_create("PL/Python procedures", 32,
! 									  &hash_ctl, HASH_ELEM | HASH_FUNCTION);
! 
! 	memset(&hash_ctl, 0, sizeof(hash_ctl));
! 	hash_ctl.keysize = sizeof(Oid);
! 	hash_ctl.entrysize = sizeof(PLyProcedureEntry);
! 	hash_ctl.hash = oid_hash;
! 	PLy_trigger_cache = hash_create("PL/Python triggers",
! 									32, &hash_ctl,
! 									HASH_ELEM | HASH_FUNCTION);
  
  	inited = true;
  }
*************** PLy_init_plpy(void)
*** 3290,3297 ****
  	PyObject   *main_mod,
  			   *main_dict,
  			   *plpy_mod;
! 	PyObject   *plpy,
! 			   *plpy_dict;
  
  	/*
  	 * initialize plpy module
--- 3368,3374 ----
  	PyObject   *main_mod,
  			   *main_dict,
  			   *plpy_mod;
! 	PyObject   *plpy;
  
  	/*
  	 * initialize plpy module
*************** PLy_init_plpy(void)
*** 3305,3322 ****
  	plpy = PyModule_Create(&PLy_module);
  #else
  	plpy = Py_InitModule("plpy", PLy_methods);
  #endif
- 	plpy_dict = PyModule_GetDict(plpy);
  
  	/* PyDict_SetItemString(plpy, "PlanType", (PyObject *) &PLy_PlanType); */
  
- 	PLy_exc_error = PyErr_NewException("plpy.Error", NULL, NULL);
- 	PLy_exc_fatal = PyErr_NewException("plpy.Fatal", NULL, NULL);
- 	PLy_exc_spi_error = PyErr_NewException("plpy.SPIError", NULL, NULL);
- 	PyDict_SetItemString(plpy_dict, "Error", PLy_exc_error);
- 	PyDict_SetItemString(plpy_dict, "Fatal", PLy_exc_fatal);
- 	PyDict_SetItemString(plpy_dict, "SPIError", PLy_exc_spi_error);
- 
  	/*
  	 * initialize main module, and add plpy
  	 */
--- 3382,3393 ----
  	plpy = PyModule_Create(&PLy_module);
  #else
  	plpy = Py_InitModule("plpy", PLy_methods);
+ 	/* for Python 3 we initialized the exceptions in PyInit_plpy */
+ 	PLy_initialize_exceptions(plpy);
  #endif
  
  	/* PyDict_SetItemString(plpy, "PlanType", (PyObject *) &PLy_PlanType); */
  
  	/*
  	 * initialize main module, and add plpy
  	 */
*************** PLy_init_plpy(void)
*** 3331,3337 ****
  /* the python interface to the elog function
   * don't confuse these with PLy_elog
   */
! static PyObject *PLy_output(volatile int, PyObject *, PyObject *);
  
  static PyObject *
  PLy_debug(PyObject *self, PyObject *args)
--- 3402,3408 ----
  /* the python interface to the elog function
   * don't confuse these with PLy_elog
   */
! static PyObject *PLy_output(int, PyObject *, PyObject *);
  
  static PyObject *
  PLy_debug(PyObject *self, PyObject *args)
*************** PLy_fatal(PyObject *self, PyObject *args
*** 3379,3387 ****
  static PyObject *
  PLy_output(volatile int level, PyObject *self, PyObject *args)
  {
! 	PyObject   *volatile so;
! 	char	   *volatile sv;
! 	volatile MemoryContext oldcontext;
  
  	if (PyTuple_Size(args) == 1)
  	{
--- 3450,3458 ----
  static PyObject *
  PLy_output(volatile int level, PyObject *self, PyObject *args)
  {
! 	PyObject			*so;
! 	char				*sv;
! 	MemoryContext		 volatile oldcontext = CurrentMemoryContext;
  
  	if (PyTuple_Size(args) == 1)
  	{
*************** PLy_output(volatile int level, PyObject
*** 3402,3432 ****
  		sv = dgettext(TEXTDOMAIN, "could not parse error message in plpy.elog");
  	}
  
! 	oldcontext = CurrentMemoryContext;
  	PG_TRY();
  	{
- 		pg_verifymbstr(sv, strlen(sv), false);
  		elog(level, "%s", sv);
  	}
  	PG_CATCH();
  	{
! 		MemoryContextSwitchTo(oldcontext);
! 		PLy_error_in_progress = CopyErrorData();
! 		FlushErrorState();
  
! 		PyErr_SetString(PLy_exc_error, sv);
  
  		/*
! 		 * Note: If sv came from PyString_AsString(), it points into storage
! 		 * owned by so.  So free so after using sv.
! 		 */
  		Py_XDECREF(so);
  
! 		/*
! 		 * returning NULL here causes the python interpreter to bail. when
! 		 * control passes back to PLy_procedure_call, we check for PG
! 		 * exceptions and re-throw the error.
! 		 */
  		return NULL;
  	}
  	PG_END_TRY();
--- 3473,3500 ----
  		sv = dgettext(TEXTDOMAIN, "could not parse error message in plpy.elog");
  	}
  
! 	pg_verifymbstr(sv, strlen(sv), false);
  	PG_TRY();
  	{
  		elog(level, "%s", sv);
  	}
  	PG_CATCH();
  	{
! 		ErrorData  *edata;
  
!         /* Must reset elog.c's state */
!         MemoryContextSwitchTo(oldcontext);
!         edata = CopyErrorData();
!         FlushErrorState();
  
  		/*
!         * Note: If sv came from PyString_AsString(), it points into storage
!         * owned by so.  So free so after using sv.
!         */
  		Py_XDECREF(so);
  
! 		/* Make Python raise the exception */
! 		PLy_exception_set(PLy_exc_error, edata->message);
  		return NULL;
  	}
  	PG_END_TRY();
*************** PLy_elog(int elevel, const char *fmt,...
*** 3528,3538 ****
  	{
  		if (fmt)
  			ereport(elevel,
! 					(errmsg("PL/Python: %s", emsg.data),
  					 (xmsg) ? errdetail("%s", xmsg) : 0));
  		else
  			ereport(elevel,
! 					(errmsg("PL/Python: %s", xmsg)));
  	}
  	PG_CATCH();
  	{
--- 3596,3606 ----
  	{
  		if (fmt)
  			ereport(elevel,
! 					(errmsg("%s", emsg.data),
  					 (xmsg) ? errdetail("%s", xmsg) : 0));
  		else
  			ereport(elevel,
! 					(errmsg("%s", xmsg)));
  	}
  	PG_CATCH();
  	{
*************** PLy_traceback(int *xlevel)
*** 3635,3653 ****
  
  /* some dumb utility functions */
  static void *
! PLy_malloc(size_t bytes)
  {
! 	void	   *ptr = malloc(bytes);
! 
! 	if (ptr == NULL)
! 		ereport(FATAL,
! 				(errcode(ERRCODE_OUT_OF_MEMORY),
! 				 errmsg("out of memory")));
! 	return ptr;
  }
  
  static void *
! PLy_malloc0(size_t bytes)
  {
  	void	   *ptr = PLy_malloc(bytes);
  
--- 3703,3716 ----
  
  /* some dumb utility functions */
  static void *
! PLy_malloc(Size bytes)
  {
! 	/* We need our allocations to be long-lived, so use TopMemoryContext */
! 	return MemoryContextAlloc(TopMemoryContext, bytes);
  }
  
  static void *
! PLy_malloc0(Size bytes)
  {
  	void	   *ptr = PLy_malloc(bytes);
  
*************** PLy_strdup(const char *str)
*** 3672,3678 ****
  static void
  PLy_free(void *ptr)
  {
! 	free(ptr);
  }
  
  /*
--- 3735,3741 ----
  static void
  PLy_free(void *ptr)
  {
! 	pfree(ptr);
  }
  
  /*
diff --git a/src/pl/plpython/sql/plpython_test.sql b/src/pl/plpython/sql/plpython_test.sql
index 7cae124..298411a 100644
*** a/src/pl/plpython/sql/plpython_test.sql
--- b/src/pl/plpython/sql/plpython_test.sql
*************** return words'
*** 25,30 ****
--- 25,39 ----
  
  select argument_test_one(users, fname, lname) from users where lname = 'doe' order by 1;
  
+ -- check module contents
+ CREATE FUNCTION module_contents() RETURNS text AS
+ $$
+ contents = list(filter(lambda x: not x.startswith("__"), dir(plpy)))
+ contents.sort()
+ return ", ".join(contents)
+ $$ LANGUAGE plpythonu;
+ 
+ select module_contents();
  
  CREATE FUNCTION elog_test() RETURNS void
  AS $$
#2Peter Eisentraut
peter_e@gmx.net
In reply to: Jan Urbański (#1)
Re: pl/python refactoring

On tor, 2010-12-23 at 14:41 +0100, Jan Urbański wrote:

It does some architectural changes to PL/Python that make it easier to
implement other features, like for instance a validator function. The
full list of changes in the patch is:

I would review this and the following patches, but I'd really prefer it
if you could split this particular patch into about 11 single-purpose
patches. I think most of the changes here are not interrelated.

#3Jan Urbański
wulczer@wulczer.org
In reply to: Peter Eisentraut (#2)
Re: pl/python refactoring

On 01/01/11 01:00, Peter Eisentraut wrote:

On tor, 2010-12-23 at 14:41 +0100, Jan Urbański wrote:

It does some architectural changes to PL/Python that make it easier to
implement other features, like for instance a validator function. The
full list of changes in the patch is:

I would review this and the following patches, but I'd really prefer it
if you could split this particular patch into about 11 single-purpose
patches. I think most of the changes here are not interrelated.

OK, I'll split this patch into even smaller chunks.

Jan

#4Peter Eisentraut
peter_e@gmx.net
In reply to: Jan Urbański (#3)
Re: pl/python refactoring

On lör, 2011-01-01 at 13:24 +0100, Jan Urbański wrote:

On 01/01/11 01:00, Peter Eisentraut wrote:

On tor, 2010-12-23 at 14:41 +0100, Jan Urbański wrote:

It does some architectural changes to PL/Python that make it easier to
implement other features, like for instance a validator function. The
full list of changes in the patch is:

I would review this and the following patches, but I'd really prefer it
if you could split this particular patch into about 11 single-purpose
patches. I think most of the changes here are not interrelated.

OK, I'll split this patch into even smaller chunks.

Thanks. Just attach them all to a single mail message. Don't create
new CF entries or something.

#5Jan Urbański
wulczer@wulczer.org
In reply to: Peter Eisentraut (#4)
1 attachment(s)
Re: pl/python refactoring

On 01/01/11 13:50, Peter Eisentraut wrote:

On lör, 2011-01-01 at 13:24 +0100, Jan Urbański wrote:

On 01/01/11 01:00, Peter Eisentraut wrote:

On tor, 2010-12-23 at 14:41 +0100, Jan Urbański wrote:

It does some architectural changes to PL/Python that make it easier to
implement other features, like for instance a validator function. The
full list of changes in the patch is:

I would review this and the following patches, but I'd really prefer it
if you could split this particular patch into about 11 single-purpose
patches. I think most of the changes here are not interrelated.

OK, I'll split this patch into even smaller chunks.

Thanks. Just attach them all to a single mail message. Don't create
new CF entries or something.

Here they are. There are 16 patches in total. They amount to what's
currently in my refactor branch (and almost to what I've sent as the
complete refactor patch, while doing the splitting I discovered a few
useless hunks that I've discarded).

Cheers,
Jan

Attachments:

0001-Use-HTABs-instead-of-Python-dictionary-objects-to-ca.patchtext/x-patch; name=0001-Use-HTABs-instead-of-Python-dictionary-objects-to-ca.patchDownload
#6Peter Eisentraut
peter_e@gmx.net
In reply to: Jan Urbański (#5)
Re: pl/python refactoring

On sön, 2011-01-02 at 12:41 +0100, Jan Urbański wrote:

Here they are. There are 16 patches in total. They amount to what's
currently in my refactor branch (and almost to what I've sent as the
complete refactor patch, while doing the splitting I discovered a few
useless hunks that I've discarded).

Committed 0001, rest later.

#7Peter Eisentraut
peter_e@gmx.net
In reply to: Peter Eisentraut (#6)
Re: pl/python refactoring

On mån, 2011-01-17 at 21:49 +0200, Peter Eisentraut wrote:

On sön, 2011-01-02 at 12:41 +0100, Jan Urbański wrote:

Here they are. There are 16 patches in total. They amount to what's
currently in my refactor branch (and almost to what I've sent as the
complete refactor patch, while doing the splitting I discovered a few
useless hunks that I've discarded).

Committed 0001, rest later.

Today committed: 3, 5, 10, 11, 12, 13

Discussion:

#2: It looks like this loses some information/formatting in the error
message. Should we keep the pointing arrow there?

--- a/src/pl/plpython/expected/plpython_error.out
+++ b/src/pl/plpython/expected/plpython_error.out
@@ -10,10 +10,7 @@ CREATE FUNCTION sql_syntax_error() RETURNS text
 SELECT sql_syntax_error();
 WARNING:  PL/Python: plpy.SPIError: unrecognized error in PLy_spi_execute_query
 CONTEXT:  PL/Python function "sql_syntax_error"
-ERROR:  syntax error at or near "syntax"
-LINE 1: syntax error
-        ^
-QUERY:  syntax error
+ERROR:  PL/Python: plpy.SPIError: syntax error at or near "syntax"
 CONTEXT:  PL/Python function "sql_syntax_error"
 /* check the handling of uncaught python exceptions
  */

#7: This is unnecessary because the remaining fields are automatically
initialized with NULL. The Python documentation uses initializations
like that. The way I understand it, newer Python versions might add
more fields at the end, and they will rely on the fact that they get
automatically initialized even if the source code was for an older
version. So I would rather not touch this, as it doesn't buy anything.

#16: This is probably pointless because pgindent will reformat this.

#8Hitoshi Harada
umi.tanuki@gmail.com
In reply to: Peter Eisentraut (#7)
Re: pl/python refactoring

2011/1/19 Peter Eisentraut <peter_e@gmx.net>:

On mån, 2011-01-17 at 21:49 +0200, Peter Eisentraut wrote:

On sön, 2011-01-02 at 12:41 +0100, Jan Urbański wrote:

Here they are. There are 16 patches in total. They amount to what's
currently in my refactor branch (and almost to what I've sent as the
complete refactor patch, while doing the splitting I discovered a few
useless hunks that I've discarded).

Committed 0001, rest later.

Today committed: 3, 5, 10, 11, 12, 13

I have reviewed this original patches for myself as the fundamental of
subsequent work, and have comments.

- This is not in the patch, but around line 184 "vis versa" in comment
seems like typo.
- -1 is used as the initial value of PLyTypeInfo.is_rowtype. Why not 0?
- PLy_(input|output)_tuple_funcs() in PLy_trigger_handler() is added.
The comment says it should check for the possibility that the
relation's tupdesc changed since last call. Is it true that tupdesc
may change even in one statement? And it also says the two functions
are responsible for not doing repetitive work, but ISTM they are not
doing something to stop redundant work. The function doesn't seem like
lightweight enough to be called on each row.
- There are elog() and PLy_elog() overall, but it looks like to me
that the usage is inconsistent. Moreover, elog(ERROR, "PLyTypeInfo
struct is initialized for a Datum"); in
PLy_(input|output)_tuple_funcs() should be Assert() not elog()?
- A line break should be added before PLy_add_exception() after "static void"
- This is also not in the patch, but the comment
/* these should only be called once at the first call
* of plpython_call_handler. initialize the python interpreter
* and global data.
*/
is bogus. PLy_init_interp() is called in _PG_init().

That's all for now. Some of them must be my misunderstanding or
trivial, but I post them for the record.

Regards,

--
Hitoshi Harada

#9Alvaro Herrera
alvherre@commandprompt.com
In reply to: Peter Eisentraut (#7)
Re: pl/python refactoring

Excerpts from Peter Eisentraut's message of mar ene 18 19:22:50 -0300 2011:

#16: This is probably pointless because pgindent will reformat this.

pgindent used to remove useless braces around single-statement blocks,
but this behavior was removed years ago because it'd break formatting
around PG_TRY blocks.

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

#10Jan Urbański
wulczer@wulczer.org
In reply to: Peter Eisentraut (#7)
Re: pl/python refactoring

On 18/01/11 23:22, Peter Eisentraut wrote:

On mån, 2011-01-17 at 21:49 +0200, Peter Eisentraut wrote:

On sön, 2011-01-02 at 12:41 +0100, Jan Urbański wrote:

Here they are. There are 16 patches in total. They amount to what's
currently in my refactor branch (and almost to what I've sent as the
complete refactor patch, while doing the splitting I discovered a few
useless hunks that I've discarded).

Committed 0001, rest later.

Today committed: 3, 5, 10, 11, 12, 13

Thanks,

#2: It looks like this loses some information/formatting in the error
message. Should we keep the pointing arrow there?

--- a/src/pl/plpython/expected/plpython_error.out
+++ b/src/pl/plpython/expected/plpython_error.out
@@ -10,10 +10,7 @@ CREATE FUNCTION sql_syntax_error() RETURNS text
SELECT sql_syntax_error();
WARNING:  PL/Python: plpy.SPIError: unrecognized error in PLy_spi_execute_query
CONTEXT:  PL/Python function "sql_syntax_error"
-ERROR:  syntax error at or near "syntax"
-LINE 1: syntax error
-        ^
-QUERY:  syntax error
+ERROR:  PL/Python: plpy.SPIError: syntax error at or near "syntax"
CONTEXT:  PL/Python function "sql_syntax_error"
/* check the handling of uncaught python exceptions
*/

Yes, the message is less informative, because the error is reported by
PL/Python, by wrapping the SPI message. I guess I could try to extract
more info from the caught ErrorData and put it in the new error that
PL/Python throws. But I'd like to do that as a refinement of the
spi-in-subxacts patch, because the current behaviour is broken anyway -
to catch the SPI error safely you need to wrap the whole thing in a
subtransaction. This patch only gets rid of the global error state, and
to do that I needed to raise some exception from the try/catch block
around SPI calls.

#7: This is unnecessary because the remaining fields are automatically
initialized with NULL. The Python documentation uses initializations
like that. The way I understand it, newer Python versions might add
more fields at the end, and they will rely on the fact that they get
automatically initialized even if the source code was for an older
version. So I would rather not touch this, as it doesn't buy anything.

OK.

#16: This is probably pointless because pgindent will reformat this.

If it does, than it's a shame. Maybe the comment should be moved above
the if() then.

Cheers,
Jan

#11Jan Urbański
wulczer@wulczer.org
In reply to: Hitoshi Harada (#8)
Re: pl/python refactoring

On 19/01/11 02:06, Hitoshi Harada wrote:

2011/1/19 Peter Eisentraut <peter_e@gmx.net>:

On mån, 2011-01-17 at 21:49 +0200, Peter Eisentraut wrote:

On sön, 2011-01-02 at 12:41 +0100, Jan Urbański wrote:

Here they are. There are 16 patches in total. They amount to what's
currently in my refactor branch (and almost to what I've sent as the
complete refactor patch, while doing the splitting I discovered a few
useless hunks that I've discarded).

Committed 0001, rest later.

Today committed: 3, 5, 10, 11, 12, 13

I have reviewed this original patches for myself as the fundamental of
subsequent work, and have comments.

- This is not in the patch, but around line 184 "vis versa" in comment
seems like typo.

Right, should certainly be "vice versa".

- -1 is used as the initial value of PLyTypeInfo.is_rowtype. Why not 0?

See the comments in struct PLyTypeInfo:

is_rowtype can be: -1 = not known yet (initial state); 0 = scalar
datatype; 1 = rowtype; 2 = rowtype, but I/O functions not set up yet

- PLy_(input|output)_tuple_funcs() in PLy_trigger_handler() is added.
The comment says it should check for the possibility that the
relation's tupdesc changed since last call. Is it true that tupdesc
may change even in one statement? And it also says the two functions
are responsible for not doing repetitive work, but ISTM they are not
doing something to stop redundant work. The function doesn't seem like
lightweight enough to be called on each row.

Hm, you may be right. I haven't touched that part of the code, but now
it seems to me that for triggers we do the I/O funcs lookup for every
row. I could try to check that and fix it, but not sure if I'll have the
time, and it's been that way before. Also, the CF is already closed in
theory...

- There are elog() and PLy_elog() overall, but it looks like to me
that the usage is inconsistent. Moreover, elog(ERROR, "PLyTypeInfo
struct is initialized for a Datum"); in
PLy_(input|output)_tuple_funcs() should be Assert() not elog()?

Well in theory PLy_elog should be used if there's a current Python
exception set that you'd like to forward to Postgres, making it a
elog(ERROR). It's true that the usage is sometimes a bit inconsistent,
some of these inconsistencies are cleaned up in the next patches, but
probably not all of them. As for the Assert/elog, I would prefer en
elog, because if there are bugs in that code, using the wrong I/O
functions could lead to unexpected results i production (where an Assert
would not be present).

- A line break should be added before PLy_add_exception() after "static void"

Oops, you're right.

- This is also not in the patch, but the comment
/* these should only be called once at the first call
* of plpython_call_handler. initialize the python interpreter
* and global data.
*/
is bogus. PLy_init_interp() is called in _PG_init().

Yep, that comment looks misguided.

That's all for now. Some of them must be my misunderstanding or
trivial, but I post them for the record.

Thanks, your feedback it's very valuable!

Cheers,
Jan

#12Hitoshi Harada
umi.tanuki@gmail.com
In reply to: Jan Urbański (#11)
Re: pl/python refactoring

2011/1/19 Jan Urbański <wulczer@wulczer.org>:

On 19/01/11 02:06, Hitoshi Harada wrote:

- -1 is used as the initial value of PLyTypeInfo.is_rowtype. Why not 0?

See the comments in struct PLyTypeInfo:

is_rowtype can be: -1 = not known yet (initial state); 0 = scalar
datatype; 1 = rowtype; 2 = rowtype, but I/O functions not set up yet

Well, I mean that 0 = inital, 1 = scalar, 2 = rowtype, 3 = rowtype but
not set up sounds more intuitive to me. Of course this is only what I
feel.

Regards,

--
Hitoshi Harada

#13David Fetter
david@fetter.org
In reply to: Jan Urbański (#11)
Re: pl/python refactoring

On Wed, Jan 19, 2011 at 11:09:56AM +0100, Jan Urbański wrote:

On 19/01/11 02:06, Hitoshi Harada wrote:

2011/1/19 Peter Eisentraut <peter_e@gmx.net>:

On mån, 2011-01-17 at 21:49 +0200, Peter Eisentraut wrote:

On sön, 2011-01-02 at 12:41 +0100, Jan Urbański wrote:

Here they are. There are 16 patches in total. They amount to what's
currently in my refactor branch (and almost to what I've sent as the
complete refactor patch, while doing the splitting I discovered a few
useless hunks that I've discarded).

Committed 0001, rest later.

Today committed: 3, 5, 10, 11, 12, 13

I have reviewed this original patches for myself as the fundamental of
subsequent work, and have comments.

- This is not in the patch, but around line 184 "vis versa" in comment
seems like typo.

Right, should certainly be "vice versa".

- -1 is used as the initial value of PLyTypeInfo.is_rowtype. Why not 0?

See the comments in struct PLyTypeInfo:

is_rowtype can be: -1 = not known yet (initial state); 0 = scalar
datatype; 1 = rowtype; 2 = rowtype, but I/O functions not set up yet

- PLy_(input|output)_tuple_funcs() in PLy_trigger_handler() is added.
The comment says it should check for the possibility that the
relation's tupdesc changed since last call. Is it true that tupdesc
may change even in one statement? And it also says the two functions
are responsible for not doing repetitive work, but ISTM they are not
doing something to stop redundant work. The function doesn't seem like
lightweight enough to be called on each row.

Hm, you may be right. I haven't touched that part of the code, but now
it seems to me that for triggers we do the I/O funcs lookup for every
row. I could try to check that and fix it, but not sure if I'll have the
time, and it's been that way before. Also, the CF is already closed in
theory...

If you're fixing things in PL/PythonU, such a change would certainly
be in scope as an update to your patch, i.e. don't let the fact that
the CF has started stop you from fixing it.

Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

#14Jan Urbański
wulczer@wulczer.org
In reply to: David Fetter (#13)
Re: pl/python refactoring

On 19/01/11 16:35, David Fetter wrote:

On Wed, Jan 19, 2011 at 11:09:56AM +0100, Jan Urbański wrote:

On 19/01/11 02:06, Hitoshi Harada wrote:

- PLy_(input|output)_tuple_funcs() in PLy_trigger_handler() is added.
The comment says it should check for the possibility that the
relation's tupdesc changed since last call. Is it true that tupdesc
may change even in one statement? And it also says the two functions
are responsible for not doing repetitive work, but ISTM they are not
doing something to stop redundant work. The function doesn't seem like
lightweight enough to be called on each row.

Hm, you may be right. I haven't touched that part of the code, but now
it seems to me that for triggers we do the I/O funcs lookup for every
row. I could try to check that and fix it, but not sure if I'll have the
time, and it's been that way before. Also, the CF is already closed in
theory...

I looked again and it seems that PLy_(input|output)_tuple_funcs does
indeed avoid repetitive work. Observe that in the loop going over the
TupleDesc attributes there's

if (arg->in.r.atts[i].typoid == desc->attrs[i]->atttypid)
continue; /* already set up this entry */

which avoids the syscache lookup and calling PLy_input_datum_func2.

Cheers,
Jan

#15Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#9)
Re: pl/python refactoring

Alvaro Herrera <alvherre@commandprompt.com> writes:

Excerpts from Peter Eisentraut's message of mar ene 18 19:22:50 -0300 2011:

#16: This is probably pointless because pgindent will reformat this.

pgindent used to remove useless braces around single-statement blocks,
but this behavior was removed years ago because it'd break formatting
around PG_TRY blocks.

Yeah. FWIW, I concur with Jan that this is a readability improvement.
A comment and a statement always look like two statements to me ---
and the fact that pgindent insists on inserting a blank line in front
of the comment in this scenario does even more to mangle the visual
impression of what the "if" controls. +1 for the braces.

regards, tom lane

#16Peter Eisentraut
peter_e@gmx.net
In reply to: Alvaro Herrera (#9)
Re: pl/python refactoring

On ons, 2011-01-19 at 00:52 -0300, Alvaro Herrera wrote:

Excerpts from Peter Eisentraut's message of mar ene 18 19:22:50 -0300 2011:

#16: This is probably pointless because pgindent will reformat this.

pgindent used to remove useless braces around single-statement blocks,
but this behavior was removed years ago because it'd break formatting
around PG_TRY blocks.

Good to know. Committed then.

#17Robert Haas
robertmhaas@gmail.com
In reply to: Peter Eisentraut (#16)
Re: pl/python refactoring

On Wed, Jan 19, 2011 at 2:59 PM, Peter Eisentraut <peter_e@gmx.net> wrote:

On ons, 2011-01-19 at 00:52 -0300, Alvaro Herrera wrote:

Excerpts from Peter Eisentraut's message of mar ene 18 19:22:50 -0300 2011:

#16: This is probably pointless because pgindent will reformat this.

pgindent used to remove useless braces around single-statement blocks,
but this behavior was removed years ago because it'd break formatting
around PG_TRY blocks.

Good to know.  Committed then.

I cracked up upon reading your commit message.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

#18Jan Urbański
wulczer@wulczer.org
In reply to: Jan Urbański (#10)
Re: pl/python refactoring

On 19/01/11 10:57, Jan Urbański wrote:

On 18/01/11 23:22, Peter Eisentraut wrote:

#2: It looks like this loses some information/formatting in the error
message. Should we keep the pointing arrow there?

CONTEXT:  PL/Python function "sql_syntax_error"
-ERROR:  syntax error at or near "syntax"
-LINE 1: syntax error
-        ^
-QUERY:  syntax error
+ERROR:  PL/Python: plpy.SPIError: syntax error at or near "syntax"
CONTEXT:  PL/Python function "sql_syntax_error"

Yes, the message is less informative, because the error is reported by
PL/Python, by wrapping the SPI message. I guess I could try to extract
more info from the caught ErrorData and put it in the new error that
PL/Python throws.

All right, I found a way to shoehorn the extra information into
SPIException, I'll post a new patch series with what's left of the
general refactoring patch soon.

Shortly after, I'll post updated patches for doing SPI in subxacts,
explicit subxact starting and generated SPI exceptions, as they will
surely be broken by these changes.

Jan

#19Jan Urbański
wulczer@wulczer.org
In reply to: Jan Urbański (#18)
1 attachment(s)
Re: pl/python refactoring

On 20/01/11 01:26, Jan Urbański wrote:

On 19/01/11 10:57, Jan Urbański wrote:

On 18/01/11 23:22, Peter Eisentraut wrote:

#2: It looks like this loses some information/formatting in the error
message. Should we keep the pointing arrow there?

CONTEXT:  PL/Python function "sql_syntax_error"
-ERROR:  syntax error at or near "syntax"
-LINE 1: syntax error
-        ^
-QUERY:  syntax error
+ERROR:  PL/Python: plpy.SPIError: syntax error at or near "syntax"
CONTEXT:  PL/Python function "sql_syntax_error"

Yes, the message is less informative, because the error is reported by
PL/Python, by wrapping the SPI message. I guess I could try to extract
more info from the caught ErrorData and put it in the new error that
PL/Python throws.

All right, I found a way to shoehorn the extra information into
SPIException, I'll post a new patch series with what's left of the
general refactoring patch soon.

Here's an updated patch series for PL/Python refactoring. It was 16
patches at first, 8 are committed, 1 got dropped, so we're down to 7.

Cheers,
Jan

Attachments:

0007-Do-not-prefix-error-messages-with-the-string-PL-Pyth.patchtext/x-patch; name=0007-Do-not-prefix-error-messages-with-the-string-PL-Pyth.patchDownload
#20Peter Eisentraut
peter_e@gmx.net
In reply to: Hitoshi Harada (#8)
Re: pl/python refactoring

On ons, 2011-01-19 at 10:06 +0900, Hitoshi Harada wrote:

- This is not in the patch, but around line 184 "vis versa" in comment
seems like typo.

Fixed.

- A line break should be added before PLy_add_exception() after "static void"

I'll add that when I get to the patch.

- This is also not in the patch, but the comment
/* these should only be called once at the first call
* of plpython_call_handler. initialize the python interpreter
* and global data.
*/
is bogus. PLy_init_interp() is called in _PG_init().

Fixed.

#21Peter Eisentraut
peter_e@gmx.net
In reply to: Jan Urbański (#19)
Re: pl/python refactoring

On tor, 2011-01-20 at 03:16 +0100, Jan Urbański wrote:

Here's an updated patch series for PL/Python refactoring. It was 16
patches at first, 8 are committed, 1 got dropped, so we're down to 7.

Refactor PLy_spi_prepare to save two levels of indentation.

Instead of checking if the arglist is NULL and then if its length is
0, do it in one step, and outside of the try/catch block.

Why is it a good idea to do the PLy_malloc calls outside of the
try/catch block? Also, why call them when nargs is 0?

#22Jan Urbański
wulczer@wulczer.org
In reply to: Peter Eisentraut (#21)
Re: pl/python refactoring

On 22/01/11 21:53, Peter Eisentraut wrote:

On tor, 2011-01-20 at 03:16 +0100, Jan Urbański wrote:

Here's an updated patch series for PL/Python refactoring. It was 16
patches at first, 8 are committed, 1 got dropped, so we're down to 7.

Refactor PLy_spi_prepare to save two levels of indentation.

Instead of checking if the arglist is NULL and then if its length is
0, do it in one step, and outside of the try/catch block.

Why is it a good idea to do the PLy_malloc calls outside of the
try/catch block? Also, why call them when nargs is 0?

I think it's better to call them outside of the try/catch, because if
palloc failed, we'de be better off just interrupting the function
execution and raising an error than catching the longjmp and turning it
into a Python exception, which will probably make Python fail with a
MemoryError really soon. And I've even seen Python segfaulting when it
ran out of memory instead of raising a MemoryError exception.

As for the nargs == 0 case, you're right, it should read

plan->types = nargs ? PLy_malloc(sizeof(Oid) * nargs) : NULL;

especially since PLy_plan_dealloc does

if (ob->types)
PLy_free(ob->types)

Cheers,
Jan

#23Peter Eisentraut
peter_e@gmx.net
In reply to: Jan Urbański (#19)
Re: pl/python refactoring

On tor, 2011-01-20 at 03:16 +0100, Jan Urbański wrote:

Here's an updated patch series for PL/Python refactoring. It was 16
patches at first, 8 are committed, 1 got dropped, so we're down to 7.

Everything(*) is now committed.

In 0006-Improve-exception-usage-in-PL-Python.patch I went for TypeError
instead of ValueError because that matched better with the behavior of
some Python built-ins. Same idea, though.

(*) In 0007-Do-not-prefix-error-messages-with-the-string-PL-Pyth.patch,
I did not commit the bit that moved pg_verifymbstr outside the TRY
block. This is debatable. I observe that there are other uses of
pg_verifymbstr in TRY blocks. Also, we document that strings in Python
code must be in the server encoding, so I would argue that this error
could be considered a Python error and thus the current code would be
correct.

#24Jan Urbański
wulczer@wulczer.org
In reply to: Peter Eisentraut (#23)
Re: pl/python refactoring

On 27/01/11 00:40, Peter Eisentraut wrote:

On tor, 2011-01-20 at 03:16 +0100, Jan Urbański wrote:

Here's an updated patch series for PL/Python refactoring. It was 16
patches at first, 8 are committed, 1 got dropped, so we're down to 7.

Everything(*) is now committed.

Great, thanks.

I'll be posting updated versions of the remaining patches to their
corresponding threads (or their review threads, if they already exist).

Jan