Plpython crashing the backend in one easy step
I'm going to look into this, but thought I'd share the misery:
CREATE FUNCTION crash() RETURNS varchar AS '
plpy.execute("syntax error")
' language 'plpython';
SELECT crash();
-Brad
I need some expert guidance here. Suppose you have:
CREATE FUNCTION crash() RETURNS varchar AS '
plpy.execute("syntax error")
' language 'plpython';
Here are three possible behaviors:
(A)
a123=# select crash();
ERROR: parser: parse error at or near "syntax"
ERROR: plpython: Call of function `__plpython_procedure_crash_41133' failed.
plpy.SPIError: Unknown error in PLy_spi_execute_query.
FATAL 2: elog: error during error recovery, giving up!
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!#
(B)
a123=# select crash();
ERROR: parser: parse error at or near "syntax"
a123=#
(C)
a123=# select crash();
ERROR: parser: parse error at or near "syntax"
ERROR: plpython: Call of function `__plpython_procedure_crash_41133' failed.
plpy.SPIError: Unknown error in PLy_spi_execute_query.
a123=#
Option (A) is the current code.
Fixing this happens near line 2290 (could be off a bit, I have some other
patches in this file), in the first if clause in PLy_spi_execute_query.
The DECLARE_EXC, SAVE_EXC, TRAP_EXC, RESTORE_EXC, RERAISE_EXC macros
are wrappers around sigsetjmp and longjmp, and are used to intercept
the elog calls occurring when plpython calls spi.
In Option (A), we return NULL, which causes the next level of code
to call elog. Elog notices that we're already in an Error state, and
shuts down the backend.
In Option (B), I replace the 'return NULL' with a RERAISE_EXC, which
allows elog to function normally, albeit without any cleanup of the
plpython environment or useful messages.
In Option (C), I set the global "InError" flag to false, and then
return NULL, causing all of the error messages to come out and
plpython to clean up gracefully, no backend crash. However, this
seems to be an unprecedented approach, and I could be missing
something big.
There's probably an Option (D) that I'm overlooking.
HELP! (thanks)
-Brad
Bradley McLean <brad@bradm.net> writes:
In Option (C), I set the global "InError" flag to false, and then
return NULL, causing all of the error messages to come out and
plpython to clean up gracefully, no backend crash. However, this
seems to be an unprecedented approach, and I could be missing
something big.
Yes, as in "it's totally unsafe". Suppressing an elog(ERROR) is
a *big* no-no at present, because way too much stuff relies on
post-abort cleanup to clean up whatever problem is being reported.
You cannot allow the transaction to continue after the error, and
you most certainly mustn't cavalierly reset the error handling state.
The only things you should be doing with longjmp trapping are
(a) doing any cleanup that Python itself has to have before you
re-propagate the longjmp, or
(b) issuing elog(NOTICE) to help identify the error location
before you re-propagate the longjmp.
plpgsql contains an example of doing (b).
Not propagating the longjmp, which is what the code seems to be doing
at present, is not acceptable. I had not looked at this code closely,
but as-is it is a huge reliability hazard. I will insist on removing
plpython from 7.2 entirely if this can't be fixed before release.
regards, tom lane
Thanks, I assumed something like that.
I can very quickly provide at implementation that meets
these criteria (It's my option "b"), but it may be
less informative in terms of messages.
I'll have a look to see what I can do to supplement
the messages.
I also need to check if there's a memory leak after I
fix it.
I was aware of the example in plpgsql.
-Brad
* Tom Lane (tgl@sss.pgh.pa.us) [011113 17:08]:
Show quoted text
Yes, as in "it's totally unsafe". Suppressing an elog(ERROR) is
a *big* no-no at present, because way too much stuff relies on
post-abort cleanup to clean up whatever problem is being reported.
You cannot allow the transaction to continue after the error, and
you most certainly mustn't cavalierly reset the error handling state.The only things you should be doing with longjmp trapping are
(a) doing any cleanup that Python itself has to have before you
re-propagate the longjmp, or
(b) issuing elog(NOTICE) to help identify the error location
before you re-propagate the longjmp.plpgsql contains an example of doing (b).
Not propagating the longjmp, which is what the code seems to be doing
at present, is not acceptable. I had not looked at this code closely,
but as-is it is a huge reliability hazard. I will insist on removing
plpython from 7.2 entirely if this can't be fixed before release.regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly
Okay, the attached patch contains the following:
- Kevin Jacob's patch to make plpython worthy of a 'trusted' language
by eliminating the ability to arbitrarily read OS files. He also did
a bunch of C declaration cleanups. (His patch has yet to appear on
either hackers or patches, and I believe is also a prerequisite for
plpython in 7.2 final).
- Changed every place that catches elog() longjmps to re-propogate.
- Added code to keep track of the current plpython function and add
it to the elog messages
- #ifdef 0 around some items that appear to be not in use.
- changes to the expected output of the regression tests, because:
Tom's requirement that we always reraise the longjmp is at odds with
the original design of plpython, which attempted to allow a plpython
function to use Python's exception handling to recover when an
embedded SPI call failed. There is no way to do this that I can
see without halting the propogation of the longjmp.
Thus, the semantics of 'execute' calls from plpython have been changed:
should the backend throw an error, the plpython function will be
summarily aborted. Previously, it could attempt to catch a python
exception.
Note that this does create some redundant exception handling paths
in upper level methods within this module that are no longer used.
I have not yet removed them, but will happily do so (in a careful
deliberate manner) once sure that there is no alternative way to
restore the python level exception handling.
Let me know if this isn't what's needed to get this up to snuff
for 7.2.
-Brad
* Tom Lane (tgl@sss.pgh.pa.us) [011113 16:49]:
Show quoted text
Yes, as in "it's totally unsafe". Suppressing an elog(ERROR) is
a *big* no-no at present, because way too much stuff relies on
post-abort cleanup to clean up whatever problem is being reported.
You cannot allow the transaction to continue after the error, and
you most certainly mustn't cavalierly reset the error handling state.The only things you should be doing with longjmp trapping are
(a) doing any cleanup that Python itself has to have before you
re-propagate the longjmp, or
(b) issuing elog(NOTICE) to help identify the error location
before you re-propagate the longjmp.plpgsql contains an example of doing (b).
Not propagating the longjmp, which is what the code seems to be doing
at present, is not acceptable. I had not looked at this code closely,
but as-is it is a huge reliability hazard. I will insist on removing
plpython from 7.2 entirely if this can't be fixed before release.regards, tom lane
Attachments:
plpython.patchtext/plain; charset=us-asciiDownload
Index: src/pl/plpython/error.expected
===================================================================
RCS file: /projects/cvsroot/pgsql/src/pl/plpython/error.expected,v
retrieving revision 1.1
diff -c -r1.1 error.expected
*** src/pl/plpython/error.expected 2001/05/09 19:54:38 1.1
--- src/pl/plpython/error.expected 2001/11/15 18:38:01
***************
*** 1,19 ****
SELECT invalid_type_uncaught('rick');
! ERROR: plpython: Call of function `__plpython_procedure_invalid_type_uncaught_1175341' failed.
plpy.SPIError: Cache lookup for type `test' failed.
SELECT invalid_type_caught('rick');
! NOTICE: ("Cache lookup for type `test' failed.",)
! invalid_type_caught
! ---------------------
!
! (1 row)
!
SELECT invalid_type_reraised('rick');
! ERROR: plpython: Call of function `__plpython_procedure_invalid_type_reraised_1175343' failed.
! plpy.Error: ("Cache lookup for type `test' failed.",)
SELECT valid_type('rick');
valid_type
------------
(1 row)
--- 1,36 ----
SELECT invalid_type_uncaught('rick');
! NOTICE: plpython: in function __plpython_procedure_invalid_type_uncaught_49801:
plpy.SPIError: Cache lookup for type `test' failed.
SELECT invalid_type_caught('rick');
! NOTICE: plpython: in function __plpython_procedure_invalid_type_caught_49802:
! plpy.SPIError: Cache lookup for type `test' failed.
SELECT invalid_type_reraised('rick');
! NOTICE: plpython: in function __plpython_procedure_invalid_type_reraised_49803:
! plpy.SPIError: Cache lookup for type `test' failed.
SELECT valid_type('rick');
valid_type
------------
+ (1 row)
+
+ SELECT read_file('/etc/passwd');
+ ERROR: plpython: Call of function `__plpython_procedure_read_file_49809' failed.
+ exceptions.IOError: can't open files in restricted mode
+ SELECT write_file('/tmp/plpython','This is very bad');
+ ERROR: plpython: Call of function `__plpython_procedure_write_file_49810' failed.
+ exceptions.IOError: can't open files in restricted mode
+ SELECT getpid();
+ ERROR: plpython: Call of function `__plpython_procedure_getpid_49811' failed.
+ exceptions.AttributeError: getpid
+ SELECT uname();
+ ERROR: plpython: Call of function `__plpython_procedure_uname_49812' failed.
+ exceptions.AttributeError: uname
+ SELECT sys_exit();
+ ERROR: plpython: Call of function `__plpython_procedure_sys_exit_49813' failed.
+ exceptions.AttributeError: exit
+ SELECT sys_argv();
+ sys_argv
+ ----------------
+ ['RESTRICTED']
(1 row)
Index: src/pl/plpython/plpython.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/pl/plpython/plpython.c,v
retrieving revision 1.12
diff -c -r1.12 plpython.c
*** src/pl/plpython/plpython.c 2001/11/05 17:46:39 1.12
--- src/pl/plpython/plpython.c 2001/11/15 18:38:03
***************
*** 188,193 ****
--- 188,197 ----
static void PLy_init_safe_interp(void);
static void PLy_init_plpy(void);
+ /* Helper functions used during initialization */
+ static int populate_methods(PyObject *klass, PyMethodDef *methods);
+ static PyObject *build_tuple(char* string_list[], int len);
+
/* error handler. collects the current Python exception, if any,
* and appends it to the error and sends it to elog
*/
***************
*** 199,204 ****
--- 203,212 ----
PLy_exception_set(PyObject *, const char *,...)
__attribute__((format(printf, 2, 3)));
+ /* Get the innermost python procedure called from the backend.
+ */
+ static char *PLy_procedure_name(PLyProcedure *);
+
/* some utility functions
*/
static void *PLy_malloc(size_t);
***************
*** 240,245 ****
--- 248,257 ----
static void PLy_output_tuple_funcs(PLyTypeInfo *, TupleDesc);
static void PLy_input_tuple_funcs(PLyTypeInfo *, TupleDesc);
+ /* RExec methods
+ */
+ static PyObject *PLy_r_open(PyObject *self, PyObject* args);
+
/* conversion functions
*/
static PyObject *PLyDict_FromTuple(PLyTypeInfo *, HeapTuple, TupleDesc);
***************
*** 255,260 ****
--- 267,277 ----
static int PLy_first_call = 1;
static volatile int PLy_call_level = 0;
+ /*
+ * Last function called by postgres backend
+ */
+ static PLyProcedure *PLy_last_procedure = NULL;
+
/* this gets modified in plpython_call_handler and PLy_elog.
* test it any old where, but do NOT modify it anywhere except
* those two functions
***************
*** 265,275 ****
static PyObject *PLy_interp_safe = NULL;
static PyObject *PLy_interp_safe_globals = NULL;
static PyObject *PLy_importable_modules = NULL;
static PyObject *PLy_procedure_cache = NULL;
! char *PLy_importable_modules_list[] = {
"array",
"bisect",
"calendar",
"cmath",
"errno",
--- 282,295 ----
static PyObject *PLy_interp_safe = NULL;
static PyObject *PLy_interp_safe_globals = NULL;
static PyObject *PLy_importable_modules = NULL;
+ static PyObject *PLy_ok_posix_names = NULL;
+ static PyObject *PLy_ok_sys_names = NULL;
static PyObject *PLy_procedure_cache = NULL;
! static char *PLy_importable_modules_list[] = {
"array",
"bisect",
+ "binascii",
"calendar",
"cmath",
"errno",
***************
*** 278,299 ****
"md5",
"mpz",
"operator",
"pickle",
"random",
"re",
"sha",
"string",
"StringIO",
"time",
"whrandom",
"zlib"
};
/* Python exceptions
*/
! PyObject *PLy_exc_error = NULL;
! PyObject *PLy_exc_fatal = NULL;
! PyObject *PLy_exc_spi_error = NULL;
/* some globals for the python module
*/
--- 298,339 ----
"md5",
"mpz",
"operator",
+ "pcre",
"pickle",
"random",
"re",
+ "regex",
+ "sre",
"sha",
"string",
"StringIO",
+ "struct",
"time",
"whrandom",
"zlib"
};
+ static char *PLy_ok_posix_names_list[] = {
+ /* None for now */
+ };
+
+ static char *PLy_ok_sys_names_list[] = {
+ "byteeorder",
+ "copyright",
+ "getdefaultencoding",
+ "getrefcount",
+ "hexrevision",
+ "maxint",
+ "maxunicode",
+ "version",
+ "version_info"
+ };
+
/* Python exceptions
*/
! static PyObject *PLy_exc_error = NULL;
! static PyObject *PLy_exc_fatal = NULL;
! static PyObject *PLy_exc_spi_error = NULL;
/* some globals for the python module
*/
***************
*** 334,340 ****
fmgr_info_cxt(functionId, finfo, TopMemoryContext);
}
-
Datum
plpython_call_handler(PG_FUNCTION_ARGS)
{
--- 374,379 ----
***************
*** 366,373 ****
}
else
PLy_restart_in_progress += 1;
! if (proc)
Py_DECREF(proc->me);
RERAISE_EXC();
}
--- 405,414 ----
}
else
PLy_restart_in_progress += 1;
! if (proc)
! {
Py_DECREF(proc->me);
+ }
RERAISE_EXC();
}
***************
*** 853,863 ****
--- 894,908 ----
PLy_procedure_call(PLyProcedure * proc, char *kargs, PyObject * vargs)
{
PyObject *rv;
+ PLyProcedure *current;
enter();
+ current = PLy_last_procedure;
+ PLy_last_procedure = proc;
PyDict_SetItemString(proc->globals, kargs, vargs);
rv = PyObject_CallFunction(proc->reval, "O", proc->code);
+ PLy_last_procedure = current;
if ((rv == NULL) || (PyErr_Occurred()))
{
***************
*** 1150,1161 ****
if ((proc->interp == NULL) || (PyErr_Occurred()))
PLy_elog(ERROR, "Unable to create rexec.RExec instance");
- /*
- * tweak the list of permitted modules
- */
- PyObject_SetAttrString(proc->interp, "ok_builtin_modules",
- PLy_importable_modules);
-
proc->reval = PyObject_GetAttrString(proc->interp, "r_eval");
if ((proc->reval == NULL) || (PyErr_Occurred()))
PLy_elog(ERROR, "Unable to get method `r_eval' from rexec.RExec");
--- 1195,1200 ----
***************
*** 1632,1640 ****
--- 1671,1682 ----
static PyObject *PLy_result_new(void);
static void PLy_result_dealloc(PyObject *);
static PyObject *PLy_result_getattr(PyObject *, char *);
+ #if 0
+ /* Appear to be unused */
static PyObject *PLy_result_fetch(PyObject *, PyObject *);
static PyObject *PLy_result_nrows(PyObject *, PyObject *);
static PyObject *PLy_result_status(PyObject *, PyObject *);
+ #endif
static int PLy_result_length(PyObject *);
static PyObject *PLy_result_item(PyObject *, int);
static PyObject *PLy_result_slice(PyObject *, int, int);
***************
*** 1650,1656 ****
static PyObject *PLy_spi_execute_fetch_result(SPITupleTable *, int, int);
! PyTypeObject PLy_PlanType = {
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
"PLyPlan", /* tp_name */
--- 1692,1698 ----
static PyObject *PLy_spi_execute_fetch_result(SPITupleTable *, int, int);
! static PyTypeObject PLy_PlanType = {
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
"PLyPlan", /* tp_name */
***************
*** 1679,1691 ****
PLy_plan_doc, /* tp_doc */
};
! PyMethodDef PLy_plan_methods[] = {
{"status", (PyCFunction) PLy_plan_status, METH_VARARGS, NULL},
{NULL, NULL, 0, NULL}
};
! PySequenceMethods PLy_result_as_sequence = {
(inquiry) PLy_result_length, /* sq_length */
(binaryfunc) 0, /* sq_concat */
(intargfunc) 0, /* sq_repeat */
--- 1721,1733 ----
PLy_plan_doc, /* tp_doc */
};
! static PyMethodDef PLy_plan_methods[] = {
{"status", (PyCFunction) PLy_plan_status, METH_VARARGS, NULL},
{NULL, NULL, 0, NULL}
};
! static PySequenceMethods PLy_result_as_sequence = {
(inquiry) PLy_result_length, /* sq_length */
(binaryfunc) 0, /* sq_concat */
(intargfunc) 0, /* sq_repeat */
***************
*** 1695,1701 ****
(intintobjargproc) PLy_result_ass_slice, /* sq_ass_slice */
};
! PyTypeObject PLy_ResultType = {
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
"PLyResult", /* tp_name */
--- 1737,1743 ----
(intintobjargproc) PLy_result_ass_slice, /* sq_ass_slice */
};
! static PyTypeObject PLy_ResultType = {
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
"PLyResult", /* tp_name */
***************
*** 1723,1736 ****
0, /* tp_xxx4 */
PLy_result_doc, /* tp_doc */
};
!
! PyMethodDef PLy_result_methods[] = {
{"fetch", (PyCFunction) PLy_result_fetch, METH_VARARGS, NULL,},
{"nrows", (PyCFunction) PLy_result_nrows, METH_VARARGS, NULL},
{"status", (PyCFunction) PLy_result_status, METH_VARARGS, NULL},
{NULL, NULL, 0, NULL}
};
!
static PyMethodDef PLy_methods[] = {
/*
--- 1765,1779 ----
0, /* tp_xxx4 */
PLy_result_doc, /* tp_doc */
};
! #if 0
! /* Appear to be unused */
! static PyMethodDef PLy_result_methods[] = {
{"fetch", (PyCFunction) PLy_result_fetch, METH_VARARGS, NULL,},
{"nrows", (PyCFunction) PLy_result_nrows, METH_VARARGS, NULL},
{"status", (PyCFunction) PLy_result_status, METH_VARARGS, NULL},
{NULL, NULL, 0, NULL}
};
! #endif
static PyMethodDef PLy_methods[] = {
/*
***************
*** 1833,1839 ****
/* result object methods
*/
! static PyObject *
PLy_result_new(void)
{
PLyResultObject *ob;
--- 1876,1882 ----
/* result object methods
*/
! PyObject *
PLy_result_new(void)
{
PLyResultObject *ob;
***************
*** 1853,1859 ****
return (PyObject *) ob;
}
! static void
PLy_result_dealloc(PyObject * arg)
{
PLyResultObject *ob = (PLyResultObject *) arg;
--- 1896,1902 ----
return (PyObject *) ob;
}
! void
PLy_result_dealloc(PyObject * arg)
{
PLyResultObject *ob = (PLyResultObject *) arg;
***************
*** 1867,1885 ****
PyMem_DEL(ob);
}
! static PyObject *
PLy_result_getattr(PyObject * self, char *attr)
{
return NULL;
}
!
! static PyObject *
PLy_result_fetch(PyObject * self, PyObject * args)
{
return NULL;
}
! static PyObject *
PLy_result_nrows(PyObject * self, PyObject * args)
{
PLyResultObject *ob = (PLyResultObject *) self;
--- 1910,1929 ----
PyMem_DEL(ob);
}
! PyObject *
PLy_result_getattr(PyObject * self, char *attr)
{
return NULL;
}
! #if 0
! /* Appear to be unused */
! PyObject *
PLy_result_fetch(PyObject * self, PyObject * args)
{
return NULL;
}
! PyObject *
PLy_result_nrows(PyObject * self, PyObject * args)
{
PLyResultObject *ob = (PLyResultObject *) self;
***************
*** 1888,1894 ****
return ob->nrows;
}
! static PyObject *
PLy_result_status(PyObject * self, PyObject * args)
{
PLyResultObject *ob = (PLyResultObject *) self;
--- 1932,1938 ----
return ob->nrows;
}
! PyObject *
PLy_result_status(PyObject * self, PyObject * args)
{
PLyResultObject *ob = (PLyResultObject *) self;
***************
*** 1896,1902 ****
Py_INCREF(ob->status);
return ob->status;
}
!
int
PLy_result_length(PyObject * arg)
{
--- 1940,1946 ----
Py_INCREF(ob->status);
return ob->status;
}
! #endif
int
PLy_result_length(PyObject * arg)
{
***************
*** 1991,1997 ****
if (!PyErr_Occurred())
PyErr_SetString(PLy_exc_spi_error,
"Unknown error in PLy_spi_prepare.");
! return NULL;
}
if (list != NULL)
--- 2035,2042 ----
if (!PyErr_Occurred())
PyErr_SetString(PLy_exc_spi_error,
"Unknown error in PLy_spi_prepare.");
! PLy_elog(NOTICE,"in function %s:",PLy_procedure_name(PLy_last_procedure));
! RERAISE_EXC();
}
if (list != NULL)
***************
*** 2187,2193 ****
if (!PyErr_Occurred())
PyErr_SetString(PLy_exc_error,
"Unknown error in PLy_spi_execute_plan");
! return NULL;
}
if (nargs)
--- 2232,2239 ----
if (!PyErr_Occurred())
PyErr_SetString(PLy_exc_error,
"Unknown error in PLy_spi_execute_plan");
! PLy_elog(NOTICE,"in function %s:",PLy_procedure_name(PLy_last_procedure));
! RERAISE_EXC();
}
if (nargs)
***************
*** 2249,2264 ****
if (TRAP_EXC())
{
RESTORE_EXC();
-
if ((!PLy_restart_in_progress) && (!PyErr_Occurred()))
PyErr_SetString(PLy_exc_spi_error,
"Unknown error in PLy_spi_execute_query.");
! return NULL;
}
rv = SPI_exec(query, limit);
RESTORE_EXC();
-
if (rv < 0)
{
PLy_exception_set(PLy_exc_spi_error,
--- 2295,2309 ----
if (TRAP_EXC())
{
RESTORE_EXC();
if ((!PLy_restart_in_progress) && (!PyErr_Occurred()))
PyErr_SetString(PLy_exc_spi_error,
"Unknown error in PLy_spi_execute_query.");
! PLy_elog(NOTICE,"in function %s:",PLy_procedure_name(PLy_last_procedure));
! RERAISE_EXC();
}
rv = SPI_exec(query, limit);
RESTORE_EXC();
if (rv < 0)
{
PLy_exception_set(PLy_exc_spi_error,
***************
*** 2311,2317 ****
"Unknown error in PLy_spi_execute_fetch_result");
Py_DECREF(result);
PLy_typeinfo_dealloc(&args);
! return NULL;
}
if (rows)
--- 2356,2362 ----
"Unknown error in PLy_spi_execute_fetch_result");
Py_DECREF(result);
PLy_typeinfo_dealloc(&args);
! RERAISE_EXC();
}
if (rows)
***************
*** 2450,2462 ****
elog(ERROR, "Unable to init plpy.");
}
void
PLy_init_safe_interp(void)
{
! PyObject *rmod;
char *rname = "rexec";
! int i,
! imax;
enter();
--- 2495,2527 ----
elog(ERROR, "Unable to init plpy.");
}
+ /*
+ * New RExec methods
+ */
+
+ PyObject*
+ PLy_r_open(PyObject *self, PyObject* args)
+ {
+ PyErr_SetString(PyExc_IOError, "can't open files in restricted mode");
+ return NULL;
+ }
+
+
+ static PyMethodDef PLy_r_exec_methods[] = {
+ {"r_open", (PyCFunction)PLy_r_open, METH_VARARGS, NULL},
+ {NULL, NULL, 0, NULL}
+ };
+
+ /*
+ * Init new RExec
+ */
+
void
PLy_init_safe_interp(void)
{
! PyObject *rmod, *rexec, *rexec_dict;
char *rname = "rexec";
! int len;
enter();
***************
*** 2467,2488 ****
PyDict_SetItemString(PLy_interp_globals, rname, rmod);
PLy_interp_safe = rmod;
! imax = sizeof(PLy_importable_modules_list) / sizeof(char *);
! PLy_importable_modules = PyTuple_New(imax);
! for (i = 0; i < imax; i++)
! {
! PyObject *m = PyString_FromString(PLy_importable_modules_list[i]);
! PyTuple_SetItem(PLy_importable_modules, i, m);
! }
PLy_interp_safe_globals = PyDict_New();
if (PLy_interp_safe_globals == NULL)
PLy_elog(ERROR, "Unable to create shared global dictionary.");
}
/* the python interface to the elog function
* don't confuse these with PLy_elog
*/
--- 2532,2627 ----
PyDict_SetItemString(PLy_interp_globals, rname, rmod);
PLy_interp_safe = rmod;
! len = sizeof(PLy_importable_modules_list) / sizeof(char *);
! PLy_importable_modules = build_tuple(PLy_importable_modules_list, len);
! len = sizeof(PLy_ok_posix_names_list) / sizeof(char *);
! PLy_ok_posix_names = build_tuple(PLy_ok_posix_names_list, len);
!
! len = sizeof(PLy_ok_sys_names_list) / sizeof(char *);
! PLy_ok_sys_names = build_tuple(PLy_ok_sys_names_list, len);
PLy_interp_safe_globals = PyDict_New();
if (PLy_interp_safe_globals == NULL)
PLy_elog(ERROR, "Unable to create shared global dictionary.");
+ /*
+ * get an rexec.RExec class
+ */
+ rexec = PyDict_GetItemString(PyModule_GetDict(rmod), "RExec");
+
+ if (rexec == NULL || !PyClass_Check(rexec))
+ PLy_elog(ERROR, "Unable to get RExec object.");
+
+
+ rexec_dict = ((PyClassObject*)rexec)->cl_dict;
+
+ /*
+ * tweak the list of permitted modules, posix and sys functions
+ */
+ PyDict_SetItemString(rexec_dict, "ok_builtin_modules", PLy_importable_modules);
+ PyDict_SetItemString(rexec_dict, "ok_posix_names", PLy_ok_posix_names);
+ PyDict_SetItemString(rexec_dict, "ok_sys_names", PLy_ok_sys_names);
+
+ /*
+ * change the r_open behavior
+ */
+ if( populate_methods(rexec, PLy_r_exec_methods) )
+ PLy_elog(ERROR, "Failed to update RExec methods.");
}
+ /* Helper function to build tuples from string lists */
+ static
+ PyObject *build_tuple(char* string_list[], int len)
+ {
+ PyObject *tup = PyTuple_New(len);
+ int i;
+ for (i = 0; i < len; i++)
+ {
+ PyObject *m = PyString_FromString(string_list[i]);
+
+ PyTuple_SetItem(tup, i, m);
+ }
+ return tup;
+ }
+
+ /* Helper function for populating a class with method wrappers. */
+ static int
+ populate_methods(PyObject *klass, PyMethodDef *methods)
+ {
+ if (!klass || !methods)
+ return 0;
+
+ for ( ; methods->ml_name; ++methods) {
+
+ /* get a wrapper for the built-in function */
+ PyObject *func = PyCFunction_New(methods, NULL);
+ PyObject *meth;
+ int status;
+
+ if (!func)
+ return -1;
+
+ /* turn the function into an unbound method */
+ if (!(meth = PyMethod_New(func, NULL, klass))) {
+ Py_DECREF(func);
+ return -1;
+ }
+
+ /* add method to dictionary */
+ status = PyDict_SetItemString( ((PyClassObject*)klass)->cl_dict,
+ methods->ml_name, meth);
+ Py_DECREF(meth);
+ Py_DECREF(func);
+ /* stop now if an error occurred, otherwise do the next method */
+ if (status)
+ return status;
+ }
+ return 0;
+ }
+
+
/* the python interface to the elog function
* don't confuse these with PLy_elog
*/
***************
*** 2566,2572 ****
* hideously.
*/
elog(FATAL, "plpython: Aiieee, elog threw an unknown exception!");
! return NULL;
}
elog(level, sv);
--- 2705,2711 ----
* hideously.
*/
elog(FATAL, "plpython: Aiieee, elog threw an unknown exception!");
! RERAISE_EXC();
}
elog(level, sv);
***************
*** 2583,2588 ****
--- 2722,2739 ----
return Py_None;
}
+
+ /* Get the last procedure name called by the backend ( the innermost,
+ * If a plpython procedure call calls the backend and the backend calls
+ * another plpython procedure )
+ */
+
+ char *PLy_procedure_name(PLyProcedure *proc)
+ {
+ if ( proc == NULL )
+ return "<unknown procedure>";
+ return proc->proname;
+ }
/* output a python traceback/exception via the postgresql elog
* function. not pretty.
Index: src/pl/plpython/plpython_error.sql
===================================================================
RCS file: /projects/cvsroot/pgsql/src/pl/plpython/plpython_error.sql,v
retrieving revision 1.1
diff -c -r1.1 plpython_error.sql
*** src/pl/plpython/plpython_error.sql 2001/05/09 19:54:38 1.1
--- src/pl/plpython/plpython_error.sql 2001/11/15 18:38:03
***************
*** 7,9 ****
--- 7,17 ----
SELECT invalid_type_caught('rick');
SELECT invalid_type_reraised('rick');
SELECT valid_type('rick');
+
+ -- Security sandbox tests
+ SELECT read_file('/etc/passwd');
+ SELECT write_file('/tmp/plpython','This is very bad');
+ SELECT getpid();
+ SELECT uname();
+ SELECT sys_exit();
+ SELECT sys_argv();
Index: src/pl/plpython/plpython_function.sql
===================================================================
RCS file: /projects/cvsroot/pgsql/src/pl/plpython/plpython_function.sql,v
retrieving revision 1.1
diff -c -r1.1 plpython_function.sql
*** src/pl/plpython/plpython_function.sql 2001/05/09 19:54:38 1.1
--- src/pl/plpython/plpython_function.sql 2001/11/15 18:38:03
***************
*** 257,262 ****
--- 257,268 ----
return None
'
LANGUAGE 'plpython';
+ /* Flat out syntax error
+ */
+ CREATE FUNCTION sql_syntax_error() RETURNS text
+ AS
+ 'plpy.execute("syntax error")'
+ LANGUAGE 'plpython';
/* check the handling of uncaught python exceptions
*/
***************
*** 287,291 ****
'
LANGUAGE 'plpython';
!
--- 293,328 ----
'
LANGUAGE 'plpython';
! CREATE OR REPLACE FUNCTION read_file(text) RETURNS text AS '
! return open(args[0]).read()
! ' LANGUAGE 'plpython';
!
! CREATE OR REPLACE FUNCTION write_file(text,text) RETURNS text AS '
! open(args[0],"w").write(args[1])
! ' LANGUAGE 'plpython';
!
! CREATE OR REPLACE FUNCTION getpid() RETURNS int4 AS '
! import os
! return os.getpid()
! ' LANGUAGE 'plpython';
!
! CREATE OR REPLACE FUNCTION uname() RETURNS int4 AS '
! import os
! return os.uname()
! ' LANGUAGE 'plpython';
!
! CREATE OR REPLACE FUNCTION sys_exit() RETURNS text AS '
! import sys
! return sys.exit()
! ' LANGUAGE 'plpython';
!
! CREATE OR REPLACE FUNCTION sys_argv() RETURNS text AS '
! import sys
! return str(sys.argv)
! ' LANGUAGE 'plpython';
!
! CREATE OR REPLACE FUNCTION sys_version() RETURNS text AS '
! import sys
! return str(sys.version)
! ' LANGUAGE 'plpython';
Bradley McLean <brad@bradm.net> writes:
Okay, the attached patch contains the following:
I have checked over and applied this patch.
It appeared that you incorporated the earlier version of Kevin's patch
rather than the later one. I made the further changes indicated by the
attached diff, which I think are all the differences between Kevin's
two submissions, but I'd appreciate it if both of you would review
what's now in CVS and confirm it's okay. (There'll probably be a beta3
release later today, so you can look at that if it's easier than CVS.)
One thing I noticed while running the selftest on a LinuxPPC box is
that the "feature" test output differed:
--- feature.expected Fri May 25 11:48:33 2001
+++ feature.output Fri Nov 16 13:00:15 2001
@@ -29,7 +29,7 @@
(1 row)
SELECT import_fail();
-NOTICE: ('import socket failed -- untrusted dynamic module: _socket',)
+NOTICE: ('import socket failed -- untrusted dynamic module: socket',)
import_fail
--------------------
failed as expected
I assume you guys both get the "expected" output? Perhaps this should
be noted as a possible platform discrepancy.
Tom's requirement that we always reraise the longjmp is at odds with
the original design of plpython, which attempted to allow a plpython
function to use Python's exception handling to recover when an
embedded SPI call failed. There is no way to do this that I can
see without halting the propogation of the longjmp.
Thus, the semantics of 'execute' calls from plpython have been changed:
should the backend throw an error, the plpython function will be
summarily aborted. Previously, it could attempt to catch a python
exception.
Note that this does create some redundant exception handling paths
in upper level methods within this module that are no longer used.
I have not yet removed them, but will happily do so (in a careful
deliberate manner) once sure that there is no alternative way to
restore the python level exception handling.
I would suggest leaving it as-is for now. Sooner or later we will
probably try to reduce the severity of most errors, and at that
point it'd become worthwhile to re-enable error trapping in plpython.
regards, tom lane
*** plpython.c~ Fri Nov 16 12:19:38 2001
--- plpython.c Fri Nov 16 12:24:43 2001
***************
*** 292,297 ****
--- 292,298 ----
"binascii",
"calendar",
"cmath",
+ "codecs",
"errno",
"marshal",
"math",
***************
*** 325,330 ****
--- 326,332 ----
"hexrevision",
"maxint",
"maxunicode",
+ "platform",
"version",
"version_info"
};
On Fri, 16 Nov 2001, Tom Lane wrote:
--- feature.expected Fri May 25 11:48:33 2001 +++ feature.output Fri Nov 16 13:00:15 2001 @@ -29,7 +29,7 @@ (1 row)SELECT import_fail(); -NOTICE: ('import socket failed -- untrusted dynamic module: _socket',) +NOTICE: ('import socket failed -- untrusted dynamic module: socket',) import_fail -------------------- failed as expectedI assume you guys both get the "expected" output? Perhaps this should
be noted as a possible platform discrepancy.
This diff is most likely a platform issue where the module printed can
change depending on the version of Python installed. Its annoying, but
harmless.
-Kevin
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com
Fax: (216) 986-0714 WWW: http://www.theopalgroup.com
* Kevin Jacobs (jacobs@penguin.theopalgroup.com) [011116 13:15]:
On Fri, 16 Nov 2001, Tom Lane wrote:
--- feature.expected Fri May 25 11:48:33 2001 +++ feature.output Fri Nov 16 13:00:15 2001 @@ -29,7 +29,7 @@ (1 row)SELECT import_fail(); -NOTICE: ('import socket failed -- untrusted dynamic module: _socket',) +NOTICE: ('import socket failed -- untrusted dynamic module: socket',) import_fail -------------------- failed as expectedI assume you guys both get the "expected" output? Perhaps this should
be noted as a possible platform discrepancy.This diff is most likely a platform issue where the module printed can
change depending on the version of Python installed. Its annoying, but
harmless.
I can confirm that: It occurs on stock python 1.5.2 (default on even the
latest RH 7.2 linux). Betweeen 1.5 and 2.0 the low level code was moved
from socket to _socket to make it possible to include a high level python
wrapper called socket.
I've reviewed the merge from CVS and it appears correct. Thanks!
-Brad
Bradley McLean <brad@bradm.net> writes:
I can confirm that: It occurs on stock python 1.5.2 (default on even the
latest RH 7.2 linux).
Bingo --- python 1.5.2 is what's installed on that machine. Seems
to pass the self-test fine other than this one item.
regards, tom lane
Has this been addressed?
---------------------------------------------------------------------------
Bradley McLean <brad@bradm.net> writes:
In Option (C), I set the global "InError" flag to false, and then
return NULL, causing all of the error messages to come out and
plpython to clean up gracefully, no backend crash. However, this
seems to be an unprecedented approach, and I could be missing
something big.Yes, as in "it's totally unsafe". Suppressing an elog(ERROR) is
a *big* no-no at present, because way too much stuff relies on
post-abort cleanup to clean up whatever problem is being reported.
You cannot allow the transaction to continue after the error, and
you most certainly mustn't cavalierly reset the error handling state.The only things you should be doing with longjmp trapping are
(a) doing any cleanup that Python itself has to have before you
re-propagate the longjmp, or
(b) issuing elog(NOTICE) to help identify the error location
before you re-propagate the longjmp.plpgsql contains an example of doing (b).
Not propagating the longjmp, which is what the code seems to be doing
at present, is not acceptable. I had not looked at this code closely,
but as-is it is a huge reliability hazard. I will insist on removing
plpython from 7.2 entirely if this can't be fixed before release.regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026