[PATCH] Add missing type conversion functions for PL/Python
Hi All,
PL/Python already has different type conversion functions to
convert PostgreSQL datum to Python object. However, the conversion
functions from Python object to PostgreSQL datum only has Boolean,
Bytea and String functions.
In this patch, we add rest of Integer and Float related datatype conversion
functions
and can increase the performance of data conversion greatly especially
when returning a large array.
We did a quick test about the performance of returning array in PL/Python:
The following UDF is used for test:
```
CREATE OR REPLACE FUNCTION pyoutfloat8(num int) RETURNS float8[] AS $$
return [x/3.0 for x in range(num)]
$$ LANGUAGE plpythonu;
```
The test command is
```
select count(pyoutfloat8(n));
```
The count is used for avoid large output, where n is the number of element
in returned array, and the size is from 1.5k to15m.
Size of Array 1.5k | 15k |
150k | 1.5m | 15m |
Origin 2.324ms | 19.834ms | 194.991ms
| 1927.28ms | 19982.1ms |
With this patch 1.168ms | 3.052ms | 21.888ms |
213.39ms | 2138.5ms |
All test for both PG and PL/Python are passed.
Thanks very much.
--
Regards,
Haozhou
Attachments:
0001-Add-missing-type-conversion-functions-for-PL-Python.patchapplication/octet-stream; name=0001-Add-missing-type-conversion-functions-for-PL-Python.patchDownload
From 56f86cd1d4513c2b2e40e19479d9cf2f64e857d4 Mon Sep 17 00:00:00 2001
From: Haozhou Wang <hawang@pivotal.io>
Date: Tue, 30 Jan 2018 17:59:23 +0800
Subject: [PATCH] Add missing type conversion functions for PL/Python
PL/Python already has different type conversion functions to
convert PostgreSQL datum to Python object. However, the conversion
functions from Python object to PostgreSQL datum only has Boolean,
Bytea and String functions. This commit add rest of Integer
and Float related datatype conversion functions, and can
increase the performance of data conversion greatly especially
when returning a large array.
Author: Haozhou Wang <hawang@pivotal.io>
Author: Paul Guo <paulguo@gmail.com>
Author: Hubert Zhang <zhanghuan929@gmail.com>
---
src/pl/plpython/plpy_typeio.c | 201 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 201 insertions(+)
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 6c6b16f..6f25103 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -46,6 +46,18 @@ static PyObject *PLyDict_FromTuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc
/* conversion from Python objects to Datums */
static Datum PLyObject_ToBool(PLyObToDatum *arg, PyObject *plrv,
bool *isnull, bool inarray);
+static Datum PLyObject_ToInt16(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToInt32(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToInt64(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToFloat4(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToFloat8(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToNumeric(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
static Datum PLyObject_ToBytea(PLyObToDatum *arg, PyObject *plrv,
bool *isnull, bool inarray);
static Datum PLyObject_ToComposite(PLyObToDatum *arg, PyObject *plrv,
@@ -398,6 +410,24 @@ PLy_output_setup_func(PLyObToDatum *arg, MemoryContext arg_mcxt,
case BOOLOID:
arg->func = PLyObject_ToBool;
break;
+ case INT2OID:
+ arg->func = PLyObject_ToInt16;
+ break;
+ case INT4OID:
+ arg->func = PLyObject_ToInt32;
+ break;
+ case INT8OID:
+ arg->func = PLyObject_ToInt64;
+ break;
+ case FLOAT4OID:
+ arg->func = PLyObject_ToFloat4;
+ break;
+ case FLOAT8OID:
+ arg->func = PLyObject_ToFloat8;
+ break;
+ case NUMERICOID:
+ arg->func = PLyObject_ToNumeric;
+ break;
case BYTEAOID:
arg->func = PLyObject_ToBytea;
break;
@@ -885,6 +915,177 @@ PLyObject_ToBool(PLyObToDatum *arg, PyObject *plrv,
}
/*
+ * Convert a Python object to a PostgreSQL int16 datum directly.
+ * If can not convert if directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt16(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Int16GetDatum((int16)PyInt_AsLong(plrv));
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
+ * Convert a Python object to a PostgreSQL int32 datum directly.
+ * If can not convert if directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt32(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Int32GetDatum((int32)PyInt_AsLong(plrv));
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
+ * Convert a Python object to a PostgreSQL int64 datum directly.
+ * If can not convert if directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt64(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Int64GetDatum((int64)PyLong_AsLong(plrv));
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
+ * Convert a Python object to a PostgreSQL float4 datum directly.
+ * If can not convert if directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToFloat4(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyFloat_Check(plrv) || PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Float4GetDatum((float)PyFloat_AsDouble(plrv));
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
+ * Convert a Python object to a PostgreSQL float8 datum directly.
+ * If can not convert if directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToFloat8(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyFloat_Check(plrv) || PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Float8GetDatum((double)PyFloat_AsDouble(plrv));
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
+ * Convert a Python object to a PostgreSQL numeric datum directly.
+ * If can not convert if directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToNumeric(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyFloat_Check(plrv) || PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ Datum fdatum;
+
+ *isnull = false;
+ fdatum = Float8GetDatum((double)PyFloat_AsDouble(plrv));
+ return DirectFunctionCall1(float8_numeric, fdatum);
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
* Convert a Python object to a PostgreSQL bytea datum. This doesn't
* go through the generic conversion function to circumvent problems
* with embedded nulls. And it's faster this way.
--
2.7.4
On Wed, 31 Jan 2018 11:57:12 +0800
Haozhou Wang <hawang@pivotal.io> wrote:
Hi All,
PL/Python already has different type conversion functions to
convert PostgreSQL datum to Python object. However, the conversion
functions from Python object to PostgreSQL datum only has Boolean,
Bytea and String functions.In this patch, we add rest of Integer and Float related datatype
conversion functions
and can increase the performance of data conversion greatly especially
when returning a large array.We did a quick test about the performance of returning array in
PL/Python:The following UDF is used for test:
```
CREATE OR REPLACE FUNCTION pyoutfloat8(num int) RETURNS float8[] AS $$
return [x/3.0 for x in range(num)]
$$ LANGUAGE plpythonu;
```The test command is
```
select count(pyoutfloat8(n));
```The count is used for avoid large output, where n is the number of
element in returned array, and the size is from 1.5k to15m.Size of Array 1.5k | 15k |
150k | 1.5m | 15m |Origin 2.324ms | 19.834ms | 194.991ms
| 1927.28ms | 19982.1ms |With this patch 1.168ms | 3.052ms |
21.888ms | 213.39ms | 2138.5ms |All test for both PG and PL/Python are passed.
Thanks very much.
Hello,
sounds like a really nice patch. I've started looking
through the code and noticed a sort of a typos (or I just couldn't
understand what did you mean) in comments.
file "src/pl/plpython/plpy_typeio.c"
the comment is
* If can not convert if directly, fallback to PLyObject_ToDatum
* to convert it
Maybe it should be something like ("it" instead of second "if")
* If can not convert it directly, fallback to PLyObject_ToDatum
* to convert it
And the same typo is repeated several times in comments.
--
Anthony Bykov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Thank you very much for your review!
I attached a new patch with typo fixed.
Regards,
Haozhou
On Mon, Feb 19, 2018 at 2:37 PM, Anthony Bykov <a.bykov@postgrespro.ru>
wrote:
Show quoted text
On Wed, 31 Jan 2018 11:57:12 +0800
Haozhou Wang <hawang@pivotal.io> wrote:Hi All,
PL/Python already has different type conversion functions to
convert PostgreSQL datum to Python object. However, the conversion
functions from Python object to PostgreSQL datum only has Boolean,
Bytea and String functions.In this patch, we add rest of Integer and Float related datatype
conversion functions
and can increase the performance of data conversion greatly especially
when returning a large array.We did a quick test about the performance of returning array in
PL/Python:The following UDF is used for test:
```
CREATE OR REPLACE FUNCTION pyoutfloat8(num int) RETURNS float8[] AS $$
return [x/3.0 for x in range(num)]
$$ LANGUAGE plpythonu;
```The test command is
```
select count(pyoutfloat8(n));
```The count is used for avoid large output, where n is the number of
element in returned array, and the size is from 1.5k to15m.Size of Array 1.5k | 15k |
150k | 1.5m | 15m |Origin 2.324ms | 19.834ms | 194.991ms
| 1927.28ms | 19982.1ms |With this patch 1.168ms | 3.052ms |
21.888ms | 213.39ms | 2138.5ms |All test for both PG and PL/Python are passed.
Thanks very much.
Hello,
sounds like a really nice patch. I've started looking
through the code and noticed a sort of a typos (or I just couldn't
understand what did you mean) in comments.file "src/pl/plpython/plpy_typeio.c"
the comment is
* If can not convert if directly, fallback to PLyObject_ToDatum
* to convert itMaybe it should be something like ("it" instead of second "if")
* If can not convert it directly, fallback to PLyObject_ToDatum
* to convert itAnd the same typo is repeated several times in comments.
--
Anthony Bykov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Attachments:
0001-Add-missing-type-conversion-functions-for-PL-Python-v2.patchapplication/octet-stream; name=0001-Add-missing-type-conversion-functions-for-PL-Python-v2.patchDownload
From b2b3c6fd503352a8fd0ff861bd6a544eb6685901 Mon Sep 17 00:00:00 2001
From: Haozhou Wang <hawang@pivotal.io>
Date: Tue, 30 Jan 2018 17:59:23 +0800
Subject: [PATCH] Add missing type conversion functions for PL/Python
PL/Python already has different type conversion functions to
convert PostgreSQL datum to Python object. However, the conversion
functions from Python object to PostgreSQL datum only has Boolean,
Bytea and String functions. This commit add rest of Integer
and Float related datatype conversion functions, and can
increase the performance of data conversion greatly especially
when returning a large array.
Author: Haozhou Wang <hawang@pivotal.io>
Author: Paul Guo <paulguo@gmail.com>
Author: Hubert Zhang <zhanghuan929@gmail.com>
---
src/pl/plpython/plpy_typeio.c | 201 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 201 insertions(+)
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 6c6b16f..2f2f566 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -46,6 +46,18 @@ static PyObject *PLyDict_FromTuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc
/* conversion from Python objects to Datums */
static Datum PLyObject_ToBool(PLyObToDatum *arg, PyObject *plrv,
bool *isnull, bool inarray);
+static Datum PLyObject_ToInt16(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToInt32(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToInt64(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToFloat4(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToFloat8(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToNumeric(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
static Datum PLyObject_ToBytea(PLyObToDatum *arg, PyObject *plrv,
bool *isnull, bool inarray);
static Datum PLyObject_ToComposite(PLyObToDatum *arg, PyObject *plrv,
@@ -398,6 +410,24 @@ PLy_output_setup_func(PLyObToDatum *arg, MemoryContext arg_mcxt,
case BOOLOID:
arg->func = PLyObject_ToBool;
break;
+ case INT2OID:
+ arg->func = PLyObject_ToInt16;
+ break;
+ case INT4OID:
+ arg->func = PLyObject_ToInt32;
+ break;
+ case INT8OID:
+ arg->func = PLyObject_ToInt64;
+ break;
+ case FLOAT4OID:
+ arg->func = PLyObject_ToFloat4;
+ break;
+ case FLOAT8OID:
+ arg->func = PLyObject_ToFloat8;
+ break;
+ case NUMERICOID:
+ arg->func = PLyObject_ToNumeric;
+ break;
case BYTEAOID:
arg->func = PLyObject_ToBytea;
break;
@@ -885,6 +915,177 @@ PLyObject_ToBool(PLyObToDatum *arg, PyObject *plrv,
}
/*
+ * Convert a Python object to a PostgreSQL int16 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt16(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Int16GetDatum((int16)PyInt_AsLong(plrv));
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
+ * Convert a Python object to a PostgreSQL int32 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt32(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Int32GetDatum((int32)PyInt_AsLong(plrv));
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
+ * Convert a Python object to a PostgreSQL int64 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt64(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Int64GetDatum((int64)PyLong_AsLong(plrv));
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
+ * Convert a Python object to a PostgreSQL float4 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToFloat4(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyFloat_Check(plrv) || PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Float4GetDatum((float)PyFloat_AsDouble(plrv));
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
+ * Convert a Python object to a PostgreSQL float8 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToFloat8(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyFloat_Check(plrv) || PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Float8GetDatum((double)PyFloat_AsDouble(plrv));
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
+ * Convert a Python object to a PostgreSQL numeric datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToDatum
+ * to convert it.
+ */
+static Datum
+PLyObject_ToNumeric(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ if (PyFloat_Check(plrv) || PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ Datum fdatum;
+
+ *isnull = false;
+ fdatum = Float8GetDatum((double)PyFloat_AsDouble(plrv));
+ return DirectFunctionCall1(float8_numeric, fdatum);
+ }
+ else
+ {
+ Oid typinput;
+ getTypeInputInfo(arg->typoid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg->mcxt);
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+ }
+}
+
+/*
* Convert a Python object to a PostgreSQL bytea datum. This doesn't
* go through the generic conversion function to circumvent problems
* with embedded nulls. And it's faster this way.
--
2.7.4
On 2/20/18 10:14 AM, Haozhou Wang wrote:
Thank you very much for your review!
I attached a new patch with typo fixed.
I think it's a bit premature to mark this Ready for Committer after a
review consisting of a few typos. Anthony only said that he started
looking at it so I've marked it Needs Review.
Anthony, were you planning to have a deeper look?
Regards,
--
-David
david@pgmasters.net
On 26.03.2018 17:19, David Steele wrote:
On 2/20/18 10:14 AM, Haozhou Wang wrote:
Thank you very much for your review!
I attached a new patch with typo fixed.
I think it's a bit premature to mark this Ready for Committer after a
review consisting of a few typos. Anthony only said that he started
looking at it so I've marked it Needs Review.
Hi.
I also have looked at this patch and found some problems.
Attached fixed 3th version of the patch:
* initialization of arg->u.scalar was moved into PLy_output_setup_func()
* added range checks for int16 and int32 types
* added subroutine PLyInt_AsLong() for correct handling OverflowError that can
be thrown from PyInt_AsLong()
* casting from Python float to PostgreSQL numeric using PyFloat_AsDouble() was
removed because it can return incorrect result for Python long and
float8_numeric() uses float8 and numeric I/O functions
* fixed whitespace
--
Nikita Glukhov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Attachments:
0001-Add-missing-type-conversion-functions-for-PL-Python-v3.patchtext/x-patch; name=0001-Add-missing-type-conversion-functions-for-PL-Python-v3.patchDownload
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index d6a6a84..3b874e1 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -46,6 +46,18 @@ static PyObject *PLyDict_FromTuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc
/* conversion from Python objects to Datums */
static Datum PLyObject_ToBool(PLyObToDatum *arg, PyObject *plrv,
bool *isnull, bool inarray);
+static Datum PLyObject_ToInt16(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToInt32(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToInt64(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToFloat4(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToFloat8(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToNumeric(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
static Datum PLyObject_ToBytea(PLyObToDatum *arg, PyObject *plrv,
bool *isnull, bool inarray);
static Datum PLyObject_ToComposite(PLyObToDatum *arg, PyObject *plrv,
@@ -397,16 +409,35 @@ PLy_output_setup_func(PLyObToDatum *arg, MemoryContext arg_mcxt,
{
case BOOLOID:
arg->func = PLyObject_ToBool;
- break;
+ return; /* no need to initialize arg->u.scalar */
case BYTEAOID:
arg->func = PLyObject_ToBytea;
+ return; /* no need to initialize arg->u.scalar */
+ case INT2OID:
+ arg->func = PLyObject_ToInt16;
+ break;
+ case INT4OID:
+ arg->func = PLyObject_ToInt32;
+ break;
+ case INT8OID:
+ arg->func = PLyObject_ToInt64;
+ break;
+ case FLOAT4OID:
+ arg->func = PLyObject_ToFloat4;
+ break;
+ case FLOAT8OID:
+ arg->func = PLyObject_ToFloat8;
+ break;
+ case NUMERICOID:
+ arg->func = PLyObject_ToNumeric;
break;
default:
arg->func = PLyObject_ToScalar;
- getTypeInputInfo(typeOid, &typinput, &arg->u.scalar.typioparam);
- fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg_mcxt);
break;
}
+
+ getTypeInputInfo(typeOid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg_mcxt);
}
}
@@ -884,6 +915,210 @@ PLyObject_ToBool(PLyObToDatum *arg, PyObject *plrv,
return BoolGetDatum(PyObject_IsTrue(plrv));
}
+/* Try to convert python Int to C long. */
+static long
+PLyInt_AsLong(PyObject *plrv, bool *err)
+{
+ long val = PyInt_AsLong(plrv);
+
+ /* If -1 is returned then OverflowError is possible. */
+ *err = val == -1 && PyErr_Occurred();
+
+ if (*err)
+ {
+ /* Catch OverflowError exception. */
+ if (!PyErr_ExceptionMatches(PyExc_OverflowError))
+ PLy_elog(ERROR, NULL);
+
+ PyErr_Clear();
+ }
+
+ return val;
+}
+
+/*
+ * Convert a Python object to a PostgreSQL int16 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt16(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ if (PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ bool err;
+ long val = PLyInt_AsLong(plrv, &err);
+
+ if (err || val < SHRT_MIN || val > SHRT_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
+ errmsg("value \"%s\" is out of range for type %s",
+ PLyObject_AsString(plrv), "smallint")));
+
+ *isnull = false;
+ return Int16GetDatum((int16) val);
+ }
+
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+}
+
+/*
+ * Convert a Python object to a PostgreSQL int32 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt32(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ if (PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ bool err;
+ long val = PLyInt_AsLong(plrv, &err);
+
+ if (err
+#ifdef HAVE_LONG_INT_64
+ || val < INT_MIN || val > INT_MAX
+#endif
+ )
+ ereport(ERROR,
+ (errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
+ errmsg("value \"%s\" is out of range for type %s",
+ PLyObject_AsString(plrv), "integer")));
+
+ *isnull = false;
+ return Int32GetDatum((int32) val);
+ }
+
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+}
+
+/*
+ * Convert a Python object to a PostgreSQL int64 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt64(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ if (PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ bool err;
+ long val = PLyInt_AsLong(plrv, &err);
+
+ if (!err)
+ {
+ *isnull = false;
+ return Int64GetDatum((int64) val);
+ }
+
+ /* try to convert via I/O */
+ }
+
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+}
+
+/*
+ * Convert a Python object to a PostgreSQL float4 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToFloat4(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ if (PyFloat_Check(plrv) || PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Float4GetDatum((float)PyFloat_AsDouble(plrv));
+ }
+
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+}
+
+/*
+ * Convert a Python object to a PostgreSQL float8 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToFloat8(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ if (PyFloat_Check(plrv) || PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ *isnull = false;
+ return Float8GetDatum((double)PyFloat_AsDouble(plrv));
+ }
+
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+}
+
+/*
+ * Convert a Python object to a PostgreSQL numeric datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToNumeric(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ if (PyInt_Check(plrv) || PyLong_Check(plrv))
+ {
+ bool err;
+ long val = PLyInt_AsLong(plrv, &err);
+
+ if (!err)
+ {
+ *isnull = false;
+ return DirectFunctionCall1(int8_numeric,
+ Int64GetDatum((int64) val));
+ }
+
+ /* try to convert via I/O */
+ }
+
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+}
+
/*
* Convert a Python object to a PostgreSQL bytea datum. This doesn't
* go through the generic conversion function to circumvent problems
Thanks Nikita!
On Tue, Mar 27, 2018 at 12:07 AM, Nikita Glukhov <n.gluhov@postgrespro.ru>
wrote:
On 26.03.2018 17:19, David Steele wrote:
On 2/20/18 10:14 AM, Haozhou Wang wrote:
Thank you very much for your review!
I attached a new patch with typo fixed.
I think it's a bit premature to mark this Ready for Committer after a
review consisting of a few typos. Anthony only said that he started
looking at it so I've marked it Needs Review.Hi.
I also have looked at this patch and found some problems.
Attached fixed 3th version of the patch:
* initialization of arg->u.scalar was moved into PLy_output_setup_func()
* added range checks for int16 and int32 types
* added subroutine PLyInt_AsLong() for correct handling OverflowError
that can
be thrown from PyInt_AsLong()
* casting from Python float to PostgreSQL numeric using
PyFloat_AsDouble() was
removed because it can return incorrect result for Python long and
float8_numeric() uses float8 and numeric I/O functions
* fixed whitespace--
Nikita GlukhovPostgres Professional: http://www.postgrespro.com
The Russian Postgres Company
--
Regards,
Haozhou
On 3/26/18 12:07 PM, Nikita Glukhov wrote:
On 26.03.2018 17:19, David Steele wrote:
On 2/20/18 10:14 AM, Haozhou Wang wrote:
Thank you very much for your review!
I attached a new patch with typo fixed.
I think it's a bit premature to mark this Ready for Committer after a
review consisting of a few typos. Anthony only said that he started
looking at it so I've marked it Needs Review.Hi.
I also have looked at this patch and found some problems.
Attached fixed 3th version of the patch:
* initialization of arg->u.scalar was moved into PLy_output_setup_func()
* added range checks for int16 and int32 types
* added subroutine PLyInt_AsLong() for correct handling OverflowError
that can
be thrown from PyInt_AsLong()
* casting from Python float to PostgreSQL numeric using
PyFloat_AsDouble() was
removed because it can return incorrect result for Python long and
float8_numeric() uses float8 and numeric I/O functions
* fixed whitespace
There's a new patch on this thread but since it was not submitted by the
author I've moved this entry to the next CF in Waiting for Author state.
Regards,
--
-David
david@pgmasters.net
Hi David,
Thanks for your email.
Just wondering will I need to re-submit this patch?
Thanks a lot!
Regards,
Haozhou
On Tue, Apr 10, 2018 at 9:35 PM, David Steele <david@pgmasters.net> wrote:
Show quoted text
On 3/26/18 12:07 PM, Nikita Glukhov wrote:
On 26.03.2018 17:19, David Steele wrote:
On 2/20/18 10:14 AM, Haozhou Wang wrote:
Thank you very much for your review!
I attached a new patch with typo fixed.
I think it's a bit premature to mark this Ready for Committer after a
review consisting of a few typos. Anthony only said that he started
looking at it so I've marked it Needs Review.Hi.
I also have looked at this patch and found some problems.
Attached fixed 3th version of the patch:
* initialization of arg->u.scalar was moved into PLy_output_setup_func()
* added range checks for int16 and int32 types
* added subroutine PLyInt_AsLong() for correct handling OverflowError
that can
be thrown from PyInt_AsLong()
* casting from Python float to PostgreSQL numeric using
PyFloat_AsDouble() was
removed because it can return incorrect result for Python long and
float8_numeric() uses float8 and numeric I/O functions
* fixed whitespaceThere's a new patch on this thread but since it was not submitted by the
author I've moved this entry to the next CF in Waiting for Author state.Regards,
--
-David
david@pgmasters.net
On 4/11/18 2:36 AM, Haozhou Wang wrote:
Thanks for your email.
Just wondering will I need to re-submit this patch?
No need to resubmit, the CF entry has been moved here:
https://commitfest.postgresql.org/18/1499/
You should have a look at Nikita's patch, though.
Regards,
--
-David
david@pgmasters.net
Hi David,
This new patch is look good for me. So I change the status to need review.
Thanks!
Regards,
Haozhou
On Wed, Apr 11, 2018 at 7:52 PM, David Steele <david@pgmasters.net> wrote:
Show quoted text
On 4/11/18 2:36 AM, Haozhou Wang wrote:
Thanks for your email.
Just wondering will I need to re-submit this patch?No need to resubmit, the CF entry has been moved here:
https://commitfest.postgresql.org/18/1499/
You should have a look at Nikita's patch, though.
Regards,
--
-David
david@pgmasters.net
On 26/03/18 19:07, Nikita Glukhov wrote:
Attached fixed 3th version of the patch:
Thanks, I'm reviewing this now. Nice speedup!
There is no test coverage for some of the added code. You can get a code
coverage report with:
./configure --enable-coverage ...
make
make -C src/pl/plpython check
make coverage-html
That produces a code coverage report in coverage/index.html. Please look
at the coverage of the new functions, and add tests to
src/pl/plpython/sql/plpython_types.sql so that all the new code is covered.
In some places, where you've already checked the object type e.g. with
PyFloat_Check(), I think you could use the less safe variants, like
PyFloat_AS_DOUBLE() instead of PyFloat_AsDouble(). Since this patch is
all about performance, seems worth doing.
Some of the conversions seem a bit questionable. For example:
/*
* Convert a Python object to a PostgreSQL float8 datum directly.
* If can not convert it directly, fallback to PLyObject_ToScalar
* to convert it.
*/
static Datum
PLyObject_ToFloat8(PLyObToDatum *arg, PyObject *plrv,
bool *isnull, bool inarray)
{
if (plrv == Py_None)
{
*isnull = true;
return (Datum) 0;
}if (PyFloat_Check(plrv) || PyInt_Check(plrv) || PyLong_Check(plrv))
{
*isnull = false;
return Float8GetDatum((double)PyFloat_AsDouble(plrv));
}return PLyObject_ToScalar(arg, plrv, isnull, inarray);
}
The conversion from Python int to C double is performed by
PyFloat_AsDouble(). But there's no error checking. And wouldn't
PyLong_AsDouble() be more appropriate in that case, since we already
checked the python type?
- Heikki
Hi Heikki,
Thank you very much for your review!
I will prepare a new patch and make it ready soon.
Regards,
Haozhou
On Thu, Jul 12, 2018 at 2:03 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 26/03/18 19:07, Nikita Glukhov wrote:
Attached fixed 3th version of the patch:
Thanks, I'm reviewing this now. Nice speedup!
There is no test coverage for some of the added code. You can get a code
coverage report with:./configure --enable-coverage ...
make
make -C src/pl/plpython check
make coverage-htmlThat produces a code coverage report in coverage/index.html. Please look
at the coverage of the new functions, and add tests to
src/pl/plpython/sql/plpython_types.sql so that all the new code is covered.In some places, where you've already checked the object type e.g. with
PyFloat_Check(), I think you could use the less safe variants, like
PyFloat_AS_DOUBLE() instead of PyFloat_AsDouble(). Since this patch is
all about performance, seems worth doing.Some of the conversions seem a bit questionable. For example:
/*
* Convert a Python object to a PostgreSQL float8 datum directly.
* If can not convert it directly, fallback to PLyObject_ToScalar
* to convert it.
*/
static Datum
PLyObject_ToFloat8(PLyObToDatum *arg, PyObject *plrv,
bool *isnull, bool inarray)
{
if (plrv == Py_None)
{
*isnull = true;
return (Datum) 0;
}if (PyFloat_Check(plrv) || PyInt_Check(plrv) || PyLong_Check(plrv))
{
*isnull = false;
return Float8GetDatum((double)PyFloat_AsDouble(plrv));
}return PLyObject_ToScalar(arg, plrv, isnull, inarray);
}The conversion from Python int to C double is performed by
PyFloat_AsDouble(). But there's no error checking. And wouldn't
PyLong_AsDouble() be more appropriate in that case, since we already
checked the python type?- Heikki
--
Regards,
Haozhou
On 11.07.2018 21:03, Heikki Linnakangas wrote:
On 26/03/18 19:07, Nikita Glukhov wrote:
Attached fixed 3th version of the patch:
Thanks, I'm reviewing this now. Nice speedup!
Thank you for your review.
There is no test coverage for some of the added code. You can get a
code coverage report with:./configure --enable-coverage ...
make
make -C src/pl/plpython check
make coverage-htmlThat produces a code coverage report in coverage/index.html. Please
look at the coverage of the new functions, and add tests to
src/pl/plpython/sql/plpython_types.sql so that all the new code is
covered.
I have added some cross-type test cases and now almost all new code is covered
(excluding several error cases which can be triggered only by custom numeric
type implementations).
In some places, where you've already checked the object type e.g. with
PyFloat_Check(), I think you could use the less safe variants, like
PyFloat_AS_DOUBLE() instead of PyFloat_AsDouble(). Since this patch is
all about performance, seems worth doing.
Fixed.
Some of the conversions seem a bit questionable. For example:
/*
* Convert a Python object to a PostgreSQL float8 datum directly.
* If can not convert it directly, fallback to PLyObject_ToScalar
* to convert it.
*/
static Datum
PLyObject_ToFloat8(PLyObToDatum *arg, PyObject *plrv,
bool *isnull, bool inarray)
{
if (plrv == Py_None)
{
*isnull = true;
return (Datum) 0;
}if (PyFloat_Check(plrv) || PyInt_Check(plrv) || PyLong_Check(plrv))
{
*isnull = false;
return Float8GetDatum((double)PyFloat_AsDouble(plrv));
}return PLyObject_ToScalar(arg, plrv, isnull, inarray);
}The conversion from Python int to C double is performed by
PyFloat_AsDouble(). But there's no error checking. And wouldn't
PyLong_AsDouble() be more appropriate in that case, since we already
checked the python type?
Yes, this might be wrong, but PyFloat_AsDouble() internally tries first to
convert number to float. Also, after gaining more experience in PL/Python
during the implementation of jsonb transforms, I found a lot of similar
problems in the code. All of them are fixed in the 4th version of the patch.
--
Nikita Glukhov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Attachments:
0001-Add-missing-type-conversion-functions-for-PL-Python-v4.patchtext/x-patch; name=0001-Add-missing-type-conversion-functions-for-PL-Python-v4.patchDownload
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index eda965a..dc1232b 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -134,6 +134,158 @@ INFO: (None, <type 'NoneType'>)
(1 row)
+CREATE FUNCTION test_type_conversion_int4_int2(x int4) RETURNS int2 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_int4_int2(100::int4);
+INFO: (100, <type 'int'>)
+ test_type_conversion_int4_int2
+--------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_int4_int2(-100::int4);
+INFO: (-100, <type 'int'>)
+ test_type_conversion_int4_int2
+--------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_int4_int2(1000000::int4);
+INFO: (1000000, <type 'int'>)
+ERROR: value "1000000" is out of range for type smallint
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_int4_int2"
+SELECT * FROM test_type_conversion_int4_int2(-1000000::int4);
+INFO: (-1000000, <type 'int'>)
+ERROR: value "-1000000" is out of range for type smallint
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_int4_int2"
+SELECT * FROM test_type_conversion_int4_int2(null);
+INFO: (None, <type 'NoneType'>)
+ test_type_conversion_int4_int2
+--------------------------------
+
+(1 row)
+
+CREATE FUNCTION test_type_conversion_int8_int2(x int8) RETURNS int2 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_int8_int2(100::int8);
+INFO: (100L, <type 'long'>)
+ test_type_conversion_int8_int2
+--------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_int8_int2(-100::int8);
+INFO: (-100L, <type 'long'>)
+ test_type_conversion_int8_int2
+--------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_int8_int2(1000000::int8);
+INFO: (1000000L, <type 'long'>)
+ERROR: value "1000000" is out of range for type smallint
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_int8_int2"
+SELECT * FROM test_type_conversion_int8_int2(-1000000::int8);
+INFO: (-1000000L, <type 'long'>)
+ERROR: value "-1000000" is out of range for type smallint
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_int8_int2"
+SELECT * FROM test_type_conversion_int8_int2(null);
+INFO: (None, <type 'NoneType'>)
+ test_type_conversion_int8_int2
+--------------------------------
+
+(1 row)
+
+CREATE FUNCTION test_type_conversion_float8_int2(x float8) RETURNS int2 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_float8_int2(100::float8);
+INFO: (100.0, <type 'float'>)
+ test_type_conversion_float8_int2
+----------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_float8_int2(-100::float8);
+INFO: (-100.0, <type 'float'>)
+ test_type_conversion_float8_int2
+----------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_float8_int2(100.345::float8);
+INFO: (100.345, <type 'float'>)
+ test_type_conversion_float8_int2
+----------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_float8_int2(-100.678::float8);
+INFO: (-100.678, <type 'float'>)
+ test_type_conversion_float8_int2
+----------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_float8_int2(1000000::float8);
+INFO: (1000000.0, <type 'float'>)
+ERROR: value "1000000.0" is out of range for type smallint
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_float8_int2"
+SELECT * FROM test_type_conversion_float8_int2(-1000000::float8);
+INFO: (-1000000.0, <type 'float'>)
+ERROR: value "-1000000.0" is out of range for type smallint
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_float8_int2"
+SELECT * FROM test_type_conversion_float8_int2(null);
+INFO: (None, <type 'NoneType'>)
+ test_type_conversion_float8_int2
+----------------------------------
+
+(1 row)
+
+CREATE FUNCTION test_type_conversion_text_int2(x text) RETURNS int2 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_text_int2('100');
+INFO: ('100', <type 'str'>)
+ test_type_conversion_text_int2
+--------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_text_int2('-100');
+INFO: ('-100', <type 'str'>)
+ test_type_conversion_text_int2
+--------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_text_int2('1000000000000000');
+INFO: ('1000000000000000', <type 'str'>)
+ERROR: value "1000000000000000" is out of range for type smallint
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_text_int2"
+SELECT * FROM test_type_conversion_text_int2('1.23');
+INFO: ('1.23', <type 'str'>)
+ERROR: invalid input syntax for integer: "1.23"
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_text_int2"
+SELECT * FROM test_type_conversion_text_int2('aaa');
+INFO: ('aaa', <type 'str'>)
+ERROR: invalid input syntax for integer: "aaa"
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_text_int2"
CREATE FUNCTION test_type_conversion_int4(x int4) RETURNS int4 AS $$
plpy.info(x, type(x))
return x
@@ -159,6 +311,132 @@ INFO: (None, <type 'NoneType'>)
(1 row)
+CREATE FUNCTION test_type_conversion_int8_int4(x int8) RETURNS int4 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_int8_int4(100);
+INFO: (100L, <type 'long'>)
+ test_type_conversion_int8_int4
+--------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_int8_int4(-100);
+INFO: (-100L, <type 'long'>)
+ test_type_conversion_int8_int4
+--------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_int8_int4(10000000000000);
+INFO: (10000000000000L, <type 'long'>)
+ERROR: value "10000000000000" is out of range for type integer
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_int8_int4"
+SELECT * FROM test_type_conversion_int8_int4(-10000000000000);
+INFO: (-10000000000000L, <type 'long'>)
+ERROR: value "-10000000000000" is out of range for type integer
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_int8_int4"
+SELECT * FROM test_type_conversion_int8_int4(null);
+INFO: (None, <type 'NoneType'>)
+ test_type_conversion_int8_int4
+--------------------------------
+
+(1 row)
+
+CREATE FUNCTION test_type_conversion_numeric_int4(x numeric) RETURNS int4 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_numeric_int4(100::numeric);
+INFO: (Decimal('100'), <class 'decimal.Decimal'>)
+ test_type_conversion_numeric_int4
+-----------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_numeric_int4(-100::numeric);
+INFO: (Decimal('-100'), <class 'decimal.Decimal'>)
+ test_type_conversion_numeric_int4
+-----------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_numeric_int4(1000000::numeric);
+INFO: (Decimal('1000000'), <class 'decimal.Decimal'>)
+ test_type_conversion_numeric_int4
+-----------------------------------
+ 1000000
+(1 row)
+
+SELECT * FROM test_type_conversion_numeric_int4(-1000000::numeric);
+INFO: (Decimal('-1000000'), <class 'decimal.Decimal'>)
+ test_type_conversion_numeric_int4
+-----------------------------------
+ -1000000
+(1 row)
+
+SELECT * FROM test_type_conversion_numeric_int4(100000000000000000000::numeric);
+INFO: (Decimal('100000000000000000000'), <class 'decimal.Decimal'>)
+ERROR: value "100000000000000000000" is out of range for type integer
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_numeric_int4"
+SELECT * FROM test_type_conversion_numeric_int4(-100000000000000000000::numeric);
+INFO: (Decimal('-100000000000000000000'), <class 'decimal.Decimal'>)
+ERROR: value "-100000000000000000000" is out of range for type integer
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_numeric_int4"
+SELECT * FROM test_type_conversion_numeric_int4(null);
+INFO: (None, <type 'NoneType'>)
+ test_type_conversion_numeric_int4
+-----------------------------------
+
+(1 row)
+
+CREATE FUNCTION test_type_conversion_long_int4(x numeric) RETURNS int4 AS $$
+plpy.info(x, type(x))
+return long(x)
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_long_int4(100::numeric);
+INFO: (Decimal('100'), <class 'decimal.Decimal'>)
+ test_type_conversion_long_int4
+--------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_long_int4(-100::numeric);
+INFO: (Decimal('-100'), <class 'decimal.Decimal'>)
+ test_type_conversion_long_int4
+--------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_long_int4(1000000::numeric);
+INFO: (Decimal('1000000'), <class 'decimal.Decimal'>)
+ test_type_conversion_long_int4
+--------------------------------
+ 1000000
+(1 row)
+
+SELECT * FROM test_type_conversion_long_int4(-1000000::numeric);
+INFO: (Decimal('-1000000'), <class 'decimal.Decimal'>)
+ test_type_conversion_long_int4
+--------------------------------
+ -1000000
+(1 row)
+
+SELECT * FROM test_type_conversion_long_int4(100000000000000000000::numeric);
+INFO: (Decimal('100000000000000000000'), <class 'decimal.Decimal'>)
+ERROR: value "100000000000000000000" is out of range for type integer
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_long_int4"
+SELECT * FROM test_type_conversion_long_int4(-100000000000000000000::numeric);
+INFO: (Decimal('-100000000000000000000'), <class 'decimal.Decimal'>)
+ERROR: value "-100000000000000000000" is out of range for type integer
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_long_int4"
CREATE FUNCTION test_type_conversion_int8(x int8) RETURNS int8 AS $$
plpy.info(x, type(x))
return x
@@ -191,6 +469,104 @@ INFO: (None, <type 'NoneType'>)
(1 row)
+CREATE FUNCTION test_type_conversion_float8_int8(x float8) RETURNS int8 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_float8_int8(100::float8);
+INFO: (100.0, <type 'float'>)
+ test_type_conversion_float8_int8
+----------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_float8_int8(-100::float8);
+INFO: (-100.0, <type 'float'>)
+ test_type_conversion_float8_int8
+----------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_float8_int8(100.345::float8);
+INFO: (100.345, <type 'float'>)
+ test_type_conversion_float8_int8
+----------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_float8_int8(-100.678::float8);
+INFO: (-100.678, <type 'float'>)
+ test_type_conversion_float8_int8
+----------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_float8_int8(100000000000.345::float8);
+INFO: (100000000000.345, <type 'float'>)
+ test_type_conversion_float8_int8
+----------------------------------
+ 100000000000
+(1 row)
+
+SELECT * FROM test_type_conversion_float8_int8(-100000000000.678::float8);
+INFO: (-100000000000.678, <type 'float'>)
+ test_type_conversion_float8_int8
+----------------------------------
+ -100000000000
+(1 row)
+
+SELECT * FROM test_type_conversion_float8_int8(100000000000000000000000::float8);
+INFO: (1e+23, <type 'float'>)
+ERROR: value "1e+23" is out of range for type bigint
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_float8_int8"
+SELECT * FROM test_type_conversion_float8_int8(-100000000000000000000000::float8);
+INFO: (-1e+23, <type 'float'>)
+ERROR: value "-1e+23" is out of range for type bigint
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_float8_int8"
+SELECT * FROM test_type_conversion_float8_int8(null);
+INFO: (None, <type 'NoneType'>)
+ test_type_conversion_float8_int8
+----------------------------------
+
+(1 row)
+
+CREATE FUNCTION test_type_conversion_text_int8(x text) RETURNS int8 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_text_int8('100');
+INFO: ('100', <type 'str'>)
+ test_type_conversion_text_int8
+--------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_text_int8('-100');
+INFO: ('-100', <type 'str'>)
+ test_type_conversion_text_int8
+--------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_text_int8('1000000000000000');
+INFO: ('1000000000000000', <type 'str'>)
+ test_type_conversion_text_int8
+--------------------------------
+ 1000000000000000
+(1 row)
+
+SELECT * FROM test_type_conversion_text_int8('100000000000000000000000');
+INFO: ('100000000000000000000000', <type 'str'>)
+ERROR: value "100000000000000000000000" is out of range for type bigint
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_text_int8"
+SELECT * FROM test_type_conversion_text_int8('aaa');
+INFO: ('aaa', <type 'str'>)
+ERROR: invalid input syntax for integer: "aaa"
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_text_int8"
CREATE FUNCTION test_type_conversion_numeric(x numeric) RETURNS numeric AS $$
# print just the class name, not the type, to avoid differences
# between decimal and cdecimal
@@ -253,6 +629,31 @@ INFO: ('None', 'NoneType')
(1 row)
+CREATE FUNCTION test_type_conversion_int4_numeric(x int4) RETURNS numeric AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_int4_numeric(100::int4);
+INFO: (100, <type 'int'>)
+ test_type_conversion_int4_numeric
+-----------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_int4_numeric(-100::int4);
+INFO: (-100, <type 'int'>)
+ test_type_conversion_int4_numeric
+-----------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_int4_numeric(null);
+INFO: (None, <type 'NoneType'>)
+ test_type_conversion_int4_numeric
+-----------------------------------
+
+(1 row)
+
CREATE FUNCTION test_type_conversion_float4(x float4) RETURNS float4 AS $$
plpy.info(x, type(x))
return x
@@ -285,6 +686,43 @@ INFO: (None, <type 'NoneType'>)
(1 row)
+CREATE FUNCTION test_type_conversion_text_float4(x text) RETURNS float4 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_text_float4('100');
+INFO: ('100', <type 'str'>)
+ test_type_conversion_text_float4
+----------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_text_float4('-100');
+INFO: ('-100', <type 'str'>)
+ test_type_conversion_text_float4
+----------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_text_float4('5000.5');
+INFO: ('5000.5', <type 'str'>)
+ test_type_conversion_text_float4
+----------------------------------
+ 5000.5
+(1 row)
+
+SELECT * FROM test_type_conversion_text_float4('aaa');
+INFO: ('aaa', <type 'str'>)
+ERROR: invalid input syntax for type real: "aaa"
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_text_float4"
+SELECT * FROM test_type_conversion_text_float4(null);
+INFO: (None, <type 'NoneType'>)
+ test_type_conversion_text_float4
+----------------------------------
+
+(1 row)
+
CREATE FUNCTION test_type_conversion_float8(x float8) RETURNS float8 AS $$
plpy.info(x, type(x))
return x
@@ -324,6 +762,171 @@ INFO: (100100100.654321, <type 'float'>)
100100100.654321
(1 row)
+CREATE FUNCTION test_type_conversion_text_float8(x text) RETURNS float8 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_text_float8('100');
+INFO: ('100', <type 'str'>)
+ test_type_conversion_text_float8
+----------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_text_float8('-100');
+INFO: ('-100', <type 'str'>)
+ test_type_conversion_text_float8
+----------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_text_float8('5000000000.5');
+INFO: ('5000000000.5', <type 'str'>)
+ test_type_conversion_text_float8
+----------------------------------
+ 5000000000.5
+(1 row)
+
+SELECT * FROM test_type_conversion_text_float8('100100100.654321');
+INFO: ('100100100.654321', <type 'str'>)
+ test_type_conversion_text_float8
+----------------------------------
+ 100100100.654321
+(1 row)
+
+SELECT * FROM test_type_conversion_text_float8(null);
+INFO: (None, <type 'NoneType'>)
+ test_type_conversion_text_float8
+----------------------------------
+
+(1 row)
+
+CREATE FUNCTION test_type_conversion_int4_float8(x int4) RETURNS float8 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_int4_float8(100);
+INFO: (100, <type 'int'>)
+ test_type_conversion_int4_float8
+----------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_int4_float8(-100);
+INFO: (-100, <type 'int'>)
+ test_type_conversion_int4_float8
+----------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_int4_float8(null);
+INFO: (None, <type 'NoneType'>)
+ test_type_conversion_int4_float8
+----------------------------------
+
+(1 row)
+
+CREATE FUNCTION test_type_conversion_int8_float8(x int8) RETURNS float8 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_int8_float8(100::int8);
+INFO: (100L, <type 'long'>)
+ test_type_conversion_int8_float8
+----------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_int8_float8(-100::int8);
+INFO: (-100L, <type 'long'>)
+ test_type_conversion_int8_float8
+----------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_int8_float8(10000000000000::int8);
+INFO: (10000000000000L, <type 'long'>)
+ test_type_conversion_int8_float8
+----------------------------------
+ 10000000000000
+(1 row)
+
+SELECT * FROM test_type_conversion_int8_float8(-10000000000000::int8);
+INFO: (-10000000000000L, <type 'long'>)
+ test_type_conversion_int8_float8
+----------------------------------
+ -10000000000000
+(1 row)
+
+SELECT * FROM test_type_conversion_int8_float8(null);
+INFO: (None, <type 'NoneType'>)
+ test_type_conversion_int8_float8
+----------------------------------
+
+(1 row)
+
+CREATE FUNCTION test_type_conversion_numeric_float8(x numeric) RETURNS float8 AS $$
+# print just the class name, not the type, to avoid differences
+# between decimal and cdecimal
+plpy.info(str(x), x.__class__.__name__)
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_numeric_float8(100);
+INFO: ('100', 'Decimal')
+ test_type_conversion_numeric_float8
+-------------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_numeric_float8(-100);
+INFO: ('-100', 'Decimal')
+ test_type_conversion_numeric_float8
+-------------------------------------
+ -100
+(1 row)
+
+SELECT * FROM test_type_conversion_numeric_float8(100.0);
+INFO: ('100.0', 'Decimal')
+ test_type_conversion_numeric_float8
+-------------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_numeric_float8(100.00);
+INFO: ('100.00', 'Decimal')
+ test_type_conversion_numeric_float8
+-------------------------------------
+ 100
+(1 row)
+
+SELECT * FROM test_type_conversion_numeric_float8(5000000000.5);
+INFO: ('5000000000.5', 'Decimal')
+ test_type_conversion_numeric_float8
+-------------------------------------
+ 5000000000.5
+(1 row)
+
+SELECT * FROM test_type_conversion_numeric_float8(1234567890.0987654321);
+INFO: ('1234567890.0987654321', 'Decimal')
+ test_type_conversion_numeric_float8
+-------------------------------------
+ 1234567890.09877
+(1 row)
+
+SELECT * FROM test_type_conversion_numeric_float8(-1234567890.0987654321);
+INFO: ('-1234567890.0987654321', 'Decimal')
+ test_type_conversion_numeric_float8
+-------------------------------------
+ -1234567890.09877
+(1 row)
+
+SELECT * FROM test_type_conversion_numeric_float8(null);
+INFO: ('None', 'NoneType')
+ test_type_conversion_numeric_float8
+-------------------------------------
+
+(1 row)
+
CREATE FUNCTION test_type_conversion_oid(x oid) RETURNS oid AS $$
plpy.info(x, type(x))
return x
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index d6a6a84..8eeefa6 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -46,6 +46,18 @@ static PyObject *PLyDict_FromTuple(PLyDatumToOb *arg, HeapTuple tuple, TupleDesc
/* conversion from Python objects to Datums */
static Datum PLyObject_ToBool(PLyObToDatum *arg, PyObject *plrv,
bool *isnull, bool inarray);
+static Datum PLyObject_ToInt16(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToInt32(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToInt64(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToFloat4(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToFloat8(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
+static Datum PLyObject_ToNumeric(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray);
static Datum PLyObject_ToBytea(PLyObToDatum *arg, PyObject *plrv,
bool *isnull, bool inarray);
static Datum PLyObject_ToComposite(PLyObToDatum *arg, PyObject *plrv,
@@ -397,16 +409,35 @@ PLy_output_setup_func(PLyObToDatum *arg, MemoryContext arg_mcxt,
{
case BOOLOID:
arg->func = PLyObject_ToBool;
- break;
+ return; /* no need to initialize arg->u.scalar */
case BYTEAOID:
arg->func = PLyObject_ToBytea;
+ return; /* no need to initialize arg->u.scalar */
+ case INT2OID:
+ arg->func = PLyObject_ToInt16;
+ break;
+ case INT4OID:
+ arg->func = PLyObject_ToInt32;
+ break;
+ case INT8OID:
+ arg->func = PLyObject_ToInt64;
+ break;
+ case FLOAT4OID:
+ arg->func = PLyObject_ToFloat4;
+ break;
+ case FLOAT8OID:
+ arg->func = PLyObject_ToFloat8;
+ break;
+ case NUMERICOID:
+ arg->func = PLyObject_ToNumeric;
break;
default:
arg->func = PLyObject_ToScalar;
- getTypeInputInfo(typeOid, &typinput, &arg->u.scalar.typioparam);
- fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg_mcxt);
break;
}
+
+ getTypeInputInfo(typeOid, &typinput, &arg->u.scalar.typioparam);
+ fmgr_info_cxt(typinput, &arg->u.scalar.typfunc, arg_mcxt);
}
}
@@ -884,6 +915,340 @@ PLyObject_ToBool(PLyObToDatum *arg, PyObject *plrv,
return BoolGetDatum(PyObject_IsTrue(plrv));
}
+static bool
+PLyNumber_AsDouble(PyObject *plrv, double *res)
+{
+ if (PyFloat_Check(plrv))
+ {
+ *res = PyFloat_AS_DOUBLE(plrv);
+ return true;
+ }
+ else if (PyInt_Check(plrv))
+ {
+ long l = PyInt_AsLong(plrv);
+
+ if (l != -1 || !PyErr_Occurred())
+ {
+ *res = (double) l;
+ return true;
+ }
+ }
+ else if (PyLong_Check(plrv))
+ {
+ *res = PyLong_AsDouble(plrv);
+
+ if (*res != -1 || !PyErr_Occurred())
+ return true;
+ }
+ else if (PyNumber_Check(plrv))
+ {
+ PyObject *f = PyNumber_Float(plrv);
+
+ if (f)
+ {
+ *res = PyFloat_AS_DOUBLE(f);
+ Py_DECREF(f);
+ return true;
+ }
+ }
+ else
+ return false;
+
+ PyErr_Clear();
+ return false;
+}
+
+/*
+ * Try to convert Python exact integer number to C long.
+ * Returns true if the conversion was successful, *overflow is set if the value
+ * overflows long type.
+ */
+static bool
+PLyNumber_AsLong(PyObject *plrv, long *val, bool *overflow)
+{
+ if (PyInt_Check(plrv))
+ *val = PyInt_AsLong(plrv);
+ else if (PyLong_Check(plrv))
+ *val = PyLong_AsLong(plrv);
+ else
+ return false;
+
+ /* If -1 is returned then OverflowError is possible. */
+ *overflow = *val == -1 && PyErr_Occurred();
+
+ if (*overflow)
+ {
+ /* Catch OverflowError exception. */
+ if (!PyErr_ExceptionMatches(PyExc_OverflowError))
+ PLy_elog(ERROR, NULL);
+
+ PyErr_Clear();
+ }
+
+ return true;
+}
+
+static bool
+PLyNumber_ToLong(PyObject *plrv, bool try_float, long *res, bool *overflow,
+ PyObject **pintval)
+{
+ PyObject *intval;
+
+ if (PLyNumber_AsLong(plrv, res, overflow))
+ return true;
+
+ if (!PyNumber_Check(plrv))
+ return false; /* not a number */
+
+ /* Try to convert Python float to int. */
+ if (try_float && PyFloat_Check(plrv))
+ {
+ double dblval = PyFloat_AS_DOUBLE(plrv);
+
+ *overflow = dblval < LONG_MIN || dblval > LONG_MAX;
+
+ if (!*overflow)
+ *res = (long) dblval;
+
+ return true;
+ }
+
+ /* Try to convert a number to Python int/long type to round its value. */
+ intval = PyNumber_Int(plrv);
+
+ if (intval)
+ {
+ bool converted = PLyNumber_AsLong(intval, res, overflow);
+
+ if (pintval)
+ *pintval = intval;
+ else
+ Py_DECREF(intval);
+
+ return converted;
+ }
+ else if (PyErr_Occurred() && PyErr_ExceptionMatches(PyExc_OverflowError))
+ {
+ PyErr_Clear();
+ *overflow = true;
+ return true;
+ }
+ else
+ {
+ PyErr_Clear();
+ return false;
+ }
+}
+
+/*
+ * Convert a Python object to a PostgreSQL int16 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt16(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ long val;
+ bool overflow;
+
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ if (PLyNumber_ToLong(plrv, true, &val, &overflow, NULL))
+ {
+ if (overflow || val < SHRT_MIN || val > SHRT_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
+ errmsg("value \"%s\" is out of range for type %s",
+ PLyObject_AsString(plrv), "smallint")));
+
+ *isnull = false;
+ return Int16GetDatum((int16) val);
+ }
+
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+}
+
+/*
+ * Convert a Python object to a PostgreSQL int32 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt32(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ long val;
+ bool overflow;
+
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ if (PLyNumber_ToLong(plrv, true, &val, &overflow, NULL))
+ {
+ if (overflow ||
+#ifdef HAVE_LONG_INT_64
+ (val < PG_INT32_MIN || val > PG_INT32_MAX)
+#endif
+ )
+ ereport(ERROR,
+ (errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
+ errmsg("value \"%s\" is out of range for type %s",
+ PLyObject_AsString(plrv), "integer")));
+
+ *isnull = false;
+ return Int32GetDatum((int32) val);
+ }
+
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+}
+
+/*
+ * Convert a Python object to a PostgreSQL int64 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToInt64(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ PyObject *ival = NULL;
+ Datum result;
+ long lval;
+ bool overflow;
+
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ /* 64-bit long can't be represented by a double without loss of precision */
+ if (PLyNumber_ToLong(plrv,
+#ifdef HAVE_LONG_INT_64
+ false,
+#else
+ true,
+#endif
+ &lval, &overflow, &ival))
+ {
+ if (!overflow)
+ {
+ Py_XDECREF(ival);
+ *isnull = false;
+ return Int64GetDatum((int64) lval);
+ }
+
+#ifdef HAVE_LONG_INT_64
+ ereport(ERROR,
+ (errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
+ errmsg("value \"%s\" is out of range for type %s",
+ PLyObject_AsString(plrv), "bigint")));
+#else
+ /* try to convert via I/O */
+#endif
+ }
+
+ /* Use integer value instead of original */
+ result = PLyObject_ToScalar(arg, ival ? ival : plrv, isnull, inarray);
+
+ Py_XDECREF(ival);
+
+ return result;
+}
+
+/*
+ * Convert a Python object to a PostgreSQL float4 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToFloat4(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ double res;
+
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ if (PLyNumber_AsDouble(plrv, &res))
+ {
+ *isnull = false;
+ return Float4GetDatum((float4) res);
+ }
+
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+}
+
+/*
+ * Convert a Python object to a PostgreSQL float8 datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToFloat8(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ double res;
+
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ if (PLyNumber_AsDouble(plrv, &res))
+ {
+ *isnull = false;
+ return Float8GetDatum((float8) res);
+ }
+
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+}
+
+/*
+ * Convert a Python object to a PostgreSQL numeric datum directly.
+ * If can not convert it directly, fallback to PLyObject_ToScalar
+ * to convert it.
+ */
+static Datum
+PLyObject_ToNumeric(PLyObToDatum *arg, PyObject *plrv,
+ bool *isnull, bool inarray)
+{
+ long val;
+ bool overflow;
+
+ if (plrv == Py_None)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+
+ if (PLyNumber_AsLong(plrv, &val, &overflow))
+ {
+ if (!overflow)
+ {
+ *isnull = false;
+ return DirectFunctionCall1(int8_numeric,
+ Int64GetDatum((int64) val));
+ }
+
+ /* try to convert via I/O */
+ }
+
+ return PLyObject_ToScalar(arg, plrv, isnull, inarray);
+}
+
/*
* Convert a Python object to a PostgreSQL bytea datum. This doesn't
* go through the generic conversion function to circumvent problems
diff --git a/src/pl/plpython/sql/plpython_types.sql b/src/pl/plpython/sql/plpython_types.sql
index cc0524e..446a4dd 100644
--- a/src/pl/plpython/sql/plpython_types.sql
+++ b/src/pl/plpython/sql/plpython_types.sql
@@ -63,6 +63,52 @@ SELECT * FROM test_type_conversion_int2(100::int2);
SELECT * FROM test_type_conversion_int2(-100::int2);
SELECT * FROM test_type_conversion_int2(null);
+CREATE FUNCTION test_type_conversion_int4_int2(x int4) RETURNS int2 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_int4_int2(100::int4);
+SELECT * FROM test_type_conversion_int4_int2(-100::int4);
+SELECT * FROM test_type_conversion_int4_int2(1000000::int4);
+SELECT * FROM test_type_conversion_int4_int2(-1000000::int4);
+SELECT * FROM test_type_conversion_int4_int2(null);
+
+CREATE FUNCTION test_type_conversion_int8_int2(x int8) RETURNS int2 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_int8_int2(100::int8);
+SELECT * FROM test_type_conversion_int8_int2(-100::int8);
+SELECT * FROM test_type_conversion_int8_int2(1000000::int8);
+SELECT * FROM test_type_conversion_int8_int2(-1000000::int8);
+SELECT * FROM test_type_conversion_int8_int2(null);
+
+CREATE FUNCTION test_type_conversion_float8_int2(x float8) RETURNS int2 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_float8_int2(100::float8);
+SELECT * FROM test_type_conversion_float8_int2(-100::float8);
+SELECT * FROM test_type_conversion_float8_int2(100.345::float8);
+SELECT * FROM test_type_conversion_float8_int2(-100.678::float8);
+SELECT * FROM test_type_conversion_float8_int2(1000000::float8);
+SELECT * FROM test_type_conversion_float8_int2(-1000000::float8);
+SELECT * FROM test_type_conversion_float8_int2(null);
+
+CREATE FUNCTION test_type_conversion_text_int2(x text) RETURNS int2 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_text_int2('100');
+SELECT * FROM test_type_conversion_text_int2('-100');
+SELECT * FROM test_type_conversion_text_int2('1000000000000000');
+SELECT * FROM test_type_conversion_text_int2('1.23');
+SELECT * FROM test_type_conversion_text_int2('aaa');
+
CREATE FUNCTION test_type_conversion_int4(x int4) RETURNS int4 AS $$
plpy.info(x, type(x))
@@ -74,6 +120,43 @@ SELECT * FROM test_type_conversion_int4(-100);
SELECT * FROM test_type_conversion_int4(null);
+CREATE FUNCTION test_type_conversion_int8_int4(x int8) RETURNS int4 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_int8_int4(100);
+SELECT * FROM test_type_conversion_int8_int4(-100);
+SELECT * FROM test_type_conversion_int8_int4(10000000000000);
+SELECT * FROM test_type_conversion_int8_int4(-10000000000000);
+SELECT * FROM test_type_conversion_int8_int4(null);
+
+CREATE FUNCTION test_type_conversion_numeric_int4(x numeric) RETURNS int4 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_numeric_int4(100::numeric);
+SELECT * FROM test_type_conversion_numeric_int4(-100::numeric);
+SELECT * FROM test_type_conversion_numeric_int4(1000000::numeric);
+SELECT * FROM test_type_conversion_numeric_int4(-1000000::numeric);
+SELECT * FROM test_type_conversion_numeric_int4(100000000000000000000::numeric);
+SELECT * FROM test_type_conversion_numeric_int4(-100000000000000000000::numeric);
+SELECT * FROM test_type_conversion_numeric_int4(null);
+
+CREATE FUNCTION test_type_conversion_long_int4(x numeric) RETURNS int4 AS $$
+plpy.info(x, type(x))
+return long(x)
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_long_int4(100::numeric);
+SELECT * FROM test_type_conversion_long_int4(-100::numeric);
+SELECT * FROM test_type_conversion_long_int4(1000000::numeric);
+SELECT * FROM test_type_conversion_long_int4(-1000000::numeric);
+SELECT * FROM test_type_conversion_long_int4(100000000000000000000::numeric);
+SELECT * FROM test_type_conversion_long_int4(-100000000000000000000::numeric);
+
+
CREATE FUNCTION test_type_conversion_int8(x int8) RETURNS int8 AS $$
plpy.info(x, type(x))
return x
@@ -84,6 +167,31 @@ SELECT * FROM test_type_conversion_int8(-100);
SELECT * FROM test_type_conversion_int8(5000000000);
SELECT * FROM test_type_conversion_int8(null);
+CREATE FUNCTION test_type_conversion_float8_int8(x float8) RETURNS int8 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_float8_int8(100::float8);
+SELECT * FROM test_type_conversion_float8_int8(-100::float8);
+SELECT * FROM test_type_conversion_float8_int8(100.345::float8);
+SELECT * FROM test_type_conversion_float8_int8(-100.678::float8);
+SELECT * FROM test_type_conversion_float8_int8(100000000000.345::float8);
+SELECT * FROM test_type_conversion_float8_int8(-100000000000.678::float8);
+SELECT * FROM test_type_conversion_float8_int8(100000000000000000000000::float8);
+SELECT * FROM test_type_conversion_float8_int8(-100000000000000000000000::float8);
+SELECT * FROM test_type_conversion_float8_int8(null);
+
+CREATE FUNCTION test_type_conversion_text_int8(x text) RETURNS int8 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_text_int8('100');
+SELECT * FROM test_type_conversion_text_int8('-100');
+SELECT * FROM test_type_conversion_text_int8('1000000000000000');
+SELECT * FROM test_type_conversion_text_int8('100000000000000000000000');
+SELECT * FROM test_type_conversion_text_int8('aaa');
CREATE FUNCTION test_type_conversion_numeric(x numeric) RETURNS numeric AS $$
# print just the class name, not the type, to avoid differences
@@ -101,6 +209,15 @@ SELECT * FROM test_type_conversion_numeric(1234567890.0987654321);
SELECT * FROM test_type_conversion_numeric(-1234567890.0987654321);
SELECT * FROM test_type_conversion_numeric(null);
+CREATE FUNCTION test_type_conversion_int4_numeric(x int4) RETURNS numeric AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_int4_numeric(100::int4);
+SELECT * FROM test_type_conversion_int4_numeric(-100::int4);
+SELECT * FROM test_type_conversion_int4_numeric(null);
+
CREATE FUNCTION test_type_conversion_float4(x float4) RETURNS float4 AS $$
plpy.info(x, type(x))
@@ -112,6 +229,17 @@ SELECT * FROM test_type_conversion_float4(-100);
SELECT * FROM test_type_conversion_float4(5000.5);
SELECT * FROM test_type_conversion_float4(null);
+CREATE FUNCTION test_type_conversion_text_float4(x text) RETURNS float4 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_text_float4('100');
+SELECT * FROM test_type_conversion_text_float4('-100');
+SELECT * FROM test_type_conversion_text_float4('5000.5');
+SELECT * FROM test_type_conversion_text_float4('aaa');
+SELECT * FROM test_type_conversion_text_float4(null);
+
CREATE FUNCTION test_type_conversion_float8(x float8) RETURNS float8 AS $$
plpy.info(x, type(x))
@@ -124,6 +252,53 @@ SELECT * FROM test_type_conversion_float8(5000000000.5);
SELECT * FROM test_type_conversion_float8(null);
SELECT * FROM test_type_conversion_float8(100100100.654321);
+CREATE FUNCTION test_type_conversion_text_float8(x text) RETURNS float8 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_text_float8('100');
+SELECT * FROM test_type_conversion_text_float8('-100');
+SELECT * FROM test_type_conversion_text_float8('5000000000.5');
+SELECT * FROM test_type_conversion_text_float8('100100100.654321');
+SELECT * FROM test_type_conversion_text_float8(null);
+
+CREATE FUNCTION test_type_conversion_int4_float8(x int4) RETURNS float8 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_int4_float8(100);
+SELECT * FROM test_type_conversion_int4_float8(-100);
+SELECT * FROM test_type_conversion_int4_float8(null);
+
+CREATE FUNCTION test_type_conversion_int8_float8(x int8) RETURNS float8 AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_int8_float8(100::int8);
+SELECT * FROM test_type_conversion_int8_float8(-100::int8);
+SELECT * FROM test_type_conversion_int8_float8(10000000000000::int8);
+SELECT * FROM test_type_conversion_int8_float8(-10000000000000::int8);
+SELECT * FROM test_type_conversion_int8_float8(null);
+
+CREATE FUNCTION test_type_conversion_numeric_float8(x numeric) RETURNS float8 AS $$
+# print just the class name, not the type, to avoid differences
+# between decimal and cdecimal
+plpy.info(str(x), x.__class__.__name__)
+return x
+$$ LANGUAGE plpythonu;
+
+SELECT * FROM test_type_conversion_numeric_float8(100);
+SELECT * FROM test_type_conversion_numeric_float8(-100);
+SELECT * FROM test_type_conversion_numeric_float8(100.0);
+SELECT * FROM test_type_conversion_numeric_float8(100.00);
+SELECT * FROM test_type_conversion_numeric_float8(5000000000.5);
+SELECT * FROM test_type_conversion_numeric_float8(1234567890.0987654321);
+SELECT * FROM test_type_conversion_numeric_float8(-1234567890.0987654321);
+SELECT * FROM test_type_conversion_numeric_float8(null);
+
CREATE FUNCTION test_type_conversion_oid(x oid) RETURNS oid AS $$
plpy.info(x, type(x))
On 12/07/18 18:06, Nikita Glukhov wrote:
I have added some cross-type test cases and now almost all new code is covered
(excluding several error cases which can be triggered only by custom numeric
type implementations).
Thanks! Some of those new tests actually fail, if you run them against
unpatched master. For example:
SELECT * FROM test_type_conversion_float8_int2(100::float8);
INFO: (100.0, <type 'float'>)
- test_type_conversion_float8_int2
-----------------------------------
- 100
-(1 row)
-
+ERROR: invalid input syntax for integer: "100.0"
+CONTEXT: while creating return value
+PL/Python function "test_type_conversion_float8_int2"
So this patch is making some subtle changes to behavior. I don't think
we want that.
- Heikki
+1, I also think that we may not change the previous behavior of plpython.
@Nikita Glukhov <n.gluhov@postgrespro.ru> maybe we just check the
whether pyobject is int or long only in related conversion functions, and
fallback otherwise?
On Fri, Jul 13, 2018 at 12:09 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 12/07/18 18:06, Nikita Glukhov wrote:
I have added some cross-type test cases and now almost all new code is
covered
(excluding several error cases which can be triggered only by custom
numeric
type implementations).
Thanks! Some of those new tests actually fail, if you run them against
unpatched master. For example:SELECT * FROM test_type_conversion_float8_int2(100::float8); INFO: (100.0, <type 'float'>) - test_type_conversion_float8_int2 ----------------------------------- - 100 -(1 row) - +ERROR: invalid input syntax for integer: "100.0" +CONTEXT: while creating return value +PL/Python function "test_type_conversion_float8_int2"So this patch is making some subtle changes to behavior. I don't think
we want that.- Heikki
--
Regards,
Haozhou
On Mon, Jul 16, 2018 at 03:35:57PM +0800, Haozhou Wang wrote:
+1, I also think that we may not change the previous behavior of plpython. @Nikita Glukhov <n.gluhov@postgrespro.ru> maybe we just check the whether pyobject is int or long only in related conversion functions, and fallback otherwise?
This patch was around for some time, and the changes in behavior are not
really nice, so this is marked as returned with feedback.
--
Michael