[patch] PL/Python is too lossy with floats
PL/Python uses str(v) to convert float data, but is lossy
by design. Only repr(v) is guaranteed to have enough
precision to make floats roundtrip properly:
https://docs.python.org/2/library/functions.html#func-repr
https://docs.python.org/2/library/functions.html#str
Example:
$ python
repr(100100100.654321)
'100100100.654321'
str(100100100.654321)
'100100100.654'
Attached patch uses PyObject_Repr() for float data.
As it's annoying-to-debug problem and the patch is simple,
perhaps it's worth backporting?
--
marko
Attachments:
plpy-float-repr.difftext/x-diff; charset=us-asciiDownload
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index 61d800b..17057a5 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -354,6 +354,14 @@ CONTEXT: PL/Python function "test_type_conversion_float8"
(1 row)
+SELECT * FROM test_type_conversion_float8(100100100.654321);
+INFO: (100100100.654321, <type 'float'>)
+CONTEXT: PL/Python function "test_type_conversion_float8"
+ test_type_conversion_float8
+-----------------------------
+ 100100100.654321
+(1 row)
+
CREATE FUNCTION test_type_conversion_oid(x oid) RETURNS oid AS $$
plpy.info(x, type(x))
return x
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 524534e..8c70c7c 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -760,6 +760,18 @@ PLyObject_ToDatum(PLyObToDatum *arg, int32 typmod, PyObject *plrv)
if (PyUnicode_Check(plrv))
plrv_bo = PLyUnicode_Bytes(plrv);
+ else if (PyFloat_Check(plrv))
+ {
+ /* use repr() for floats, str() is lossy */
+#if PY_MAJOR_VERSION >= 3
+ PyObject *s = PyObject_Repr(plrv);
+
+ plrv_bo = PLyUnicode_Bytes(s);
+ Py_XDECREF(s);
+#else
+ plrv_bo = PyObject_Repr(plrv);
+#endif
+ }
else
{
#if PY_MAJOR_VERSION >= 3
diff --git a/src/pl/plpython/sql/plpython_types.sql b/src/pl/plpython/sql/plpython_types.sql
index d9d0db6..19d920d 100644
--- a/src/pl/plpython/sql/plpython_types.sql
+++ b/src/pl/plpython/sql/plpython_types.sql
@@ -122,6 +122,7 @@ SELECT * FROM test_type_conversion_float8(100);
SELECT * FROM test_type_conversion_float8(-100);
SELECT * FROM test_type_conversion_float8(5000000000.5);
SELECT * FROM test_type_conversion_float8(null);
+SELECT * FROM test_type_conversion_float8(100100100.654321);
CREATE FUNCTION test_type_conversion_oid(x oid) RETURNS oid AS $$
On 3/3/15 9:32 AM, Marko Kreen wrote:
PL/Python uses str(v) to convert float data, but is lossy
by design. Only repr(v) is guaranteed to have enough
precision to make floats roundtrip properly:
committed
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Mar 11, 2015 at 9:49 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On 3/3/15 9:32 AM, Marko Kreen wrote:
PL/Python uses str(v) to convert float data, but is lossy
by design. Only repr(v) is guaranteed to have enough
precision to make floats roundtrip properly:committed
In 9.3 and before, numeric is converted to float in PL/Python.
Please reconsider backporting.
--
marko
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 6/1/15 5:34 AM, Marko Kreen wrote:
On Wed, Mar 11, 2015 at 9:49 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On 3/3/15 9:32 AM, Marko Kreen wrote:
PL/Python uses str(v) to convert float data, but is lossy
by design. Only repr(v) is guaranteed to have enough
precision to make floats roundtrip properly:committed
In 9.3 and before, numeric is converted to float in PL/Python.
Please reconsider backporting.
It's been like this forever, so I don't think it's appropriate to
backpatch this.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Peter Eisentraut <peter_e@gmx.net> writes:
On 6/1/15 5:34 AM, Marko Kreen wrote:
Please reconsider backporting.
It's been like this forever, so I don't think it's appropriate to
backpatch this.
http://www.postgresql.org/docs/devel/static/plpython-data.html
states in so many words that floats are converted with str().
Presumably this needs an update in HEAD, but I would say that
the current behavior is per specification in all back branches.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers