hstore_plpython regression test does not work on Python 3
As exhibited for instance here:
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=spoonbill&dt=2015-05-16%2011%3A00%3A07
I've been able to replicate this on a Fedora 21 box: works fine with
Python 2, fails with Python 3. Seems like we still have an issue
with reliance on a system-provided sort method.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 5/16/15 12:06 PM, Tom Lane wrote:
As exhibited for instance here:
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=spoonbill&dt=2015-05-16%2011%3A00%3A07
I've been able to replicate this on a Fedora 21 box: works fine with
Python 2, fails with Python 3. Seems like we still have an issue
with reliance on a system-provided sort method.
Pushed a fix, tested with 2.3 .. 3.4.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
* Peter Eisentraut wrote:
On 5/16/15 12:06 PM, Tom Lane wrote:
As exhibited for instance here:
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=spoonbill&dt=2015-05-16%2011%3A00%3A07
I've been able to replicate this on a Fedora 21 box: works fine with
Python 2, fails with Python 3. Seems like we still have an issue
with reliance on a system-provided sort method.Pushed a fix, tested with 2.3 .. 3.4.
There is still a sorting problem (of sorts). jaguarundi [1]http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=jaguarundi&br=HEAD keeps
failing intermittently like this:
*** 47,53 ****
return len(val)
$$;
SELECT test1arr(array['aa=>bb, cc=>NULL'::hstore, 'dd=>ee']);
! INFO: [{'aa': 'bb', 'cc': None}, {'dd': 'ee'}]
CONTEXT: PL/Python function "test1arr"
test1arr
----------
--- 47,53 ----
return len(val)
$$;
SELECT test1arr(array['aa=>bb, cc=>NULL'::hstore, 'dd=>ee']);
! INFO: [{'cc': None, 'aa': 'bb'}, {'dd': 'ee'}]
CONTEXT: PL/Python function "test1arr"
test1arr
----------I cannot find any other animal that does the same, but I doubt it's due
to CCA this time.
Should dict tests perhaps output sorted(thedict.items()) instead?
Testing dict formatting could be done with single-item dicts.
[1]: http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=jaguarundi&br=HEAD
--
Christian
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
22.05.2015, 09:44, Christian Ullrich kirjoitti:
* Peter Eisentraut wrote:
On 5/16/15 12:06 PM, Tom Lane wrote:
As exhibited for instance here:
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=spoonbill&dt=2015-05-16%2011%3A00%3A07
I've been able to replicate this on a Fedora 21 box: works fine with
Python 2, fails with Python 3. Seems like we still have an issue
with reliance on a system-provided sort method.Pushed a fix, tested with 2.3 .. 3.4.
There is still a sorting problem (of sorts). jaguarundi [1] keeps
failing intermittently like this:*** 47,53 **** return len(val) $$; SELECT test1arr(array['aa=>bb, cc=>NULL'::hstore, 'dd=>ee']); ! INFO: [{'aa': 'bb', 'cc': None}, {'dd': 'ee'}] CONTEXT: PL/Python function "test1arr" test1arr ---------- --- 47,53 ---- return len(val) $$; SELECT test1arr(array['aa=>bb, cc=>NULL'::hstore, 'dd=>ee']); ! INFO: [{'cc': None, 'aa': 'bb'}, {'dd': 'ee'}] CONTEXT: PL/Python function "test1arr" test1arr ----------I cannot find any other animal that does the same, but I doubt it's due
to CCA this time.Should dict tests perhaps output sorted(thedict.items()) instead?
Testing dict formatting could be done with single-item dicts.[1] http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=jaguarundi&br=HEAD
Looks like that animal uses Python 3.4. Python 3.3 and newer versions
default to using a random seed for hashing objects into dicts which
makes the order of dict elements random; see
https://docs.python.org/3/using/cmdline.html#cmdoption-R
The test case could be changed to use sorted(dict.items()) always, but
there are multiple places where it would need to be applied. Setting
the PYTHONHASHSEED environment variable to a stable value would probably
be easier.
/ Oskari
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 5/26/15 5:19 PM, Oskari Saarenmaa wrote:
[1] http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=jaguarundi&br=HEAD
Looks like that animal uses Python 3.4. Python 3.3 and newer versions
default to using a random seed for hashing objects into dicts which
makes the order of dict elements random; see
https://docs.python.org/3/using/cmdline.html#cmdoption-R
Ah, good catch. That explains the, well, randomness. I can reproduce
the test failures with PYTHONHASHSEED=2.
But I haven't been successful getting that environment variable set so
that it works in the installcheck case. Instead, I have rewritten the
tests to use asserts instead of textual comparisons. See attached
patch. Comments?
Attachments:
hstore-plpython-test.patchtext/x-diff; name=hstore-plpython-test.patchDownload
diff --git a/contrib/hstore_plpython/expected/hstore_plpython.out b/contrib/hstore_plpython/expected/hstore_plpython.out
index 6252836..b7a6a92 100644
--- a/contrib/hstore_plpython/expected/hstore_plpython.out
+++ b/contrib/hstore_plpython/expected/hstore_plpython.out
@@ -43,12 +43,10 @@ CREATE FUNCTION test1arr(val hstore[]) RETURNS int
LANGUAGE plpythonu
TRANSFORM FOR TYPE hstore
AS $$
-plpy.info(repr(val))
+assert(val == [{'aa': 'bb', 'cc': None}, {'dd': 'ee'}])
return len(val)
$$;
SELECT test1arr(array['aa=>bb, cc=>NULL'::hstore, 'dd=>ee']);
-INFO: [{'aa': 'bb', 'cc': None}, {'dd': 'ee'}]
-CONTEXT: PL/Python function "test1arr"
test1arr
----------
2
@@ -88,18 +86,14 @@ LANGUAGE plpythonu
TRANSFORM FOR TYPE hstore
AS $$
rv = plpy.execute("SELECT 'aa=>bb, cc=>NULL'::hstore AS col1")
-plpy.info(repr(rv[0]["col1"]))
+assert(rv[0]["col1"] == {'aa': 'bb', 'cc': None})
val = {'a': 1, 'b': 'boo', 'c': None}
plan = plpy.prepare("SELECT $1::text AS col1", ["hstore"])
rv = plpy.execute(plan, [val])
-plpy.info(repr(rv[0]["col1"]))
+assert(rv[0]["col1"] == '"a"=>"1", "b"=>"boo", "c"=>NULL')
$$;
SELECT test3();
-INFO: {'aa': 'bb', 'cc': None}
-CONTEXT: PL/Python function "test3"
-INFO: '"a"=>"1", "b"=>"boo", "c"=>NULL'
-CONTEXT: PL/Python function "test3"
test3
-------
@@ -118,7 +112,7 @@ CREATE FUNCTION test4() RETURNS trigger
LANGUAGE plpythonu
TRANSFORM FOR TYPE hstore
AS $$
-plpy.info("Trigger row: {'a': %r, 'b': %r}" % (TD["new"]["a"], TD["new"]["b"]))
+assert(TD["new"] == {'a': 1, 'b': {'aa': 'bb', 'cc': None}})
if TD["new"]["a"] == 1:
TD["new"]["b"] = {'a': 1, 'b': 'boo', 'c': None}
@@ -126,8 +120,6 @@ return "MODIFY"
$$;
CREATE TRIGGER test4 BEFORE UPDATE ON test1 FOR EACH ROW EXECUTE PROCEDURE test4();
UPDATE test1 SET a = a;
-INFO: Trigger row: {'a': 1, 'b': {'aa': 'bb', 'cc': None}}
-CONTEXT: PL/Python function "test4"
SELECT * FROM test1;
a | b
---+---------------------------------
diff --git a/contrib/hstore_plpython/sql/hstore_plpython.sql b/contrib/hstore_plpython/sql/hstore_plpython.sql
index 872d8c6..9ff2ebc 100644
--- a/contrib/hstore_plpython/sql/hstore_plpython.sql
+++ b/contrib/hstore_plpython/sql/hstore_plpython.sql
@@ -37,7 +37,7 @@ CREATE FUNCTION test1arr(val hstore[]) RETURNS int
LANGUAGE plpythonu
TRANSFORM FOR TYPE hstore
AS $$
-plpy.info(repr(val))
+assert(val == [{'aa': 'bb', 'cc': None}, {'dd': 'ee'}])
return len(val)
$$;
@@ -74,12 +74,12 @@ CREATE FUNCTION test3() RETURNS void
TRANSFORM FOR TYPE hstore
AS $$
rv = plpy.execute("SELECT 'aa=>bb, cc=>NULL'::hstore AS col1")
-plpy.info(repr(rv[0]["col1"]))
+assert(rv[0]["col1"] == {'aa': 'bb', 'cc': None})
val = {'a': 1, 'b': 'boo', 'c': None}
plan = plpy.prepare("SELECT $1::text AS col1", ["hstore"])
rv = plpy.execute(plan, [val])
-plpy.info(repr(rv[0]["col1"]))
+assert(rv[0]["col1"] == '"a"=>"1", "b"=>"boo", "c"=>NULL')
$$;
SELECT test3();
@@ -94,7 +94,7 @@ CREATE FUNCTION test4() RETURNS trigger
LANGUAGE plpythonu
TRANSFORM FOR TYPE hstore
AS $$
-plpy.info("Trigger row: {'a': %r, 'b': %r}" % (TD["new"]["a"], TD["new"]["b"]))
+assert(TD["new"] == {'a': 1, 'b': {'aa': 'bb', 'cc': None}})
if TD["new"]["a"] == 1:
TD["new"]["b"] = {'a': 1, 'b': 'boo', 'c': None}
Peter Eisentraut <peter_e@gmx.net> writes:
On 5/26/15 5:19 PM, Oskari Saarenmaa wrote:
Looks like that animal uses Python 3.4. Python 3.3 and newer versions
default to using a random seed for hashing objects into dicts which
makes the order of dict elements random; see
https://docs.python.org/3/using/cmdline.html#cmdoption-R
Ah, good catch. That explains the, well, randomness. I can reproduce
the test failures with PYTHONHASHSEED=2.
But I haven't been successful getting that environment variable set so
that it works in the installcheck case.
Yeah, there's pretty much no chance of controlling the postmaster's
environment in installcheck cases.
Instead, I have rewritten the tests to use asserts instead of textual
comparisons. See attached patch. Comments?
If that works back to Python 2.3 or whatever is the oldest we support,
sounds good to me.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
29.05.2015, 03:12, Peter Eisentraut kirjoitti:
On 5/26/15 5:19 PM, Oskari Saarenmaa wrote:
[1] http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=jaguarundi&br=HEAD
Looks like that animal uses Python 3.4. Python 3.3 and newer versions
default to using a random seed for hashing objects into dicts which
makes the order of dict elements random; see
https://docs.python.org/3/using/cmdline.html#cmdoption-RAh, good catch. That explains the, well, randomness. I can reproduce
the test failures with PYTHONHASHSEED=2.But I haven't been successful getting that environment variable set so
that it works in the installcheck case. Instead, I have rewritten the
tests to use asserts instead of textual comparisons. See attached
patch. Comments?
Looks good to me.
/ Oskari
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers