hstore_plpython regression test does not work on Python 3

On 5/16/15 12:06 PM, Tom Lane wrote:

As exhibited for instance here:

http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=spoonbill&dt=2015-05-16%2011%3A00%3A07

I've been able to replicate this on a Fedora 21 box: works fine with
Python 2, fails with Python 3. Seems like we still have an issue
with reliance on a system-provided sort method.

Pushed a fix, tested with 2.3 .. 3.4.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

* Peter Eisentraut wrote:

On 5/16/15 12:06 PM, Tom Lane wrote:

As exhibited for instance here:

http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=spoonbill&dt=2015-05-16%2011%3A00%3A07

I've been able to replicate this on a Fedora 21 box: works fine with
Python 2, fails with Python 3. Seems like we still have an issue
with reliance on a system-provided sort method.

Pushed a fix, tested with 2.3 .. 3.4.

There is still a sorting problem (of sorts). jaguarundi [1]http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=jaguarundi&br=HEAD keeps
failing intermittently like this:

*** 47,53 ****
   return len(val)
   $$;
   SELECT test1arr(array['aa=>bb, cc=>NULL'::hstore, 'dd=>ee']);
! INFO:  [{'aa': 'bb', 'cc': None}, {'dd': 'ee'}]
   CONTEXT:  PL/Python function "test1arr"
    test1arr
   ----------
--- 47,53 ----
   return len(val)
   $$;
   SELECT test1arr(array['aa=>bb, cc=>NULL'::hstore, 'dd=>ee']);
! INFO:  [{'cc': None, 'aa': 'bb'}, {'dd': 'ee'}]
   CONTEXT:  PL/Python function "test1arr"
    test1arr
   ----------

I cannot find any other animal that does the same, but I doubt it's due
to CCA this time.

Should dict tests perhaps output sorted(thedict.items()) instead?
Testing dict formatting could be done with single-item dicts.

[1]: http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=jaguarundi&br=HEAD

--
Christian

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

22.05.2015, 09:44, Christian Ullrich kirjoitti:

* Peter Eisentraut wrote:

On 5/16/15 12:06 PM, Tom Lane wrote:

As exhibited for instance here:

http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=spoonbill&dt=2015-05-16%2011%3A00%3A07

I've been able to replicate this on a Fedora 21 box: works fine with
Python 2, fails with Python 3. Seems like we still have an issue
with reliance on a system-provided sort method.

Pushed a fix, tested with 2.3 .. 3.4.

There is still a sorting problem (of sorts). jaguarundi [1] keeps
failing intermittently like this:

*** 47,53 ****
return len(val)
$$;
SELECT test1arr(array['aa=>bb, cc=>NULL'::hstore, 'dd=>ee']);
! INFO:  [{'aa': 'bb', 'cc': None}, {'dd': 'ee'}]
CONTEXT:  PL/Python function "test1arr"
test1arr
----------
--- 47,53 ----
return len(val)
$$;
SELECT test1arr(array['aa=>bb, cc=>NULL'::hstore, 'dd=>ee']);
! INFO:  [{'cc': None, 'aa': 'bb'}, {'dd': 'ee'}]
CONTEXT:  PL/Python function "test1arr"
test1arr
----------

I cannot find any other animal that does the same, but I doubt it's due
to CCA this time.

Should dict tests perhaps output sorted(thedict.items()) instead?
Testing dict formatting could be done with single-item dicts.

[1] http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=jaguarundi&br=HEAD

Looks like that animal uses Python 3.4. Python 3.3 and newer versions
default to using a random seed for hashing objects into dicts which
makes the order of dict elements random; see
https://docs.python.org/3/using/cmdline.html#cmdoption-R

The test case could be changed to use sorted(dict.items()) always, but
there are multiple places where it would need to be applied. Setting
the PYTHONHASHSEED environment variable to a stable value would probably
be easier.

/ Oskari

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 5/26/15 5:19 PM, Oskari Saarenmaa wrote:

[1] http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=jaguarundi&br=HEAD

Looks like that animal uses Python 3.4. Python 3.3 and newer versions
default to using a random seed for hashing objects into dicts which
makes the order of dict elements random; see
https://docs.python.org/3/using/cmdline.html#cmdoption-R

Ah, good catch. That explains the, well, randomness. I can reproduce
the test failures with PYTHONHASHSEED=2.

But I haven't been successful getting that environment variable set so
that it works in the installcheck case. Instead, I have rewritten the
tests to use asserts instead of textual comparisons. See attached
patch. Comments?

Peter Eisentraut <peter_e@gmx.net> writes:

On 5/26/15 5:19 PM, Oskari Saarenmaa wrote:

Looks like that animal uses Python 3.4. Python 3.3 and newer versions
default to using a random seed for hashing objects into dicts which
makes the order of dict elements random; see
https://docs.python.org/3/using/cmdline.html#cmdoption-R

Ah, good catch. That explains the, well, randomness. I can reproduce
the test failures with PYTHONHASHSEED=2.

But I haven't been successful getting that environment variable set so
that it works in the installcheck case.

Yeah, there's pretty much no chance of controlling the postmaster's
environment in installcheck cases.

Instead, I have rewritten the tests to use asserts instead of textual
comparisons. See attached patch. Comments?

If that works back to Python 2.3 or whatever is the oldest we support,
sounds good to me.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

29.05.2015, 03:12, Peter Eisentraut kirjoitti:

On 5/26/15 5:19 PM, Oskari Saarenmaa wrote:

[1] http://pgbuildfarm.org/cgi-bin/show_history.pl?nm=jaguarundi&amp;br=HEAD

Looks like that animal uses Python 3.4. Python 3.3 and newer versions
default to using a random seed for hashing objects into dicts which
makes the order of dict elements random; see
https://docs.python.org/3/using/cmdline.html#cmdoption-R

Ah, good catch. That explains the, well, randomness. I can reproduce
the test failures with PYTHONHASHSEED=2.

But I haven't been successful getting that environment variable set so
that it works in the installcheck case. Instead, I have rewritten the
tests to use asserts instead of textual comparisons. See attached
patch. Comments?

Looks good to me.

/ Oskari

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers