pgsql: Transforms for jsonb to PL/Python

On 3/28/18 08:39, Peter Eisentraut wrote:

Transforms for jsonb to PL/Python

Needs fixes for Windows build. I can give it a blind attempt, but help
would be welcome.

I'm looking into the regression test failures that appear to be minor
output differences.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:

I'm looking into the regression test failures that appear to be minor
output differences.

gaur (python 2.5) is showing a different output pathology:

*** 246,253 ****
  return x
  $$;
  SELECT test1inf();
! ERROR:  could not convert value "inf" to jsonb
! CONTEXT:  while creating return value
  PL/Python function "test1inf"
  -- test python -infinity -> jsonb
  CREATE FUNCTION test2inf() RETURNS jsonb
--- 246,255 ----
  return x
  $$;
  SELECT test1inf();
! ERROR:  ValueError: invalid literal for float(): inf
! CONTEXT:  Traceback (most recent call last):
!   PL/Python function "test1inf", line 2, in <module>
!     x = float('inf')
  PL/Python function "test1inf"
  -- test python -infinity -> jsonb
  CREATE FUNCTION test2inf() RETURNS jsonb
***************

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=gaur&amp;dt=2018-03-31%2002%3A50%3A13

regards, tom lane

I wrote:

gaur (python 2.5) is showing a different output pathology:

A bit of google research indicates that Python did not have reliable
handling of infinity or NaN floats until 2.6; before that, you could
get failures or platform-specific results from code like float('inf').
So this test ain't gonna work as-is on old versions.

I see a few options:

1. Just remove the tests involving inf/nan. As far as I can see,
there's no explicit handling of these cases in jsonb_plpython,
so we're not really buying any test coverage of our own code anyway.

2. Maintain additional expected-output files. One problem with this
is it's not clear how many we'd need.

3. Somehow tweak the test cases in a different way, so that they're
not exercised before 2.6.

4. Move our minimum supported Python version up to 2.6 (from 2.4
where it is now).

#4 seems like an overreaction at first sight, but on the other
hand, I wonder how many people really care about pre-2.6 anymore
anyway. (2.4 was released Nov. 2004, 2.6 Oct 2008.) AFAIR, the only
reason I'm running 2.5 on gaur/pademelon is we claim to support it.

regards, tom lane

Hi,

On Sun, 2018-04-01 at 18:07 -0400, Tom Lane wrote:

4. Move our minimum supported Python version up to 2.6 (from 2.4
where it is now).

#4 seems like an overreaction at first sight, but on the other
hand, I wonder how many people really care about pre-2.6 anymore
anyway. (2.4 was released Nov. 2004, 2.6 Oct 2008.) AFAIR, the only
reason I'm running 2.5 on gaur/pademelon is we claim to support it.

FWIW, 2.6 is also the version that RHEL 6 has, so we still can support it.

Regards,
--
Devrim Gündüz
EnterpriseDB: https://www.enterprisedb.com
PostgreSQL Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <devrim@gunduz.org> writes:

On Sun, 2018-04-01 at 18:07 -0400, Tom Lane wrote:

4. Move our minimum supported Python version up to 2.6 (from 2.4
where it is now).

#4 seems like an overreaction at first sight, but on the other
hand, I wonder how many people really care about pre-2.6 anymore
anyway. (2.4 was released Nov. 2004, 2.6 Oct 2008.) AFAIR, the only
reason I'm running 2.5 on gaur/pademelon is we claim to support it.

FWIW, 2.6 is also the version that RHEL 6 has, so we still can support it.

I poked around in the buildfarm logs to see what versions we're actually
testing. It's hard to tell in some cases. (I notice that configure does
actually extract python_version, but doesn't display it ... should we make
it do so?) However, it's clear from version numbers embedded in places
like library paths that we are testing with python 2.4.x on aholehole,
arapaima, and prairiedog, and with 2.5.x on gaur/pademelon and locust.
aholehole and arapaima are evidently using the platform-supplied version
on RHEL 5, and locust the platform-supplied version on macOS 10.5.8.

So while those are surely trailing-edge platforms, I'm no longer thinking
that we can realistically claim there's zero remaining real-world interest
in these Python versions, and so option #4 doesn't seem workable.

I think option #1 (drop the tests with inf/nan) is the most attractive
of the remaining choices. It's surely the least work, and AFAICS
we aren't actually buying any code coverage with these test cases.
(It is interesting though that we have four other pre-2.6 platforms that
are passing these tests; apparently the pre-2.6 behavior wasn't broken
everywhere.)

Unless someone volunteers to undertake one of the other options,
I'll go remove these tests shortly.

regards, tom lane

Hi,

On Mon, 2018-04-02 at 10:59 -0400, Tom Lane wrote:

aholehole and arapaima are evidently using the platform-supplied version
on RHEL 5, and locust the platform-supplied version on macOS 10.5.8.

Yeah, I'm running arapaima and aholehole, they have stock RHEL 5 packages
(Python 2.4)

So while those are surely trailing-edge platforms, I'm no longer thinking
that we can realistically claim there's zero remaining real-world interest
in these Python versions, and so option #4 doesn't seem workable.

RHEL 5 is already EOL, last supported (community) RPM release was 9.6, so it
won't hurt to drop Python 2.4 support from RPM point of view.

Regards,

--
Devrim Gündüz
EnterpriseDB: https://www.enterprisedb.com
PostgreSQL Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR

On 4/1/18 18:07, Tom Lane wrote:

1. Just remove the tests involving inf/nan. As far as I can see,
there's no explicit handling of these cases in jsonb_plpython,
so we're not really buying any test coverage of our own code anyway.

That makes sense.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:

On 4/1/18 18:07, Tom Lane wrote:

1. Just remove the tests involving inf/nan. As far as I can see,
there's no explicit handling of these cases in jsonb_plpython,
so we're not really buying any test coverage of our own code anyway.

That makes sense.

OK, done.

regards, tom lane