improve type conversion of SPI_processed in Python

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:

Here is a patch to improves how PL/Python deals with very large values
of SPI_processed. The previous code converts anything that does not fit
into a C long into a Python float. But Python long has unlimited
precision, so we should be using that instead. And in Python 3, int and
long as the same, so there is no need to deal with any variations anymore.

I took a quick look at this. +1 for returning Python long all the time,
but I wonder why the Python version dependency. Our existing function
PLyLong_FromInt64() believes that PyLong_FromLongLong is unconditionally
available. I'd be inclined to code PLyObject_FromUint64() as an exact
analog of PLyLong_FromInt64(), ie

/* on 32 bit platforms "unsigned long" may be too small */
if (sizeof(uint64) > sizeof(unsigned long))
return PyLong_FromUnsignedLongLong(DatumGetUInt64(d));
else
return PyLong_FromUnsignedLong(DatumGetUInt64(d));

and let Python worry about how to optimize the conversion.

So far as I can tell from
https://docs.python.org/2/c-api/long.html
these functions are available as far back as we could need.
If the buildfarm tells us otherwise, we could deal with it then.

regards, tom lane

On 1/9/18 15:54, Tom Lane wrote:

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:

Here is a patch to improves how PL/Python deals with very large values
of SPI_processed. The previous code converts anything that does not fit
into a C long into a Python float. But Python long has unlimited
precision, so we should be using that instead. And in Python 3, int and
long as the same, so there is no need to deal with any variations anymore.

I took a quick look at this. +1 for returning Python long all the time,
but I wonder why the Python version dependency.

To keep returning an int in Python 2 in the cases it fits. Maybe that's
overkill.

Our existing function
PLyLong_FromInt64() believes that PyLong_FromLongLong is unconditionally
available.

Interesting. I had coded this to account for the possibility that long
long does not exist on a 64-bit platform, but that situation probably
died with the first Alpha or something.

I'd be inclined to code PLyObject_FromUint64() as an exact
analog of PLyLong_FromInt64(), ie

/* on 32 bit platforms "unsigned long" may be too small */
if (sizeof(uint64) > sizeof(unsigned long))
return PyLong_FromUnsignedLongLong(DatumGetUInt64(d));
else
return PyLong_FromUnsignedLong(DatumGetUInt64(d));

and let Python worry about how to optimize the conversion.

Would that even be necessary? Why not use the LongLong variant all the
time then?

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:

On 1/9/18 15:54, Tom Lane wrote:

I'd be inclined to code PLyObject_FromUint64() as an exact
analog of PLyLong_FromInt64(), ie

/* on 32 bit platforms "unsigned long" may be too small */
if (sizeof(uint64) > sizeof(unsigned long))
return PyLong_FromUnsignedLongLong(DatumGetUInt64(d));
else
return PyLong_FromUnsignedLong(DatumGetUInt64(d));

and let Python worry about how to optimize the conversion.

Would that even be necessary? Why not use the LongLong variant all the
time then?

I've not looked at the Python internals to see if one is noticeably faster
than the other, but if it isn't, yeah we could simplify that. In any case
my main point is let's keep these two functions using similar approaches.

regards, tom lane

On 1/12/18 11:06, Tom Lane wrote:

Would that even be necessary? Why not use the LongLong variant all the
time then?

I've not looked at the Python internals to see if one is noticeably faster
than the other, but if it isn't, yeah we could simplify that. In any case
my main point is let's keep these two functions using similar approaches.

I ran extensive tests on this and went ahead with the simplified versions.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services