Python 3.0 does not work with PL/Python

Started by Peter Eisentrautover 17 years ago26 messageshackers
Jump to latest
#1Peter Eisentraut
peter_e@gmx.net

I have recently fixed the configure script to recognize Python 3.0. But
note that building and running PL/Python with Python 3.0 does not
actually work. It looks like several symbols have been removed or
changed. It would be good if the Python pundits around here could take
a look.

(I have found Python 3.0 to be very quick and easy to install from
source, in case your distribution doesn't have it packaged yet.)

#2Joshua D. Drake
jd@commandprompt.com
In reply to: Peter Eisentraut (#1)
Re: Python 3.0 does not work with PL/Python

On Thu, 2009-01-08 at 11:38 +0200, Peter Eisentraut wrote:

I have recently fixed the configure script to recognize Python 3.0. But
note that building and running PL/Python with Python 3.0 does not
actually work. It looks like several symbols have been removed or
changed. It would be good if the Python pundits around here could take
a look.

Yeah I mentioned that to Bruce a couple of days ago. I am trying to fix
it. Or I should say, I am trying to get Alvaro to hold my hand while I
fix it.

Joshua D. Drake

(I have found Python 3.0 to be very quick and easy to install from
source, in case your distribution doesn't have it packaged yet.)

--
PostgreSQL
Consulting, Development, Support, Training
503-667-4564 - http://www.commandprompt.com/
The PostgreSQL Company, serving since 1997

#3Bruce Momjian
bruce@momjian.us
In reply to: Peter Eisentraut (#1)
Re: Python 3.0 does not work with PL/Python

Peter Eisentraut wrote:

I have recently fixed the configure script to recognize Python 3.0. But
note that building and running PL/Python with Python 3.0 does not
actually work. It looks like several symbols have been removed or
changed. It would be good if the Python pundits around here could take
a look.

(I have found Python 3.0 to be very quick and easy to install from
source, in case your distribution doesn't have it packaged yet.)

Any progress on this?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

#4Peter Eisentraut
peter_e@gmx.net
In reply to: Bruce Momjian (#3)
Re: Python 3.0 does not work with PL/Python

Bruce Momjian wrote:

Peter Eisentraut wrote:

I have recently fixed the configure script to recognize Python 3.0. But
note that building and running PL/Python with Python 3.0 does not
actually work. It looks like several symbols have been removed or
changed. It would be good if the Python pundits around here could take
a look.

(I have found Python 3.0 to be very quick and easy to install from
source, in case your distribution doesn't have it packaged yet.)

Any progress on this?

Won't happen for 8.4, it looks like.

#5Tom Lane
tgl@sss.pgh.pa.us
In reply to: Peter Eisentraut (#1)
Re: Python 3.0 does not work with PL/Python

Peter Eisentraut <peter_e@gmx.net> writes:

I have recently fixed the configure script to recognize Python 3.0. But
note that building and running PL/Python with Python 3.0 does not
actually work. It looks like several symbols have been removed or
changed. It would be good if the Python pundits around here could take
a look.

(I have found Python 3.0 to be very quick and easy to install from
source, in case your distribution doesn't have it packaged yet.)

I thought I would experiment with this a bit. I got past Python's
"configure; make; make install" okay, but got no further than here
with building PG:

checking for python... /home/tgl/python3.0.1/bin/python
checking for Python distutils module... ./configure: line 6946: 21044 Aborted "${PYTHON}" -c 'import distutils' 2>&-
no
configure: error: distutils module not found
$

Okay, but some research revealed that there does not exist any
working distutils for Python 3.0.1 yet:
http://regebro.wordpress.com/2009/02/01/setuptools-and-easy_install-for-python-3/

If the language is still at the point where they're breaking fundamental
tools with each dot-release, I don't think it's really stable enough for
us to spend effort on :-(

I also found out that my favorite distro is just *starting* to think
about what it will take to migrate to Python 3, and they seem to think
that it's not going to be viable till around Fedora 13 (a year away):
https://www.redhat.com/archives/fedora-devel-list/2009-April/msg00085.html

So my conclusion is that Python 3.0 is much too wet behind the ears for
us to worry about in PG 8.4. I'd guess that we should come back to the
issue towards the end of 2009, and perhaps think about back-porting
after we have something working in 8.5.

BTW, there is some useful info here:
http://docs.python.org/3.0/howto/cporting.html
for whenever we do get around to trying to make the code work.
It looks like we'll need not-trivial code changes. But there's
no point until the language is a bit more stable.

regards, tom lane

#6Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Tom Lane (#5)
Re: Python 3.0 does not work with PL/Python

Tom Lane wrote:

I thought I would experiment with this a bit. I got past Python's
"configure; make; make install" okay, but got no further than here
with building PG:

Consequently, I have removed the Python 3.0 item from the open items
list and added a link to this thread to the TODO item that was already
there.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

#7Marko Kreen
markokr@gmail.com
In reply to: Tom Lane (#5)
Re: Python 3.0 does not work with PL/Python

On 4/4/09, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Peter Eisentraut <peter_e@gmx.net> writes:

I have recently fixed the configure script to recognize Python 3.0. But
note that building and running PL/Python with Python 3.0 does not
actually work. It looks like several symbols have been removed or
changed. It would be good if the Python pundits around here could take
a look.

(I have found Python 3.0 to be very quick and easy to install from
source, in case your distribution doesn't have it packaged yet.)

I thought I would experiment with this a bit. I got past Python's
"configure; make; make install" okay, but got no further than here
with building PG:

checking for python... /home/tgl/python3.0.1/bin/python
checking for Python distutils module... ./configure: line 6946: 21044 Aborted "${PYTHON}" -c 'import distutils' 2>&-
no
configure: error: distutils module not found
$

Okay, but some research revealed that there does not exist any
working distutils for Python 3.0.1 yet:
http://regebro.wordpress.com/2009/02/01/setuptools-and-easy_install-for-python-3/

If the language is still at the point where they're breaking fundamental
tools with each dot-release, I don't think it's really stable enough for
us to spend effort on :-(

Well, the point of 3.0 was to "break the world"...

I also found out that my favorite distro is just *starting* to think
about what it will take to migrate to Python 3, and they seem to think
that it's not going to be viable till around Fedora 13 (a year away):
https://www.redhat.com/archives/fedora-devel-list/2009-April/msg00085.html

So my conclusion is that Python 3.0 is much too wet behind the ears for
us to worry about in PG 8.4. I'd guess that we should come back to the
issue towards the end of 2009, and perhaps think about back-porting
after we have something working in 8.5.

It is not "wet" (the new interfaces should be stable), but it is break
from 2.x series. This means that users of PL/Python should not expect
PL/Python to automatically work with 3.0. Supporting 3.0 will be a new
feature. So it's OK to drop it from 8.4.

--
marko

#8Tom Lane
tgl@sss.pgh.pa.us
In reply to: Marko Kreen (#7)
Re: Python 3.0 does not work with PL/Python

Marko Kreen <markokr@gmail.com> writes:

On 4/4/09, Tom Lane <tgl@sss.pgh.pa.us> wrote:

So my conclusion is that Python 3.0 is much too wet behind the ears for
us to worry about in PG 8.4. I'd guess that we should come back to the
issue towards the end of 2009, and perhaps think about back-porting
after we have something working in 8.5.

It is not "wet" (the new interfaces should be stable), but it is break
from 2.x series.

Hm, did you read the link I cited? It's not so surprising that 3.0
should have broken distutils, but what I found distressing is that they
fixed distutils and then 3.0.1 broke it *again*. I stand by my opinion
that Python 3 isn't stable yet.

This means that users of PL/Python should not expect PL/Python to
automatically work with 3.0. Supporting 3.0 will be a new feature.
So it's OK to drop it from 8.4.

One other thing that we'll have to seriously consider is whether we
should package python3 as a separate PL, so that people can keep using
their 2.x plpython functions without fear of breakage. I know that the
Fedora guys are currently debating whether to treat it that way, and
I suppose other distros are having or will soon have the same
conversation. Six months from now, there will be some precedents and
some track record for us to look at in making that choice.

regards, tom lane

#9James William Pye
lists@jwp.name
In reply to: Tom Lane (#8)
Re: Python 3.0 does not work with PL/Python

On Apr 5, 2009, at 8:54 AM, Tom Lane wrote:

Hm, did you read the link I cited? It's not so surprising that 3.0
should have broken distutils, but what I found distressing is that
they
fixed distutils and then 3.0.1 broke it *again*. I stand by my
opinion
that Python 3 isn't stable yet.

Yeah, actually. From some of the talk I've seen on python-dev, it
sounds like 3.0.2 will be the last 3.0 release. 3.1 is in alpha, and
ready to start cleaning things up, afaict.

This means that users of PL/Python should not expect PL/Python to
automatically work with 3.0. Supporting 3.0 will be a new feature.
So it's OK to drop it from 8.4.

One other thing that we'll have to seriously consider is whether we
should package python3 as a separate PL, so that people can keep using
their 2.x plpython functions without fear of breakage. I know that
the
Fedora guys are currently debating whether to treat it that way, and
I suppose other distros are having or will soon have the same
conversation. Six months from now, there will be some precedents and
some track record for us to look at in making that choice.

I think this would be wise.

Any thoughts on the acceptability of a complete rewrite for Python 3?
I've been fiddling with a HEAD branch including the plpy code in a
github repo. (nah it dunt compile yet: bitrot and been busy with a 3.x
driver. ;)

#10Tom Lane
tgl@sss.pgh.pa.us
In reply to: James William Pye (#9)
Re: Python 3.0 does not work with PL/Python

James Pye <lists@jwp.name> writes:

Any thoughts on the acceptability of a complete rewrite for Python 3?

I've always thought that plpython.c was a bit on the hackish side.
If we do decide we have to make plpython2 and plpython3 separate
languages, it'd be pretty easy to just start over with a whole new
implementation for python3 ...

regards, tom lane

#11David Blewett
david@dawninglight.net
In reply to: Peter Eisentraut (#1)
Re: Python 3.0 does not work with PL/Python

On Sun, Apr 5, 2009 at 7:10 PM, James Pye <lists@jwp.name> wrote:

Any thoughts on the acceptability of a complete rewrite for Python 3? I've
been fiddling with a HEAD branch including the plpy code in a github repo.
(nah it dunt compile yet: bitrot and been busy with a 3.x driver. ;)

I'd love to see this. I can't help with the C stuff, but I can try to
help test things as you go.

David Blewett

#12David Blewett
david@dawninglight.net
In reply to: Peter Eisentraut (#1)
Re: Python 3.0 does not work with PL/Python

On Thu, Apr 30, 2009 at 8:30 AM, James Pye <lists@jwp.name> wrote:

On Apr 30, 2009, at 5:09 AM, David Blewett wrote:

I'd love to see this.

yep, once I get 0.9 of the driver out the door, I'll probably focus on this.

It's the perfect time for a rewrite.. I really don't want to see the 2.x
version ported. =\

I can't help with the C stuff, but I can try to
help test things as you go.

I will probably be taking you up on that offer. =)

I'll be dev'ing on osx with some occasional compiles/tests on fbsd, so an
extra pair of eyes on builds/test runs would be a *huge* help..

I have access to the current version of ubuntu, centos 5, gentoo and
opensolaris. Just let me know what you'd like tested.

David Blewett

#13Peter Eisentraut
peter_e@gmx.net
In reply to: James William Pye (#9)
Re: Python 3.0 does not work with PL/Python

On Monday 06 April 2009 02:10:59 James Pye wrote:

Any thoughts on the acceptability of a complete rewrite for Python 3?

http://www.joelonsoftware.com/articles/fog0000000069.html

#14James William Pye
lists@jwp.name
In reply to: Peter Eisentraut (#13)
Re: Python 3.0 does not work with PL/Python

On May 3, 2009, at 11:02 PM, Peter Eisentraut wrote:

http://www.joelonsoftware.com/articles/fog0000000069.html

Good read. =)

However, complete rewrite being relative in this case:

WIP: http://github.com/jwp/postgresql-plpython3/tree/c804e693b6a0df98c0e5c465e75ba2e9014ebf37/src/pl/plpython3

That is, from pgsql's perspective it would be a complete rewrite.

From my perspective it would be a refactored version of plpy for
Python 3.x and pgsql 8.5/head. Additionally, plpy originally came from
plpythonu(7.3 or 7.4), but was *massively* refactored(read:
effectively rewritten).

I wouldn't want to start from scratch, actually.

#15Bruce Momjian
bruce@momjian.us
In reply to: Peter Eisentraut (#13)
Re: Python 3.0 does not work with PL/Python

Peter Eisentraut wrote:

On Monday 06 April 2009 02:10:59 James Pye wrote:

Any thoughts on the acceptability of a complete rewrite for Python 3?

http://www.joelonsoftware.com/articles/fog0000000069.html

You usually have to rewrite when you have not done refactoring as part
of development; PGDG does refactoring regularly.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

#16Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Bruce Momjian (#15)
Re: Python 3.0 does not work with PL/Python

Bruce Momjian escribi�:

Peter Eisentraut wrote:

On Monday 06 April 2009 02:10:59 James Pye wrote:

Any thoughts on the acceptability of a complete rewrite for Python 3?

http://www.joelonsoftware.com/articles/fog0000000069.html

You usually have to rewrite when you have not done refactoring as part
of development; PGDG does refactoring regularly.

Except that plpython stagnates, save for minor hacks here and there.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

#17Andrew Dunstan
andrew@dunslane.net
In reply to: Alvaro Herrera (#16)
Re: Python 3.0 does not work with PL/Python

Alvaro Herrera wrote:

Bruce Momjian escribi�:

Peter Eisentraut wrote:

On Monday 06 April 2009 02:10:59 James Pye wrote:

Any thoughts on the acceptability of a complete rewrite for Python 3?

http://www.joelonsoftware.com/articles/fog0000000069.html

You usually have to rewrite when you have not done refactoring as part
of development; PGDG does refactoring regularly.

Except that plpython stagnates, save for minor hacks here and there.

Does Python 3 have some sort of usable sandbox that would mean we could
have a trusted plpython?

Otherwise, I'm not too keen simply to throw Python 2.x overboard until
it's no longer common on platforms people are likely to want to install
Postgres on, if that's what's implied by the original question.

cheers

andrew

#18Caleb Welton
cwelton@greenplum.com
In reply to: Andrew Dunstan (#17)
Re: Python 3.0 does not work with PL/Python

No. Still no sandbox.

-Caleb

On 5/28/09 6:06 PM, "Andrew Dunstan" <andrew@dunslane.net> wrote:

Does Python 3 have some sort of usable sandbox that would mean we could
have a trusted plpython?

Otherwise, I'm not too keen simply to throw Python 2.x overboard until
it's no longer common on platforms people are likely to want to install
Postgres on, if that's what's implied by the original question.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#19Peter Eisentraut
peter_e@gmx.net
In reply to: Alvaro Herrera (#16)
Re: Python 3.0 does not work with PL/Python

On Friday 29 May 2009 03:53:17 Alvaro Herrera wrote:

Bruce Momjian escribió:

Peter Eisentraut wrote:

On Monday 06 April 2009 02:10:59 James Pye wrote:

Any thoughts on the acceptability of a complete rewrite for Python 3?

http://www.joelonsoftware.com/articles/fog0000000069.html

You usually have to rewrite when you have not done refactoring as part
of development; PGDG does refactoring regularly.

Except that plpython stagnates, save for minor hacks here and there.

But that doesn't mean that there is anything wrong with it. Of course there
is, but those are isolated problems that can be fixed when desired. For
example, it might just be that those who use it don't have use of INOUT
parameters or table returns.

#20Peter Eisentraut
peter_e@gmx.net
In reply to: Andrew Dunstan (#17)
Re: Python 3.0 does not work with PL/Python

On Friday 29 May 2009 04:06:14 Andrew Dunstan wrote:

Otherwise, I'm not too keen simply to throw Python 2.x overboard until
it's no longer common on platforms people are likely to want to install
Postgres on, if that's what's implied by the original question.

My guess is that we will need to keep around a Python 2.x version for at least
another three years, meaning two or three major PostgreSQL releases.

That also means that maintaining a separate, parallel code base for a Python 3
variant can only be acceptable if it gives major advantages.

#21Sam Mason
sam@samason.me.uk
In reply to: Andrew Dunstan (#17)
#22David Blewett
david@dawninglight.net
In reply to: Andrew Dunstan (#17)
#23Andrew Dunstan
andrew@dunslane.net
In reply to: David Blewett (#22)
#24Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andrew Dunstan (#23)
#25Hannu Krosing
hannu@tm.ee
In reply to: Peter Eisentraut (#19)
#26James William Pye
lists@jwp.name
In reply to: Peter Eisentraut (#20)