What's our minimum supported Python version?
Our fine manual claims the answer to $SUBJECT is 3.2.
However, there is room to doubt that our code still actually
works with 3.2, because the oldest version I can find being
tested by the buildfarm is topminnow's 3.4.2. The next oldest
is shelduck's 3.4.10, and everything else has 3.5.2 or newer.
topminnow hasn't reported in a couple months, so it may
actually be dead.
The reason I bring this up is that I found out the hard way
that src/test/modules/oauth_validator fails on RHEL8, because
its oauth_server.py script is not compatible with the 3.6.8
version of Python supplied by this distro. (There are 22
buildfarm animals running 3.6.8, presumably mostly also Red
Hat-derived platforms, so I'm not going to apologize for
my workstation being a little long in the tooth.)
I think we need to do some combination of moving our
minimum-supported-version goalposts forward, making sure that
whatever we claim is the minimum Python version is actually
being tested in the buildfarm, and fixing oauth_server.py
so that it works on that version.
I am not familiar enough with the Python landscape to
have an informed opinion about what the minimum supported
version should be. But I'll present this data scraped
from the buildfarm about how many animals are running what
(counting animals that have reported since 2025-01-01):
Count Version
1 3.4.2
1 3.4.10
2 3.5.2
3 3.5.3
2 3.6.5
22 3.6.8
3 3.6.9
5 3.6.15
1 3.7.1
2 3.7.3
1 3.7.10
2 3.7.16
1 3.8.8
4 3.8.10
15 3.9.2
6 3.9.16
6 3.9.18
1 3.9.19
2 3.9.20
13 3.9.21
6 3.10.x
18 3.11.x
21 3.12.x
22 3.13.x
160 total
Thoughts?
regards, tom lane
On 19 Apr 2025, at 7:17 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Our fine manual claims the answer to $SUBJECT is 3.2.
However, there is room to doubt that our code still actually
works with 3.2, because the oldest version I can find being
tested by the buildfarm is topminnow's 3.4.2. The next oldest
is shelduck's 3.4.10, and everything else has 3.5.2 or newer.
topminnow hasn't reported in a couple months, so it may
actually be dead.The reason I bring this up is that I found out the hard way
that src/test/modules/oauth_validator fails on RHEL8, because
its oauth_server.py script is not compatible with the 3.6.8
version of Python supplied by this distro. (There are 22
buildfarm animals running 3.6.8, presumably mostly also Red
Hat-derived platforms, so I'm not going to apologize for
my workstation being a little long in the tooth.)I think we need to do some combination of moving our
minimum-supported-version goalposts forward, making sure that
whatever we claim is the minimum Python version is actually
being tested in the buildfarm, and fixing oauth_server.py
so that it works on that version.I am not familiar enough with the Python landscape to
have an informed opinion about what the minimum supported
version should be. But I'll present this data scraped
from the buildfarm about how many animals are running what
(counting animals that have reported since 2025-01-01):Count Version
1 3.4.2
1 3.4.10
2 3.5.2
3 3.5.3
2 3.6.5
22 3.6.8
3 3.6.9
5 3.6.15
1 3.7.1
2 3.7.3
1 3.7.10
2 3.7.16
1 3.8.8
4 3.8.10
15 3.9.2
6 3.9.16
6 3.9.18
1 3.9.19
2 3.9.20
13 3.9.21
6 3.10.x
18 3.11.x
21 3.12.x
22 3.13.x160 total
Thoughts?
regards, tom lane
From an Python ecosystem perspective,
3.9 is the usual minimum that people use in CI matrices nowdays.
So if it was up to me, that’s what I’d choose.
Mabe 3.7 if I was too liberal, but <3.6 and older it’s probably too old to bother.
Then again, I’m not a buildfarm power user, so my 2 cents are worth a bit less.
Florents Tselai <florents.tselai@gmail.com> writes:
On 19 Apr 2025, at 7:17 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
I think we need to do some combination of moving our
minimum-supported-version goalposts forward, making sure that
whatever we claim is the minimum Python version is actually
being tested in the buildfarm, and fixing oauth_server.py
so that it works on that version.
From an Python ecosystem perspective,
3.9 is the usual minimum that people use in CI matrices nowdays.
So if it was up to me, that’s what I’d choose.
Per these numbers, that would be cutting off 31% of the buildfarm,
including a lot of still-in-support distros such as RHEL8.
So I would say that's not likely to be our choice.
regards, tom lane
On 19 Apr 2025, at 18:17, Tom Lane <tgl@sss.pgh.pa.us> wrote:
The reason I bring this up is that I found out the hard way
that src/test/modules/oauth_validator fails on RHEL8, because
its oauth_server.py script is not compatible with the 3.6.8
version of Python supplied by this distro.
Do you have the error message/log for the failure handy?
--
Daniel Gustafsson
Daniel Gustafsson <daniel@yesql.se> writes:
On 19 Apr 2025, at 18:17, Tom Lane <tgl@sss.pgh.pa.us> wrote:
The reason I bring this up is that I found out the hard way
that src/test/modules/oauth_validator fails on RHEL8, because
its oauth_server.py script is not compatible with the 3.6.8
version of Python supplied by this distro.
Do you have the error message/log for the failure handy?
The first problem is that this Python version seems not to
like assignments embedded in if statements:
File "t/oauth_server.py", line 319
if err := self._get_param("error_code", None):
^
SyntaxError: invalid syntax
I was able to work around that with the attached quick hack.
But then I get
Traceback (most recent call last):
File "t/oauth_server.py", line 19, in <module>
class OAuthHandler(http.server.BaseHTTPRequestHandler):
File "t/oauth_server.py", line 26, in OAuthHandler
JsonObject = dict[str, object] # TypeAlias is not available until 3.10
TypeError: 'type' object is not subscriptable
which I have no idea how to work around.
regards, tom lane
Attachments:
python-syntax-hack.patchtext/x-diff; charset=us-ascii; name=python-syntax-hack.patchDownload
diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py
index 5bc30be87fd..133fe496cfc 100755
--- a/src/test/modules/oauth_validator/t/oauth_server.py
+++ b/src/test/modules/oauth_validator/t/oauth_server.py
@@ -316,11 +316,13 @@ class OAuthHandler(http.server.BaseHTTPRequestHandler):
return resp
def token(self) -> JsonObject:
- if err := self._get_param("error_code", None):
+ err = self._get_param("error_code", None)
+ if err:
self._response_code = self._get_param("error_status", 400)
resp = {"error": err}
- if desc := self._get_param("error_desc", ""):
+ desc = self._get_param("error_desc", "")
+ if desc:
resp["error_description"] = desc
return resp
On Tue, Apr 22, 2025 at 8:05 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
The first problem is that this Python version seems not to
like assignments embedded in if statements: [...]I was able to work around that with the attached quick hack.
But then I get [...]
Thanks, I'll put a patch together. Sorry -- IIRC, both of those
features are probably too new for me to have used, regardless of what
we decide is the minimum version for 18.
As for picking a version... 3.6 will have been EOL for almost three
years by the time 18 releases. It seems like we would drop it happily,
were it not for RHEL8. But RHEL also supports side-by-side Python,
right? In general, I'd be more than willing to try plumbing that
through the new tests, if it lets us drop end-of-life versions for new
code.
(I don't know if we'll need to drop 3.6 for PG18, specifically, but I
also think we shouldn't plan to support 3.6 for the full RHEL8
lifetime.)
Thanks,
--Jacob
Jacob Champion <jacob.champion@enterprisedb.com> writes:
As for picking a version... 3.6 will have been EOL for almost three
years by the time 18 releases. It seems like we would drop it happily,
were it not for RHEL8.
Agreed, but RHEL8 is out there and I don't think we can just drop
support for it. I'm also not excited by the idea that an incidental
test script gets to dictate what the cutoff is.
I do think we should stop claiming that Python 3.2 will work.
(Maybe it does, but we don't know that.) I see that the configure
script only checks for Python >= 3, and it doesn't look like the
meson scripts check explicitly at all, although there's a comment
saying that our meson version cutoff is intended to allow working
with Python 3.5.
Maybe it's sufficient to make a documentation change here, and
say we support Python >= 3.5? I'd be okay with saying 3.6.8
too, on the grounds that if anything older fails to work we'd
almost certainly just say "too bad". But RHEL8 is widespread
enough that I think we need to keep making the effort for 3.6.8.
regards, tom lane
On Tue, Apr 22, 2025 at 9:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I'm also not excited by the idea that an incidental
test script gets to dictate what the cutoff is.
I agree, and I don't intend for the script to dictate that.
Maybe it's sufficient to make a documentation change here, and
say we support Python >= 3.5? I'd be okay with saying 3.6.8
too, on the grounds that if anything older fails to work we'd
almost certainly just say "too bad".
I'm definitely on board with pulling the minimum up, as high as we
think we can get away with. If you're comfortable with 3.6(.8), I'd
say let's start there.
But RHEL8 is widespread
enough that I think we need to keep making the effort for 3.6.8.
Even if we can get side-by-side versions working? RHEL8 has a bunch of
newer Python 3 versions available, according to the docs. And
virtualenvs can do a lot of lifting for us in practice.
Thanks,
--Jacob
Jacob Champion <jacob.champion@enterprisedb.com> writes:
On Tue, Apr 22, 2025 at 9:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
But RHEL8 is widespread
enough that I think we need to keep making the effort for 3.6.8.
Even if we can get side-by-side versions working?
That would be off-loading our problem onto the users (and the
buildfarm owners). If we really had little other choice,
we could go there; but it doesn't sound like this is that
hard to work around.
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> writes:
Florents Tselai <florents.tselai@gmail.com> writes:
On 19 Apr 2025, at 7:17 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
I think we need to do some combination of moving our
minimum-supported-version goalposts forward, making sure that
whatever we claim is the minimum Python version is actually
being tested in the buildfarm, and fixing oauth_server.py
so that it works on that version.From an Python ecosystem perspective,
3.9 is the usual minimum that people use in CI matrices nowdays.
So if it was up to me, that’s what I’d choose.Per these numbers, that would be cutting off 31% of the buildfarm,
including a lot of still-in-support distros such as RHEL8.
So I would say that's not likely to be our choice.regards, tom lane
Also from a python world perspective, we really don't want to run EOL
versions. Besides the security reasons, bugs in thirdy-party
dependencies usually are not fixed and the whole system seems to rot
very quickly.
Of course, this does not happen if one does not rely on third-party
deps. Even in this case, I would say that it is wise to support a
relatively small set because the built-in libs will vary over versions,
and a wide range of versions will cause more troubles like this one.
The oldest non EOL version is 3.9 right now. My suggestion is to follow
the official supported releases. I'm not familiar with the buildfarm,
but I know that python3.6 does not come installed by default in RHEL8,
so instead of installing this version we can simply install 3.9, 3.11 or
3.12.
In a persistent workstation, I believe the following should do the
trick (tested on a 8.5 box):
sudo dnf install python3.12
sudo dnf remove python36
BTW, we are used to create a virtualenv for everything, but that is another topic.
Best regards,
Renan Fonseca
On Tue, Apr 22, 2025 at 8:29 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
On Tue, Apr 22, 2025 at 8:05 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
The first problem is that this Python version seems not to
like assignments embedded in if statements: [...]I was able to work around that with the attached quick hack.
But then I get [...]Thanks, I'll put a patch together. Sorry -- IIRC, both of those
features are probably too new for me to have used, regardless of what
we decide is the minimum version for 18.
Splitting this off into its own $SUBJECT.
I coupled oauth_server to new Python stuff without realizing it, sorry:
- urllib's reclassification of `~` as a safe URL character (3.7)
- walrus operator `:=` (3.8)
- dict[type, type] annotations (3.9)
- str.removeprefix/suffix() (3.9)
Attached patch gets rid of all that.
This is tested against Python 3.6.15 (3.6 went EOL at the end of
2021). I'm working on getting Rocky 8 installed locally to test
against. If it's decided we want to support downwards to 3.5, I will
test there too (but I hope we don't; see parent thread).
Thanks,
--Jacob
Attachments:
0001-oauth-Support-Python-3.6-in-tests.patchapplication/octet-stream; name=0001-oauth-Support-Python-3.6-in-tests.patchDownload
From 5531cdd78a60882626e2e3bf827b8f24a94f2e23 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Tue, 22 Apr 2025 11:59:36 -0700
Subject: [PATCH] oauth: Support Python 3.6 in tests
RHEL8 ships a patched 3.6.8 as its base Python version, and I
accidentally let some newer Python-isms creep into oauth_server.py
during development.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/16098.1745079444%40sss.pgh.pa.us
---
.../modules/oauth_validator/t/oauth_server.py | 26 ++++++++++++-------
1 file changed, 17 insertions(+), 9 deletions(-)
diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py
index 5bc30be87fd..20b3a9506cb 100755
--- a/src/test/modules/oauth_validator/t/oauth_server.py
+++ b/src/test/modules/oauth_validator/t/oauth_server.py
@@ -14,6 +14,7 @@ import sys
import time
import urllib.parse
from collections import defaultdict
+from typing import Dict
class OAuthHandler(http.server.BaseHTTPRequestHandler):
@@ -23,7 +24,7 @@ class OAuthHandler(http.server.BaseHTTPRequestHandler):
documentation for BaseHTTPRequestHandler.
"""
- JsonObject = dict[str, object] # TypeAlias is not available until 3.10
+ JsonObject = Dict[str, object] # TypeAlias is not available until 3.10
def _check_issuer(self):
"""
@@ -35,14 +36,16 @@ class OAuthHandler(http.server.BaseHTTPRequestHandler):
)
self._parameterized = self.path.startswith("/param/")
+ # Strip off the magic path segment. (The more readable
+ # str.removeprefix()/removesuffix() aren't available until Py3.9.)
if self._alt_issuer:
# The /alternate issuer uses IETF-style .well-known URIs.
if self.path.startswith("/.well-known/"):
- self.path = self.path.removesuffix("/alternate")
+ self.path = self.path[: -len("/alternate")]
else:
- self.path = self.path.removeprefix("/alternate")
+ self.path = self.path[len("/alternate") :]
elif self._parameterized:
- self.path = self.path.removeprefix("/param")
+ self.path = self.path[len("/param") :]
def _check_authn(self):
"""
@@ -58,8 +61,10 @@ class OAuthHandler(http.server.BaseHTTPRequestHandler):
if method != "Basic":
raise RuntimeError(f"client used {method} auth; expected Basic")
- username = urllib.parse.quote_plus(self.client_id)
- password = urllib.parse.quote_plus(secret)
+ # TODO: Remove "~" from the safe list after Py3.6 support is removed.
+ # 3.7 does this by default.
+ username = urllib.parse.quote_plus(self.client_id, safe="~")
+ password = urllib.parse.quote_plus(secret, safe="~")
expected_creds = f"{username}:{password}"
if creds.encode() != base64.b64encode(expected_creds.encode()):
@@ -83,7 +88,7 @@ class OAuthHandler(http.server.BaseHTTPRequestHandler):
self._send_json(resp)
- def _parse_params(self) -> dict[str, str]:
+ def _parse_params(self) -> Dict[str, str]:
"""
Parses apart the form-urlencoded request body and returns the resulting
dict. For use by do_POST().
@@ -316,11 +321,14 @@ class OAuthHandler(http.server.BaseHTTPRequestHandler):
return resp
def token(self) -> JsonObject:
- if err := self._get_param("error_code", None):
+ err = self._get_param("error_code", None)
+ if err:
self._response_code = self._get_param("error_status", 400)
resp = {"error": err}
- if desc := self._get_param("error_desc", ""):
+
+ desc = self._get_param("error_desc", "")
+ if desc:
resp["error_description"] = desc
return resp
--
2.34.1
On Tue, Apr 22, 2025 at 12:09 PM Renan Alves Fonseca
<renanfonseca@gmail.com> wrote:
The oldest non EOL version is 3.9 right now. My suggestion is to follow
the official supported releases.
I think this policy is too aggressive. Many operating systems we
support are going to ship Python versions past their EOL date (and
keep them supported for a long while with security patches). There's a
middle ground to be found, IMO.
--Jacob
Jacob Champion <jacob.champion@enterprisedb.com> writes:
This is tested against Python 3.6.15 (3.6 went EOL at the end of
2021). I'm working on getting Rocky 8 installed locally to test
against. If it's decided we want to support downwards to 3.5, I will
test there too (but I hope we don't; see parent thread).
Thanks! I confirm this works for me on RHEL8 with 3.6.8.
regards, tom lane
Jacob Champion <jacob.champion@enterprisedb.com> writes:
On Tue, Apr 22, 2025 at 12:09 PM Renan Alves Fonseca
<renanfonseca@gmail.com> wrote:The oldest non EOL version is 3.9 right now. My suggestion is to follow
the official supported releases.
I think this policy is too aggressive. Many operating systems we
support are going to ship Python versions past their EOL date (and
keep them supported for a long while with security patches).
Yeah, that. The distros that are still shipping older Pythons
are LTS distros where part of the deal is that the vendor
is supposed to back-patch security fixes, even if the upstream
has moved on. Maybe a particular vendor is falling down on that
job, but it's not for us to rate their performance. So I don't
buy that "but security!" is a good argument here.
(Full disclosure: I worked for Red Hat for more than a decade,
and still preferentially use their distros. So I've pretty well
bought into that model.)
regards, tom lane
On Tue, Apr 22, 2025 at 12:36 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Thanks! I confirm this works for me on RHEL8 with 3.6.8.
Great, thanks for the quick review!
I'm considering committing this myself. Do you mind if it takes a day
or so to get this in, while I quadruple-check everything, or are you
blocked on this?
--Jacob
Jacob Champion <jacob.champion@enterprisedb.com> writes:
I'm considering committing this myself.
Please do.
Do you mind if it takes a day
or so to get this in, while I quadruple-check everything, or are you
blocked on this?
I'm not in a hurry. Push when you're ready.
regards, tom lane
tl;dr I think requiring support of Python 3.9 for PG18 would probably
be reasonable and save us a bunch of maintenance burden over the next
5 years.
On Tue, 22 Apr 2025 at 21:41, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Yeah, that. The distros that are still shipping older Pythons
are LTS distros where part of the deal is that the vendor
is supposed to back-patch security fixes, even if the upstream
has moved on. Maybe a particular vendor is falling down on that
job, but it's not for us to rate their performance. So I don't
buy that "but security!" is a good argument here.
I totally agree that "security" is not the discussion that we should
be having here. I think the Python version decision (or really any
minimum version decision) should be based on some analysis of costs to
us for maintaining support vs benefits to the users.
I'm pretty sure most users of RHEL expect most modern software not to
compile/work completely effortlessly on their distros without some
effort on their part or on the part of RHEL packagers. That's kinda
what you're signing up for if you choose a distro like that.
It might very well be that supporting Python 3.6 does not require many
code changes. But it's so old that installing it on most of the recent
OSes is not trivial, e.g. uv does not support installing it[1]https://github.com/astral-sh/uv/issues/9833. By
saying we support it, it means that all developers need to go through
serious hoops to be able to test that their python scripts work
correctly on their own machines. And given our own support policy,
we'd probably have to do that for 5 more years (unless we want to
change supported python versions in minor PG releases, which I highly
doubt we want).
So that leaves the question, how much are we actually helping users by
spending all that effort on keeping support for old Python versions?
Based on the Red Hat docs, it's pretty easy to install newer Python3
versions on RHEL8. Depending on the X in RHEL 8.X you can install
different Python3 versions from the official repos[2]https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/configuring_basic_system_settings/installing-and-using-dynamic-programming-languages_configuring-basic-system-settings#assembly_installing-and-using-python_installing-and-using-dynamic-programming-languages. And based on
the official RHEL EOL policy[3]https://access.redhat.com/support/policy/updates/errata#RHEL8_Planning_Guide, only RHEL 8.4, 8.6, 8.8 and 8.10 are
still getting some level of support. With 8.4 and 8.6 only getting the
final level of support called "Update Services for SAP Solutions", and
with 8.8 reaching that same final support level before PG18 is
released (and 8.4 becoming unsupported before PG18).
Based on that information (only taking into account only RHEL8, not
other distros). It seems like we could at least bump the minimum
Python support to Python 3.9, which is the newest python that's
installable in RHEL 8.4 and 8.6. Anyone that's running a RHEL version
that's still supported by Red Hat can then simply do "yum install
python39" and run the tests (assuming we set up meson/autoconf to
correctly detect the "python3.9" binary).
I'd even be curious how much resistance we'd get from RHEL users if
we'd bump the Python requirement for PG18 to Python 3.12, which is the
newest version that RHEL 8.10 supports. RHEL 8.10 is the only RHEL 8
version that is getting regular support from Red Hat.
(Full disclosure: I've used close to EOL versions of RHEL7
professionally, and I hated every second it. I constantly felt like I
was walking through a software museum. Having to work around bugs that
had been fixed for ages in upstream)
Looking a little further than RHEL:
- Ubuntu 20.04 can easily install Python3.9 from the official package
repos. Currently this is still supported by Cristoph his pgdg repos,
but looks like that support might be removed before PG18 given it's
transition to "Expanded Security Maintenance" next month.
- Ubuntu 22.04 can easily install Python3.11 from the official package
repos. Still has full upstream support for 2 years.
- Debian Bullseye (EOL in 1.5 years) can only install Python3.9 from
the official package repos.
- Debian Bookworm (EOL in 3 years) has Pytho
- Python 3.8 itself is declared EOL by upstream
- Python 3.10 will be declared EOL by upstream around the release of PG19
Based on all this info, I think that it sounds quite reasonable to
start requiring Python3.9 for PG18. And for PG19 we could probably
start requiring Python3.11
[1]: https://github.com/astral-sh/uv/issues/9833
[2]: https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/configuring_basic_system_settings/installing-and-using-dynamic-programming-languages_configuring-basic-system-settings#assembly_installing-and-using-python_installing-and-using-dynamic-programming-languages
[3]: https://access.redhat.com/support/policy/updates/errata#RHEL8_Planning_Guide
P.S. I CC-ed Devrim and Christoph, for insights into this from the
packagers perspective.
Re: Jelte Fennema-Nio
- Ubuntu 20.04 can easily install Python3.9 from the official package
repos. Currently this is still supported by Cristoph his pgdg repos,
but looks like that support might be removed before PG18 given it's
transition to "Expanded Security Maintenance" next month.
20.04 (focal) is already too old for PG18 because of llvm version
woes, so we are already not building it there. (I forgot what the
precise problem was, but I figured focal would be EOL before the PG18
release anyway.)
- Ubuntu 22.04 can easily install Python3.11 from the official package
repos. Still has full upstream support for 2 years.
- Debian Bullseye (EOL in 1.5 years) can only install Python3.9 from
the official package repos.
- Debian Bookworm (EOL in 3 years) has Pytho
^ n 3.11
- Python 3.8 itself is declared EOL by upstream
- Python 3.10 will be declared EOL by upstream around the release of PG19
Based on all this info, I think that it sounds quite reasonable to
start requiring Python3.9 for PG18. And for PG19 we could probably
start requiring Python3.11
Ack. Bullseye LTS will be supported until August 2026, so very close
to the PG19 release, but it should work out not to support PG19 there.
Christoph
Jacob Champion <jacob.champion@enterprisedb.com> writes:
This is tested against Python 3.6.15 (3.6 went EOL at the end of
2021). I'm working on getting Rocky 8 installed locally to test
against. If it's decided we want to support downwards to 3.5, I will
test there too (but I hope we don't; see parent thread).
I did a quick test on 3.5. It seems to work after removing
f-strings... At this point, I would be really happy with 3.6.
On Tue, Apr 22, 2025 at 2:28 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
I'm pretty sure most users of RHEL expect most modern software not to
compile/work completely effortlessly on their distros without some
effort on their part or on the part of RHEL packagers. That's kinda
what you're signing up for if you choose a distro like that.
This is the core of my position, too. I think it's reasonable to
support Python versions for some time after they go EOL, but I don't
think that we need to treat "users who want bleeding-edge Postgres
with long-EOL dependencies" as something to cater to, at the expense
of the committer testing matrix.
Note that Meson itself has updated to Python 3.7 as a minimum version
(as it now warns you, loudly).
Thanks,
--Jacob
On Tue, Apr 22, 2025 at 2:43 PM Renan Alves Fonseca
<renanfonseca@gmail.com> wrote:
I did a quick test on 3.5.
Thanks!
It seems to work after removing
f-strings... At this point, I would be really happy with 3.6.
Ah, makes sense. Well, I'm starting with the 3.6 fix regardless, since
it's preventing RHEL8 testing. If we decide we must support 3.5, I can
patch f-strings too. Fingers crossed...
--Jacob
Jelte Fennema-Nio <postgres@jeltef.nl> writes:
I totally agree that "security" is not the discussion that we should
be having here. I think the Python version decision (or really any
minimum version decision) should be based on some analysis of costs to
us for maintaining support vs benefits to the users.
[ shrug... ] This thread in itself has already consumed more effort
than we've spent thinking about old-Python problems for the last
several years. (Otherwise, we would probably have updated that
supported-version claim in installation.sgml some time ago.)
So I doubt that there's all that much in cost savings to us that'd
outweigh benefits to users. The fact that Jacob was able to fix
oauth_server.py without (it seemed like) very much work seems to
bear out that opinion.
This calculus might change if we start to rely more heavily on
Python for build and test infrastructure than we do today. But
I think we can have that conversation when it happens.
Worth noting also is that there are different requirements in
different places. For example,
contrib/unaccent/generate_unaccent_rules.py is probably only ever run
by developers, so I doubt anyone is going to make a fuss if it doesn't
work on hoary Pythons. Test infrastructure is more of a problem,
since we want to be able to test on any platform that we consider
supported at all.
Anyway, what I propose that we do for now is replace the
installation.sgml text
The minimum required version is Python 3.2.
with
The minimum supported version is Python 3.6.8.
where I use "supported" advisedly, as in "it might work with
something older, but we don't promise to fix it if not".
regards, tom lane
On Tue, 22 Apr 2025 at 21:26, Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
- str.removeprefix/suffix() (3.9)
The way you replaced this does not have the same behaviour in the case
where the prefix/suffix is not part of the string. removeprefix/suffix
will not remove any characters in that case, but your code will always
remove the number of characters that the suffix/prefix is long. Maybe
your other checks and definition of the OAuth spec ensure that these
prefixes/suffixes are present when you remove them, but the code is
not the same in the general case (you'd need to add a hasprefix/suffix
check before removing the characters.
+ # Strip off the magic path segment. (The more readable
+ # str.removeprefix()/removesuffix() aren't available until Py3.9.)
if self._alt_issuer:
# The /alternate issuer uses IETF-style .well-known URIs.
if self.path.startswith("/.well-known/"):
- self.path = self.path.removesuffix("/alternate")
+ self.path = self.path[: -len("/alternate")]
else:
- self.path = self.path.removeprefix("/alternate")
+ self.path = self.path[len("/alternate") :]
elif self._parameterized:
- self.path = self.path.removeprefix("/param")
+ self.path = self.path[len("/param") :]On Tue, Apr 22, 2025 at 3:04 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
The fact that Jacob was able to fix
oauth_server.py without (it seemed like) very much work seems to
bear out that opinion.
I think my ability to do it should not be used as evidence of "ease".
I knew where to download 3.6, that I should build it with Clang to
avoid GCC segfaults, that I should set the rpaths appropriately, and
to wrap it all in a virtualenv for Meson to find. And I had to kick
Meson to get it to clear its cached Python information, or else the
contrib modules refused to build...
I suspect the only way most developers are going to want to test this
is by running EL8.
This calculus might change if we start to rely more heavily on
Python for build and test infrastructure than we do today. But
I think we can have that conversation when it happens.
As long as the need to backport to PG18 doesn't freeze that
conversation in place, I suppose.
Anyway, what I propose that we do for now is replace the
installation.sgml textThe minimum required version is Python 3.2.
with
The minimum supported version is Python 3.6.8.
where I use "supported" advisedly, as in "it might work with
something older, but we don't promise to fix it if not".
No objection to that as a first step. I'm still hopeful that Devrim
has some evidence in favor of bumping to 3.8 or 3.9. :)
Thanks,
--Jacob
On Tue, Apr 22, 2025 at 3:17 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
The way you replaced this does not have the same behaviour in the case
where the prefix/suffix is not part of the string. removeprefix/suffix
will not remove any characters in that case, but your code will always
remove the number of characters that the suffix/prefix is long. Maybe
your other checks and definition of the OAuth spec ensure that these
prefixes/suffixes are present when you remove them
Correct. Directly above the changed code are the prefix checks, which
set self._alt_issuer/_parameterized.
Thanks!
--Jacob
Hi,
On Tue, 2025-04-22 at 15:33 -0700, Jacob Champion wrote:
I'm still hopeful that Devrim has some evidence in favor of bumping
to 3.8 or 3.9. :)
I would love to have such an evidence -- but I don't have :) In the last
couple of weeks I've also been thinking of bumping every single Python
piece in the PGDG RPM repository to 3.9 (at least) on RHEL 8, but that
will break many things immediately. It is still a very major platform
for users and such a breakage is not welcome.
Regards,
--
Devrim Gündüz
Open Source Solution Architect, PostgreSQL Major Contributor
BlueSky: @devrim.gunduz.org , @gunduz.org
On Wed, 23 Apr 2025 at 10:15, Devrim Gündüz <devrim@gunduz.org> wrote:
I would love to have such an evidence -- but I don't have :) In the last
couple of weeks I've also been thinking of bumping every single Python
piece in the PGDG RPM repository to 3.9 (at least) on RHEL 8, but that
will break many things immediately. It is still a very major platform
for users and such a breakage is not welcome.
Sad... Sounds like a bump to 3.9 is probably out of the question for PG18 then.
Can you share some examples of the things that break? Because if (and
I'm hopeful that Jacob will be adding some) for PG19 we get more
testing infrastructure in Python, I wouldn't want all that extra
testing infra to be stuck on Python3.6 for the next 5 years.
I think the UX that we should make work for compiling/testing infra on
RHEL8 is the following:
yum install python39
./configure PYTHON=python39 # or equivalent meson
make check-world
That seems like a reasonable hurdle for people using RHEL8 to jump
over if they want to compile themselves. Especially since when
installing from pgdg-rpm the package could simply depend on the
python39 package.
I haven't tried, but I'm pretty sure we're not there yet. At minimum I
believe we'd have to change our shebang's from:
#!/usr/bin/env python3
To something that takes into account the passed in PYTHON option.
Hi,
On Wed, 2025-04-23 at 10:47 +0200, Jelte Fennema-Nio wrote:
Can you share some examples of the things that break?
The most notable one would be Psycopg (2 and 3). Plus Patroni
dependencies. There may be a couple of more.
Regards,
--
Devrim Gündüz
Open Source Solution Architect, PostgreSQL Major Contributor
BlueSky: @devrim.gunduz.org , @gunduz.org
On Wed, 23 Apr 2025 at 10:59, Devrim Gündüz <devrim@gunduz.org> wrote:
The most notable one would be Psycopg (2 and 3). Plus Patroni
dependencies. There may be a couple of more.
I meant more in what ways do these things break? Since you're actually
the one that's packaging them, I'd expect that you could make them
depend on the python39 package and be mostly done.
On Wed, 23 Apr 2025 at 00:33, Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
As long as the need to backport to PG18 doesn't freeze that
conversation in place, I suppose.
I'm confused. Are you intending to backport new test infra to PG18?
Looking at the amount of python code that we currently have, I agree
with Tom: Making the few scripts that we have compatible with
Python3.6 seems the best solution for PG18. (especially since you
already have a patch that fixes the known issues).
Given the purpose and pretty small size of the scripts, I expect that
it'd be extremely rare for us to backport changes to them. e.g. I
doubt your oauth server would need many changes to keep working
correctly for the lifetime of PG18. Maybe it gets some minor fixes in
18.1 and/or 18.2 but other than that it seems really unlikely.
Hi,
On Wed, 2025-04-23 at 11:03 +0200, Jelte Fennema-Nio wrote:
I meant more in what ways do these things break? Since you're actually
the one that's packaging them, I'd expect that you could make them
depend on the python39 package and be mostly done.
You are right, our side is fixable. However many packages in the
upstream also depend on Psycopg. I don't want to create a Linux
distribution based on RHEL 8 built against Python 3.9 (or 3.1x) :-)
Regards,
--
Devrim Gündüz
Open Source Solution Architect, PostgreSQL Major Contributor
BlueSky: @devrim.gunduz.org , @gunduz.org
On Wed, 23 Apr 2025 at 11:13, Devrim Gündüz <devrim@gunduz.org> wrote:
You are right, our side is fixable. However many packages in the
upstream also depend on Psycopg. I don't want to create a Linux
distribution based on RHEL 8 built against Python 3.9 (or 3.1x) :-)
I'm confused. The upstream RHEL8 repo depends on packages in pgdg???
Or are you saying that psycopg (and patroni) are not part of the
pgdg-repo, and instead part of the upstream-repo. And so that other
packages in pgdg that depend on upstream psycopg, would always need to
support python 3.6 (because that's what upstream psycopg uses).
Hi,
On Wed, 2025-04-23 at 11:46 +0200, Jelte Fennema-Nio wrote:
I'm confused. The upstream RHEL8 repo depends on packages in pgdg???
Or are you saying that psycopg (and patroni) are not part of the
pgdg-repo, and instead part of the upstream-repo. And so that other
packages in pgdg that depend on upstream psycopg, would always need to
support python 3.6 (because that's what upstream psycopg uses).
psycopg is included in RHEL 8, but PGDG packages are up2date (2.7.5 vs
2.9.5) so they override OS packages. That is why things will break.
A solution would be creating our own psycopg2 (likely for Python 3.12)
package, update all PGDG packages that depend on psycopg2 to use that
package. That is not impossible (I have already psycopg2 built against
Python 3.11 on SLES 15), but I don't know how much work it will be and
its impact as that set of update should go to both RHEL 8, 9 and 10
(RHEL 10 already includes Python 3.12 by default)
I can go for this solution if it is *absolutely* required. We already
have custom packages to support PostGIS, so that is not a new topic for
me.
Regards,
--
Devrim Gündüz
Open Source Solution Architect, PostgreSQL Major Contributor
BlueSky: @devrim.gunduz.org , @gunduz.org
On Wed, 23 Apr 2025 at 13:24, Devrim Gündüz <devrim@gunduz.org> wrote:
psycopg is included in RHEL 8, but PGDG packages are up2date (2.7.5 vs
2.9.5) so they override OS packages. That is why things will break.A solution would be creating our own psycopg2 (likely for Python 3.12)
package, update all PGDG packages that depend on psycopg2 to use that
package. That is not impossible (I have already psycopg2 built against
Python 3.11 on SLES 15), but I don't know how much work it will be and
its impact as that set of update should go to both RHEL 8, 9 and 10
(RHEL 10 already includes Python 3.12 by default)
Thanks for that explanation, now I understand the problem.
I can go for this solution if it is *absolutely* required. We already
have custom packages to support PostGIS, so that is not a new topic for
me.
I don't really think this would be required to bump Postgres its
minimum supported postgres version. psycopg is a client and postgresql
is the server, it seems fine for them to use different Python
versions. The postgresql-server package could depend on python3.9,
while psycopg packages would happily use python3.6.
On Tue, Apr 22, 2025 at 12:45 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
Great, thanks for the quick review!
And pushed. Thanks everyone!
--Jacob
On 23 Apr 2025, at 22:26, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
On Tue, Apr 22, 2025 at 12:45 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:Great, thanks for the quick review!
And pushed. Thanks everyone!
Congratulations on your first commit, looking forward to many more to come!
--
Daniel Gustafsson
On Wed, Apr 23, 2025 at 4:24 AM Devrim Gündüz <devrim@gunduz.org> wrote:
psycopg is included in RHEL 8, but PGDG packages are up2date (2.7.5 vs
2.9.5) so they override OS packages. That is why things will break.
Thanks, this is super helpful!
--Jacob
On Wed, Apr 23, 2025 at 2:11 AM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
I'm confused. Are you intending to backport new test infra to PG18?
Not particularly (though, if the barriers were low enough, wouldn't
that be nice?). I'm talking about hazards in oauth_server.py here.
Given the purpose and pretty small size of the scripts, I expect that
it'd be extremely rare for us to backport changes to them. e.g. I
doubt your oauth server would need many changes to keep working
correctly for the lifetime of PG18. Maybe it gets some minor fixes in
18.1 and/or 18.2 but other than that it seems really unlikely.
That assumes there are no bugs in my code to backport tests for, and I
don't like assuming that. The Perl and Python sides of the
oauth_validator tests are pretty tightly coupled out of necessity.
On Wed, Apr 23, 2025 at 7:54 AM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
I don't really think this would be required to bump Postgres its
minimum supported postgres version. psycopg is a client and postgresql
is the server, it seems fine for them to use different Python
versions. The postgresql-server package could depend on python3.9,
while psycopg packages would happily use python3.6.
I'm not sure that's the mental model we would want, since clients can
be used server-side. Maybe a safer way to split, if and when we want
to decouple this on RHEL, would be between clients of only the Python
language (tests), and clients of the Python ABI (PL/Python, psycopg,
etc.). Like, imagine a world where backported tests could use new
goodies, but extensions remained solidly on platform defaults. That
seems kind of nice to me, at a surface level.
The Python stable ABI may change the calculus, too, but I'm not going
to look into that right now...
--Jacob
On 22.04.25 18:04, Tom Lane wrote:
Jacob Champion <jacob.champion@enterprisedb.com> writes:
As for picking a version... 3.6 will have been EOL for almost three
years by the time 18 releases. It seems like we would drop it happily,
were it not for RHEL8.Agreed, but RHEL8 is out there and I don't think we can just drop
support for it. I'm also not excited by the idea that an incidental
test script gets to dictate what the cutoff is.I do think we should stop claiming that Python 3.2 will work.
(Maybe it does, but we don't know that.) I see that the configure
script only checks for Python >= 3, and it doesn't look like the
meson scripts check explicitly at all, although there's a comment
saying that our meson version cutoff is intended to allow working
with Python 3.5.Maybe it's sufficient to make a documentation change here, and
say we support Python >= 3.5? I'd be okay with saying 3.6.8
too, on the grounds that if anything older fails to work we'd
almost certainly just say "too bad". But RHEL8 is widespread
enough that I think we need to keep making the effort for 3.6.8.
There are a lot of different things that are meant by Python support
nowadays.
It used to be the case that the minimum Python version was (1) the
oldest version that was required to get plpython to compile, and (2) the
range of versions for which we would maintain "expected" files for the
plpython tests (see history of src/pl/plpython/expected/README).
#2 isn't really a problem anymore, it seems. It used to require yearly
attention, but not anymore. (Maybe Python itself has gotten more
stable, or we have changed our tests to be less sensitive to this,
probably a combination of both.)
#1 is also less of a hot-spot, as indicated that we still claim to
support back to Python 3.2. Also, starting in PG18, we are using the
Python "Limited API", so this is being enforced on the Python side.
That means, in principle, if plpython compiles successfully right now on
Python 3.13, then it should also compile successfully on Python 3.2.
But now the new meanings are also, (3) what version of Python is
required by the oldest supported Meson version, and (4) what version of
Python can be assumed by various build and test scripts. There is
actually (4a) scripts that are only run from a meson build, which should
surely align with #3, and (4b) scripts that are also run by a make
build, which is what oauth_server.py is, for which we can pick anything.
The answer to #3 is currently Python 3.5 (see top of top-level
meson.build).
And then there is meaning (5) what version has anyone actually tested.
Note that #2, #3, and #4 are build-time and #1 also has a run-time
component. It would be plausible to say that you need a certain newer
Python to build, but plpython can still run on older versions. If we
were to make any changes, we need to make sure that the documentation is
precise about this and matches the code (see #define Py_LIMITED_API).
The cut-off in practice for these things is usually RHEL. PG18
currently still supports RHEL 7, which appears to come with Python 3.6.
Seeing that the small problem with the test script was easily fixed, I
think we should stick with that for now. There might be value in
dropping support for RHEL 7 sometime, but that should be done across the
board (meson version, openssl version, perl version, etc.), and requires
some buildfarm changes, so I wouldn't do this right now.
On Thu, 24 Apr 2025 at 10:54, Peter Eisentraut <peter@eisentraut.org> wrote:
The cut-off in practice for these things is usually RHEL. PG18
currently still supports RHEL 7, which appears to come with Python 3.6.
Seeing that the small problem with the test script was easily fixed, I
think we should stick with that for now. There might be value in
dropping support for RHEL 7 sometime, but that should be done across the
board (meson version, openssl version, perl version, etc.), and requires
some buildfarm changes, so I wouldn't do this right now.
RHEL7 is not supported by Devrim for PG16 and PG17 already[1]https://yum.postgresql.org/news/rhel7-postgresql-rpms-end-of-life/. Did you
mean something different with "PG18 currently still supports RHEL 7"?
[1]: https://yum.postgresql.org/news/rhel7-postgresql-rpms-end-of-life/
On 24.04.25 13:16, Jelte Fennema-Nio wrote:
On Thu, 24 Apr 2025 at 10:54, Peter Eisentraut <peter@eisentraut.org> wrote:
The cut-off in practice for these things is usually RHEL. PG18
currently still supports RHEL 7, which appears to come with Python 3.6.
Seeing that the small problem with the test script was easily fixed, I
think we should stick with that for now. There might be value in
dropping support for RHEL 7 sometime, but that should be done across the
board (meson version, openssl version, perl version, etc.), and requires
some buildfarm changes, so I wouldn't do this right now.RHEL7 is not supported by Devrim for PG16 and PG17 already[1]. Did you
mean something different with "PG18 currently still supports RHEL 7"?[1]: https://yum.postgresql.org/news/rhel7-postgresql-rpms-end-of-life/
There are approximately 6 buildfarm members with RHEL 7 or CentOS 7, so
in that sense, "support" means that everything is expected to work
there. And some amount of work has been done recently to keep that
support alive.
(But I'm confused now because I see Python 3.6 on both the RHEL 7 and
the RHEL 8 animals. So it's not clear how representative these animals
are especially with respect to the Python versions.)
Peter Eisentraut <peter@eisentraut.org> writes:
There are approximately 6 buildfarm members with RHEL 7 or CentOS 7, so
in that sense, "support" means that everything is expected to work
there. And some amount of work has been done recently to keep that
support alive.
Yeah. Devrim's choice of what to package is not project policy,
it's just his own personal allocation of resources.
(But I'm confused now because I see Python 3.6 on both the RHEL 7 and
the RHEL 8 animals. So it's not clear how representative these animals
are especially with respect to the Python versions.)
Per wikipedia, RHEL7 was released mid-2014, so the latest Python
version 7.0 could possibly have shipped with is 3.4 (released
2014-03) and much more likely they shipped 3.3 (2012-09). However,
you can if you choose install later Python versions, and you
have control over which version /usr/bin/python3 points at.
(At least this is true on the RHEL8 box I am looking at, and
I'm fairly sure it was so on RHEL7 too.) Similarly, Python 3.6
seems to be the baseline default on RHEL8 --- and the timing
more or less matches up, as 3.7 was released not long before
they would have been freezing the RHEL8 feature set. But you can
install 3.8, 3.9, 3.11, and/or 3.12 without going outside the
universe of Red-Hat-supported packages.
So what's that mean for us? "We still want to support RHEL7" turns
out to be squishier than one might think. But I don't think we
really want to promise to work with Python 3.3, especially given
the lack of any buildfarm representation of that. In the other
direction, yeah we could insist that RHEL users install some
later Python version, but I think we'd get push-back. The point
of using an LTS distro is, well, stability. The users want to
decide which packages they are comfortable with updating.
I'm still content with the idea of deciding that 3.6 is now our
cutoff. If someone comes along and complains because
oauth_server.py doesn't work on an older version, it's up to them
to provide a patch. If/when we want to include some Python code
that can't reasonably be made to work on 3.6, we can reconsider.
regards, tom lane
On Thu, Apr 24, 2025 at 7:59 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I'm still content with the idea of deciding that 3.6 is now our
cutoff.
Seems like no one is pushing hard for an earlier version, yet, so
here's a patch with your suggested wording from upthread. I'm not sure
if this meets Peter's request for precision. (Though I'm not really
excited about documenting more precision than we are testing for...)
If someone comes along and complains because
oauth_server.py doesn't work on an older version, it's up to them
to provide a patch.
As an aside, EL7's standard libcurl is too old to pass our current
configure checks, so I think someone would really have to go out of
their way to get to that point.
--Jacob
Attachments:
0001-Bump-the-minimum-supported-Python-version-to-3.6.8.patchapplication/octet-stream; name=0001-Bump-the-minimum-supported-Python-version-to-3.6.8.patchDownload
From ff0c77b016a74d8c92d34acf16d67ca67e1222bb Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Thu, 24 Apr 2025 08:41:43 -0700
Subject: [PATCH] Bump the minimum supported Python version to 3.6.8
Python 3.2 is no longer tested by the buildfarm, and there are only a
handful of buildfarm animals running versions older than 3.6, which
itself went end-of-life in 2021. Python 3.6.8 is the default version
shipped in RHEL8, so that seems like a reasonable baseline for PG18.
Now that we use the Python Limited API as of 0793ab810, older versions
of Python should continue functioning for users of PL/Python in
particular, so soften the language from "required" to "supported".
Suggested-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: TODO
Discussion: https://postgr.es/m/16098.1745079444%40sss.pgh.pa.us
---
doc/src/sgml/installation.sgml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index 077bcc20759..e7ffb942bbd 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -241,7 +241,7 @@
language, you need a <productname>Python</productname>
installation with the header files and
the <application>sysconfig</application> module. The minimum
- required version is <productname>Python</productname> 3.2.
+ supported version is <productname>Python</productname> 3.6.8.
</para>
<para>
--
2.34.1
On 24.04.25 18:20, Jacob Champion wrote:
On Thu, Apr 24, 2025 at 7:59 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I'm still content with the idea of deciding that 3.6 is now our
cutoff.Seems like no one is pushing hard for an earlier version, yet, so
here's a patch with your suggested wording from upthread. I'm not sure
if this meets Peter's request for precision. (Though I'm not really
excited about documenting more precision than we are testing for...)
I like the change to "supported", that's useful.
I would just write 3.6 instead of 3.6.8. We've never tracked the third
version component for Python.
Peter Eisentraut <peter@eisentraut.org> writes:
On 24.04.25 18:20, Jacob Champion wrote:
Seems like no one is pushing hard for an earlier version, yet, so
here's a patch with your suggested wording from upthread. I'm not sure
if this meets Peter's request for precision. (Though I'm not really
excited about documenting more precision than we are testing for...)
I like the change to "supported", that's useful.
I would just write 3.6 instead of 3.6.8. We've never tracked the third
version component for Python.
On the reading that "supported" means "we'll try to fix a problem
rather than telling you to use a newer Python", I suspect that the
correct thing to say is 3.6.8 not 3.6. There may be no difference
in practice; but if push comes to shove I don't think we'd support
a 3.6.x Python version that appears in no LTS distro.
regards, tom lane
On 29.04.25 17:16, Tom Lane wrote:
Peter Eisentraut <peter@eisentraut.org> writes:
On 24.04.25 18:20, Jacob Champion wrote:
Seems like no one is pushing hard for an earlier version, yet, so
here's a patch with your suggested wording from upthread. I'm not sure
if this meets Peter's request for precision. (Though I'm not really
excited about documenting more precision than we are testing for...)I like the change to "supported", that's useful.
I would just write 3.6 instead of 3.6.8. We've never tracked the third
version component for Python.On the reading that "supported" means "we'll try to fix a problem
rather than telling you to use a newer Python", I suspect that the
correct thing to say is 3.6.8 not 3.6. There may be no difference
in practice; but if push comes to shove I don't think we'd support
a 3.6.x Python version that appears in no LTS distro.
Ok, that sounds sensible.
FWIW, 3.6.8 is the "final bugfix release for 3.6", so at least it's not
just some random intermediate version.
On Tue, Apr 29, 2025 at 8:46 AM Peter Eisentraut <peter@eisentraut.org> wrote:
On the reading that "supported" means "we'll try to fix a problem
rather than telling you to use a newer Python", I suspect that the
correct thing to say is 3.6.8 not 3.6. There may be no difference
in practice; but if push comes to shove I don't think we'd support
a 3.6.x Python version that appears in no LTS distro.Ok, that sounds sensible.
Cool, I will plan to push this Sometime Soon, then.
Thanks!
--Jacob
On Tue, Apr 29, 2025 at 8:57 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
Cool, I will plan to push this Sometime Soon, then.
This is now committed. Thanks everybody!
--Jacob
On 29.04.25 22:27, Jacob Champion wrote:
On Tue, Apr 29, 2025 at 8:57 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:Cool, I will plan to push this Sometime Soon, then.
This is now committed. Thanks everybody!
Since we now require Python 3.6, we can also remove PL/Python test
alternative expected files for earlier Python versions. See attached patch.
Attachments:
0001-plpython-Remove-obsolete-test-expected-file.patchtext/plain; charset=UTF-8; name=0001-plpython-Remove-obsolete-test-expected-file.patchDownload
From f3f155e62db55c8782abe084d9ba46925c068cda Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter@eisentraut.org>
Date: Thu, 5 Jun 2025 09:47:15 +0200
Subject: [PATCH] plpython: Remove obsolete test expected file
Move plpython_error_5.out to plpython_error.out, since the pre-3.5
version is no longer needed, since we raised the Python requirement to
3.6 (commit 45363fca637).
---
src/pl/plpython/expected/README | 3 -
src/pl/plpython/expected/plpython_error.out | 2 +-
src/pl/plpython/expected/plpython_error_5.out | 460 ------------------
3 files changed, 1 insertion(+), 464 deletions(-)
delete mode 100644 src/pl/plpython/expected/README
delete mode 100644 src/pl/plpython/expected/plpython_error_5.out
diff --git a/src/pl/plpython/expected/README b/src/pl/plpython/expected/README
deleted file mode 100644
index 388c553a589..00000000000
--- a/src/pl/plpython/expected/README
+++ /dev/null
@@ -1,3 +0,0 @@
-Guide to alternative expected files:
-
-plpython_error_5.out Python 3.5 and newer
diff --git a/src/pl/plpython/expected/plpython_error.out b/src/pl/plpython/expected/plpython_error.out
index 68722b00097..fd9cd73be74 100644
--- a/src/pl/plpython/expected/plpython_error.out
+++ b/src/pl/plpython/expected/plpython_error.out
@@ -243,7 +243,7 @@ $$
plpy.nonexistent
$$ LANGUAGE plpython3u;
SELECT toplevel_attribute_error();
-ERROR: AttributeError: 'module' object has no attribute 'nonexistent'
+ERROR: AttributeError: module 'plpy' has no attribute 'nonexistent'
CONTEXT: Traceback (most recent call last):
PL/Python function "toplevel_attribute_error", line 2, in <module>
plpy.nonexistent
diff --git a/src/pl/plpython/expected/plpython_error_5.out b/src/pl/plpython/expected/plpython_error_5.out
deleted file mode 100644
index fd9cd73be74..00000000000
--- a/src/pl/plpython/expected/plpython_error_5.out
+++ /dev/null
@@ -1,460 +0,0 @@
--- test error handling, i forgot to restore Warn_restart in
--- the trigger handler once. the errors and subsequent core dump were
--- interesting.
-/* Flat out Python syntax error
- */
-CREATE FUNCTION python_syntax_error() RETURNS text
- AS
-'.syntaxerror'
- LANGUAGE plpython3u;
-ERROR: could not compile PL/Python function "python_syntax_error"
-DETAIL: SyntaxError: invalid syntax (<string>, line 2)
-/* With check_function_bodies = false the function should get defined
- * and the error reported when called
- */
-SET check_function_bodies = false;
-CREATE FUNCTION python_syntax_error() RETURNS text
- AS
-'.syntaxerror'
- LANGUAGE plpython3u;
-SELECT python_syntax_error();
-ERROR: could not compile PL/Python function "python_syntax_error"
-DETAIL: SyntaxError: invalid syntax (<string>, line 2)
-/* Run the function twice to check if the hashtable entry gets cleaned up */
-SELECT python_syntax_error();
-ERROR: could not compile PL/Python function "python_syntax_error"
-DETAIL: SyntaxError: invalid syntax (<string>, line 2)
-RESET check_function_bodies;
-/* Flat out syntax error
- */
-CREATE FUNCTION sql_syntax_error() RETURNS text
- AS
-'plpy.execute("syntax error")'
- LANGUAGE plpython3u;
-SELECT sql_syntax_error();
-ERROR: spiexceptions.SyntaxError: syntax error at or near "syntax"
-LINE 1: syntax error
- ^
-QUERY: syntax error
-CONTEXT: Traceback (most recent call last):
- PL/Python function "sql_syntax_error", line 1, in <module>
- plpy.execute("syntax error")
-PL/Python function "sql_syntax_error"
-/* check the handling of uncaught python exceptions
- */
-CREATE FUNCTION exception_index_invalid(text) RETURNS text
- AS
-'return args[1]'
- LANGUAGE plpython3u;
-SELECT exception_index_invalid('test');
-ERROR: IndexError: list index out of range
-CONTEXT: Traceback (most recent call last):
- PL/Python function "exception_index_invalid", line 1, in <module>
- return args[1]
-PL/Python function "exception_index_invalid"
-/* check handling of nested exceptions
- */
-CREATE FUNCTION exception_index_invalid_nested() RETURNS text
- AS
-'rv = plpy.execute("SELECT test5(''foo'')")
-return rv[0]'
- LANGUAGE plpython3u;
-SELECT exception_index_invalid_nested();
-ERROR: spiexceptions.UndefinedFunction: function test5(unknown) does not exist
-LINE 1: SELECT test5('foo')
- ^
-HINT: No function matches the given name and argument types. You might need to add explicit type casts.
-QUERY: SELECT test5('foo')
-CONTEXT: Traceback (most recent call last):
- PL/Python function "exception_index_invalid_nested", line 1, in <module>
- rv = plpy.execute("SELECT test5('foo')")
-PL/Python function "exception_index_invalid_nested"
-/* a typo
- */
-CREATE FUNCTION invalid_type_uncaught(a text) RETURNS text
- AS
-'if "plan" not in SD:
- q = "SELECT fname FROM users WHERE lname = $1"
- SD["plan"] = plpy.prepare(q, [ "test" ])
-rv = plpy.execute(SD["plan"], [ a ])
-if len(rv):
- return rv[0]["fname"]
-return None
-'
- LANGUAGE plpython3u;
-SELECT invalid_type_uncaught('rick');
-ERROR: spiexceptions.UndefinedObject: type "test" does not exist
-CONTEXT: Traceback (most recent call last):
- PL/Python function "invalid_type_uncaught", line 3, in <module>
- SD["plan"] = plpy.prepare(q, [ "test" ])
-PL/Python function "invalid_type_uncaught"
-/* for what it's worth catch the exception generated by
- * the typo, and return None
- */
-CREATE FUNCTION invalid_type_caught(a text) RETURNS text
- AS
-'if "plan" not in SD:
- q = "SELECT fname FROM users WHERE lname = $1"
- try:
- SD["plan"] = plpy.prepare(q, [ "test" ])
- except plpy.SPIError as ex:
- plpy.notice(str(ex))
- return None
-rv = plpy.execute(SD["plan"], [ a ])
-if len(rv):
- return rv[0]["fname"]
-return None
-'
- LANGUAGE plpython3u;
-SELECT invalid_type_caught('rick');
-NOTICE: type "test" does not exist
- invalid_type_caught
----------------------
-
-(1 row)
-
-/* for what it's worth catch the exception generated by
- * the typo, and reraise it as a plain error
- */
-CREATE FUNCTION invalid_type_reraised(a text) RETURNS text
- AS
-'if "plan" not in SD:
- q = "SELECT fname FROM users WHERE lname = $1"
- try:
- SD["plan"] = plpy.prepare(q, [ "test" ])
- except plpy.SPIError as ex:
- plpy.error(str(ex))
-rv = plpy.execute(SD["plan"], [ a ])
-if len(rv):
- return rv[0]["fname"]
-return None
-'
- LANGUAGE plpython3u;
-SELECT invalid_type_reraised('rick');
-ERROR: plpy.Error: type "test" does not exist
-CONTEXT: Traceback (most recent call last):
- PL/Python function "invalid_type_reraised", line 6, in <module>
- plpy.error(str(ex))
-PL/Python function "invalid_type_reraised"
-/* no typo no messing about
- */
-CREATE FUNCTION valid_type(a text) RETURNS text
- AS
-'if "plan" not in SD:
- SD["plan"] = plpy.prepare("SELECT fname FROM users WHERE lname = $1", [ "text" ])
-rv = plpy.execute(SD["plan"], [ a ])
-if len(rv):
- return rv[0]["fname"]
-return None
-'
- LANGUAGE plpython3u;
-SELECT valid_type('rick');
- valid_type
-------------
-
-(1 row)
-
-/* error in nested functions to get a traceback
-*/
-CREATE FUNCTION nested_error() RETURNS text
- AS
-'def fun1():
- plpy.error("boom")
-
-def fun2():
- fun1()
-
-def fun3():
- fun2()
-
-fun3()
-return "not reached"
-'
- LANGUAGE plpython3u;
-SELECT nested_error();
-ERROR: plpy.Error: boom
-CONTEXT: Traceback (most recent call last):
- PL/Python function "nested_error", line 10, in <module>
- fun3()
- PL/Python function "nested_error", line 8, in fun3
- fun2()
- PL/Python function "nested_error", line 5, in fun2
- fun1()
- PL/Python function "nested_error", line 2, in fun1
- plpy.error("boom")
-PL/Python function "nested_error"
-/* raising plpy.Error is just like calling plpy.error
-*/
-CREATE FUNCTION nested_error_raise() RETURNS text
- AS
-'def fun1():
- raise plpy.Error("boom")
-
-def fun2():
- fun1()
-
-def fun3():
- fun2()
-
-fun3()
-return "not reached"
-'
- LANGUAGE plpython3u;
-SELECT nested_error_raise();
-ERROR: plpy.Error: boom
-CONTEXT: Traceback (most recent call last):
- PL/Python function "nested_error_raise", line 10, in <module>
- fun3()
- PL/Python function "nested_error_raise", line 8, in fun3
- fun2()
- PL/Python function "nested_error_raise", line 5, in fun2
- fun1()
- PL/Python function "nested_error_raise", line 2, in fun1
- raise plpy.Error("boom")
-PL/Python function "nested_error_raise"
-/* using plpy.warning should not produce a traceback
-*/
-CREATE FUNCTION nested_warning() RETURNS text
- AS
-'def fun1():
- plpy.warning("boom")
-
-def fun2():
- fun1()
-
-def fun3():
- fun2()
-
-fun3()
-return "you''ve been warned"
-'
- LANGUAGE plpython3u;
-SELECT nested_warning();
-WARNING: boom
- nested_warning
---------------------
- you've been warned
-(1 row)
-
-/* AttributeError at toplevel used to give segfaults with the traceback
-*/
-CREATE FUNCTION toplevel_attribute_error() RETURNS void AS
-$$
-plpy.nonexistent
-$$ LANGUAGE plpython3u;
-SELECT toplevel_attribute_error();
-ERROR: AttributeError: module 'plpy' has no attribute 'nonexistent'
-CONTEXT: Traceback (most recent call last):
- PL/Python function "toplevel_attribute_error", line 2, in <module>
- plpy.nonexistent
-PL/Python function "toplevel_attribute_error"
-/* Calling PL/Python functions from SQL and vice versa should not lose context.
- */
-CREATE OR REPLACE FUNCTION python_traceback() RETURNS void AS $$
-def first():
- second()
-
-def second():
- third()
-
-def third():
- plpy.execute("select sql_error()")
-
-first()
-$$ LANGUAGE plpython3u;
-CREATE OR REPLACE FUNCTION sql_error() RETURNS void AS $$
-begin
- select 1/0;
-end
-$$ LANGUAGE plpgsql;
-CREATE OR REPLACE FUNCTION python_from_sql_error() RETURNS void AS $$
-begin
- select python_traceback();
-end
-$$ LANGUAGE plpgsql;
-CREATE OR REPLACE FUNCTION sql_from_python_error() RETURNS void AS $$
-plpy.execute("select sql_error()")
-$$ LANGUAGE plpython3u;
-SELECT python_traceback();
-ERROR: spiexceptions.DivisionByZero: division by zero
-CONTEXT: Traceback (most recent call last):
- PL/Python function "python_traceback", line 11, in <module>
- first()
- PL/Python function "python_traceback", line 3, in first
- second()
- PL/Python function "python_traceback", line 6, in second
- third()
- PL/Python function "python_traceback", line 9, in third
- plpy.execute("select sql_error()")
-PL/Python function "python_traceback"
-SELECT sql_error();
-ERROR: division by zero
-CONTEXT: SQL statement "select 1/0"
-PL/pgSQL function sql_error() line 3 at SQL statement
-SELECT python_from_sql_error();
-ERROR: spiexceptions.DivisionByZero: division by zero
-CONTEXT: Traceback (most recent call last):
- PL/Python function "python_traceback", line 11, in <module>
- first()
- PL/Python function "python_traceback", line 3, in first
- second()
- PL/Python function "python_traceback", line 6, in second
- third()
- PL/Python function "python_traceback", line 9, in third
- plpy.execute("select sql_error()")
-PL/Python function "python_traceback"
-SQL statement "select python_traceback()"
-PL/pgSQL function python_from_sql_error() line 3 at SQL statement
-SELECT sql_from_python_error();
-ERROR: spiexceptions.DivisionByZero: division by zero
-CONTEXT: Traceback (most recent call last):
- PL/Python function "sql_from_python_error", line 2, in <module>
- plpy.execute("select sql_error()")
-PL/Python function "sql_from_python_error"
-/* check catching specific types of exceptions
- */
-CREATE TABLE specific (
- i integer PRIMARY KEY
-);
-CREATE FUNCTION specific_exception(i integer) RETURNS void AS
-$$
-from plpy import spiexceptions
-try:
- plpy.execute("insert into specific values (%s)" % (i or "NULL"));
-except spiexceptions.NotNullViolation as e:
- plpy.notice("Violated the NOT NULL constraint, sqlstate %s" % e.sqlstate)
-except spiexceptions.UniqueViolation as e:
- plpy.notice("Violated the UNIQUE constraint, sqlstate %s" % e.sqlstate)
-$$ LANGUAGE plpython3u;
-SELECT specific_exception(2);
- specific_exception
---------------------
-
-(1 row)
-
-SELECT specific_exception(NULL);
-NOTICE: Violated the NOT NULL constraint, sqlstate 23502
- specific_exception
---------------------
-
-(1 row)
-
-SELECT specific_exception(2);
-NOTICE: Violated the UNIQUE constraint, sqlstate 23505
- specific_exception
---------------------
-
-(1 row)
-
-/* SPI errors in PL/Python functions should preserve the SQLSTATE value
- */
-CREATE FUNCTION python_unique_violation() RETURNS void AS $$
-plpy.execute("insert into specific values (1)")
-plpy.execute("insert into specific values (1)")
-$$ LANGUAGE plpython3u;
-CREATE FUNCTION catch_python_unique_violation() RETURNS text AS $$
-begin
- begin
- perform python_unique_violation();
- exception when unique_violation then
- return 'ok';
- end;
- return 'not reached';
-end;
-$$ language plpgsql;
-SELECT catch_python_unique_violation();
- catch_python_unique_violation
--------------------------------
- ok
-(1 row)
-
-/* manually starting subtransactions - a bad idea
- */
-CREATE FUNCTION manual_subxact() RETURNS void AS $$
-plpy.execute("savepoint save")
-plpy.execute("create table foo(x integer)")
-plpy.execute("rollback to save")
-$$ LANGUAGE plpython3u;
-SELECT manual_subxact();
-ERROR: plpy.SPIError: SPI_execute failed: SPI_ERROR_TRANSACTION
-CONTEXT: Traceback (most recent call last):
- PL/Python function "manual_subxact", line 2, in <module>
- plpy.execute("savepoint save")
-PL/Python function "manual_subxact"
-/* same for prepared plans
- */
-CREATE FUNCTION manual_subxact_prepared() RETURNS void AS $$
-save = plpy.prepare("savepoint save")
-rollback = plpy.prepare("rollback to save")
-plpy.execute(save)
-plpy.execute("create table foo(x integer)")
-plpy.execute(rollback)
-$$ LANGUAGE plpython3u;
-SELECT manual_subxact_prepared();
-ERROR: plpy.SPIError: SPI_execute_plan failed: SPI_ERROR_TRANSACTION
-CONTEXT: Traceback (most recent call last):
- PL/Python function "manual_subxact_prepared", line 4, in <module>
- plpy.execute(save)
-PL/Python function "manual_subxact_prepared"
-/* raising plpy.spiexception.* from python code should preserve sqlstate
- */
-CREATE FUNCTION plpy_raise_spiexception() RETURNS void AS $$
-raise plpy.spiexceptions.DivisionByZero()
-$$ LANGUAGE plpython3u;
-DO $$
-BEGIN
- SELECT plpy_raise_spiexception();
-EXCEPTION WHEN division_by_zero THEN
- -- NOOP
-END
-$$ LANGUAGE plpgsql;
-/* setting a custom sqlstate should be handled
- */
-CREATE FUNCTION plpy_raise_spiexception_override() RETURNS void AS $$
-exc = plpy.spiexceptions.DivisionByZero()
-exc.sqlstate = 'SILLY'
-raise exc
-$$ LANGUAGE plpython3u;
-DO $$
-BEGIN
- SELECT plpy_raise_spiexception_override();
-EXCEPTION WHEN SQLSTATE 'SILLY' THEN
- -- NOOP
-END
-$$ LANGUAGE plpgsql;
-/* test the context stack trace for nested execution levels
- */
-CREATE FUNCTION notice_innerfunc() RETURNS int AS $$
-plpy.execute("DO LANGUAGE plpython3u $x$ plpy.notice('inside DO') $x$")
-return 1
-$$ LANGUAGE plpython3u;
-CREATE FUNCTION notice_outerfunc() RETURNS int AS $$
-plpy.execute("SELECT notice_innerfunc()")
-return 1
-$$ LANGUAGE plpython3u;
-\set SHOW_CONTEXT always
-SELECT notice_outerfunc();
-NOTICE: inside DO
-CONTEXT: PL/Python anonymous code block
-SQL statement "DO LANGUAGE plpython3u $x$ plpy.notice('inside DO') $x$"
-PL/Python function "notice_innerfunc"
-SQL statement "SELECT notice_innerfunc()"
-PL/Python function "notice_outerfunc"
- notice_outerfunc
-------------------
- 1
-(1 row)
-
-/* test error logged with an underlying exception that includes a detail
- * string (bug #18070).
- */
-CREATE FUNCTION python_error_detail() RETURNS SETOF text AS $$
- plan = plpy.prepare("SELECT to_date('xy', 'DD') d")
- for row in plpy.cursor(plan):
- yield row['d']
-$$ LANGUAGE plpython3u;
-SELECT python_error_detail();
-ERROR: error fetching next item from iterator
-DETAIL: spiexceptions.InvalidDatetimeFormat: invalid value "xy" for "DD"
-CONTEXT: Traceback (most recent call last):
-PL/Python function "python_error_detail"
--
2.49.0
Peter Eisentraut <peter@eisentraut.org> writes:
Since we now require Python 3.6, we can also remove PL/Python test
alternative expected files for earlier Python versions. See attached patch.
+1. So nice to get rid of src/pl/plpython/expected/README.
regards, tom lane
On Fri, Jun 6, 2025 at 7:17 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Peter Eisentraut <peter@eisentraut.org> writes:
Since we now require Python 3.6, we can also remove PL/Python test
alternative expected files for earlier Python versions. See attached patch.+1. So nice to get rid of src/pl/plpython/expected/README.
Awesome! LGTM.
--Jacob
On 06.06.25 17:46, Jacob Champion wrote:
On Fri, Jun 6, 2025 at 7:17 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Peter Eisentraut <peter@eisentraut.org> writes:
Since we now require Python 3.6, we can also remove PL/Python test
alternative expected files for earlier Python versions. See attached patch.+1. So nice to get rid of src/pl/plpython/expected/README.
Awesome! LGTM.
done