doc: virtual envs with Pl/Python
I think there should be a mention of virtual environments in the plpython
docs.
Something like $attached maybe?
Attachments:
v1-0001-Add-tip-on-using-virtual-envs-with-Pl-Python.patchapplication/octet-stream; name=v1-0001-Add-tip-on-using-virtual-envs-with-Pl-Python.patchDownload
From 6d271aee716bf5d7a4a5dc632db88b6418ac8b59 Mon Sep 17 00:00:00 2001
From: Florents Tselai <florents.tselai@gmail.com>
Date: Thu, 17 Oct 2024 11:30:20 +0300
Subject: [PATCH v1] Add tip on using virtual envs with Pl/Python
---
doc/src/sgml/plpython.sgml | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/doc/src/sgml/plpython.sgml b/doc/src/sgml/plpython.sgml
index e5d51d6e9f..9bea992e0f 100644
--- a/doc/src/sgml/plpython.sgml
+++ b/doc/src/sgml/plpython.sgml
@@ -1393,5 +1393,15 @@ plpy.execute("UPDATE tbl SET %s = %s WHERE key = %s" % (
the <command>python</command> man page are only effective in a
command-line interpreter and not an embedded Python interpreter.)
</para>
+
+ <tip>
+ <para>
+ When installing <application>PL/Python</application>,
+ package managers usually pick up the system-wide interpreter.
+ In Python it is generally recommended to use virtual environments.
+ In that case you can set <envar>PYTHONPATH</envar> to something like
+ <literal>PYTHONPATH=/path/to/venv/lib/python3.XY</literal>.
+ </para>
+ </tip>
</sect1>
</chapter>
--
2.39.5 (Apple Git-154)
Florents Tselai <florents.tselai@gmail.com> writes:
I think there should be a mention of virtual environments in the plpython
docs.
Why? We are not here to teach people how to use Python. More, this
patch seems like not merely teaching Python, but teaching the use of
specific Python installation patterns that might or might not apply
on a particular platform. That's not going to scale.
regards, tom lane
On 17.10.24 15:56, Tom Lane wrote:
Florents Tselai <florents.tselai@gmail.com> writes:
I think there should be a mention of virtual environments in the plpython
docs.Why? We are not here to teach people how to use Python. More, this
patch seems like not merely teaching Python, but teaching the use of
specific Python installation patterns that might or might not apply
on a particular platform. That's not going to scale.
Virtual environments are a standard part of Python now, so maybe there
is something to this. But personally I don't understand this. Do you
mean for this to select the Python environment that PL/Python is built
against, or do you want to use the virtual environment at run time? The
text says "installing" and talks about package managers. It's unclear
to me at exactly what step this is supposed to apply.
On 17 Oct 2024, at 5:10 PM, Peter Eisentraut <peter@eisentraut.org> wrote:
On 17.10.24 15:56, Tom Lane wrote:
Florents Tselai <florents.tselai@gmail.com> writes:
I think there should be a mention of virtual environments in the plpython
docs.Why? We are not here to teach people how to use Python. More, this
patch seems like not merely teaching Python, but teaching the use of
specific Python installation patterns that might or might not apply
on a particular platform. That's not going to scale.Virtual environments are a standard part of Python now, so maybe there is something to this. But personally I don't understand this. Do you mean for this to select the Python environment that PL/Python is built against, or do you want to use the virtual environment at run time? The text says "installing" and talks about package managers. It's unclear to me at exactly what step this is supposed to apply.
this
want to use the virtual environment at run time?
OK, I should have explained this a bit more.
Here’s the story:
Unless someone builds from source, chances are, the default Python interpreter is something like /usr/bin/python3 .
Which sometimes may even point to python3.10/11/12 .
Plus, If you have multiple Postgres versions (not unusual if you’ve had the same PG server for years),
the version matrix gets too complicated.
Which means that if you want non-standard packages in your Pl/Python code,
you’ll have to `sudo python3 -m pip install` things, which in turn will break things in different versions.
In fact, in the latest versions standard Python actively discourages that by having your
say `pip install … —break-system-packages`
The practice now is to build a virtual `python3 -m venv`, which, as you say, is standard.
And indeed, it is not just a specific installation pattern.
I’m not getting into fancy things like poetry et. al. here.
Just this https://docs.python.org/3/library/venv.html
The documentation as-is mentions the availability of env vars that can be set and stops there.
As-is, someone who reads the documentation is left with the impression that
the only way to use non-standard Python libs is by `sudo python3 -m install package` because
that’s the python3 interpreter that plpython3u picks-up during installation.
The proposed note goes a step further and aims to say how to support this not-so-rare pattern.
I hope that’s more clear now.
PS: fwiw, It’s practically impossible to build —with-python=/path/to/venv because most virtualenvs creators don’t bother handling header files correctly. config/python.m4 can’t support that. But that’s another story irrelevant here.
On Thu, Oct 17, 2024 at 1:10 PM Florents Tselai
<florents.tselai@gmail.com> wrote:
Which means that if you want non-standard packages in your Pl/Python code,
you’ll have to `sudo python3 -m pip install` things
Or `sudo apt install python3-<package>`, or `pip install --user
<package>` as the database user, or...
Virtualenvs are great -- I use them daily -- but there are plenty of
different ways to use the standard interpreter, with other pros and
cons.
--Jacob
Jacob Champion <jacob.champion@enterprisedb.com> writes:
On Thu, Oct 17, 2024 at 1:10 PM Florents Tselai
<florents.tselai@gmail.com> wrote:Which means that if you want non-standard packages in your Pl/Python code,
you’ll have to `sudo python3 -m pip install` things
Or `sudo apt install python3-<package>`, or `pip install --user
<package>` as the database user, or...
Virtualenvs are great -- I use them daily -- but there are plenty of
different ways to use the standard interpreter, with other pros and
cons.
What I'm still confused about is whether this is anything that
a typical user of Python shouldn't be expected to know already.
I understand the idea of setting up a virtualenv that is decoupled
from the system-supplied package set, but if the user has done
that, wouldn't they already know about setting PYTHONPATH?
I can believe that they might not be totally clear on how to
set it in a way that will affect the Postgres server --- but
the proposed patch provides no details about that, and hardly
could since it'd vary so much from one server packaging to
another.
regards, tom lane
On Thu, Oct 17, 2024 at 2:03 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
What I'm still confused about is whether this is anything that
a typical user of Python shouldn't be expected to know already.
I mean... I think I agree, but also it seems like people regularly
find ways to shoot themselves in the foot, so I'm not opposed to some
tips in theory. But they should be more generally applicable IMO.
I understand the idea of setting up a virtualenv that is decoupled
from the system-supplied package set, but if the user has done
that, wouldn't they already know about setting PYTHONPATH?
Agreed.
I can believe that they might not be totally clear on how to
set it in a way that will affect the Postgres server --- but
the proposed patch provides no details about that, and hardly
could since it'd vary so much from one server packaging to
another.
Yeah. I think this patch is _way_ too prescriptive, and maybe belongs
in a wiki on "Ways I Get Complex Python Setups Working" or similar,
where it can share space with alternatives.
--Jacob