Quick Extensions Question
It's about dependences.
If my extension requires a procedural language, will adding that language to the `requires` control key do what I think it should do?
If not, how should one require a PL? Come to think of it, how might I require other features that might not be included in a particular build, like xpath()?
Thanks,
David
"David E. Wheeler" <david@kineticode.com> writes:
If my extension requires a procedural language, will adding that language to the `requires` control key do what I think it should do?
No.
Probably in future the standard PLs will be packaged as extensions, and
then it will work. The main reason that it won't happen for 9.1 is that
right now we require superuser privilege to install an extension, which
would be a regression compared to the privilege requirements for
installing standard PLs in existing releases. And relaxing that
requirement is a research project :-(
regards, tom lane
On Mar 2, 2011, at 11:00 PM, Tom Lane wrote:
"David E. Wheeler" <david@kineticode.com> writes:
If my extension requires a procedural language, will adding that language to the `requires` control key do what I think it should do?
No.
Probably in future the standard PLs will be packaged as extensions, and
then it will work. The main reason that it won't happen for 9.1 is that
right now we require superuser privilege to install an extension, which
would be a regression compared to the privilege requirements for
installing standard PLs in existing releases. And relaxing that
requirement is a research project :-(
Okay. I guess the to do list should be updated?
http://wiki.postgresql.org/wiki/Todo
Best,
David
Tom Lane <tgl@sss.pgh.pa.us> writes:
Probably in future the standard PLs will be packaged as extensions, and
then it will work. The main reason that it won't happen for 9.1 is that
right now we require superuser privilege to install an extension, which
would be a regression compared to the privilege requirements for
installing standard PLs in existing releases. And relaxing that
requirement is a research project :-(
Meanwhile, is it possible to have CREATE LANGUAGE internally register an
extension? It's a kludge but I somehow though I'd mention it.
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
Dimitri Fontaine <dimitri@2ndQuadrant.fr> writes:
Tom Lane <tgl@sss.pgh.pa.us> writes:
Probably in future the standard PLs will be packaged as extensions, and
then it will work. The main reason that it won't happen for 9.1 is that
right now we require superuser privilege to install an extension, which
would be a regression compared to the privilege requirements for
installing standard PLs in existing releases. And relaxing that
requirement is a research project :-(
Meanwhile, is it possible to have CREATE LANGUAGE internally register an
extension? It's a kludge but I somehow though I'd mention it.
Hmmm ... it definitely is a kluge, but ...
You'd need to work out how the CREATE OR REPLACE and DROP cases would
work.
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> writes:
Meanwhile, is it possible to have CREATE LANGUAGE internally register an
extension? It's a kludge but I somehow though I'd mention it.Hmmm ... it definitely is a kluge, but ...
You'd need to work out how the CREATE OR REPLACE and DROP cases would
work.
Maybe the fever ain't gone far enough, but I'd just do nothing in the
first case and internally cascade to the extension in the second case.
In fact internally the drop case would be redirected on the extension
and the dependencies would get rid of the PL, right?
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
Dimitri Fontaine <dimitri@2ndQuadrant.fr> writes:
Tom Lane <tgl@sss.pgh.pa.us> writes:
You'd need to work out how the CREATE OR REPLACE and DROP cases would
work.
Maybe the fever ain't gone far enough, but I'd just do nothing in the
first case and internally cascade to the extension in the second case.
In fact internally the drop case would be redirected on the extension
and the dependencies would get rid of the PL, right?
Not sure it's that easy. I think DROP LANGUAGE can't assume that the
language it's been told to drop is extension-ified. (Even if we do this
for all the core ones, there are a dozen non-core ones that might not
all get with the program right away.) How do we make this work in a way
that covers both cases, but doesn't turn DROP LANGUAGE into a security
hole that lets non-superusers drop random extensions?
It may all work pretty easily, but I'm still caffeine-deprived so I'm
not sure ...
regards, tom lane
On Thu, Mar 3, 2011 at 10:31 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
It may all work pretty easily, but I'm still caffeine-deprived so I'm
not sure ...
I think that it's not a good idea to devote too much energy to this
problem right now, anyway. We have crammed a gigantic pile of code
into the source tree in the last month, and there are bound to be bugs
even in what we already have. We need to get that code out into the
field in the form of alpha and beta releases and start getting it
tested; and even apart from bugs, we need to get some field experience
with it, so that we learn which things work well enough in practice
and which things are really problems. Then we can come back to the
table and engineer better solutions for 9.2.
It is also important that we get to the point of being able to start
accepting 9.2 patches on other topics as quickly as possible. Unless
we do something radically different than what we have done in previous
releases, we are now about to enter a ~4 month period during which
there will be no CommitFests and very little 9.2 work underway, at
least publicly. Even with the improvements we have made in getting
CommitFests started and finished on time (the present case being,
fortunately, an exception, and yeah I know it could have been a lot
worse, but it could also have been better), the long quiet period that
is required to get a release out the door is still problematic for
many of our developers.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
[ slightly more awake now ... ]
I wrote:
Not sure it's that easy. I think DROP LANGUAGE can't assume that the
language it's been told to drop is extension-ified. (Even if we do this
for all the core ones, there are a dozen non-core ones that might not
all get with the program right away.)
No, wait, that's nonsense. With what you're talking about, there would
never be a non-extension-ified language, because CREATE LANGUAGE would
force it to be attached to an extension. So maybe that problem isn't
so hard after all.
However, what *is* looking a bit hard is dump/restore behavior. By
default, pg_dump would proceed to dump an installed language as a CREATE
EXTENSION command, and that would fail on restore, especially if you
were trying to restore as non superuser. The behavior of
--binary-upgrade would be differently unpleasant: it would make the
extension, then try to CREATE LANGUAGE, and that would fail because the
extension name already exists. No doubt we could kluge those behaviors
too, but it's starting to look pretty messy.
On the whole I'd rather spend time trying to fix the real problem, which
is allowing CREATE EXTENSION to non-superusers. The general case of
that looks quite hard to me, but maybe we could get something that works
for the single case of an extension containing just a language.
regards, tom lane
Robert Haas <robertmhaas@gmail.com> writes:
I think that it's not a good idea to devote too much energy to this
problem right now, anyway. [ we need to get to beta ASAP, instead ]
I hear you, but once we get to beta, or even the last alpha, it's going
to be very hard to make changes that would interfere with people doing
upgrades or dump/restores. If we don't do something about the language-
as-extension situation right now, the window will be closed until 9.2.
Most of the other things that are on our plates for beta are internal
changes that would be unlikely to break the upgrade path for beta
testers.
You may well be right that there's no way to fix this with an amount of
effort that would be appropriate to spend now, but I don't want to just
walk away from the problem without considering whether there is a way
that we can fix it with a day or so's additional effort.
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> writes:
Not sure it's that easy. I think DROP LANGUAGE can't assume that the
language it's been told to drop is extension-ified. (Even if we do this
If CREATE LANGUAGE creates an extension of the same name, then DROP
LANGUAGE can assume that there's an extension of the same name, right?
for all the core ones, there are a dozen non-core ones that might not
all get with the program right away.) How do we make this work in a way
that covers both cases, but doesn't turn DROP LANGUAGE into a security
hole that lets non-superusers drop random extensions?
We could check that the extension named the same as the language only
contains one object of class pg_language.
It may all work pretty easily, but I'm still caffeine-deprived so I'm
not sure ...
It does not look like a big deal to me either. If you don't have the
time too, I could propose a patch.
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
On Thu, Mar 3, 2011 at 11:19 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
I think that it's not a good idea to devote too much energy to this
problem right now, anyway. [ we need to get to beta ASAP, instead ]I hear you, but once we get to beta, or even the last alpha, it's going
to be very hard to make changes that would interfere with people doing
upgrades or dump/restores. If we don't do something about the language-
as-extension situation right now, the window will be closed until 9.2.
So what? AFAIK the extension patch hasn't broken anything here that
used to work. People can still install languages the way they always
have. What we're talking about here is a way of installing languages
that is arguably nicer than what they are doing now. The window for
feature enhancements is already closed until 9.2, unless you want to
go back and start working through every patch we marked Returned with
Feedback during this last CommitFest.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Thu, Mar 3, 2011 at 4:30 PM, Robert Haas <robertmhaas@gmail.com> wrote:
So what? AFAIK the extension patch hasn't broken anything here that
used to work. People can still install languages the way they always
have. What we're talking about here is a way of installing languages
that is arguably nicer than what they are doing now. The window for
feature enhancements is already closed until 9.2, unless you want to
go back and start working through every patch we marked Returned with
Feedback during this last CommitFest.
No, what is being talked about isn't intended as a "way of installing
languages that is ... nicer". What is being talked about is allowing
an "Extension" that is being installed know that it's going to blow up
because it's required language (plpgsql, for instance) isn't
installed.
Maybe it's a problem with extensions that isn't easily solvable, but
that means extension authors are going to have a readme in their
extension with the followign text:
EXTENSION "mystuff" requires that pl/pgsql be installed in the
database. There is no way for the extension to check this before
it is installed, so make sure it's installed, or be prepared to
cope with errors during the installation.
And make sure you don't try and drop pl/pgsql language when
the extension is installed either.
Maybe that's enough for 9.1.
a.
--
Aidan Van Dyk Create like a god,
aidan@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.
On 03.03.2011 18:30, Robert Haas wrote:
On Thu, Mar 3, 2011 at 11:19 AM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
Robert Haas<robertmhaas@gmail.com> writes:
I think that it's not a good idea to devote too much energy to this
problem right now, anyway. [ we need to get to beta ASAP, instead ]I hear you, but once we get to beta, or even the last alpha, it's going
to be very hard to make changes that would interfere with people doing
upgrades or dump/restores. If we don't do something about the language-
as-extension situation right now, the window will be closed until 9.2.So what? AFAIK the extension patch hasn't broken anything here that
used to work. People can still install languages the way they always
have. What we're talking about here is a way of installing languages
that is arguably nicer than what they are doing now.
IMHO the main advantage of having languages as extensions is that you
could define a dependency on a language.
We've been talking about PLs, but what about the other thing David
asked: could we have extension entries for compile-time options like SSL
or libxml, so that you could define a dependency on them?
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
On Thu, Mar 3, 2011 at 11:39 AM, Aidan Van Dyk <aidan@highrise.ca> wrote:
No, what is being talked about isn't intended as a "way of installing
languages that is ... nicer". What is being talked about is allowing
an "Extension" that is being installed know that it's going to blow up
because it's required language (plpgsql, for instance) isn't
installed.Maybe it's a problem with extensions that isn't easily solvable, but
that means extension authors are going to have a readme in their
extension with the followign text:
EXTENSION "mystuff" requires that pl/pgsql be installed in the
database. There is no way for the extension to check this before
it is installed, so make sure it's installed, or be prepared to
cope with errors during the installation.And make sure you don't try and drop pl/pgsql language when
the extension is installed either.Maybe that's enough for 9.1.
Well, in 9.0, what'll happen is you'll type psql -f somefile.sql and
it'll spit out a stream of error messages if things aren't in the
prerequisite state. Unless you happen to have taken the precaution of
wrapping the whole thing in a transaction, you may end up with the
extension half-installed and some fun manual cleanup to do. I'm not
going to argue that this is perfect, but it is already better than it
was. The extensions patch didn't change much between mid-Decemeber
when we wrapped up CF#3 and early February when Tom picked it up. If
he wanted to have a chance to do lots more refinement after the
initial commit, he had the means within his grasp: he could have
picked it up in December. It is no more fair for Tom to hold up this
release to work on extensions than it is for Simon to hold it up to
work on sync rep. Jeff Davis would have loved to get range types in,
Alvaro would have liked to do finish his work on foreign key locks,
and there are other examples as well. We can't expect anyone to be
willing to step aside graciously when time has expired unless everyone
is willing to do it.
By the way, I don't question Tom's decision to leave this patch until
February. He worked on other things. Who am I to say that they were
any less valuable than this? But you can't have your cake and eat it
too. There's a part of me that wonders whether we'd get the same
number of features per release if we shortened the release cycle by
one CommitFest. Most large patches get done in 2 or 3 CommitFests, so
it would still be entirely reasonable to get a major feature done in
one release. Right now we seem to have two classes of people: the
ones who start working at the beginning of the cycle, and are done by
CF#2/#3, and the ones who start at the end of the cycle, and hold
things up at the end. Neither group would get any less done on a
shorter cycle. Contrariwise I bet if we went to 5 CommitFests we'd
see only a slight increase in patches - they'd just be spread out over
more calendar time. It's already the case that the middle two
CommitFests are smaller than the first and last.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Tom Lane <tgl@sss.pgh.pa.us> writes:
On the whole I'd rather spend time trying to fix the real problem, which
is allowing CREATE EXTENSION to non-superusers. The general case of
that looks quite hard to me, but maybe we could get something that works
for the single case of an extension containing just a language.
Then, what about a control file property to cover that?
pl_language = plpgsql
Then when running the script any object attached to the extension that
is not a 'pg_catalog.pg_language'::regclass is an ERROR. And only when
the pl_language property is used then the superuser-only check is
bypassed.
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
We've been talking about PLs, but what about the other thing David asked:
could we have extension entries for compile-time options like SSL or libxml,
so that you could define a dependency on them?
Then those should be marked "System" and only get displayed with \dxS,
or this will completely bloat the extension listings. Also if we get
there, what about listing all the SQL Standard Features (optional only
maybe) that are provided by the server?
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
On Mar 3, 2011, at 9:47 AM, Dimitri Fontaine wrote:
Then, what about a control file property to cover that?
pl_language = plpgsql
Then when running the script any object attached to the extension that
is not a 'pg_catalog.pg_language'::regclass is an ERROR. And only when
the pl_language property is used then the superuser-only check is
bypassed.
More simply, I think there are two kinds of dependencies:
* Other extensions
* Core features
Notwithstanding that PLs might be extensions, now or in the future, the necessity to require other stuff from core, like libxml support or SSL, together with your example, leads me to think that we ought to think about having two ways of specifying dependencies: requires and core_requires. The latter might look something like:
core_requires = plpgsql libxml
The downside of course is that then there would need to be a second infrastructure for tracking core dependencies, and it would need to be kept up-to-date. But I think something like this will be essential -- even if it only supports core PLs for now.
Best,
David
On Mar 3, 2011, at 9:55 AM, Dimitri Fontaine wrote:
Then those should be marked "System" and only get displayed with \dxS,
or this will completely bloat the extension listings. Also if we get
there, what about listing all the SQL Standard Features (optional only
maybe) that are provided by the server?
I think server version is sufficient for this. If we go with my previous proposal, I might add something like this to the explanation extension I released on PGXN last week:
core_requires = 9.0, plpgsql, libxml
Best,
David
"David E. Wheeler" <david@kineticode.com> writes:
core_requires = 9.0, plpgsql, libxml
As soon as you get there you need an or operator to be able to say 9.0 |
9.1, or maybe some comparison operators to say >= 9.0. Remember that
about all extensions we have are source-compatible with many different
releases of PostgreSQL.
So having the PostgreSQL server itself as an extension so that you can
require it is 9.2 material at best in my opinion.
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
