Dumping an Extension's Script
Hi,
Please find attached to this email an RFC patch implementing the basics
of the pg_dump --extension-script option. After much discussion around
the concept of an inline extension, we decided last year that a good
first step would be pg_dump support for an extension's script.
The approach I've been using here is to dump the script from the catalog
current dependencies, which mean that a sequence of CREATE EXTENSION
followed by a number of ALTER EXTENSION … UPDATE … will be consolidated
into a single CREATE EXTENSION command in the dump, much the same as
with CREATE TABLE then ALTER TABLE … ADD COLUMN and the like.
Currently the option behavior is the following, that looks sane to me,
and is open for discussion: the dump's schema always include the CREATE
EXTENSION commands you need. The extensions listed in the -X option
(that you can use more than once) will get dumped with their's current
member objects in a script, inline.
To try the attached patch, you could do as following:
createdb foo
psql -c "create extension hstore" -d foo
pg_dump -X hstore -f /tmp/foo.sql foo
createdb bar
psql -1 -f /tmp/foo.sql -d bar
To be able to restore the dump, I've been adding some basic support to
the CREATE EXTENSION command so that it will find the data it needs from
the SQL command rather than the control file.
Note that the extension control file only contains information about how
to install an extension from a script file on disk. That's something we
don't need at all when installing the extension from a dump, using
either pg_restore or psql. We have some exceptions to that principle,
namely: requires (sets the search_path) and relocatable (found in the
catalogs, needs to survive dump/restore).
Given positive feedback on that way to attack the problem, the TODO list
includes:
- document the new pg_dump --extension-script switch
- add support for ALTER EXTENSION … WITH $$ <script here> $$;
The ALTER EXTENSION support is optional as far as pg_dump support goes,
it would be good to have it to make the User Interface complete.
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
Attachments:
dump_extension_script.patchtext/x-patchDownload+373-159
On Mon, Nov 12, 2012 at 11:00 AM, Dimitri Fontaine
<dimitri@2ndquadrant.fr> wrote:
Please find attached to this email an RFC patch implementing the basics
of the pg_dump --extension-script option. After much discussion around
the concept of an inline extension, we decided last year that a good
first step would be pg_dump support for an extension's script.
Do you have a link to the original thread? I have to confess I don't
remember what the purpose of this was and, heh heh, there are no
documentation changes in the patch itself either.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes:
Please find attached to this email an RFC patch implementing the basics
of the pg_dump --extension-script option. After much discussion around
the concept of an inline extension, we decided last year that a good
first step would be pg_dump support for an extension's script.Do you have a link to the original thread? I have to confess I don't
remember what the purpose of this was and, heh heh, there are no
documentation changes in the patch itself either.
My notes include those links to the original thread:
http://archives.postgresql.org/message-id/3157.1327298440@sss.pgh.pa.us
http://archives.postgresql.org/pgsql-hackers/2012-01/msg01311.php
https://commitfest.postgresql.org/action/patch_view?id=746
I could of course work on documenting the changes prior to the
reviewing, the thing is that I've been taking a different implementation
route towards the pg_dump --extension-script idea we talked about, that
I think is much simpler than anything else.
So I'd like to know if that approach is deemed acceptable by the
Guardians Of The Code before expanding any more hour on this…
It basically boils down to this hunk in dumpExtension():
output CREATE EXTENSION x WITH … AS $x$
/*
* Have another archive for this extension: this allows us to simply
* walk the extension's dependencies and use the existing pg_dump code
* to get the object create statement to be added in the script.
*
*/
eout = CreateArchive(NULL, archNull, 0, archModeAppend);
EH = (ArchiveHandle *) eout;
/* grab existing connection and remote version information */
EH->connection = ((ArchiveHandle *)fout)->connection;
eout->remoteVersion = fout->remoteVersion;
/* dump all objects for this extension, that have been sorted out in
* the right order following dependencies etc */
...
/* restore the eout Archive into the local buffer */
for (te = EH->toc->next; te != EH->toc; te = te->next)
{
if (strlen(te->defn) > 0)
appendPQExpBuffer(q, "%s", te->defn);
}
CloseArchive(eout);
output $x$;
What do you think?
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
On Thu, Nov 15, 2012 at 3:09 PM, Dimitri Fontaine
<dimitri@2ndquadrant.fr> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
Please find attached to this email an RFC patch implementing the basics
of the pg_dump --extension-script option. After much discussion around
the concept of an inline extension, we decided last year that a good
first step would be pg_dump support for an extension's script.Do you have a link to the original thread? I have to confess I don't
remember what the purpose of this was and, heh heh, there are no
documentation changes in the patch itself either.My notes include those links to the original thread:
http://archives.postgresql.org/message-id/3157.1327298440@sss.pgh.pa.us
http://archives.postgresql.org/pgsql-hackers/2012-01/msg01311.php
https://commitfest.postgresql.org/action/patch_view?id=746I could of course work on documenting the changes prior to the
reviewing, the thing is that I've been taking a different implementation
route towards the pg_dump --extension-script idea we talked about, that
I think is much simpler than anything else.So I'd like to know if that approach is deemed acceptable by the
Guardians Of The Code before expanding any more hour on this…It basically boils down to this hunk in dumpExtension():
output CREATE EXTENSION x WITH … AS $x$
/*
* Have another archive for this extension: this allows us to simply
* walk the extension's dependencies and use the existing pg_dump code
* to get the object create statement to be added in the script.
*
*/
eout = CreateArchive(NULL, archNull, 0, archModeAppend);EH = (ArchiveHandle *) eout;
/* grab existing connection and remote version information */
EH->connection = ((ArchiveHandle *)fout)->connection;
eout->remoteVersion = fout->remoteVersion;/* dump all objects for this extension, that have been sorted out in
* the right order following dependencies etc */
.../* restore the eout Archive into the local buffer */
for (te = EH->toc->next; te != EH->toc; te = te->next)
{
if (strlen(te->defn) > 0)
appendPQExpBuffer(q, "%s", te->defn);
}
CloseArchive(eout);output $x$;
What do you think?
That approach seems likely to break things for the hoped-for parallel
pg_dump feature, though I'm not sure exactly in what way.
Beyond that, I think much of the appeal of the extension feature is
that it dumps as "CREATE EXTENSION hstore;" and nothing more. That
allows you to migrate a dump between systems with different but
compatible versions of the hstore and have things work as intended.
I'm not opposed to the idea of being able to make extensions without
files on disk work ... but I consider it a niche use case; the
behavior we have right now works well for me and hopefully for others
most of the time.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes:
That approach seems likely to break things for the hoped-for parallel
pg_dump feature, though I'm not sure exactly in what way.
Will the parallel dump solve the dependencies and extension membership
properties in parallel too?
Beyond that, I think much of the appeal of the extension feature is
that it dumps as "CREATE EXTENSION hstore;" and nothing more. That
allows you to migrate a dump between systems with different but
compatible versions of the hstore and have things work as intended.
Yes. That's the only use case supported so far. The contrib/ use case.
I'm not opposed to the idea of being able to make extensions without
files on disk work ... but I consider it a niche use case; the
behavior we have right now works well for me and hopefully for others
most of the time.
I hear you. I'm not doing that on my free time, it's not a hobby, I have
customers that want it bad enough to be willing to sponsor my work here.
I hope that helps you figuring about the use case being a niche or not.
The current extension support has been targeted at a single use case,
because that's how you bootstrap that kind of feature. We have request
for extensions that will not include a part written in C.
We've been around the topic last year, we spent much energy trying to
come up with something easy enough to accept as a first step in that
direction, and the conclusion at the time was that we want to be able to
dump an extension's script. That's what my current patch is all about.
More about use cases. Consider PL/Proxy. By the way, that should really
really get in core and be called a FOREIGN FUNCTION, but let's get back
on topic. So I have customers with between 8 and 256 plproxy partitions,
that means each database upgrade has to reach that many databases.
Now, I've built a automatic system that will fetch the PL function code
from the staging database area, put them into files depending on the
schema they live in, package those files into a single one that can be
used by the CREATE EXTENSION command, automatically create an upgrade
file to be able to ALTER EXTENSION … TO VERSION …, and create a bunch of
debian packages out of that (a single debian source package that will
build as many binary packages as we have extensions).
Then, the system will push those packages to an internal repository, run
apt-get update on all the database hosts, then connect to each partition
and run the upgrade command.
All of that could get simplified to getting the PL code into a single
SQL command then running it on all the members of the cluster by using a
plproxy RUN ON ALL command, now that it's a self-contained single SQL
command.
Of course that's only one use case, but that email is already only too
long for what it does: rehashing a story we already ran last year.
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
Robert Haas <robertmhaas@gmail.com> writes:
I'm not opposed to the idea of being able to make extensions without
files on disk work ... but I consider it a niche use case; the
behavior we have right now works well for me and hopefully for others
most of the time.
Apparently I'm not the only one doing extensions without anything to
compile, all SQL:
http://keithf4.com/extension_tips_3
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
On 19 November 2012 16:25, Robert Haas <robertmhaas@gmail.com> wrote:
Beyond that, I think much of the appeal of the extension feature is
that it dumps as "CREATE EXTENSION hstore;" and nothing more. That
allows you to migrate a dump between systems with different but
compatible versions of the hstore and have things work as intended.
I'm not opposed to the idea of being able to make extensions without
files on disk work ... but I consider it a niche use case; the
behavior we have right now works well for me and hopefully for others
most of the time.
Distributing software should only happen by files?
So why does Stackbuilder exist on the Windows binary?
Why does yum exist? What's wrong with ftp huh?
Why does CPAN?
I've a feeling this case might be a sensible way forwards, not a niche at all.
--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On 20.11.2012 21:25, Simon Riggs wrote:
On 19 November 2012 16:25, Robert Haas<robertmhaas@gmail.com> wrote:
Beyond that, I think much of the appeal of the extension feature is
that it dumps as "CREATE EXTENSION hstore;" and nothing more. That
allows you to migrate a dump between systems with different but
compatible versions of the hstore and have things work as intended.
I'm not opposed to the idea of being able to make extensions without
files on disk work ... but I consider it a niche use case; the
behavior we have right now works well for me and hopefully for others
most of the time.Distributing software should only happen by files?
So why does Stackbuilder exist on the Windows binary?
Why does yum exist? What's wrong with ftp huh?
Why does CPAN?
I've a feeling this case might be a sensible way forwards, not a niche at all.
I have to join Robert in scratching my head over this. I don't
understand what the use case is. Can you explain? I don't understand the
comparison with stackbuilder, yum, ftp and CPAN. CPAN seems close to
pgxn, but what does that have to do with this patch?
On 20.11.2012 11:08, Dimitri Fontaine wrote:
Apparently I'm not the only one doing extensions without anything to
compile, all SQL:
No doubt about that. I'm sure extensions written in pure SQL or PL/pgSQL
are very common. But what does that have to do with this patch?
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Heikki Linnakangas <hlinnakangas@vmware.com> writes:
No doubt about that. I'm sure extensions written in pure SQL or PL/pgSQL are
very common. But what does that have to do with this patch?
This patch is all about enabling users to create extension without
having to ship them as root on the file system of the database(s)
server(s) first.
When you're having to code your extension in C, you know you're in for
shipping an executable binary (.so, .dylib or .dll), and for security
reasons it's well understood that you will have to get root privileges
on the server's file system to ship your binary before to be able to ask
PostgreSQL to please load it and execute the code in there.
When you can code your extension using dynamic code such as SQL or
PL/pgSQL, PL/pythonu or PL/perl, there's absolutely no good reason to
have to do the "ship on the server's file system first" that I can see.
Technically creating an extension "inline" (sending its definition in
the CREATE EXTENSION query itself) solves the problem of having to
access the server's file system as root.
Then, next pg_dump will include "CREATE EXTENSION foo;" as usual and at
pg_restore time that access files on the file systems. But maybe you
still are not granted access to the server's file system as root on the
pg_restore target, right? So now you need to be able to include the
extension's script into the dump.
Now, we don't want to have more than one kind of extensions. That's what
we learnt all together when reviewing my proposal from last year. Having
more than one way to ship an extension is good, having two different
animals with two different incompatible behaviors named the same thing
is bad. The solution we found is then to be able to include an
extension's script into pg_dump's output, and that's what my current
patch implements, per last year review.
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 05.12.2012 12:22, Dimitri Fontaine wrote:
Heikki Linnakangas<hlinnakangas@vmware.com> writes:
No doubt about that. I'm sure extensions written in pure SQL or PL/pgSQL are
very common. But what does that have to do with this patch?This patch is all about enabling users to create extension without
having to ship them as root on the file system of the database(s)
server(s) first.
...
When you can code your extension using dynamic code such as SQL or
PL/pgSQL, PL/pythonu or PL/perl, there's absolutely no good reason to
have to do the "ship on the server's file system first" that I can see.Technically creating an extension "inline" (sending its definition in
the CREATE EXTENSION query itself) solves the problem of having to
access the server's file system as root.
Ok, I'm with you this far.
Then, next pg_dump will include "CREATE EXTENSION foo;" as usual and at
pg_restore time that access files on the file systems. But maybe you
still are not granted access to the server's file system as root on the
pg_restore target, right? So now you need to be able to include the
extension's script into the dump.
Now you lost me. I can see the need to install an extension without
access to the filesystem - but it does not follow that you need to be
able to dump an extension script. In general, I think you're confusing
three things:
1. The way an extension is deployed. It could be by copying the files to
the file system, by sending them over libpq, or shipped in .rpms by the
OS, or something else.
2. The way an extension's files are laid out before it's deployed.
Typically, you want to keep an extension's source code (whether it's C
or SQL or plpython) in a version control system.
3. Being able to deploy extensions to the server without superuser or
root access
I think it would make this discussion a lot clearer if we keep those
concerns separate. It's useful to have a mechanism to deploy an
extension over libpq. It's not clear to me if you're envisioning to
change 2. I don't think we should; having a .sql file and a .control
file seems perfectly fine to me.
I'd suggest that we just need a way to upload an extension to the server
via libpq. Something like "UPLOAD EXTENSION foo", which goes into COPY
mode and you can stream over a zip file containing the .sql and .control
file that make up the extension. The server would unzip the file into
the right directory.
Now, point 3 is yet another issue. If you need to copy the extension
files to /usr/share/, you need root (or similar) access on the
filesystem. We could allow extensions to be located somewhere in the
data directory instead. Like $PGDATA/extensions. But again, that would
be an independent change from 1 and 2.
And I still don't understand why pg_dump needs to know about any of this...
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2012-12-05 19:13:10 +0200, Heikki Linnakangas wrote:
On 05.12.2012 12:22, Dimitri Fontaine wrote:
Heikki Linnakangas<hlinnakangas@vmware.com> writes:
No doubt about that. I'm sure extensions written in pure SQL or PL/pgSQL are
very common. But what does that have to do with this patch?This patch is all about enabling users to create extension without
having to ship them as root on the file system of the database(s)
server(s) first.
...
When you can code your extension using dynamic code such as SQL or
PL/pgSQL, PL/pythonu or PL/perl, there's absolutely no good reason to
have to do the "ship on the server's file system first" that I can see.Technically creating an extension "inline" (sending its definition in
the CREATE EXTENSION query itself) solves the problem of having to
access the server's file system as root.Ok, I'm with you this far.
Then, next pg_dump will include "CREATE EXTENSION foo;" as usual and at
pg_restore time that access files on the file systems. But maybe you
still are not granted access to the server's file system as root on the
pg_restore target, right? So now you need to be able to include the
extension's script into the dump.Now you lost me. I can see the need to install an extension without access
to the filesystem - but it does not follow that you need to be able to dump
an extension script. In general, I think you're confusing three things:1. The way an extension is deployed. It could be by copying the files to the
file system, by sending them over libpq, or shipped in .rpms by the OS, or
something else.2. The way an extension's files are laid out before it's deployed.
Typically, you want to keep an extension's source code (whether it's C or
SQL or plpython) in a version control system.3. Being able to deploy extensions to the server without superuser or root
accessI think it would make this discussion a lot clearer if we keep those
concerns separate. It's useful to have a mechanism to deploy an extension
over libpq. It's not clear to me if you're envisioning to change 2. I don't
think we should; having a .sql file and a .control file seems perfectly fine
to me.I'd suggest that we just need a way to upload an extension to the server via
libpq. Something like "UPLOAD EXTENSION foo", which goes into COPY mode and
you can stream over a zip file containing the .sql and .control file that
make up the extension. The server would unzip the file into the right
directory.
Not sure what is better here. Dimitri's way seems to be easier to manage
for people who maintain their database in update scripts and such and
your's seems to be a bit simpler from the backend perspective.
Now, point 3 is yet another issue. If you need to copy the extension files
to /usr/share/, you need root (or similar) access on the filesystem. We
could allow extensions to be located somewhere in the data directory
instead. Like $PGDATA/extensions. But again, that would be an independent
change from 1 and 2.
I think installing them into some global space is not a sensible
interim-step. Having a UPLOAD EXTENSION in one database affect all other
databases or even clusters (because you e.g. updated the version) would
be really confusing.
Which leads to:
And I still don't understand why pg_dump needs to know about any of this...
Extensions should be fully per-database and we want pg_dump backups to
be restorable into another database/clusters/servers. So having a mode
for pg_dump that actually makes dumps that are usable for recovering
after a disaster seems sensible to me. Otherwise you need to redeploy
from the VCS or whatever, which isn't really what you want when
restoring a database backup.
Comparing the situation to the one where you have extensions provided by
the packaging system or by /contrib or whatever doesn't seem to be all
that valid to me.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 05.12.2012 19:27, Andres Freund wrote:
And I still don't understand why pg_dump needs to know about any of this...
Extensions should be fully per-database and we want pg_dump backups to
be restorable into another database/clusters/servers. So having a mode
for pg_dump that actually makes dumps that are usable for recovering
after a disaster seems sensible to me. Otherwise you need to redeploy
from the VCS or whatever, which isn't really what you want when
restoring a database backup.
Ok - but that it yet another issue, not to be confused with how you
deploy extensions. If we are to have such a mode in pg_dump, it should
be able to dump *all* extensions, regardless of how they were deployed.
(ok, might be difficult for extensions that include .so files or
similar, but certainly for an extension that only contains a .sql file
and a .control file, it shouldn't matter how it was deployed).
And whether extension control files (or the same information stored in a
table or wherever) should be per-database or per cluster - that's *yet*
another separate issue. You could argue for either behavior.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2012-12-05 19:40:58 +0200, Heikki Linnakangas wrote:
On 05.12.2012 19:27, Andres Freund wrote:
And I still don't understand why pg_dump needs to know about any of this...
Extensions should be fully per-database and we want pg_dump backups to
be restorable into another database/clusters/servers. So having a mode
for pg_dump that actually makes dumps that are usable for recovering
after a disaster seems sensible to me. Otherwise you need to redeploy
from the VCS or whatever, which isn't really what you want when
restoring a database backup.Ok - but that it yet another issue, not to be confused with how you deploy
extensions. If we are to have such a mode in pg_dump, it should be able to
dump *all* extensions, regardless of how they were deployed. (ok, might be
difficult for extensions that include .so files or similar, but certainly
for an extension that only contains a .sql file and a .control file, it
shouldn't matter how it was deployed).
For me it seems pretty natural to support dumping extension the way they
got created. I.e. a plain CREATE EXTENSION ...; if the extension was
preinstalled and some form that includes the extension source if you
installed it via the connection.
Extensions that were installed in some global manner *should* not be
dumped with ones installed over the connection. E.g. dumping /contrib or
packaged modules seems to be a bad idea to me.
That would possibly be useful as a debugging tool, but I don't see much
point besides that.
And whether extension control files (or the same information stored in a
table or wherever) should be per-database or per cluster - that's *yet*
another separate issue. You could argue for either behavior.
What would be the case for the per-cluster in the case of uploaded
extensions?
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Heikki Linnakangas <hlinnakangas@vmware.com> writes:
Ok - but that it yet another issue, not to be confused with how you deploy
extensions. If we are to have such a mode in pg_dump, it should be able to
dump *all* extensions, regardless of how they were deployed. (ok, might be
difficult for extensions that include .so files or similar, but certainly
for an extension that only contains a .sql file and a .control file, it
shouldn't matter how it was deployed).
That's what you have in the current patch. Try
=> create extension 'hstore';
$ pg_dump --extension-script hstore
It works as far as the script is concerned, and the control file is not
needed any more because the script as dumped does not need it, except
for the two parameters 'require' and 'relocatable', that are added in
the SQL command.
The binary file is not taken care of by this mechanism. Remember that in
most cases pg_restore will not be granted to deploy it at the right
place anyway, for security reasons.
And whether extension control files (or the same information stored in a
table or wherever) should be per-database or per cluster - that's *yet*
another separate issue. You could argue for either behavior.
At the SQL level, extensions do live in a database. The only reason why
we currently have them on the file system is binary executables (.so,
.dylib, .dll). And those are not per database, not even per cluster, not
even per major version, they are *per server*. It's something that makes
me very sad, and that I want to have the chance to fix later, but that
won't happen in 9.3, and certainly not in that very patch…
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Andres Freund <andres@2ndquadrant.com> writes:
On 2012-12-05 19:13:10 +0200, Heikki Linnakangas wrote:
And I still don't understand why pg_dump needs to know about any of this...
Extensions should be fully per-database and we want pg_dump backups to
be restorable into another database/clusters/servers.
Wait a minute. I haven't bought into either of those statements, and
most particularly not the first one.
Upthread, Dimitri claimed that he wasn't creating two different kinds of
extensions with this patch, but the more I read about it the more it
seems that he *is* making a fundamentally different kind of animal.
And I don't think it's necessarily a good idea, especially not if we
still call it an extension.
I kind of like Heikki's idea of leaving CREATE EXTENSION alone and
inventing a separate "UPLOAD EXTENSION" operation, but there's a problem
with that: in many, probably most, installations, the server does not
and should not have permission to scribble on the directories where the
extension scripts are stored. Possibly we could circumvent that by
creating an auxiliary extensions directory under $PGDATA. (But then
it starts to seem like pg_dumpall --- not pg_dump --- ought to include
those files in its output...)
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Heikki Linnakangas <hlinnakangas@vmware.com> writes:
And whether extension control files (or the same information stored in a
table or wherever) should be per-database or per cluster - that's *yet*
another separate issue. You could argue for either behavior.
I think anyone arguing for the former is confusing an installed
extension with a not-installed one. Maybe it would help if we adopted
different terminologies. Perhaps call the control+sql files a "template",
while using "extension" for the installed entity?
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2012-12-05 12:55:42 -0500, Tom Lane wrote:
Andres Freund <andres@2ndquadrant.com> writes:
On 2012-12-05 19:13:10 +0200, Heikki Linnakangas wrote:
And I still don't understand why pg_dump needs to know about any of this...
Extensions should be fully per-database and we want pg_dump backups to
be restorable into another database/clusters/servers.Wait a minute. I haven't bought into either of those statements, and
most particularly not the first one.
Ok.
Upthread, Dimitri claimed that he wasn't creating two different kinds of
extensions with this patch, but the more I read about it the more it
seems that he *is* making a fundamentally different kind of animal.
And I don't think it's necessarily a good idea, especially not if we
still call it an extension.
I have to admit I haven't read the whole discussion about this. And I
also have to say that I have no idea yet whether I like the current
implementation because I haven't looked at it yet. I just wanted to give
input to the separate problems Heikki listed. Because I wished for
something roughly like this for years...
To me it seems to be sensible that extensions which are preinstalled on
the system are global and extensions which a single user inside a single
database created are per database.
Imo that doesn't make them all that fundamentally different.
I kind of like Heikki's idea of leaving CREATE EXTENSION alone and
inventing a separate "UPLOAD EXTENSION" operation, but there's a problem
with that: in many, probably most, installations, the server does not
and should not have permission to scribble on the directories where the
extension scripts are stored. Possibly we could circumvent that by
creating an auxiliary extensions directory under $PGDATA. (But then
it starts to seem like pg_dumpall --- not pg_dump --- ought to include
those files in its output...)
UPLOAD EXTENSION seems to be a good idea.
But I really really would like them to go to a per-database directory
not a per-cluster one. Otherwise the coordination between different
database "owners" inside a cluster will get really hairy. I want to be
able to install different versions of an application into different
databases.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 05.12.2012 20:07, Tom Lane wrote:
Heikki Linnakangas<hlinnakangas@vmware.com> writes:
And whether extension control files (or the same information stored in a
table or wherever) should be per-database or per cluster - that's *yet*
another separate issue. You could argue for either behavior.I think anyone arguing for the former is confusing an installed
extension with a not-installed one. Maybe it would help if we adopted
different terminologies. Perhaps call the control+sql files a "template",
while using "extension" for the installed entity?
+1 on the naming.
You could still argue that templates should be per-database. It would
make life easier for someone who is database owner but not superuser,
for example, allowing you to install an extension that only affects your
own database (assuming we set up the permissions so that that's
possible, of course).
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Dimitri Fontaine <dimitri@2ndQuadrant.fr> writes:
At the SQL level, extensions do live in a database. The only reason why
we currently have them on the file system is binary executables (.so,
.dylib, .dll). And those are not per database, not even per cluster, not
even per major version, they are *per server*. It's something that makes
me very sad, and that I want to have the chance to fix later, but that
won't happen in 9.3, and certainly not in that very patch…
I think you're wasting your time to imagine that that case will ever be
"fixed". Allowing the server to scribble on executable files would set
off all kinds of security alarm bells, and rightly so. If Postgres ever
did ship with such a thing, I rather imagine that I'd be required to
patch it out of Red Hat releases (not that SELinux wouldn't prevent
it from happening anyway).
I do see an argument for allowing SQL-only extensions to be installed
this way, since that doesn't allow the execution of anything the user
couldn't execute anyway. There's no need to worry about anything except
control and script files though.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 05.12.2012 20:13, Andres Freund wrote:
But I really really would like them to go to a per-database directory
not a per-cluster one. Otherwise the coordination between different
database "owners" inside a cluster will get really hairy. I want to be
able to install different versions of an application into different
databases.
Extension authors should be careful to maintain backwards-compatibility,
so that it would be enough to have the latest version installed. If you
break compatibility, you probably should rename the extension.
That said, I can understand that in practice you'd want to have
different versions installed at the same time, so that you don't need to
re-test everything when upgrading an extension, and don't need to trust
that the extension author didn't accidentally break
backwards-compatibility anyway.
If you really meant "different versions of an application", and not
"different versions of an extension", then it seems to me that you're
abusing the extension infrastructure for something else. If you have
some functions that you consider part of the application, even if those
functions might be useful in other applications too, you probably don't
want to treat them as an extension.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
