[PATCH] Store Extension Options

On Tue, Dec 31, 2013 at 9:08 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

Hello

I am looking on this patch

ALTER TABLE foo SET (ext.somext.do_replicate=true);

Why is there fixed prefix "ext" ?

This feature is similar to attaching setting to function

CREATE OR REPLACE FUNCTION ... SET var = ...;

We can use someprefix.someguc without problems there.

Hi,

We use the prefix "ext" (aka namespace) to distinguish these options which
are related to "extensions".

Have you seen the previous thread [1]/messages/by-id/CAFcNs+rqCq1H5eXW-cvdti6T-xo2STEkhjREx=OdmAkK5tiOOw@mail.gmail.com ?

Regards,

[1]: /messages/by-id/CAFcNs+rqCq1H5eXW-cvdti6T-xo2STEkhjREx=OdmAkK5tiOOw@mail.gmail.com
/messages/by-id/CAFcNs+rqCq1H5eXW-cvdti6T-xo2STEkhjREx=OdmAkK5tiOOw@mail.gmail.com

--
Fabrízio de Royes Mello
Consultoria/Coaching PostgreSQL

2013/12/31 Fabrízio de Royes Mello <fabriziomello@gmail.com>

On Tue, Dec 31, 2013 at 9:08 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

Hello

I am looking on this patch

ALTER TABLE foo SET (ext.somext.do_replicate=true);

Why is there fixed prefix "ext" ?

This feature is similar to attaching setting to function

CREATE OR REPLACE FUNCTION ... SET var = ...;

We can use someprefix.someguc without problems there.

Hi,

We use the prefix "ext" (aka namespace) to distinguish these options which
are related to "extensions".

Have you seen the previous thread [1] ?

yes, but I don't understand why it is necessary? I use a analogy with
custom GUC - and there we don't use similar prefix. Only any prefix is
required - and it can contain a dot.

Regards

Pavel

On Tue, Dec 31, 2013 at 9:38 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

2013/12/31 Fabrízio de Royes Mello <fabriziomello@gmail.com>

On Tue, Dec 31, 2013 at 9:08 AM, Pavel Stehule <pavel.stehule@gmail.com>

wrote:

Hello

I am looking on this patch

ALTER TABLE foo SET (ext.somext.do_replicate=true);

Why is there fixed prefix "ext" ?

This feature is similar to attaching setting to function

CREATE OR REPLACE FUNCTION ... SET var = ...;

We can use someprefix.someguc without problems there.

Hi,

We use the prefix "ext" (aka namespace) to distinguish these options

which are related to "extensions".

Have you seen the previous thread [1] ?

yes, but I don't understand why it is necessary? I use a analogy with

custom GUC - and there we don't use similar prefix. Only any prefix is
required - and it can contain a dot.

We use the namespace "ext" to the internal code
(src/backend/access/common/reloptions.c) skip some validations and store
the custom GUC.

Do you think we don't need to use the "ext" namespace?

Regards,

--
Fabrízio de Royes Mello
Consultoria/Coaching PostgreSQL

2013/12/31 Fabrízio de Royes Mello <fabriziomello@gmail.com>

On Tue, Dec 31, 2013 at 9:38 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

2013/12/31 Fabrízio de Royes Mello <fabriziomello@gmail.com>

On Tue, Dec 31, 2013 at 9:08 AM, Pavel Stehule <pavel.stehule@gmail.com>

wrote:

Hello

I am looking on this patch

ALTER TABLE foo SET (ext.somext.do_replicate=true);

Why is there fixed prefix "ext" ?

This feature is similar to attaching setting to function

CREATE OR REPLACE FUNCTION ... SET var = ...;

We can use someprefix.someguc without problems there.

Hi,

We use the prefix "ext" (aka namespace) to distinguish these options

which are related to "extensions".

Have you seen the previous thread [1] ?

yes, but I don't understand why it is necessary? I use a analogy with

custom GUC - and there we don't use similar prefix. Only any prefix is
required - and it can contain a dot.

We use the namespace "ext" to the internal code
(src/backend/access/common/reloptions.c) skip some validations and store
the custom GUC.

Do you think we don't need to use the "ext" namespace?

yes - there be same mechanism as we use for GUC

Pavel

On Tue, Dec 31, 2013 at 10:37 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

2013/12/31 Fabrízio de Royes Mello <fabriziomello@gmail.com>

On Tue, Dec 31, 2013 at 9:38 AM, Pavel Stehule <pavel.stehule@gmail.com>

wrote:

2013/12/31 Fabrízio de Royes Mello <fabriziomello@gmail.com>

On Tue, Dec 31, 2013 at 9:08 AM, Pavel Stehule <

pavel.stehule@gmail.com> wrote:

Hello

I am looking on this patch

ALTER TABLE foo SET (ext.somext.do_replicate=true);

Why is there fixed prefix "ext" ?

This feature is similar to attaching setting to function

CREATE OR REPLACE FUNCTION ... SET var = ...;

We can use someprefix.someguc without problems there.

Hi,

We use the prefix "ext" (aka namespace) to distinguish these options

which are related to "extensions".

Have you seen the previous thread [1] ?

yes, but I don't understand why it is necessary? I use a analogy with

custom GUC - and there we don't use similar prefix. Only any prefix is
required - and it can contain a dot.

We use the namespace "ext" to the internal code

(src/backend/access/common/reloptions.c) skip some validations and store
the custom GUC.

Do you think we don't need to use the "ext" namespace?

yes - there be same mechanism as we use for GUC

If we going to that way then we can expand the use of this patch to store
custom GUCs to functions also, and we can wrote a function (like
current_setting) to get specific GUC values, like:

ALTER TABLE foo SET (myextension.option=on);

SELECT current_setting('foo'::regclass, 'myextension.option');

Comments?

--
Fabrízio de Royes Mello
Consultoria/Coaching PostgreSQL

2013/12/31 Fabrízio de Royes Mello <fabriziomello@gmail.com>

On Tue, Dec 31, 2013 at 10:37 AM, Pavel Stehule <pavel.stehule@gmail.com>
wrote:

2013/12/31 Fabrízio de Royes Mello <fabriziomello@gmail.com>

On Tue, Dec 31, 2013 at 9:38 AM, Pavel Stehule <pavel.stehule@gmail.com>

wrote:

2013/12/31 Fabrízio de Royes Mello <fabriziomello@gmail.com>

On Tue, Dec 31, 2013 at 9:08 AM, Pavel Stehule <

pavel.stehule@gmail.com> wrote:

Hello

I am looking on this patch

ALTER TABLE foo SET (ext.somext.do_replicate=true);

Why is there fixed prefix "ext" ?

This feature is similar to attaching setting to function

CREATE OR REPLACE FUNCTION ... SET var = ...;

We can use someprefix.someguc without problems there.

Hi,

We use the prefix "ext" (aka namespace) to distinguish these options

which are related to "extensions".

Have you seen the previous thread [1] ?

yes, but I don't understand why it is necessary? I use a analogy with

custom GUC - and there we don't use similar prefix. Only any prefix is
required - and it can contain a dot.

We use the namespace "ext" to the internal code

(src/backend/access/common/reloptions.c) skip some validations and store
the custom GUC.

Do you think we don't need to use the "ext" namespace?

yes - there be same mechanism as we use for GUC

If we going to that way then we can expand the use of this patch to store
custom GUCs to functions also, and we can wrote a function (like
current_setting) to get specific GUC values, like:

ALTER TABLE foo SET (myextension.option=on);

SELECT current_setting('foo'::regclass, 'myextension.option');

I like it

Pavel

On 2013-12-31 13:37:59 +0100, Pavel Stehule wrote:

We use the namespace "ext" to the internal code
(src/backend/access/common/reloptions.c) skip some validations and store
the custom GUC.

Do you think we don't need to use the "ext" namespace?

yes - there be same mechanism as we use for GUC

There is no existing mechanism to handle conflicts for GUCs. The
difference is that for GUCs nearly no "namespaced" GUCs exist (plperl,
plpgsql have some), but postgres defines at least autovacuum. and
toast. namespaces for relation options.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Jan 2, 2014 at 7:19 AM, Andres Freund <andres@2ndquadrant.com>
wrote:

On 2013-12-31 13:37:59 +0100, Pavel Stehule wrote:

We use the namespace "ext" to the internal code
(src/backend/access/common/reloptions.c) skip some validations and

store

the custom GUC.

Do you think we don't need to use the "ext" namespace?

yes - there be same mechanism as we use for GUC

There is no existing mechanism to handle conflicts for GUCs. The
difference is that for GUCs nearly no "namespaced" GUCs exist (plperl,
plpgsql have some), but postgres defines at least autovacuum. and
toast. namespaces for relation options.

autovacuum. namespace ???

The HEAP_RELOPT_NAMESPACES (src/include/access/reloptions.h) constant
define only "toast" and "null" as a valid relation option namespace.

I missed something?

Regards,

--
Fabrízio de Royes Mello
Consultoria/Coaching PostgreSQL

On 2014-01-02 08:26:20 -0200, Fabrízio de Royes Mello wrote:

On Thu, Jan 2, 2014 at 7:19 AM, Andres Freund <andres@2ndquadrant.com>
wrote:

On 2013-12-31 13:37:59 +0100, Pavel Stehule wrote:

We use the namespace "ext" to the internal code
(src/backend/access/common/reloptions.c) skip some validations and

store

the custom GUC.

Do you think we don't need to use the "ext" namespace?

yes - there be same mechanism as we use for GUC

There is no existing mechanism to handle conflicts for GUCs. The
difference is that for GUCs nearly no "namespaced" GUCs exist (plperl,
plpgsql have some), but postgres defines at least autovacuum. and
toast. namespaces for relation options.

autovacuum. namespace ???

Yea, right, it's autovacuum_...

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Thu, Jan 2, 2014 at 4:19 AM, Andres Freund <andres@2ndquadrant.com> wrote:

On 2013-12-31 13:37:59 +0100, Pavel Stehule wrote:

We use the namespace "ext" to the internal code
(src/backend/access/common/reloptions.c) skip some validations and store
the custom GUC.

Do you think we don't need to use the "ext" namespace?

yes - there be same mechanism as we use for GUC

There is no existing mechanism to handle conflicts for GUCs. The
difference is that for GUCs nearly no "namespaced" GUCs exist (plperl,
plpgsql have some), but postgres defines at least autovacuum. and
toast. namespaces for relation options.

I continue to think that the case for having this feature at all has
not been well-made.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Enviado via iPhone

Em 02/01/2014, às 22:16, Robert Haas <robertmhaas@gmail.com> escreveu:

On Thu, Jan 2, 2014 at 4:19 AM, Andres Freund <andres@2ndquadrant.com> wrote:
On 2013-12-31 13:37:59 +0100, Pavel Stehule wrote:

We use the namespace "ext" to the internal code
(src/backend/access/common/reloptions.c) skip some validations and store
the custom GUC.

Do you think we don't need to use the "ext" namespace?

yes - there be same mechanism as we use for GUC

There is no existing mechanism to handle conflicts for GUCs. The
difference is that for GUCs nearly no "namespaced" GUCs exist (plperl,
plpgsql have some), but postgres defines at least autovacuum. and
toast. namespaces for relation options.

I continue to think that the case for having this feature at all has
not been well-made.

We can use this feature to store any custom GUC for relations, attributes and functions also.

Some use cases:
* extension options
* config for external apps (frameworks, third part software)

Comments?

Regards,

Fabrízio Mello

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Sat, Jan 4, 2014 at 11:07 AM, Fabrizio Mello <fabriziomello@gmail.com> wrote:

I continue to think that the case for having this feature at all has
not been well-made.

We can use this feature to store any custom GUC for relations, attributes and functions also.

Some use cases:
* extension options
* config for external apps (frameworks, third part software)

Comments?

Well, as I said before, somebody can make their own configuration
table and store stuff there, rather than using pg_class.reloptions.
As I recall, the only response to that proposal was "well, they might
not want to do it that way", which does not strike me as a sufficient
reason.

What we've basically settled into for GUCs is that you can register a
custom GUC, but unless the module is loaded we'll accept any value for
that GUC without checking it. We'd presumably need a similar
mechanism here, or maybe you're proposing that we accept any reloption
at all with any associated value whatsoever, so long as the prefix is
ext. The first seems like an extension of an existing kludge of
which I'm not overly found, and the second is an even larger kludge.
In my experience as a software developer, there are very few places
where it's useful to accept and store user input without any
validation whatsoever, and I doubt that this is one of them.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2014-01-04 11:54:46 -0500, Robert Haas wrote:

On Sat, Jan 4, 2014 at 11:07 AM, Fabrizio Mello <fabriziomello@gmail.com> wrote:

I continue to think that the case for having this feature at all has
not been well-made.

We can use this feature to store any custom GUC for relations, attributes and functions also.

Some use cases:
* extension options
* config for external apps (frameworks, third part software)

Comments?

Well, as I said before, somebody can make their own configuration
table and store stuff there, rather than using pg_class.reloptions.
As I recall, the only response to that proposal was "well, they might
not want to do it that way", which does not strike me as a sufficient
reason.

Well, there's some things you get by that integration:
* Proper dependency tracking when relations are dropped & renamed
* Sensible integration into pg_dump, with only the relevant options
being dumped/restored on partial dump/restores
* Caching of values, with proper cache invalidation

Sure, you can implement both using event triggers and relcache
invalidation callbacks, but that's not something we want several
extensions to do independently.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund <andres@2ndquadrant.com> writes:

On 2014-01-04 11:54:46 -0500, Robert Haas wrote:

Well, as I said before, somebody can make their own configuration
table and store stuff there, rather than using pg_class.reloptions.
As I recall, the only response to that proposal was "well, they might
not want to do it that way", which does not strike me as a sufficient
reason.

Well, there's some things you get by that integration:
* Proper dependency tracking when relations are dropped & renamed
* Sensible integration into pg_dump, with only the relevant options
being dumped/restored on partial dump/restores
* Caching of values, with proper cache invalidation

If you have some settings that need to be table-specific, then
I agree that the reloptions infrastructure is a nice place to track them.
What's actually missing here is some compelling examples of such settings
for plausible extensions. (The original example was pure handwaving, as
I don't believe it's possible to build a "replication extension" with no
core-code changes. As long as you need some of those, patching in a few
more standard reloptions is hardly a deal-breaker.)

Assuming that such examples are forthcoming, though, I think my main
objection to this proposal is the "ext." prefix, which seems precisely
100% useless, not to mention inconsistent with the naming of custom GUCs,
which the same extension might well have some of. I think that custom
reloptions should just have names like "extension_name.option_name", the
same as custom GUCs do. We have enough experience now with custom GUCs
that I don't think it's unreasonable to model the behavior of custom
reloptions on them as closely as possible.

I would suggest addressing Robert's concern about lack of error checking
by refusing to allow a custom reloption to be set unless the relevant
extension is loaded and checks it. Unlike the postgresql.conf problem,
I don't see any very good use-case for allowing an unchecked ALTER TABLE
to occur.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2014-01-04 13:00:03 -0500, Tom Lane wrote:

Andres Freund <andres@2ndquadrant.com> writes:

On 2014-01-04 11:54:46 -0500, Robert Haas wrote:

Well, as I said before, somebody can make their own configuration
table and store stuff there, rather than using pg_class.reloptions.
As I recall, the only response to that proposal was "well, they might
not want to do it that way", which does not strike me as a sufficient
reason.

Well, there's some things you get by that integration:
* Proper dependency tracking when relations are dropped & renamed
* Sensible integration into pg_dump, with only the relevant options
being dumped/restored on partial dump/restores
* Caching of values, with proper cache invalidation

If you have some settings that need to be table-specific, then
I agree that the reloptions infrastructure is a nice place to track them.
What's actually missing here is some compelling examples of such settings
for plausible extensions.

I don't know about others, but I would like to use it for bdr to
configure which table is replicated where. I.e. something like
bdr.replication_set=a,b,c,

(The original example was pure handwaving, as
I don't believe it's possible to build a "replication extension" with no
core-code changes. As long as you need some of those, patching in a few
more standard reloptions is hardly a deal-breaker.)

Well, slony et al exist, so it's certainly possible. And even if you
want to talk about logical replication, there aren't that much core
changes required - and all of them have been submitted. They might not
make it into 9.4 but I certainly plan to pursue things further so it's
possible to stuff without patching core.

Assuming that such examples are forthcoming, though, I think my main
objection to this proposal is the "ext." prefix, which seems precisely
100% useless, not to mention inconsistent with the naming of custom GUCs,
which the same extension might well have some of.

Well, the argument is/was that it avoid conflicts with future core code
adding more namespaces - like the already existing toast. prefix. If we
say we can live with the possibility of such conflicts, it seems
appropriate not to use ext. as a prefix.

I would suggest addressing Robert's concern about lack of error checking
by refusing to allow a custom reloption to be set unless the relevant
extension is loaded and checks it. Unlike the postgresql.conf problem,
I don't see any very good use-case for allowing an unchecked ALTER TABLE

Fine with me.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund <andres@2ndquadrant.com> writes:

On 2014-01-04 13:00:03 -0500, Tom Lane wrote:

Assuming that such examples are forthcoming, though, I think my main
objection to this proposal is the "ext." prefix, which seems precisely
100% useless, not to mention inconsistent with the naming of custom GUCs,
which the same extension might well have some of.

Well, the argument is/was that it avoid conflicts with future core code
adding more namespaces - like the already existing toast. prefix. If we
say we can live with the possibility of such conflicts, it seems
appropriate not to use ext. as a prefix.

And if we have ext. as a prefix, exactly what prevents conflicts in the
second part of the name? Nothing, that's what. It's useless.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2014-01-04 14:06:19 -0500, Tom Lane wrote:

Andres Freund <andres@2ndquadrant.com> writes:

On 2014-01-04 13:00:03 -0500, Tom Lane wrote:

Assuming that such examples are forthcoming, though, I think my main
objection to this proposal is the "ext." prefix, which seems precisely
100% useless, not to mention inconsistent with the naming of custom GUCs,
which the same extension might well have some of.

Well, the argument is/was that it avoid conflicts with future core code
adding more namespaces - like the already existing toast. prefix. If we
say we can live with the possibility of such conflicts, it seems
appropriate not to use ext. as a prefix.

And if we have ext. as a prefix, exactly what prevents conflicts in the
second part of the name? Nothing, that's what. It's useless.

Uh? We are certainly not going to add core code that defines relation
options with ext. in the name like we've introduced toast.fillfactor et
al?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund <andres@2ndquadrant.com> writes:

On 2014-01-04 14:06:19 -0500, Tom Lane wrote:

And if we have ext. as a prefix, exactly what prevents conflicts in the
second part of the name? Nothing, that's what. It's useless.

Uh? We are certainly not going to add core code that defines relation
options with ext. in the name like we've introduced toast.fillfactor et
al?

If this feature is of any use, surely we should assume that more than
one extension will use it. If those extensions are separately developed,
there's nothing preventing name conflicts. I would rank the odds of
two people writing "my_replication_extension" a lot higher than the odds
of the core code deciding to use such a prefix.

What's more, what happens if we decide to migrate some such extension
into core? A hard and fast division between names allowed to external
and internal features is just going to bite us on the rear eventually.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Sat, Jan 4, 2014 at 1:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I would suggest addressing Robert's concern about lack of error checking
by refusing to allow a custom reloption to be set unless the relevant
extension is loaded and checks it. Unlike the postgresql.conf problem,
I don't see any very good use-case for allowing an unchecked ALTER TABLE
to occur.

How do you plan to resolve the associated dump/restore hazard? AIUI,
that's why we allow people define any old this.that GUC that they want
without checking it - because the relevant shared library might not be
loaded at the time of definition, but only by time of use.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers