Why conf.d should be default, and auto.conf and recovery.conf should be in it

Started by Josh Berkusabout 12 years ago36 messageshackers
Jump to latest
#1Josh Berkus
josh@agliodbs.com

Hackers,

ALTER SYSTEM SET has been committed and recovery.conf GUCs are being
reviewed. I'm going to make a last case for conf.d in relation to these
two patches before 9.4 goes out the door.

In 9.3, we added support for a config directory (conf.d), but have it
disabled by default. For tool authors, this makes conf.d useless since
you never know, on any given installation, whether it will be
present/enabled or not. While we don't want to prevent users from
disabling it, conf.d only becomes useful if it's present by default.
There's a simple reason why: if you want to write a tool which manages
postgresql.conf, you don't want the user to have to manually edit
postgresql.conf (and create a directory) in order to enable the tool.

I'm particularly thinking about this in relation to the merger of
recovery.conf and postgresql.conf. There are several tools already
(RepMgr, OminPITR, HandyRep, pgPool, etc.) which manage recovery.conf
separately from postgresql.conf. These tools will want to continue
managing recovery.conf as a separate file, even if it's /included in
postgresql.conf now.

If conf.d exists by default, and is enabled in postgresql.conf by
default, this is easy: the tool just drops a recovery.conf file into
conf.d. Changing file locations and variable names is a fairly simple
exercise in backwards compatibility.

If conf.d does NOT exist by default, things become complicated, and
backwards compatibility for replication management tools becomes much
harder.

Yes, I'm also arguing that postgresql.auto.conf should go into conf.d.
I said I'd bring that up again after ALTER SYSTEM SET was committed, and
here it is.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#2Stephen Frost
sfrost@snowman.net
In reply to: Josh Berkus (#1)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

Josh,

* Josh Berkus (josh@agliodbs.com) wrote:

In 9.3, we added support for a config directory (conf.d), but have it
disabled by default. For tool authors, this makes conf.d useless since
you never know, on any given installation, whether it will be
present/enabled or not. While we don't want to prevent users from
disabling it, conf.d only becomes useful if it's present by default.
There's a simple reason why: if you want to write a tool which manages
postgresql.conf, you don't want the user to have to manually edit
postgresql.conf (and create a directory) in order to enable the tool.

I don't buy this argument one bit- certainly, on Debian, the defaults
are overridden for a number of variables already and could be done to
enable a conf.d directory as well. Directory creation would, of
course, also be able to be handled by the Debian package. Any tool
which is being packaged for Debian would simply have to Depend on the
version of the PG package that enabled the conf.d option by default.

This doesn't deal with upgrade cases or where the user has disabled the
feature and so the tool would need to check for a directory's
existance, but changing our default isn't going to completely address
that issue either. Indeed, the Debian package would at least be able to
indicate, through the existance or not of the directory, whether a given
cluster was set up by default with the conf.d structure or not.

I'm particularly thinking about this in relation to the merger of
recovery.conf and postgresql.conf. There are several tools already
(RepMgr, OminPITR, HandyRep, pgPool, etc.) which manage recovery.conf
separately from postgresql.conf. These tools will want to continue
managing recovery.conf as a separate file, even if it's /included in
postgresql.conf now.

Certainly an interesting thought.

If conf.d exists by default, and is enabled in postgresql.conf by
default, this is easy: the tool just drops a recovery.conf file into
conf.d. Changing file locations and variable names is a fairly simple
exercise in backwards compatibility.

This isn't quite that simple on at least Debian, and I believe RH, as
there's going to be multiple directories involved (one for each cluster
which exists on the system). Also, there are rules which packages need
to follow on Debian regarding conf file changes, which these certainly
would be.

If conf.d does NOT exist by default, things become complicated, and
backwards compatibility for replication management tools becomes much
harder.

My recommendation would be to argue the point with the package
maintainers. Even if we *had* a default, I'm not convinced that the
package maintainers would keep it that way until and unless they update
their scripts and whatnot to handle it.

Yes, I'm also arguing that postgresql.auto.conf should go into conf.d.
I said I'd bring that up again after ALTER SYSTEM SET was committed, and
here it is.

I disagree with enabling that by default.

Thanks,

Stephen

#3Peter Eisentraut
peter_e@gmx.net
In reply to: Josh Berkus (#1)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

On 1/15/14, 1:53 PM, Josh Berkus wrote:

I'm particularly thinking about this in relation to the merger of
recovery.conf and postgresql.conf. There are several tools already
(RepMgr, OminPITR, HandyRep, pgPool, etc.) which manage recovery.conf
separately from postgresql.conf. These tools will want to continue
managing recovery.conf as a separate file, even if it's /included in
postgresql.conf now.

That seems like a very general argument. We'd need to consider exactly
what these tools want to do, and how that affects the recovery.conf
merger. My personal proposal was to allow postgresql.conf to contain
recovery parameters, but read recovery.conf last anyway. That would
solve that problem.

If conf.d exists by default, and is enabled in postgresql.conf by
default, this is easy: the tool just drops a recovery.conf file into
conf.d.

That would just give false hope, I think. My standard operating
procedure is to delete the default postgresql.conf and write a new one
from scratch (or have deployment tools do it). I don't want to
subscribe to a policy that the only endorsed way to maintain
postgresql.conf is to start with the default one and edit around carefully.

The only way you could achieve your goal is to read the configuration
directory automatically, but that would come with its own set of problems.

Yes, I'm also arguing that postgresql.auto.conf should go into conf.d.
I said I'd bring that up again after ALTER SYSTEM SET was committed, and
here it is.

Independent of the above, I don't agree with that. postgresql.auto.conf
is a special thing and should have its own special place. For once
thing, when putting configuration files in a separate directory
structure (/etc/ vs /var), then postgresql.auto.conf should stay in the
data directory.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Josh Berkus
josh@agliodbs.com
In reply to: Josh Berkus (#1)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

On 01/15/2014 11:10 AM, Stephen Frost wrote:

I don't buy this argument one bit- certainly, on Debian, the defaults
are overridden for a number of variables already and could be done to
enable a conf.d directory as well. Directory creation would, of
course, also be able to be handled by the Debian package. Any tool
which is being packaged for Debian would simply have to Depend on the
version of the PG package that enabled the conf.d option by default.

However, Debian is *never* going to add conf.d to the packages if we
don't recommend it as an upstream project. And, frankly, I use the
apt.postgresql.org packages anyway, which would certainly include
anything which was decided on this list.

This doesn't deal with upgrade cases or where the user has disabled the
feature and so the tool would need to check for a directory's
existance, but changing our default isn't going to completely address
that issue either.

There's a HUGE difference between "This tool depends on conf.d, so
please don't disable it" and "please edit postgresql.conf in the
following way, and create this directory with these permissions ..."

I'm particularly thinking about this in relation to the merger of
recovery.conf and postgresql.conf. There are several tools already
(RepMgr, OminPITR, HandyRep, pgPool, etc.) which manage recovery.conf
separately from postgresql.conf. These tools will want to continue
managing recovery.conf as a separate file, even if it's /included in
postgresql.conf now.

Certainly an interesting thought.

It's more than in interesting thought. It's the difference between
having 20 lines of backwards compatibility code, and having 150 plus a
bunch of additional user documentation and setup.

If conf.d exists by default, and is enabled in postgresql.conf by
default, this is easy: the tool just drops a recovery.conf file into
conf.d. Changing file locations and variable names is a fairly simple
exercise in backwards compatibility.

This isn't quite that simple on at least Debian, and I believe RH, as
there's going to be multiple directories involved (one for each cluster
which exists on the system). Also, there are rules which packages need
to follow on Debian regarding conf file changes, which these certainly
would be.

Again, from the perspective of the tool (assuming it already supports
Debian), this is just a directory change. ("if version == 9.4,
directory = directory + "/conf.d/")

My recommendation would be to argue the point with the package
maintainers. Even if we *had* a default, I'm not convinced that the
package maintainers would keep it that way until and unless they update
their scripts and whatnot to handle it.

This is disengenuous. The package maintainers are never, ever going to
add conf.d and enable it by default unless it's the official
recommendation of the PostgreSQL project that they do so. Exactly how
far would I get with "I couldn't get pgsql-hackers to agree to this, so
I'm asking you"?

On 01/15/2014 11:40 AM, Peter Eisentraut wrote:

That seems like a very general argument. We'd need to consider exactly
what these tools want to do, and how that affects the recovery.conf
merger. My personal proposal was to allow postgresql.conf to contain
recovery parameters, but read recovery.conf last anyway. That would
solve that problem.

Three issues:

a) if postgresql is still going to look for a recovery.conf file in the
usual place, but we are changing the names and meaning of some of the
parameters, then aren't we making the upgrade problem much worse?

b) what if the admin *does* want to disable reading recovery.conf in
order to prevent old utilities from mistakenly including one? How will
they do that?

c) would this be in the configdir, datadir, or both?

I'd thought that part of the idea of the merger was to remove the
"magic" status of recovery.conf.

If conf.d exists by default, and is enabled in postgresql.conf by
default, this is easy: the tool just drops a recovery.conf file into
conf.d.

That would just give false hope, I think. My standard operating
procedure is to delete the default postgresql.conf and write a new one
from scratch (or have deployment tools do it). I don't want to
subscribe to a policy that the only endorsed way to maintain
postgresql.conf is to start with the default one and edit around

carefully.

The only way you could achieve your goal is to read the configuration
directory automatically, but that would come with its own set of problems.

Frankly, I'm not concerned about people who rewrite their
postgresql.conf from scratch already; it's *easy* to tell those people
to re-add the conf.d reference to that file. I'm talking about the vast
majority of our users who never edit pg.conf by hand.

Independent of the above, I don't agree with that. postgresql.auto.conf
is a special thing and should have its own special place. For once
thing, when putting configuration files in a separate directory
structure (/etc/ vs /var), then postgresql.auto.conf should stay in the
data directory.

Ah, I'd forgotten about that line of argument. Where is auto.conf now?

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#5Stephen Frost
sfrost@snowman.net
In reply to: Josh Berkus (#4)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

Josh,

* Josh Berkus (josh@agliodbs.com) wrote:

However, Debian is *never* going to add conf.d to the packages if we
don't recommend it as an upstream project. And, frankly, I use the
apt.postgresql.org packages anyway, which would certainly include
anything which was decided on this list.

Those are both categorial false claims. Debian certainly does not ship
with 'trust' auth, nor do our PGDG packages. They also move the conf
files out of the data directory. Last, but not least, they absolutely
enable conf.d directories even when that is not the default upstream.

Additionally, I fully expect and hope that the apt.postgresql.org
packages to follow the Debian/Ubuntu package management- having those
diverge would absolutely be a horrible idea and cause a great deal of
trouble for our users. Ideally, we can all agree, but this notion that
the PGDG must follow what happens on -hackers is simply wrong, we need
include and coordinate with the OS package maintainers.

It's more than in interesting thought. It's the difference between
having 20 lines of backwards compatibility code, and having 150 plus a
bunch of additional user documentation and setup.

If I was writing the tool, I'm pretty sure that I'd be writing all that
code either way.

Thanks,

Stephen

#6Tom Lane
tgl@sss.pgh.pa.us
In reply to: Peter Eisentraut (#3)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

Peter Eisentraut <peter_e@gmx.net> writes:

On 1/15/14, 1:53 PM, Josh Berkus wrote:

Yes, I'm also arguing that postgresql.auto.conf should go into conf.d.
I said I'd bring that up again after ALTER SYSTEM SET was committed, and
here it is.

Independent of the above, I don't agree with that. postgresql.auto.conf
is a special thing and should have its own special place. For once
thing, when putting configuration files in a separate directory
structure (/etc/ vs /var), then postgresql.auto.conf should stay in the
data directory.

It seems like we still aren't all on the same page as to whether the
conf.d directory (and contained files) should be expected to be writable
by the postgres server or not. I think it's hopeless to proceed further
unless there's a strong consensus on that.

My vote would be that the server should *not* be expected to be able to
write these files. It might physically be able to, in a quick-and-dirty
installation, but in a setup prepared by a packaging system I'd not
expect that the config files would be postgres-owned.

Given that assumption, it's clear that postgresql.auto.conf can't live
in conf.d. However, if you reject that assumption then maybe it should.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#7Peter Eisentraut
peter_e@gmx.net
In reply to: Tom Lane (#6)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

On 1/15/14, 4:35 PM, Tom Lane wrote:

Peter Eisentraut <peter_e@gmx.net> writes:

On 1/15/14, 1:53 PM, Josh Berkus wrote:

Yes, I'm also arguing that postgresql.auto.conf should go into conf.d.
I said I'd bring that up again after ALTER SYSTEM SET was committed, and
here it is.

Independent of the above, I don't agree with that. postgresql.auto.conf
is a special thing and should have its own special place. For once
thing, when putting configuration files in a separate directory
structure (/etc/ vs /var), then postgresql.auto.conf should stay in the
data directory.

It seems like we still aren't all on the same page as to whether the
conf.d directory (and contained files) should be expected to be writable
by the postgres server or not. I think it's hopeless to proceed further
unless there's a strong consensus on that.

You can turn this around and ask, why should it be writable? The server
has no need to write anything there.

In my mind, a conf.d directory is an extension of a single-file
configuration, and so it should be handled that way.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#8Peter Eisentraut
peter_e@gmx.net
In reply to: Josh Berkus (#4)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

On 1/15/14, 3:01 PM, Josh Berkus wrote:

Three issues:

a) if postgresql is still going to look for a recovery.conf file in the
usual place, but we are changing the names and meaning of some of the
parameters, then aren't we making the upgrade problem much worse?

That assumes that we are changing the names and meanings of some of the
parameters, which I don't see a reason for.

b) what if the admin *does* want to disable reading recovery.conf in
order to prevent old utilities from mistakenly including one? How will
they do that?

That assumes that there is a reason for doing that, which goes away if
point (a) is addressed.

c) would this be in the configdir, datadir, or both?

That might depend on the parameter and what a tool wants to do with it.

There is also the consideration of whether some of those tools couldn't
be changed to use ALTER SYSTEM.

I'd thought that part of the idea of the merger was to remove the
"magic" status of recovery.conf.

Well, clearly, everyone has their own ideas about that. I have several
non-overlapping ones of my own. ;-) But my point is that we should look
what actually comes out of that discussion before we start designing
other facilities that interact with it.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#9Stephen Frost
sfrost@snowman.net
In reply to: Peter Eisentraut (#7)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

* Peter Eisentraut (peter_e@gmx.net) wrote:

In my mind, a conf.d directory is an extension of a single-file
configuration, and so it should be handled that way.

I'm apparently out on some funny limb with this thought, but I'll throw
it out there anyway- in my head, the 'postgresql.auto.conf' thing that
essentially ends up included as part of 'postgresql.conf' should be
handled the same way a single 'postgresql.conf' or 'conf.d' directory
is. Now, I've never particularly agreed with it, but at least on
Debian/Ubuntu, the /etc conf directories are owned by the postgres user
by default. I dislike the idea of separating the PG config into things
in /etc and things in PGDATA as it will make life more difficult for the
poor sysadmins trying to figure out what's going on.

Thanks,

Stephen

#10Amit Kapila
amit.kapila16@gmail.com
In reply to: Josh Berkus (#4)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

On Thu, Jan 16, 2014 at 1:31 AM, Josh Berkus <josh@agliodbs.com> wrote:

On 01/15/2014 11:10 AM, Stephen Frost wrote:

Independent of the above, I don't agree with that. postgresql.auto.conf
is a special thing and should have its own special place. For once
thing, when putting configuration files in a separate directory
structure (/etc/ vs /var), then postgresql.auto.conf should stay in the
data directory.

Ah, I'd forgotten about that line of argument. Where is auto.conf now?

Data Directory

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#11Craig Ringer
craig@2ndquadrant.com
In reply to: Josh Berkus (#1)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

On 01/16/2014 02:53 AM, Josh Berkus wrote:

Hackers,

ALTER SYSTEM SET has been committed and recovery.conf GUCs are being
reviewed. I'm going to make a last case for conf.d in relation to these
two patches before 9.4 goes out the door.

Personally, I'd find conf.d greatly more useful when enabled by default.
The need to hack postgresql.conf to use it eliminates half the point of
having it.

I haven't seen any compelling reason why conf.d should NOT be enabled by
default.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#12Peter Eisentraut
peter_e@gmx.net
In reply to: Stephen Frost (#9)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

On 1/15/14, 11:23 PM, Stephen Frost wrote:

* Peter Eisentraut (peter_e@gmx.net) wrote:

In my mind, a conf.d directory is an extension of a single-file
configuration, and so it should be handled that way.

I'm apparently out on some funny limb with this thought, but I'll throw
it out there anyway- in my head, the 'postgresql.auto.conf' thing that
essentially ends up included as part of 'postgresql.conf' should be
handled the same way a single 'postgresql.conf' or 'conf.d' directory
is.

Then one might as well argue that the pg_db_role_setting table be
relocated to /etc. It's the same facility, only on a slightly different
level. The fact that postgresql.auto.conf looks the same as a
plain-text configuration file is an implementation detail. We could
have chosen some binary format instead.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#13Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Peter Eisentraut (#7)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

Peter Eisentraut wrote:

In my mind, a conf.d directory is an extension of a single-file
configuration, and so it should be handled that way.

+1 on this. This means

1. it is to be read automatically by the server without need for an
"include_dir conf.d" option in the main postgresql.conf.

2. it is to be placed alongside postgresql.conf, not necessarily in
PGDATA.

3. it might or might not be writable by the running server; it's an
operating-system-owned configuration location.

4. there is no point in "disabling" it, and thus we offer no mechanism
to do that.

5. its entries override what is set in postgresql.conf, and are in turn
overridden by what is set in postgresql.auto.conf.

Taken together, these properties guarantee that it's an useful mechanism
to be used by system-level deployment/configuration tools such as Puppet
et al.

I also think, but this is a secondary point, that initdb should write
its modified settings into a file in conf.d instead of generating a
custom postgresql.conf.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#14Stephen Frost
sfrost@snowman.net
In reply to: Peter Eisentraut (#12)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

Peter,

* Peter Eisentraut (peter_e@gmx.net) wrote:

Then one might as well argue that the pg_db_role_setting table be
relocated to /etc. It's the same facility, only on a slightly different
level. The fact that postgresql.auto.conf looks the same as a
plain-text configuration file is an implementation detail. We could
have chosen some binary format instead.

I argued quite a bit that we should be looking at moving more things
into the catalog tables and pulling them out of postgresql.conf,
reducing that down to just what's required to be there, rather than this
nasty hack where SQL commands are modifying text config files.

Thanks,

Stephen

#15Andres Freund
andres@anarazel.de
In reply to: Stephen Frost (#14)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

On 2014-01-16 08:34:23 -0500, Stephen Frost wrote:

Peter,

* Peter Eisentraut (peter_e@gmx.net) wrote:

Then one might as well argue that the pg_db_role_setting table be
relocated to /etc. It's the same facility, only on a slightly different
level. The fact that postgresql.auto.conf looks the same as a
plain-text configuration file is an implementation detail. We could
have chosen some binary format instead.

I argued quite a bit that we should be looking at moving more things
into the catalog tables and pulling them out of postgresql.conf,
reducing that down to just what's required to be there, rather than this
nasty hack where SQL commands are modifying text config files.

Given that the majority didn't seem to be convinced by this and that the
feature was written differently this isn't a convincing argument for the
location of the file given the current feature, no?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#16Christian Kruse
christian@2ndquadrant.com
In reply to: Alvaro Herrera (#13)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

Hi Alvaro,

On 16/01/14 10:21, Alvaro Herrera wrote:

1. it is to be read automatically by the server without need for an
"include_dir conf.d" option in the main postgresql.conf.

+1

4. there is no point in "disabling" it, and thus we offer no mechanism
to do that.

Not only there is „no point“ in disabling it, it makes this feature
nearly useless. One can't rely on it if the distro may disable
it. There are so many out there, it will never be a reliable feature
if it can be disabled.

5. its entries override what is set in postgresql.conf, and are in turn
overridden by what is set in postgresql.auto.conf.

+1

Best regards,

--
Christian Kruse http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

#17Stephen Frost
sfrost@snowman.net
In reply to: Alvaro Herrera (#13)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

* Alvaro Herrera (alvherre@2ndquadrant.com) wrote:

Peter Eisentraut wrote:

In my mind, a conf.d directory is an extension of a single-file
configuration, and so it should be handled that way.

+1 on this. This means

1. it is to be read automatically by the server without need for an
"include_dir conf.d" option in the main postgresql.conf.

I am not thrilled with the idea that we're claiming ownership of the
directory which postgresql.conf happens to reside in and you had better
not have any 'conf.d' directory in there unless it's the one for PG.

2. it is to be placed alongside postgresql.conf, not necessarily in
PGDATA.

I certainly agree with this idea- but I feel it should be configurable.
The path which is passed in should be relative to the location of
postgresql.conf.

3. it might or might not be writable by the running server; it's an
operating-system-owned configuration location.

Works well enough, I suppose.

4. there is no point in "disabling" it, and thus we offer no mechanism
to do that.

No. There should be a way to disable it.

5. its entries override what is set in postgresql.conf, and are in turn
overridden by what is set in postgresql.auto.conf.

For my part, I'd rather my configurations be deterministic and would
prefer that we error on obvious misconfigurations where values are set
and then reset.

Taken together, these properties guarantee that it's an useful mechanism
to be used by system-level deployment/configuration tools such as Puppet
et al.

It will be a useful mechanism for puppet even with the ability to
disable it in postgresql.conf (you're probably managing that with puppet
also, or keeping it at the system default which will likely include the
include_dir option by default- but, yes, you'd need to check that) and
without the "override previous definition" option.

I also think, but this is a secondary point, that initdb should write
its modified settings into a file in conf.d instead of generating a
custom postgresql.conf.

This makes sense.

Thanks,

Stephen

#18Stephen Frost
sfrost@snowman.net
In reply to: Andres Freund (#15)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

* Andres Freund (andres@2ndquadrant.com) wrote:

Given that the majority didn't seem to be convinced by this and that the
feature was written differently this isn't a convincing argument for the
location of the file given the current feature, no?

I'll start by pointing out that the only reason this ALTER SYSTEM thing
exists is because there are options that we can't set in the catalogs.

Now, I can't claim to conjecture on what the majority opinion is when it
disagreed with my own. I remain of the opinion that having settings
which admins are familiar with being overridden by some file in the data
directory is going to cause confusion and problems. Therefore, I feel
that the 'auto' file should be where postgresql.conf lives because
admins will at least have some hope of finding it there (and then
realizing they need to disable that functionality and lock down the
postgresql.conf permissions to be root-owned...).

Having the log files be sufficiently detailed to help an admin is better
than nothing, but it certainly won't be as good as having the config
files in the same directory when the admin is trying to figure out why
the cluster won't start any more.

Thanks,

Stephen

#19Tom Lane
tgl@sss.pgh.pa.us
In reply to: Stephen Frost (#17)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

Stephen Frost <sfrost@snowman.net> writes:

* Alvaro Herrera (alvherre@2ndquadrant.com) wrote:

1. it is to be read automatically by the server without need for an
"include_dir conf.d" option in the main postgresql.conf.

I am not thrilled with the idea that we're claiming ownership of the
directory which postgresql.conf happens to reside in and you had better
not have any 'conf.d' directory in there unless it's the one for PG.

If we were to do this, I'd argue that the location of this hard-wired
config directory ought to be selected by a configure option. And
in fact, I'd argue that the default behavior with no such option be
that there's no such hard-wired directory. That puts it entirely
on the packager to decide what makes sense as a location. In the end
it's going to be the packager's decision anyway --- it's just a matter
of how painful we make it for him to configure it to his distro's
conventions.

On the whole though, I find the argument that we can't configure this
in postgresql.conf to be exceedingly weak. In particular, the idea
that you can puppet-ify a configuration without any knowledge of the
distro you're targeting is ridiculous on its face, as is the idea
that we/postgres can dictate configuration file location conventions
to packagers who have distro rules to follow. There isn't going to
be a "one location to rule them all", and that means the argument
that a location determined by postgresql.conf is too unstable does
not really hold water.

Another point here is that hard-wiring a config directory location
into the executables completely breaks many scenarios for running
multiple clusters with the same executables. Yeah, maybe you'd
like to share most of the same config across all your clusters.
But then again, maybe you wouldn't. The proposed behavior would
make it practically impossible for a test cluster to not pick up
random subsets of the system primary cluster's configuration.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#20Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Stephen Frost (#17)
Re: Why conf.d should be default, and auto.conf and recovery.conf should be in it

Stephen Frost wrote:

* Alvaro Herrera (alvherre@2ndquadrant.com) wrote:

Peter Eisentraut wrote:

In my mind, a conf.d directory is an extension of a single-file
configuration, and so it should be handled that way.

+1 on this. This means

1. it is to be read automatically by the server without need for an
"include_dir conf.d" option in the main postgresql.conf.

I am not thrilled with the idea that we're claiming ownership of the
directory which postgresql.conf happens to reside in and you had better
not have any 'conf.d' directory in there unless it's the one for PG.

Please explain. I don't see a reason not to. Are you saying you expect
to put, say, Apache configuration files in the same directory as
PostgreSQL's? That doesn't sound really tenable to me, and it doesn't
sounds like what any sane admin would do.

In /etc you have /etc/postfix/ where you have Postfix files, and
/etc/exim4 where you have Exim files, and /etc/apache2 or whatever. And
if each of those software packages uses a "conf.d" or "conf-enabled"
directory, they use a hardcoded name for a place inside their own
subdirectory, not one directly in /etc.

What you don't have is /etc/exim4.conf and /etc/postfix.conf and
/etc/apache.conf, where each of those files specifies that I would
please like to have extra files loaded from /etc/exim.conf.d and
/etc/apache.conf.d, and that packages please figure out by parsing my
config file in whatever random format I have invented to figure out
where snippets are to be stored.

2. it is to be placed alongside postgresql.conf, not necessarily in
PGDATA.

I certainly agree with this idea- but I feel it should be configurable.
The path which is passed in should be relative to the location of
postgresql.conf.

What would be the point of having it be configurable?

4. there is no point in "disabling" it, and thus we offer no mechanism
to do that.

No. There should be a way to disable it.

What's you rationale for this?

5. its entries override what is set in postgresql.conf, and are in turn
overridden by what is set in postgresql.auto.conf.

For my part, I'd rather my configurations be deterministic and would
prefer that we error on obvious misconfigurations where values are set
and then reset.

Any convention we establish will make configurations deterministic.
Random variations such as making the conf.d directory be configurable
would make them non-deterministic. ISTM you're contradicting yourself
here.

Taken together, these properties guarantee that it's an useful mechanism
to be used by system-level deployment/configuration tools such as Puppet
et al.

It will be a useful mechanism for puppet even with the ability to
disable it in postgresql.conf (you're probably managing that with puppet
also, or keeping it at the system default which will likely include the
include_dir option by default- but, yes, you'd need to check that) and
without the "override previous definition" option.

Meh. External packages (say, postgis) will be able to add their own
files into conf.d without having to check what its actual location is,
and without having to check whether the thing is enabled in the first
place. If we make it disable-able, packages will not be able to do
that; or rather, they will be able to add files, but they will have no
effect.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#21Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#19)
#22Stephen Frost
sfrost@snowman.net
In reply to: Alvaro Herrera (#20)
#23Stephen Frost
sfrost@snowman.net
In reply to: Andres Freund (#21)
#24Andres Freund
andres@anarazel.de
In reply to: Stephen Frost (#23)
#25Stephen Frost
sfrost@snowman.net
In reply to: Andres Freund (#24)
#26Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Stephen Frost (#25)
#27Stephen Frost
sfrost@snowman.net
In reply to: Alvaro Herrera (#26)
#28Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alvaro Herrera (#26)
#29Josh Berkus
josh@agliodbs.com
In reply to: Alvaro Herrera (#26)
#30Stephen Frost
sfrost@snowman.net
In reply to: Josh Berkus (#29)
#31Josh Berkus
josh@agliodbs.com
In reply to: Josh Berkus (#1)
#32Amit Kapila
amit.kapila16@gmail.com
In reply to: Tom Lane (#28)
#33Robert Haas
robertmhaas@gmail.com
In reply to: Amit Kapila (#32)
#34Amit Kapila
amit.kapila16@gmail.com
In reply to: Robert Haas (#33)
#35Tom Lane
tgl@sss.pgh.pa.us
In reply to: Amit Kapila (#34)
#36Amit Kapila
amit.kapila16@gmail.com
In reply to: Tom Lane (#35)