Explicit config patch 7.2B4

mlw <markw@mohawksoft.com> writes:

I guess I will maintain this for people who want it.

I'd definitely encourage you to try to get it into 7.3 when that
opens--the list seems to be generally positive about it, and as long
as existing setups work as-is (which seems to be the case) I think
it's a valuable addition.

-Doug
--
Let us cross over the river, and rest under the shade of the trees.
--T. J. Jackson, 1863

On Sun, Dec 16, 2001 at 09:35:58AM -0500, mlw wrote:

This allows

postmaster -C /etc/pgsql/mydb.conf

Nice.

In the config file there are two more options:

datadir = '/u01/postgres'
hbaconfig = '/etc/pgsql/pg_hba.conf'

What about pg_ident.conf?

--
marko

mlw writes:

This allows

postmaster -C /etc/pgsql/mydb.conf

The "-C" option specifies a configuration file.

I'm still not happy about this, because given a pre-configured or already
running system it is difficult or impossible to find out which
configuration file is being used. This offsets in many ways the improved
usability you're trying to achieve.

I think an 'include' directive for postgresql.conf would solve this
problem more generally (since it allows many more sharing models) and
would also give us a good tool when we get to the configuration of
alternative storage locations.

Probably a command-line option could prove useful for testing purposes,
etc., but I feel that by default the configuration should be written down
in some easy-to-find file. This is consistent with the move away from
command-line options that we have made with postgresql.conf.

Probably we could make the option -C to mean "imagine an include directive
written at the very start [or end?] of $PGDATA/postgresql.conf". With the
default empty file this would achieve exactly the same thing as you're
trying.

Comments?

--
Peter Eisentraut peter_e@gmx.net

Peter Eisentraut wrote:

mlw writes:

This allows

postmaster -C /etc/pgsql/mydb.conf

The "-C" option specifies a configuration file.

I'm still not happy about this, because given a pre-configured or already
running system it is difficult or impossible to find out which
configuration file is being used. This offsets in many ways the improved
usability you're trying to achieve.

I do not agree. A command line option which points to a configuration file IS
the standard way to start a server under UNIX.

I think an 'include' directive for postgresql.conf would solve this
problem more generally (since it allows many more sharing models) and
would also give us a good tool when we get to the configuration of
alternative storage locations.

An include directive would be useful, obviously, but it is not in exclusion of
a more flexible configuration file.

Probably a command-line option could prove useful for testing purposes,
etc., but I feel that by default the configuration should be written down
in some easy-to-find file. This is consistent with the move away from
command-line options that we have made with postgresql.conf.

I am having the hardest time understanding your antipathy toward an explicit
configuration file. I just don't have any idea of why you are fighting it so
hardly. As far as I can see there is no reason not to do it, and every other
important server on UNIX supports this construct.

Again, I just don't get it. Standards are standards, and an explicit
configuration file is a defacto standard.

Probably we could make the option -C to mean "imagine an include directive
written at the very start [or end?] of $PGDATA/postgresql.conf". With the
default empty file this would achieve exactly the same thing as you're
trying.

The WHOLE idea is to get away from a configuration file mixed with the data. I
think the notion of having configuration contained in the same location as data
is bad. Furthermore, forcing this construct is worse.

Comments?

I really don't understand why you don't want this. There isn't a single
important UNIX server which forces its configuration file to be contained
within its data / operational directory. Not one. Why is postgresql "better"
for being less flexible?

What is the harm in including this functionality?

On Sunday 16 December 2001 05:23 pm, Peter Eisentraut wrote:

I'm still not happy about this, because given a pre-configured or already
running system it is difficult or impossible to find out which
configuration file is being used. This offsets in many ways the improved
usability you're trying to achieve.

Would -C and its argument not be written to $PGDATA/postmaster.opts if
started with pg_ctl? I've not looked at the patch directly, but it would be
regular behaviour for that to show up, right? If it's in postmaster.opts,
then it's trivial to examine and determin where things are coming from.

I think an 'include' directive for postgresql.conf would solve this
problem more generally (since it allows many more sharing models) and
would also give us a good tool when we get to the configuration of
alternative storage locations.

I like an include directive. But not for the reasons you like it.

Probably we could make the option -C to mean "imagine an include directive
written at the very start [or end?] of $PGDATA/postgresql.conf". With the
default empty file this would achieve exactly the same thing as you're
trying.

No -- he's trying to get the config out of $PGDATA. The config
_does_not_belong_with_the_data_. While it _could_ be put there if the admin
chooses, it should not be tied to $PGDATA.

I'll have to echo Mark's query, though: Why are you fighting this, Peter?
This functionality mirrors the standard behaviour for daemons. Name a
standard daemon package other than postgresql that automatically assumes the
config is with dynamic data, and overwrites an existing config when the
dynamic data area is reinitialized.

I'm willing to compromise to a degree on this issue. However, Mark already
has working code. Sounds like a candidate for a patch -- maybe even a
feature patch. And I'll admit to some desire to apply this patch for the
RPMset -- but I probably won't, as the RPMset shouldn't do anything that
drastic. However, it wouldn't surprize me in the least for a distributor
such as Red Hat to apply this patch.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen <lamar.owen@wgcr.org> writes:

I'll have to echo Mark's query, though: Why are you fighting this, Peter?

Peter's not the only one who's unhappy.

This functionality mirrors the standard behaviour for daemons.

That's been Mark's primary argument all along, and what it ignores is
that the standard behavior for daemons is designed around the assumption
that a system is running only one copy of any given daemon. That's a
fine assumption for most daemons but an unacceptable one for Postgres.

I'm prepared to accept some kind of compromise on this issue, but I'm
really tired of hearing the useless "other daemons do it this way"
argument. Could we hear some more-relevant argument?

I rather liked Peter's idea of treating the feature as an implicit
inclusion. Maybe there's an even-better approach out there, but so far
that's the best idea I've heard.

Name a standard daemon package other than postgresql that
automatically assumes the config is with dynamic data, and overwrites
an existing config when the dynamic data area is reinitialized.

initdb will not overwrite an existing config. Try it.

However, it wouldn't surprize me in the least for a distributor
such as Red Hat to apply this patch.

Oh, I doubt it...

regards, tom lane
Red Hat Database project

Tom Lane <tgl@sss.pgh.pa.us> writes:

That's been Mark's primary argument all along, and what it ignores is
that the standard behavior for daemons is designed around the assumption
that a system is running only one copy of any given daemon. That's a
fine assumption for most daemons but an unacceptable one for Postgres.

I'd say that's not completely accurate. I've seen and run sites with
more than one {httpd, sendmail} running. The basic idea is:

* If no config file is specified, either look for it in a standard
place, or complain bitterly. Sendmail looks for /etc/sendmail.cf
(usually); Apache looks in a place configured at compile time
(/etc/httpd/conf/httpd.conf on RedHat systems).

* If a config file *is* specified, use it. It tells you where to look
for other stuff (queue directory, webserver root or whatever).

The above scheme is used by many different daemons and is *perfectly*
conducive to running multiple copies. What makes you say it isn't?

I'm prepared to accept some kind of compromise on this issue, but I'm
really tired of hearing the useless "other daemons do it this way"
argument. Could we hear some more-relevant argument?

How is a patch that (a) perfectly preserves existing behavior, and (b)
allows for more flexibility in configuration, a bad thing?

I'm not going to lose a lot of sleep if Mark's patch isn't adopted. I
will say, however, that as a long-time Un*x sysadmin (Ultrix, Irix,
Solaris, BSD, Linux) PG's method of configuration struck me as a bit
weird when I first saw it. It obviously does the job, but I like the
idea of giving users and packagers a configuration method that's still
sufficiently flexible and is more familiar to some.

My $0.02...

-Doug
--
Let us cross over the river, and rest under the shade of the trees.
--T. J. Jackson, 1863

Tom Lane wrote:

Lamar Owen <lamar.owen@wgcr.org> writes:

I'll have to echo Mark's query, though: Why are you fighting this, Peter?

Peter's not the only one who's unhappy.

I still do not understand why.

This functionality mirrors the standard behaviour for daemons.

That's been Mark's primary argument all along, and what it ignores is
that the standard behavior for daemons is designed around the assumption
that a system is running only one copy of any given daemon. That's a
fine assumption for most daemons but an unacceptable one for Postgres.

It makes NO such argument at all! It allows an admin to explicitly share a
configuration file (or not). I allows for one central place in which multiple
configuration files can be stored and backed up. It allows for the explicit
sharing of pg_hba.conf files. (It was noted that pg_ident.conf should be added,
I am looking at it.)

Most of all it does these things WITHOUT symlinks. Most admins I know like
symlinks as a method of working around a short coming in a product, but would
rather have the configurability designed into the product.

I wrote this patch to make it easier for me to administer multiple databases on
one machine. To make it easier for UNIX admins to follow the dba's trail. I
wrote this so a system was a bit more self documenting without having to follow
symlinks.

I'm prepared to accept some kind of compromise on this issue, but I'm
really tired of hearing the useless "other daemons do it this way"
argument. Could we hear some more-relevant argument?

This something else I don't understand. Why would you NOT give an admin the
sort of configurability they expect?

I rather liked Peter's idea of treating the feature as an implicit
inclusion. Maybe there's an even-better approach out there, but so far
that's the best idea I've heard.

In my eyes, "implicit" means unexpected. I love the idea of an include
directive, that is a great idea, but it does not address the separation of data
and config.

People new to PostgreSQL will look for he config in /usr/local/pgsql/etc, but
it won't be there. You can specify the config directory with configure
(sysconfdir), but it is never used.

Name a standard daemon package other than postgresql that
automatically assumes the config is with dynamic data, and overwrites
an existing config when the dynamic data area is reinitialized.

initdb will not overwrite an existing config. Try it.

That's good to know.

However, it wouldn't surprize me in the least for a distributor
such as Red Hat to apply this patch.

Oh, I doubt it...

Tom, I really don't understand the resistance. It seem irrational to me, and I
am really trying to figure it out.

Obviously you must have your reasons, but all I've read thus far is the
argument that "PostgreSQL is different than other daemons." Maybe I'm pushing
too hard for this and making people defensive, but I just don't get it.

The core reasons for this patch:
(1) Move configuration out of $PGDATA and into some centralized location. I do
not know of a single admin that would object to this.
(2) Allow admins to share the pg_hba.conf file.

I try to get pg_ident when I get a chance, that's a good idea.

The one thing about this I do not like is that it is a "share all" or "share
none" solution. An include directive would be helpful here, but I think it is a
good start.

Tom Lane <tgl@sss.pgh.pa.us> writes:

That's been Mark's primary argument all along, and what it ignores is
that the standard behavior for daemons is designed around the assumption
that a system is running only one copy of any given daemon. That's a
fine assumption for most daemons but an unacceptable one for Postgres.

I don't think there would be that much problems with it, but you can
always have defaults in one location and allow them to be overridable.

However, it wouldn't surprize me in the least for a distributor
such as Red Hat to apply this patch.

Oh, I doubt it...

regards, tom lane
Red Hat Database project

Don't. I'd do it in a heartbeat - I'd love to have /etc/postgresql/
with defaults. Configuration files should not be located in /var.

regards, Trond Eivind Glomsr�d
developer, Red Hat Linux developer
--
Trond Eivind Glomsr�d
Red Hat, Inc.

On Mon, 2001-12-17 at 12:22, mlw wrote:

That's been Mark's primary argument all along, and what it ignores is
that the standard behavior for daemons is designed around the assumption
that a system is running only one copy of any given daemon. That's a
fine assumption for most daemons but an unacceptable one for Postgres.

It makes NO such argument at all! It allows an admin to explicitly share a
configuration file (or not). I allows for one central place in which multiple
configuration files can be stored and backed up. It allows for the explicit
sharing of pg_hba.conf files. (It was noted that pg_ident.conf should be added,
I am looking at it.)

Most of all it does these things WITHOUT symlinks. Most admins I know like
symlinks as a method of working around a short coming in a product, but would
rather have the configurability designed into the product.

I wrote this patch to make it easier for me to administer multiple databases on
one machine. To make it easier for UNIX admins to follow the dba's trail. I
wrote this so a system was a bit more self documenting without having to follow
symlinks.

As Debian packager I would very much like to see a more convenient and
controllable configuration system.

Ross described last week the Debian method of symlinking the real config
files in /etc/postgresql to $PGDATA/; this method is forced on us by
Debian policy, which requires that all config files be in /etc. The
undesired result is that it is not convenient to run multiple servers on
a single machine using the Debian package.

I am currently considering a method of getting round this by creating a
subdirectory of /etc/postgresql for each server instance, but it would
be nice to be able to share global options in more convenient ways than
by multiplying symlinks.

--
Oliver Elphick Oliver.Elphick@lfix.co.uk
Isle of Wight http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C

"For I say, through the grace given unto me, to every
man that is among you: Do not think of yourself more
highly than you ought, but rather think of yourself
with sober judgement, in accordance with the measure
of faith God has given you." Romans 12:3

On Sunday 16 December 2001 11:13 pm, Tom Lane wrote:

Lamar Owen <lamar.owen@wgcr.org> writes:

This functionality mirrors the standard behaviour for daemons.

That's been Mark's primary argument all along, and what it ignores is
that the standard behavior for daemons is designed around the assumption
that a system is running only one copy of any given daemon. That's a
fine assumption for most daemons but an unacceptable one for Postgres.

Really? Most daemons allow a default config location, and then either allow
or require a config file path on the command line. AOLserver _requires_ the
path on the command line; named allows it, sendmail allows it, etc.

I routinely run multiple nameds on machines behind NAT. Just a simple config
file path away. I routinely run multiple instances of other programs as well.

Note that neither I, Mark, Doug, or anyone else is asking for a change in the
default behavior. I personally just want it to be _allowed_ behavior.
Nothing more; nothing less.

I rather liked Peter's idea of treating the feature as an implicit
inclusion. Maybe there's an even-better approach out there, but so far
that's the best idea I've heard.

I rather dislike the idea that $PGDATA is where the config file lives. I
quite particularly dislike the 'implicit include' idea.

Name a standard daemon package other than postgresql that
automatically assumes the config is with dynamic data, and overwrites
an existing config when the dynamic data area is reinitialized.

initdb will not overwrite an existing config. Try it.

Ok, I'll concede that. But having postgresql.conf in $PGDATA makes it more
difficult for an admin to wipe $PGDATA and start over. For that matter,
pg_hba.conf is there, too, and I disagree that users should be forced to put
it in $PGDATA.

However, it wouldn't surprize me in the least for a distributor
such as Red Hat to apply this patch.

Oh, I doubt it...

regards, tom lane
Red Hat Database project

Having seen Trond's reply, I have to laugh....as I _know_ he disagrees with
you (and knew such before he replied -- this has been a thorn in his side for
awhile. Might prove to be an interesting 'discussion' inside RH over it.
But, again, an 'optional' command line switch should be a no-brainer. It
just seems to me to be rather unproductive to not allow people more
flexibility in a regular way. Symlinks are not the answer, either.

But RH is not the only distributor of PostgreSQL. And, for the record, the
Debian packages already do something very similar. Wouldn't it be better for
the core package to support this, rather than each distributor doing it
differently from each other?
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Marko Kreen wrote:

On Sun, Dec 16, 2001 at 09:35:58AM -0500, mlw wrote:

This allows

postmaster -C /etc/pgsql/mydb.conf

Nice.

In the config file there are two more options:

datadir = '/u01/postgres'
hbaconfig = '/etc/pgsql/pg_hba.conf'

What about pg_ident.conf?

I will submit a complete patch that includes pg_ident.conf as well.

Here is a question for the great minds here:

If a user has used the "-C" option, as in:

postmaster -C /etc/pgsql/mydb.conf

Should I then, and first, see if there is a "/etc/pgsql/pg_hba.conf" or
"/etc/pgsql/pg_ident.conf" and use it as an explicit path?

How about:

postmaster -C /etc/pgsql

Should I then look for:

/etc/pgsql/postgresql.conf
/etc/pgsql/pg_hba.conf
/etc/pgsql/pg_ident.conf

?

Just curious what you think.

On Monday 17 December 2001 12:56 pm, mlw wrote:

Here is a question for the great minds here:
If a user has used the "-C" option, as in:

postmaster -C /etc/pgsql/mydb.conf

Should I then, and first, see if there is a "/etc/pgsql/pg_hba.conf" or
"/etc/pgsql/pg_ident.conf" and use it as an explicit path?

How about:
postmaster -C /etc/pgsql

I agree to both; of course, this should be overrideable in the conf file.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

I was looking around, and for this to be a good patch, I have to add it
to "postgres" as well as "postmaster."

In the 7.1.3 source, there is a command line option "-C" (don't show
version number) which seems to have been depricated in 7.1.x (no longer
documented, but still active.)

Is is "-C" dangerous at this stage? Should I use another command line
option?

Trond Eivind Glomsr�d writes:

Don't. I'd do it in a heartbeat - I'd love to have /etc/postgresql/
with defaults. Configuration files should not be located in /var.

This problem has already been solved with symlinks a long time ago.
Nothing new here. In fact, for users that are accustomed to the
"original" source distribution it'd be a much easier-to-understand
approach. (Whether it's "better" in the general sense is being
discussed.)

--
Peter Eisentraut peter_e@gmx.net

On Monday 17 December 2001 03:58 pm, Peter Eisentraut wrote:

Trond Eivind Glomsr�d writes:

Don't. I'd do it in a heartbeat - I'd love to have /etc/postgresql/
with defaults. Configuration files should not be located in /var.

This problem has already been solved with symlinks a long time ago.
Nothing new here.

Using symlinks is a workaround, not a solution.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

mlw writes:

Is is "-C" dangerous at this stage? Should I use another command line
option?

Use -C and eliminate/ignore the other use.

--
Peter Eisentraut peter_e@gmx.net

On Mon, Dec 17, 2001 at 12:56:53PM -0500, mlw wrote:

Marko Kreen wrote:

On Sun, Dec 16, 2001 at 09:35:58AM -0500, mlw wrote:

In the config file there are two more options:

datadir = '/u01/postgres'
hbaconfig = '/etc/pgsql/pg_hba.conf'

I will submit a complete patch that includes pg_ident.conf as well.

Here is a question for the great minds here:

If a user has used the "-C" option, as in:

postmaster -C /etc/pgsql/mydb.conf

Should I then, and first, see if there is a "/etc/pgsql/pg_hba.conf" or
"/etc/pgsql/pg_ident.conf" and use it as an explicit path?

It should search them only if they are not mentioned in .conf, I
guess.

How about:

postmaster -C /etc/pgsql

Should I then look for:

/etc/pgsql/postgresql.conf
/etc/pgsql/pg_hba.conf
/etc/pgsql/pg_ident.conf

One suggestion to you: try to think how different approaches
look when documented. User must be able to predict program
behaviour only by looking at docs. If 3 lines of C add
2 messy paragraphs to docs, then it is probably unnecessary
'feature'. Also what if I say '-C /etc/pgsql/db.conf' and there
is no pg_hba.conf there. It should give error, not go secretly
searching it in $PGDATA.

--
marko

mlw writes:

If a user has used the "-C" option, as in:

postmaster -C /etc/pgsql/mydb.conf

Should I then, and first, see if there is a "/etc/pgsql/pg_hba.conf" or
"/etc/pgsql/pg_ident.conf" and use it as an explicit path?

How about:

postmaster -C /etc/pgsql

Should I then look for:

/etc/pgsql/postgresql.conf
/etc/pgsql/pg_hba.conf
/etc/pgsql/pg_ident.conf

I like the latter better because of the symmetry of -C with -D.

Btw., the following issues still need to be addressed:

- Location of SSL certificate files (Are they appropriate for /etc? What
does Apache do?)

- Location of secondary password files. By default they should probably
track pg_hba.conf. Is that enough?

- Location of charset.conf and the associated recode tables

For a start, putting all of these under -C is probably sufficient. More
complicated setups can be achieved using symlinks. ;-)

--
Peter Eisentraut peter_e@gmx.net