What is the motivation of include directive and configuration files outside PGDATA

Takayuki Tsunakawa wrote:

Hello,

Let me ask about the background of configuration files. I couldn't
find the relevant information in the 8.2 documentation. I'm sorry to
cause you trouble.

In section "17.1. Setting Parameters", include directive is described.
Why was this directive prepared? What usage is assumed? Is it for
GUI tools, or for placing custom parameters in other files?

In section "17.2. File Locations", the following parameters are
described:

data_directory (string)
config_file (string)
hba_file (string)
ident_file (string)
external_pid_file (string)

What demand is assumed for placing configuration files outside the
data directory? Is it for placing configuration files in /etc to
conform to the FHS (Filesystem Hierarchy Standard, as documented
below) to enable the backup of all configuration files on the system?

It was designed for people who have multiple postmasters, but and want
to centralize some of that configuation.

--
Bruce Momjian bruce@momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

From: "Bruce Momjian" <bruce@momjian.us>

Takayuki Tsunakawa wrote:

In section "17.1. Setting Parameters", include directive is

described.

Why was this directive prepared? What usage is assumed? Is it for
GUI tools, or for placing custom parameters in other files?

In section "17.2. File Locations", the following parameters are
described:
What demand is assumed for placing configuration files outside the
data directory? Is it for placing configuration files in /etc to
conform to the FHS (Filesystem Hierarchy Standard, as documented
below) to enable the backup of all configuration files on the

system?

It was designed for people who have multiple postmasters, but and

want

to centralize some of that configuation.

Thank you. But I'm still a bit unclear.
Do you mean those functionalities were designed for, e.g.,
organizations that provide hosting service -- runs multiple database
instances on a single large machine? The image is:

/DataDirForCustomer1/postgresql.conf
[content]
include '/etc/postgresql_common.conf'
hba_file='/etc/pg_hba.conf'
...settings specific to customer 1

/DataDirForCustomer2/postgresql.conf
[content]
include '/etc/postgresql_common.conf'
hba_file='/etc/pg_hba.conf'
...settings specific to customer 2

/etc/postgresql_common.conf
[content]
...settings common among all customers

Was this mechanism invented in response to the demand from users, or
invented from the idea of PostgreSQL developers?

Still, I don't understand well why config files need to be placed
outside the data directory, except for daring conform to FHS.

"Takayuki Tsunakawa" <tsunakawa.takay@jp.fujitsu.com> writes:

Still, I don't understand well why config files need to be placed
outside the data directory, except for daring conform to FHS.

The killer argument for it is that most of what is in $PGDATA should be
excluded from your normal filesystem backup method, because you need to
be using some database-aware mechanism for backing up the database. But
the config files are perfectly suited for standard filesystem backup,
and indeed will *not* be covered by, say, pg_dumpall. So putting them
somewhere else helps in creating a coherent backup strategy.

regards, tom lane

From: "Tom Lane" <tgl@sss.pgh.pa.us>

"Takayuki Tsunakawa" <tsunakawa.takay@jp.fujitsu.com> writes:

Still, I don't understand well why config files need to be placed
outside the data directory, except for daring conform to FHS.

The killer argument for it is that most of what is in $PGDATA should

be

excluded from your normal filesystem backup method, because you need

to

be using some database-aware mechanism for backing up the database.

But

the config files are perfectly suited for standard filesystem

backup,

and indeed will *not* be covered by, say, pg_dumpall. So putting

them

somewhere else helps in creating a coherent backup strategy.

Thank you, I've understood the reason for placement. As I supposed,
it is provided mainly for placing config files in /etc to allow
frequent backup of the configuration as a whole system, because most
config files of UNIX are stored in /etc (recommended so in FHS) and
/etc is small.

Tom Lane wrote:

"Takayuki Tsunakawa" <tsunakawa.takay@jp.fujitsu.com> writes:

Still, I don't understand well why config files need to be placed
outside the data directory, except for daring conform to FHS.

The killer argument for it is that most of what is in $PGDATA should be
excluded from your normal filesystem backup method, because you need to
be using some database-aware mechanism for backing up the database. But
the config files are perfectly suited for standard filesystem backup,
and indeed will *not* be covered by, say, pg_dumpall. So putting them
somewhere else helps in creating a coherent backup strategy.

My backup setup excludes subdirectories of the data dir, rather than the
data dir itself.

Meeting FHS requirements is no bad thing, though. And the ability to
include a common configuration set in multiple instances is surely
useful to a number of people. After all, you aren't forced to use these
facilities - I typically don't.

cheers

andrew

From: "Andrew Dunstan" <andrew@dunslane.net>

Meeting FHS requirements is no bad thing, though. And the ability to
include a common configuration set in multiple instances is surely
useful to a number of people. After all, you aren't forced to use

these

facilities - I typically don't.

Thank you, Andrew-san.
What I want to know is a more concreet thing.

How useful are those facilities to what kind of users in what cases?
Is there a reason why users in the real world positively use those
facilities?

If I want to develop an add-on tool that manipulates PostgreSQL
settings even when PostgreSQL is stopped, should that tool support the
environments where include directive is used and config files are
placed outside the data directory? If the need for include directive
and flexible config file placement is very low or vague, I thought I
could say "this tool does not support those environments".

Are any facilities (functions) provided in PostgreSQL that external
tool developers can use to read/write config files? Suppose a user
created a database instance manually, with the config files as
follows:

[$PGDATA/postgresql.conf]
...
max_connections=10
include '/some_dir/some.conf'
include '/other_dir/other.conf'

[/some_dir/some.conf]
...
max_connections=50

[/other_dir/other.conf]
...
max_connections=100

If an external tool wants to determine the current value of
max_connections or modify the setting, what should the tool do?
Should the tool parse and write out the files by itself?

Takayuki Tsunakawa wrote:

From: "Andrew Dunstan" <andrew@dunslane.net>

Meeting FHS requirements is no bad thing, though. And the ability to
include a common configuration set in multiple instances is surely
useful to a number of people. After all, you aren't forced to use

these

facilities - I typically don't.

Thank you, Andrew-san.
What I want to know is a more concreet thing.

How useful are those facilities to what kind of users in what cases?
Is there a reason why users in the real world positively use those
facilities?

If you want to find out about usage patterns this is probably not the
best place to survey users - the hackers are not very representative of
users in general. I have a sneaking suspicion that include directives
are not used a whole lot, but alternative file locations are much more
widely used.

My off-the-cuff advice about your tool would be to process any include
directives when reading the config file, but initially just write out
new settings to the top level file (at the end, so they don't get
overridden by the includes).

cheers

andrew