Server tries to read a different config file than it is supposed to

On Sat, 2015-05-23 at 04:23 -0700, twoflower wrote:

I thought I understood how specifying a config file path for the
server works, but that's apparently not the case.

The cluster data is at /storage/postgresql/9.4/data.

The config files are at /etc/postgresql/9.4/main (this is the default
location on Ubuntu).

This is how the beginning of /etc/postgresql/9.4/main/postgresql.conf
looks like:

data_directory = '/storage/postgresql/9.4/data'
hba_file = '/etc/postgresql/9.4/main/pg_hba.conf'
ident_file = '/etc/postgresql/9.4/main/pg_ident.conf'

So I wrote a few scripts to make my life easier, e.g. pg94start.sh:

su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl
-D /storage/postgresql/9.4/data -o '-c
config_file=/etc/postgresql/9.4/main/postgresql.conf'"

But running this script did not work, the server would not start. So I
checked the log file and there was:

FATAL: could not open file
"/storage/postgresql/9.4/data/postgresql.conf": Permission denied

After fixing the ownership of this file, it worked.

What's the reason the server was trying to access that file? Why does
not the override given by the config_file parameter work?

Thank you.

______________________________________________________________________
View this message in context: Server tries to read a different config
file than it is supposed to
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

Similar hassles with Debian. I use scripts like this to start the
server.

#! /bin/sh
PGDATA=/home/postgres/data94;export PGDATA
PATH=$PATH:/usr/lib/postgresql/9.4/bin;export PATH
PGBINARIES=/usr/lib/postgresql/9.4/bin;export PGBINARIES
cd /usr/lib/postgresql/9.4/bin
./pg_ctl start
exit 0

I have different scripts depending on the version of Postgres that I
need. I know that I could have a generic script and just pass in the
variables. I run them as the postgres user.
It reads the conf file from the PGDATA path.

HTH.

Robert

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

On 05/23/2015 04:23 AM, twoflower wrote:

I thought I understood how specifying a config file path for the server
works, but that's apparently not the case.

The cluster data is at */storage/postgresql/9.4/data*.

The config files are at */etc/postgresql/9.4/main* (this is the default
location on Ubuntu).

This is how the beginning of */etc/postgresql/9.4/main/postgresql.conf*
looks like:

|data_directory = '/storage/postgresql/9.4/data'
hba_file = '/etc/postgresql/9.4/main/pg_hba.conf'
ident_file = '/etc/postgresql/9.4/main/pg_ident.conf' |

So I wrote a few scripts to make my life easier, e.g. *pg94start.sh*:

|su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D
/storage/postgresql/9.4/data -o '-c
config_file=/etc/postgresql/9.4/main/postgresql.conf'"|

But running this script did not work, the server would not start. So I
checked the log file and there was:

*FATAL: could not open file
"/storage/postgresql/9.4/data/postgresql.conf": Permission denied*

After fixing the ownership of this file, it worked.

What's the reason the server was trying to access that file? Why does
not the override given by the *|config_file|* parameter work?

I think you are seeing the effects of the below:

http://www.postgresql.org/docs/9.4/static/runtime-config-file-locations.html

"If you wish to keep the configuration files elsewhere than the data
directory, the postgres -D command-line option or PGDATA environment
variable must point to the directory containing the configuration files,
and the data_directory parameter must be set in postgresql.conf (or on
the command line) to show where the data directory is actually located.
Notice that data_directory overrides -D and PGDATA for the location of
the data directory, but not for the location of the configuration files."

By specifying -D you pointing at /storage/postgresql/9.4/data as the
source for the conf files, then changing that with the -o switch. Try
your command without the -D switch and let the postgresql.conf file in
/etc/postgresql/9.4/main/ provide the data_directory.

Thank you.
------------------------------------------------------------------------
View this message in context: Server tries to read a different config
file than it is supposed to
<http://postgresql.nabble.com/Server-tries-to-read-a-different-config-file-than-it-is-supposed-to-tp5850752.html&gt;
Sent from the PostgreSQL - general mailing list archive
<http://postgresql.nabble.com/PostgreSQL-general-f1843780.html&gt; at
Nabble.com.

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

On 05/23/2015 07:01 AM, rob stone wrote:

On Sat, 2015-05-23 at 04:23 -0700, twoflower wrote:

I thought I understood how specifying a config file path for the
server works, but that's apparently not the case.

The cluster data is at /storage/postgresql/9.4/data.

The config files are at /etc/postgresql/9.4/main (this is the default
location on Ubuntu).

This is how the beginning of /etc/postgresql/9.4/main/postgresql.conf
looks like:

data_directory = '/storage/postgresql/9.4/data'
hba_file = '/etc/postgresql/9.4/main/pg_hba.conf'
ident_file = '/etc/postgresql/9.4/main/pg_ident.conf'

So I wrote a few scripts to make my life easier, e.g. pg94start.sh:

su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl
-D /storage/postgresql/9.4/data -o '-c
config_file=/etc/postgresql/9.4/main/postgresql.conf'"

But running this script did not work, the server would not start. So I
checked the log file and there was:

FATAL: could not open file
"/storage/postgresql/9.4/data/postgresql.conf": Permission denied

After fixing the ownership of this file, it worked.

What's the reason the server was trying to access that file? Why does
not the override given by the config_file parameter work?

Thank you.

______________________________________________________________________
View this message in context: Server tries to read a different config
file than it is supposed to
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

Similar hassles with Debian. I use scripts like this to start the
server.

#! /bin/sh
PGDATA=/home/postgres/data94;export PGDATA
PATH=$PATH:/usr/lib/postgresql/9.4/bin;export PATH
PGBINARIES=/usr/lib/postgresql/9.4/bin;export PGBINARIES
cd /usr/lib/postgresql/9.4/bin
./pg_ctl start
exit 0

I have different scripts depending on the version of Postgres that I
need. I know that I could have a generic script and just pass in the
variables. I run them as the postgres user.
It reads the conf file from the PGDATA path.

Any particular reason you do not use pg_ctlcluster?:

https://wiki.debian.org/PostgreSql#pg_ctl_replacement

HTH.

Robert

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

twoflower <standa.kurik@gmail.com> writes:

So I wrote a few scripts to make my life easier, e.g. *pg94start.sh*:
su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D
/storage/postgresql/9.4/data -o '-c
config_file=/etc/postgresql/9.4/main/postgresql.conf'"
But running this script did not work, the server would not start.

Testing this, the problem appears to be that you forgot the keyword
"start", so pg_ctl didn't really do anything. It's always a good
idea to redirect pg_ctl's stdout/stderr somewhere, so that you can
look at it in event of problems. (It will *not* magically go to the
server log file.)

So I
checked the log file and there was:
*FATAL: could not open file "/storage/postgresql/9.4/data/postgresql.conf":
Permission denied*

I suspect this was left over from some previous attempt.

After fixing the ownership of this file, it worked.

I can't explain that claim, but for me, -c config_file=something
seems to work as you'd expect, and a look at the server source
code confirms that it should honor that (cf SelectConfigFiles()).
I think the documentation Adrian pointed to is a bit out of date,
or at least oversimplified.

One possible theory is that you had an "include" directive in
the config file in /etc, causing it to try to read the other one?

regards, tom lane

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Testing this, the problem appears to be that you forgot the
keyword"start", so pg_ctl didn't really do anything.

I am sorry, that was just a mistake on my part here, it is in the script.

I suspect this was left over from some previous attempt.

It doesn't look like it. I tried several times, always looked in the log
file after that. The timestamp also suggests it was a result of my attempts.

One possible theory is that you had an "include" directive in the config
file in /etc, causing it to try to read the other one?

I checked now, no "include" in there.

--
View this message in context: http://postgresql.nabble.com/Server-tries-to-read-a-different-config-file-than-it-is-supposed-to-tp5850752p5850763.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

On 05/23/2015 08:06 AM, Tom Lane wrote:

twoflower <standa.kurik@gmail.com> writes:

So I wrote a few scripts to make my life easier, e.g. *pg94start.sh*:
su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D
/storage/postgresql/9.4/data -o '-c
config_file=/etc/postgresql/9.4/main/postgresql.conf'"
But running this script did not work, the server would not start.

Testing this, the problem appears to be that you forgot the keyword
"start", so pg_ctl didn't really do anything. It's always a good
idea to redirect pg_ctl's stdout/stderr somewhere, so that you can
look at it in event of problems. (It will *not* magically go to the
server log file.)

So I
checked the log file and there was:
*FATAL: could not open file "/storage/postgresql/9.4/data/postgresql.conf":
Permission denied*

I suspect this was left over from some previous attempt.

After fixing the ownership of this file, it worked.

I can't explain that claim, but for me, -c config_file=something
seems to work as you'd expect, and a look at the server source
code confirms that it should honor that (cf SelectConfigFiles()).
I think the documentation Adrian pointed to is a bit out of date,
or at least oversimplified.

So order on the the command line is not important, the start up code
sets its own precedence?

One possible theory is that you had an "include" directive in
the config file in /etc, causing it to try to read the other one?

regards, tom lane

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Adrian Klaver <adrian.klaver@aklaver.com> writes:

On 05/23/2015 08:06 AM, Tom Lane wrote:

I can't explain that claim, but for me, -c config_file=something
seems to work as you'd expect, and a look at the server source
code confirms that it should honor that (cf SelectConfigFiles()).
I think the documentation Adrian pointed to is a bit out of date,
or at least oversimplified.

So order on the the command line is not important, the start up code
sets its own precedence?

Order on the command line would only matter if you wrote -c config_file
twice. The point here is that an explicit setting for config_file on the
command line overrides the default based on -D. Or at least it's supposed
to. I'm not sure now what's going wrong for the OP.

regards, tom lane

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

On 05/23/2015 08:23 AM, twoflower wrote:

Testing this, the problem appears to be that you forgot the keyword
"start", so pg_ctl didn't really do anything.

I am sorry, that was just a mistake on my part here, it is in the script.

I suspect this was left over from some previous attempt.

It doesn't look like it. I tried several times, always looked in the log
file after that. The timestamp also suggests it was a result of my
attempts.

So on my Ubuntu installs it does not set up the postgres user to allow login, so how are you getting to:

su postgres -c ...

Have you overridden the Ubuntu settings or created your own postgres user?

The reason I ask is that I can the below to work on Ubuntu 14.04:

aklaver@arkansas:~/postgres_test$ sudo su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D /home/aklaver/postgres_test/9.4/main/ -o '-c config_file=/etc/postgresql/9.4/main/postgresql.conf' start"
server starting

aklaver@arkansas:~/postgres_test$ 2015-05-23 14:02:15 PDT [20033-1] LOG: database system was shut down at 2015-05-23 13:49:04 PDT
2015-05-23 14:02:15 PDT [20037-1] LOG: autovacuum launcher started
2015-05-23 14:02:15 PDT [20032-1] LOG: database system is ready to accept connections

One possible theory is that you had an "include" directive in the
config file in /etc, causing it to try to read the other one?

I checked now, no "include" in there.
------------------------------------------------------------------------
View this message in context: Re: Server tries to read a different
config file than it is supposed to
<http://postgresql.nabble.com/Server-tries-to-read-a-different-config-file-than-it-is-supposed-to-tp5850752p5850763.html&gt;
Sent from the PostgreSQL - general mailing list archive
<http://postgresql.nabble.com/PostgreSQL-general-f1843780.html&gt; at
Nabble.com.

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Adrian Klaver <adrian.klaver@aklaver.com> writes:

So on my Ubuntu installs it does not set up the postgres user to allow login, so how are you getting to:
su postgres -c ...

From root, presumably ...

I thought of a different theory: maybe the server's complaint is not due
to trying to read that file as a config file, but it's just because there
is an unreadable/unwritable file in the data directory. See Christoph
Berg's complaint at
/messages/by-id/20150523172627.GA24277@msg.df7cb.de

This would only apply if the OP was trying to use this week's releases
though. Also, I thought the fsync-everything code would only run if
the server had been shut down uncleanly. Which maybe it was, but that
bit of info wasn't provided either.

regards, tom lane

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

On 05/23/2015 02:42 PM, Tom Lane wrote:

Adrian Klaver <adrian.klaver@aklaver.com> writes:

So on my Ubuntu installs it does not set up the postgres user to allow login, so how are you getting to:
su postgres -c ...

From root, presumably ...

Aargh, so used to the little voice telling me not to do things as root I
forgot that possibility.

I thought of a different theory: maybe the server's complaint is not due
to trying to read that file as a config file, but it's just because there
is an unreadable/unwritable file in the data directory. See Christoph
Berg's complaint at
/messages/by-id/20150523172627.GA24277@msg.df7cb.de

This would only apply if the OP was trying to use this week's releases
though. Also, I thought the fsync-everything code would only run if
the server had been shut down uncleanly. Which maybe it was, but that
bit of info wasn't provided either.

regards, tom lane

--
Adrian Klaver
adrian.klaver@aklaver.com

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

From root, presumably ...

Yes

I thought of a different theory: maybe the server's complaint is not due
to trying to read that file as a config file, but it's just because there
is an unreadable/unwritable file in the data directory. See Christoph
Berg's complaint at
/messages/by-id/20150523172627.GA24277@...This would
only apply if the OP was trying to use this week's releases though. Also,
I thought the fsync-everything code would only run if the server had been
shut down uncleanly. Which maybe it was, but that bit of info wasn't
provided either.

I was doing this after I upgraded to 9.4.2, yes. As for the shut down: I
suspect the server was rebooted without explicitly stopping Postgres. Not
sure how this plays out in terms of cleanliness. This is everything relevant
in the log file after I ran the start script:
2015-05-23 10:36:39.999 GMT [2102][0]: [1] LOG: database system was
interrupted; last known up at 2015-05-23 08:59:41 GMT
2015-05-23 10:36:40.053 GMT [2102][0]: [2] FATAL: could not open file
"/storage/postgresql/9.4/data/postgresql.conf": Permission denied
2015-05-23 10:36:40.054 GMT [2100][0]: [3] LOG: startup process (PID 2102)
exited with exit code 1
2015-05-23 10:36:40.054 GMT [2100][0]: [4] LOG: aborting startup due to
startup process failure
I also tried the same situation on two other Ubuntu servers with the same
version of Postgres (also upgraded to 9.4.2) and the same directory layout -
made *postgresql.conf* in the data directory unaccessible, even renamed it,
and everything worked fine. The only difference is that these are
streaming-replicated standby servers. They also had been restarted without
explicitly terminating Postgres.

--
View this message in context: http://postgresql.nabble.com/Server-tries-to-read-a-different-config-file-than-it-is-supposed-to-tp5850752p5850829.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.

twoflower <standa.kurik@gmail.com> writes:

I was doing this after I upgraded to 9.4.2, yes. As for the shut down: I
suspect the server was rebooted without explicitly stopping Postgres. Not
sure how this plays out in terms of cleanliness. This is everything relevant
in the log file after I ran the start script:
2015-05-23 10:36:39.999 GMT [2102][0]: [1] LOG: database system was
interrupted; last known up at 2015-05-23 08:59:41 GMT
2015-05-23 10:36:40.053 GMT [2102][0]: [2] FATAL: could not open file
"/storage/postgresql/9.4/data/postgresql.conf": Permission denied
2015-05-23 10:36:40.054 GMT [2100][0]: [3] LOG: startup process (PID 2102)
exited with exit code 1
2015-05-23 10:36:40.054 GMT [2100][0]: [4] LOG: aborting startup due to
startup process failure

Yeah, so this is long after the real config file has been read.

I think that that unwritable postgresql.conf file had probably been
hanging around in your data directory for some time. It was not causing
any particular problem until we decided we ought to fsync everything in
the data directory after a crash. So this is indeed the same case
Christoph was complaining about. But really you should remove that file
not just change its permissions; as is it's just causing confusion.

I also tried the same situation on two other Ubuntu servers with the same
version of Postgres (also upgraded to 9.4.2) and the same directory layout -
made *postgresql.conf* in the data directory unaccessible, even renamed it,
and everything worked fine. The only difference is that these are
streaming-replicated standby servers. They also had been restarted without
explicitly terminating Postgres.

Hm. I wonder why we aren't fsync'ing on crash restart on standby servers
as well.

regards, tom lane

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

I think that that unwritable postgresql.conf file had probably been
hanging around in your data directory for some time. It was not causing
any particular problem until we decided we ought to fsync everything in
the data directory after a crash. So this is indeed the same case
Christoph was complaining about. But really you should remove that file
not just change its permissions; as is it's just causing confusion.

Thank you very much Tom for explaining the cause, now I will make sure I am
not leaving any unused files in the data directory anymore.

--
View this message in context: http://postgresql.nabble.com/Server-tries-to-read-a-different-config-file-than-it-is-supposed-to-tp5850752p5850896.html
Sent from the PostgreSQL - general mailing list archive at Nabble.com.