Postgres won't start. Nothing in the log.

Hi!

pgsql-general-owner@postgresql.org wrote:

Unfortunately there is nothing anywhere telling me what the
problem is. The log file is empty, there is nothing in the
/var/log/messages or /var/log/syslog either. The only output
I get is this.

* Starting PostgreSQL 8.3 database server
* The PostgreSQL server failed to start. Please check the log output.
What can I do to figure out why it won't start?

Did you check if the directory PostgreSQL wants to write the logfile to
has appropriate rights to allow the user PostgreSQL runs under to write
to it? Another fairly common issue would be a postgresql.conf that is
not readable by the PostgreSQL-user. In case of such an outright failure
to start, it's usually something fairly simple to fix such as file or
directory permissions.

Kind regards

Markus

Computec Media AG
Sitz der Gesellschaft und Registergericht: Furth (HRB 8818)
Vorstandsmitglieder: Albrecht Hengstenberg (Vorsitzender) und Rainer Rosenbusch
Vorsitzender des Aufsichtsrates: Jurg Marquard
Umsatzsteuer-Identifikationsnummer: DE 812 575 276

2009/10/5 Markus Wollny <Markus.Wollny@computec.de>

Hi!

pgsql-general-owner@postgresql.org wrote:

Unfortunately there is nothing anywhere telling me what the
problem is. The log file is empty, there is nothing in the
/var/log/messages or /var/log/syslog either. The only output
I get is this.

* Starting PostgreSQL 8.3 database server
* The PostgreSQL server failed to start. Please check the log output.
What can I do to figure out why it won't start?

Was it a minor-release upgrade? (i.e. 8.3.1 to 8.3.7 and not 8.2. to 8.3)
If not, you would have needed to do a complete dump and restore of your
data.

To force a log file, try pg_ctl start -l /path/to/logfile

You might as well use /tmp/pg.log for the logfile for diagnosis. This
should tell you what the problem is.

Thom

"Markus Wollny" <Markus.Wollny@computec.de> writes:

pgsql-general-owner@postgresql.org wrote:

Unfortunately there is nothing anywhere telling me what the
problem is. The log file is empty, there is nothing in the
/var/log/messages or /var/log/syslog either. The only output
I get is this.

* Starting PostgreSQL 8.3 database server
* The PostgreSQL server failed to start. Please check the log output.
What can I do to figure out why it won't start?

Did you check if the directory PostgreSQL wants to write the logfile to
has appropriate rights to allow the user PostgreSQL runs under to write
to it? Another fairly common issue would be a postgresql.conf that is
not readable by the PostgreSQL-user. In case of such an outright failure
to start, it's usually something fairly simple to fix such as file or
directory permissions.

As Markus suggests, it's highly unlikely that Postgres is failing
without any message whatsoever. What is more likely is that the message
is being sent somewhere you're not looking, or there is a permissions
problem or something with where it was told to send it. Take a look at
the init script you're using to start the server, and see where it sends
the postmaster's stderr. You need to check that in case of failures
that occur before the configured log destination has taken effect.
(I have seen some scripts that redirect stderr to /dev/null :-( ...
if you have one of those, change it, and file a bug report with the
packager.)

regards, tom lane

On Mon, 2009-10-05 at 17:39 +1300, Tim Uckun wrote:

Unfortunately there is nothing anywhere telling me what the problem
is. The log file is empty, there is nothing in the /var/log/messages
or /var/log/syslog either. The only output I get is this.

* Starting PostgreSQL 8.3 database server
* The PostgreSQL server failed to start. Please check the log output.

Is this Debian/Ubuntu? If so, the logs should be
at /var/log/postgresql/postgresql-8.3-main.log (or so).
--
Devrim GÜNDÜZ, RHCE
Command Prompt - http://www.CommandPrompt.com
devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz

did you install (i should say register) postgres and pg_ctl binaries as postgres user
are you executing postgres and pg_ctl binaries as postgres user
?
Martin Gainty
______________________________________________
Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité

Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité pour le contenu fourni.

Subject: Re: [GENERAL] Postgres won't start. Nothing in the log.
From: devrim@gunduz.org
To: timuckun@gmail.com
CC: pgsql-general@postgresql.org
Date: Mon, 5 Oct 2009 17:16:00 +0300

On Mon, 2009-10-05 at 17:39 +1300, Tim Uckun wrote:

Unfortunately there is nothing anywhere telling me what the problem
is. The log file is empty, there is nothing in the /var/log/messages
or /var/log/syslog either. The only output I get is this.

* Starting PostgreSQL 8.3 database server
* The PostgreSQL server failed to start. Please check the log output.

Is this Debian/Ubuntu? If so, the logs should be
at /var/log/postgresql/postgresql-8.3-main.log (or so).
--
Devrim GÜNDÜZ, RHCE
Command Prompt - http://www.CommandPrompt.com
devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz

_________________________________________________________________
Hotmail: Free, trusted and rich email service.
http://clk.atdmt.com/GBL/go/171222984/direct/01/

Tim Uckun <timuckun@gmail.com> wrote:

Unfortunately there is nothing anywhere telling me what the problem
is. The log file is empty, there is nothing in the /var/log/messages
or /var/log/syslog either. The only output I get is this.

* Starting PostgreSQL 8.3 database server
* The PostgreSQL server failed to start. Please check the log output.

What can I do to figure out why it won't start?

I know, from IRC, the problem has been solved, there was no space on the
disk ...

Unfortunately, i haven't logs.

Andreas
--
Really, I'm not out to destroy Microsoft. That will just be a completely
unintentional side effect. (Linus Torvalds)
"If I was god, I would recompile penguin with --enable-fly." (unknown)
Kaufbach, Saxony, Germany, Europe. N 51.05082�, E 13.56889�

I know, from IRC, the problem has been solved, there was no space on the
disk ...

Unfortunately, i haven't logs.

Yes. Thanks to everybody on the IRC who helped me out.

The suggestion that was most helpful was to call the posgres binary
directly. /usr/lib/postgresql/8.3/bin/postgres. Calling the
/etc/init.d postgres starter gave me no information at all but calling
the binary told me what the problem was.

In a nutshell somehow the restoring of the log files stopped and the
disk started filling up with the unprocessed log files. Eventually the
disk filled up.

I have since set up monitoring the disk space and the postgres
replication lag using zabbix. The zabbix agent reports the disk space
and I report the replication lag using a ruby script.

In case anybody is using zabbix here is the script.

#! /usr/local/bin/ruby

require 'date'
require 'yaml'
require 'socket'
require 'timeout'
require "base64"

def send_item_to_zabbix (server, port, host, key, value)
response = ""
request = "<req><host>"
request << Base64.encode64(host)
request << "</host><key>"
request << Base64.encode64(key)
request << "</key><data>"
request << Base64.encode64(value)
request << "</data></req>"

status = Timeout::timeout(30){
socket = TCPSocket.new(server, port)
socket.puts request
response = socket.gets
if response == "OK" || response.nil?
response = ""
else
response = "Monitor rejected key='#{key}' value='#{value}'
host='#{host}' with response #{response}"
end
} #timeout
return response
rescue Timeout::Error
return "can't connect to monitoring server at #{server}:#{port}"
rescue
return $!
end #send item to zabbix

output =`/usr/lib/postgresql/8.3/bin/pg_controldata
/var/lib/postgresql/8.3/main`

data = YAML.load output
last_checkpoint = DateTime.parse( data['Time of latest checkpoint'])
hours, mins, secs, ignore_fractions =
Date::day_fraction_to_time(DateTime.now - last_checkpoint)

lag = hours * 60 * 60 + mins * 60 + secs
output = send_item_to_zabbix 'my.zabbix..host.com', '10051',
'postgres_host_name'', 'postgres.replication.lag', lag.to_s

~