PQstatus() detect change in connection...
Greetings,
PostgreSQL 7.1.3, FreeBSD-4.3-RELEASE, gcc 2.95.3
I'm trying to attempt to detect a failed backend connection, but a call to
PQstatus() always returns the state of the backend when the call was
made. For example, take this test code:
PGconn *pgConn;
PGresult *pgRes;
int fdPGconn;
int i = 0;
int iNewState = 0;
int iOldState = 60;
pgConn = PQconnectdb("dbname=pglogd user=postgres");
while ( i == 0 )
{
iNewState = PQstatus(pgConn);
if ( iNewState != iOldState )
{
iOldState = iNewState;
printf("Connection State [%d]\n", iNewState);
fdPGconn = PQsocket(pgConn);
printf("Connection Socket [%d]\n", fdPGconn);
}
sleep(1);
}
PQfinish(pgConn);
If you start this with the backend running, the status is CONNECTION_OK,
then pull the plug on the backend, the call to PQstatus() will still return
CONNECTION_OK, even though the backend is not running. Start this program
with the backend not running, then start the backend, PQstatus() never sees
the backend come to life...
Am I reading PQstatus() wrong? Is there any way to detect when the backend
goes down or comes back up?
Thanks,
Matthew
I presume you are trying to re-establish a connection automatically...if
that doesn't apply, ignore the rest of this email :)
The way I interpreted the docs was that you can use the return codes from
PQexec() to establish whether the command was sent to the backend correctly.
PQresultStatus() returns whether the command was syntactically
correct/executed OK.
I've attached a chunk of code from a back-end independent DB driver
(supports Oracle, PgSQL, MySQL through the same front end API), which
implements this auto-reconnect. Take a look at the sqlExec() method.
This code successfully recovers when used in a client connection pool in the
following sequence:
1) start postmaster
2) connect through pool/driver
3) issue SQL statements
4) kill postmaster
5) start postmaster
6) issue SQL statements
7) driver detects connection invalid, reconnects and re-issues
automatically.
Perhaps those infinitely more knowledgeable on the list have a better/more
correct way of doing things?
Cheers,
Mark Pritchard
Show quoted text
-----Original Message-----
From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org]On Behalf Of Matthew Hagerty
Sent: Thursday, 18 October 2001 10:47 AM
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] PQstatus() detect change in connection...Greetings,
PostgreSQL 7.1.3, FreeBSD-4.3-RELEASE, gcc 2.95.3
I'm trying to attempt to detect a failed backend connection, but
a call to
PQstatus() always returns the state of the backend when the call was
made. For example, take this test code:PGconn *pgConn;
PGresult *pgRes;
int fdPGconn;int i = 0;
int iNewState = 0;
int iOldState = 60;pgConn = PQconnectdb("dbname=pglogd user=postgres");
while ( i == 0 )
{
iNewState = PQstatus(pgConn);if ( iNewState != iOldState )
{
iOldState = iNewState;
printf("Connection State [%d]\n", iNewState);fdPGconn = PQsocket(pgConn);
printf("Connection Socket [%d]\n", fdPGconn);
}sleep(1);
}PQfinish(pgConn);
If you start this with the backend running, the status is CONNECTION_OK,
then pull the plug on the backend, the call to PQstatus() will
still return
CONNECTION_OK, even though the backend is not running. Start
this program
with the backend not running, then start the backend, PQstatus()
never sees
the backend come to life...Am I reading PQstatus() wrong? Is there any way to detect when
the backend
goes down or comes back up?Thanks,
Matthew---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
Attachments:
DBDriverPostgres.cppapplication/octet-stream; name=DBDriverPostgres.cppDownload
/***************************************************************************
DBDriverPostgres.cpp
Date Author Description
20001114 MP Created
20010529 OB Mods to make this TRULY part of TSAUtils
***************************************************************************/
#include "DBDriverPostgres.h"
TSAUTILS_NAMESPACE_BEGIN
/*
***** Internal function declarations
*/
// Trims trailing spaces from string
char * _rTrim(char *in_string);
/*
***** Internal function definitions
*/
// _rTrim()
char * _rTrim(char *in_string)
{
long lPos;
// Exit if null
if (!in_string)
return "";
// Find last char in string so we can loop backward
lPos = strlen(in_string)-1;
// Loop backward finding last non-space char
while ((lPos >= 0) && (in_string[lPos] == ' '))
lPos--;
// Move forward one place (back on to the spaces) and set null
lPos++;
in_string[lPos] = 0;
return in_string;
}
/*
***** DBDriverPostgres
*/
// DBDriverPostgres::_initResultSet()
void DBDriverPostgres::_initResultSet()
{
// Clear the column names
m_columnNames.clear();
// Free the result set
PQclear(m_resultSet);
m_resultSet = NULL;
// Indicate we did not return rows
m_blDidReturnRows = false;
// Reset the row number
m_lCurrentRow = -1;
}
// DBDriverPostgres::_validateFieldAccess()
int DBDriverPostgres::_validateFieldAccess(const string &in_sFldName)
{
int iFieldIndex;
// Check if we have rows
if (!didReturnRows())
throw TSAUTILS::xExceptionGeneric("No records returned");
// Check if the field name is valid and return the index if it is
iFieldIndex = PQfnumber(m_resultSet,in_sFldName.c_str());
if (iFieldIndex == -1)
throw TSAUTILS::xExceptionGeneric("Invalid field name %s",in_sFldName.c_str());
return iFieldIndex;
}
// DBDriverPostgres::colNames
vector <string> DBDriverPostgres::colNames()
{
// Check if we have rows
if (!didReturnRows())
throw TSAUTILS::xExceptionGeneric("Column names unavailable - query did not return rows");
// Return the list of column names
return m_columnNames;
}
// DBDriverPostgres::DBDriverPostgres
DBDriverPostgres::DBDriverPostgres(const char *pchHost, const char *pchDB, const char *pchUser, const char *pchPwd)
{
// Pre-init
m_resultSet = NULL;
// Init
_initResultSet();
// Build connect string
sprintf(m_pchTemp,"host=%s dbname=%s user=%s password=%s",pchHost,pchDB,pchUser,pchPwd);
// Connect to DB
m_connection = PQconnectdb(m_pchTemp);
if (PQstatus(m_connection) == CONNECTION_BAD)
throw TSAUTILS::xExceptionGeneric("Unable to connect to %s on %s. Error - %s",pchDB,pchHost,PQerrorMessage(m_connection));
}
// DBDriverPostgres::~DBDriverPostgres
DBDriverPostgres::~DBDriverPostgres()
{
// Free result set
_initResultSet();
// Free connection
PQfinish(m_connection);
}
// DBDriverPostgres::fldBool
bool DBDriverPostgres::fldBool(const string &in_sFldName)
{
// Return whether int <> 0
return fldInt(in_sFldName);
}
// DBDriverPostgres::fldDbl
double DBDriverPostgres::fldDbl(const string &in_sFldName)
{
int iFieldIndex = _validateFieldAccess(in_sFldName);
// Return field value
return atof(PQgetvalue(m_resultSet,m_lCurrentRow,iFieldIndex));
}
// DBDriverPostgres::fldDT
Date DBDriverPostgres::fldDT(const string &in_sFldName)
{
// Grab field index and data
int iFieldIndex = _validateFieldAccess(in_sFldName);
string sDateTime = PQgetvalue(m_resultSet,m_lCurrentRow,iFieldIndex);
// Trim off the timezone
size_t iTZstart = sDateTime.find('+');
if (iTZstart < sDateTime.length())
sDateTime = sDateTime.substr(0,iTZstart);
// Return a new date object
return Date(sDateTime);
}
// DBDriverPostgres::fldFlt
float DBDriverPostgres::fldFlt(const string &in_sFldName)
{
int iFieldIndex = _validateFieldAccess(in_sFldName);
// Return field value
return atof(PQgetvalue(m_resultSet,m_lCurrentRow,iFieldIndex));
}
// DBDriverPostgres::fldInt
int DBDriverPostgres::fldInt(const string &in_sFldName)
{
int iFieldIndex = _validateFieldAccess(in_sFldName);
// Return field value
return atoi(PQgetvalue(m_resultSet,m_lCurrentRow,iFieldIndex));
}
// DBDriverPostgres::fldLong
long DBDriverPostgres::fldLong(const string &in_sFldName)
{
int iFieldIndex = _validateFieldAccess(in_sFldName);
// Return field value
return atol(PQgetvalue(m_resultSet,m_lCurrentRow,iFieldIndex));
}
// DBDriverPostgres::fldStr
string DBDriverPostgres::fldStr(const string &in_sFldName)
{
string sRet = "";
int iFieldIndex = _validateFieldAccess(in_sFldName);
// Return field value
char *pchTmp = PQgetvalue(m_resultSet,m_lCurrentRow,iFieldIndex);
if (pchTmp)
sRet = _rTrim(pchTmp);
return sRet;
}
// DBDriverPostgres::didReturnRows
bool DBDriverPostgres::didReturnRows()
{
return m_blDidReturnRows;
}
// DBDriverPostgres::recordNext
bool DBDriverPostgres::recordNext()
{
// Return false if we didn't return any rows from the query or we are at end
if ((!didReturnRows()) || (m_lCurrentRow >= PQntuples(m_resultSet)))
return false;
// Move to the next row
m_lCurrentRow++;
return (m_lCurrentRow < PQntuples(m_resultSet));
}
// DBDriverPostgres::sqlExec
bool DBDriverPostgres::sqlExec(const char *in_pchSQL)
{
ExecStatusType execStatus;
int iFieldCount;
// Init
_initResultSet();
// Check if we have SQL
if (!in_pchSQL)
throw TSAUTILS::xExceptionGeneric("Unable to execute query - no SQL specified");
// Attempt to execute it
m_resultSet = PQexec(m_connection,in_pchSQL);
if (!m_resultSet)
{
// Query execution failed. Attempt to reconnect and re-execute
PQreset(m_connection);
m_resultSet = PQexec(m_connection,in_pchSQL);
}
// Check the result set status
execStatus = PQresultStatus(m_resultSet);
if ((!m_resultSet) || ((execStatus != PGRES_COMMAND_OK) && (execStatus != PGRES_TUPLES_OK)))
throw TSAUTILS::xExceptionGeneric("Error executing query [%s]. Error = %s",in_pchSQL,PQerrorMessage(m_connection));
// Exit if we executed a query that doesn't return rows
if (execStatus == PGRES_COMMAND_OK)
return didReturnRows();
// Exit if we executed a query that returns rows, but didn't in this case (select with no matching records)
if (PQntuples(m_resultSet) == 0)
return didReturnRows();
// We have data
m_blDidReturnRows = true;
// Init row counter
m_lCurrentRow = -1;
// Fetch the list of column names
iFieldCount = PQnfields(m_resultSet);
for (int i = 0; i < iFieldCount; i++)
m_columnNames.push_back(string(PQfname(m_resultSet,i)));
// Indicate we returned rows
return didReturnRows();
}
// DBDriverPostgres::transCommit
void DBDriverPostgres::transCommit()
{
_initResultSet();
// Start the transaction
m_resultSet = PQexec(m_connection, "COMMIT");
if ((!m_resultSet) || (PQresultStatus(m_resultSet) != PGRES_COMMAND_OK))
throw TSAUTILS::xExceptionGeneric("Unable to start transaction. Error - %s",PQerrorMessage(m_connection));
_initResultSet();
}
// DBDriverPostgres::transRollback
void DBDriverPostgres::transRollback()
{
_initResultSet();
// Start the transaction
m_resultSet = PQexec(m_connection, "ROLLBACK");
if ((!m_resultSet) || (PQresultStatus(m_resultSet) != PGRES_COMMAND_OK))
throw TSAUTILS::xExceptionGeneric("Unable to rollback transaction. Error - %s",PQerrorMessage(m_connection));
_initResultSet();
}
// DBDriverPostgres::transStart
void DBDriverPostgres::transStart()
{
_initResultSet();
// Start the transaction
m_resultSet = PQexec(m_connection, "BEGIN");
if ((!m_resultSet) || (PQresultStatus(m_resultSet) != PGRES_COMMAND_OK))
throw TSAUTILS::xExceptionGeneric("Unable to start transaction. Error - %s",PQerrorMessage(m_connection));
_initResultSet();
}
TSAUTILS_NAMESPACE_END
DBDriverPostgres.happlication/octet-stream; name=DBDriverPostgres.hDownload
#ifndef __TSAUTILS_DB_DRIVER_POSTGRES_H__
#define __TSAUTILS_DB_DRIVER_POSTGRES_H__
/***************************************************************************
DBDriverPostgres.h
DBDriverPostgres implements Postgres functionality
Date Author Description
20001114 MP Created
***************************************************************************/
#include <postgres/libpq-fe.h>
#include "DBDriverGeneric.h"
TSAUTILS_NAMESPACE_BEGIN
/*
***** DBDriverPostgres
*/
class DBDriverPostgres : public DBDriverGeneric
{
private:
// Whether the last query returned rows
bool m_blDidReturnRows;
// Connection to Postgres database
PGconn *m_connection;
// Result set from Postgres
PGresult *m_resultSet;
// Column names in the result set
vector <string> m_columnNames;
// Current row number
long m_lCurrentRow;
// Initialises result set data
void _initResultSet();
// Validates OK to access fields
int _validateFieldAccess(const string &in_sFldName);
public:
// Constructor/destructor
DBDriverPostgres(const char *pchHost, const char *pchDB, const char *pchUser, const char *pchPwd);
virtual ~DBDriverPostgres();
// Returns column names
vector <string> colNames();
// Retrieves field data
bool fldBool(const string &in_sFldName);
double fldDbl(const string &in_sFldName);
Date fldDT(const string &in_sFldName);
float fldFlt(const string &in_sFldName);
int fldInt(const string &in_sFldName);
long fldLong(const string &in_sFldName);
string fldStr(const string &in_sFldName);
// Record navigation
bool didReturnRows();
bool recordNext();
// SQL manipulation
bool sqlExec(const char *in_pchSQL);
// Transaction control
void transCommit();
void transRollback();
void transStart();
};
TSAUTILS_NAMESPACE_END
#endif
I am trying to re-establish a connection, however, I cannot afford to issue
a query to determine if the connection still exists. I'm writing a server
that uses the asynchronous query processing functions and speed is an
issue. Queries are slow compared to what the server does and it cannot
wait around for a query to finish just to see if another query *should* be
attempted based on the connection status.
I've been digging into the libpq code to see what is going on, maybe I can
gleam a little hint or two there... Anyone know a good *fast* way to test
if a socket is still valid?
Thanks,
Matthew
At 11:51 AM 10/18/2001 +1000, Mark Pritchard wrote:
Show quoted text
I presume you are trying to re-establish a connection automatically...if
that doesn't apply, ignore the rest of this email :)The way I interpreted the docs was that you can use the return codes from
PQexec() to establish whether the command was sent to the backend correctly.
PQresultStatus() returns whether the command was syntactically
correct/executed OK.I've attached a chunk of code from a back-end independent DB driver
(supports Oracle, PgSQL, MySQL through the same front end API), which
implements this auto-reconnect. Take a look at the sqlExec() method.This code successfully recovers when used in a client connection pool in the
following sequence:1) start postmaster
2) connect through pool/driver
3) issue SQL statements
4) kill postmaster
5) start postmaster
6) issue SQL statements
7) driver detects connection invalid, reconnects and re-issues
automatically.Perhaps those infinitely more knowledgeable on the list have a better/more
correct way of doing things?Cheers,
Mark Pritchard
-----Original Message-----
From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org]On Behalf Of Matthew Hagerty
Sent: Thursday, 18 October 2001 10:47 AM
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] PQstatus() detect change in connection...Greetings,
PostgreSQL 7.1.3, FreeBSD-4.3-RELEASE, gcc 2.95.3
I'm trying to attempt to detect a failed backend connection, but
a call to
PQstatus() always returns the state of the backend when the call was
made. For example, take this test code:PGconn *pgConn;
PGresult *pgRes;
int fdPGconn;int i = 0;
int iNewState = 0;
int iOldState = 60;pgConn = PQconnectdb("dbname=pglogd user=postgres");
while ( i == 0 )
{
iNewState = PQstatus(pgConn);if ( iNewState != iOldState )
{
iOldState = iNewState;
printf("Connection State [%d]\n", iNewState);fdPGconn = PQsocket(pgConn);
printf("Connection Socket [%d]\n", fdPGconn);
}sleep(1);
}PQfinish(pgConn);
If you start this with the backend running, the status is CONNECTION_OK,
then pull the plug on the backend, the call to PQstatus() will
still return
CONNECTION_OK, even though the backend is not running. Start
this program
with the backend not running, then start the backend, PQstatus()
never sees
the backend come to life...Am I reading PQstatus() wrong? Is there any way to detect when
the backend
goes down or comes back up?Thanks,
Matthew---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
Matthew Hagerty <mhagerty@voyager.net> writes:
Anyone know a good *fast* way to test
if a socket is still valid?
What exactly are you trying to defend against?
In general, I don't believe that there is any way of discovering whether
the server is still up, other than to send it a query. (FWIW, an empty
query string bounces back very quickly, with little processing.)
For particular scenarios it's possible that some notification has been
delivered to the client, but if you have had (say) a loss of network
connectivity then there just is no other alternative. Your end isn't
going to discover the connectivity loss until it tries to send a
message.
regards, tom lane
At 02:10 PM 10/18/2001 -0400, Tom Lane wrote:
Matthew Hagerty <mhagerty@voyager.net> writes:
Anyone know a good *fast* way to test
if a socket is still valid?What exactly are you trying to defend against?
In general, I don't believe that there is any way of discovering whether
the server is still up, other than to send it a query. (FWIW, an empty
query string bounces back very quickly, with little processing.)For particular scenarios it's possible that some notification has been
delivered to the client, but if you have had (say) a loss of network
connectivity then there just is no other alternative. Your end isn't
going to discover the connectivity loss until it tries to send a
message.regards, tom lane
I was using PQstatus() under the assumption that it actually *checked* the
connection, however I have since discovered that is simply returns the
value in a structure, and that value only gets updated in pqReadData() or
pqReadReady() (both of which are internal function calls.)
What I'm doing is using the asynchronous processing to write a server that
does not have to wait around for a query to finish (which is a slow process
compared to what the rest of the server does.) So, using a query to test
if the connection is up seems rather redundant and slow... I was hoping to
come up with a faster more simple solution. If the connection is down I
need to write - what would have been a query - to a temporary place and
attempt a reconnect, all while going off and doing other things.
This all came about when my main select() bailed because the backend went
down and the socket's file-descriptor became invalid. I could probably
catch the error in that loop, but I also want to check the connection
*before* submitting a query... Basically, I hope to avoid a huge rewrite
based on my assumption of how PQstatus() was actually working. ;-)
Currently I'm looking at fnctl() or a dedicated select() call (similar to
what pgReadReady() does), but I'm not sure of the OS overhead of these
solutions compared to each other or an empty query. Any insight would be
greatly appreciated.
Thanks,
Matthew
Matthew Hagerty writes:
I am trying to re-establish a connection, however, I cannot afford to issue
a query to determine if the connection still exists.
But requesting that the server do something *is* the only way to know
whether it's still alive. Another question to ask, of course, would be,
why is your server always going down?
I've been digging into the libpq code to see what is going on, maybe I can
gleam a little hint or two there... Anyone know a good *fast* way to test
if a socket is still valid?
Try to send or receive something.
--
Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
Matthew Hagerty <mhagerty@voyager.net> writes:
but I also want to check the connection
*before* submitting a query...
This strikes me as utterly pointless. You'll need to be able to recover
from query failure anyway, so what's the value of testing beforehand?
Send the query and see if it works or not.
regards, tom lane
Matthew Hagerty <mhagerty@voyager.net> writes:
but I also want to check the connection *before* submitting a
query...
If you mean directly before the query, then forget it, as Tom already
said :-)
This strikes me as utterly pointless. You'll need to be able to
recover
from query failure anyway, so what's the value of testing beforehand?
Send the query and see if it works or not.
I see a value in checking connection status before you start doing
loads of local work after a long idle time, that results in a query.
In this situation I guess it is good enough to send an empty query
even if it takes a little.
In our projects we recv 0 bytes from the socket every x seconds
during long idle periods to detect connection problems early.
While it is not 100% reliable (since it does not transfer
anything over the network) it does detect some common error situations.
I am not 100% sure, but I think PQstatus could be patched to do that.
Andreas
Import Notes
Resolved by subject fallback