BUG #2724: Could not check connection status with "ssl=on"

Started by Alexey Zayatsover 19 years ago8 messageshackersbugs
Jump to latest
#1Alexey Zayats
alexey.zayats@gmail.com
hackersbugs

The following bug has been logged online:

Bug reference: 2724
Logged by: Alexey Zayats
Email address: alexey.zayats@gmail.com
PostgreSQL version: 8.1.5
Operating system: SuSE 10.0 Linux 2.6.13 x86-64
Description: Could not check connection status with "ssl=on"
Details:

I'm wrote a programm using libpq.
Postgres(8.1.5) build and configured with ssl(OpenSSL 0.9.7g 11 Apr 2005)
support.

Os: SuSE Linux 10.0
Kernel: 2.6.13
Arch: x86-64

In case until postgresql backend restarted, all is work fine, but if it
happens, i could not get correct connection status,
programm recieve SIGPIPE and crush.

I've got next backtrace from core file:
--------------------------
Program received signal SIGPIPE, Broken pipe.
[Switching to Thread 46912549615488 (LWP 28743)]
0x00002aaaad07ae32 in __write_nocancel () from /lib64/tls/libpthread.so.0
(gdb) bt
#0  0x00002aaaad07ae32 in __write_nocancel () from
/lib64/tls/libpthread.so.0
#1  0x00002aaaad9e2187 in BIO_sock_should_retry ()
from /usr/lib64/libcrypto.so.0.9.7
#2  0x00002aaaad9e0751 in BIO_write () from /usr/lib64/libcrypto.so.0.9.7
#3  0x00002aaaad82b8a4 in ssl3_change_cipher_state ()
from /usr/lib64/libssl.so.0.9.7
#4  0x00002aaaad82b9a2 in ssl3_dispatch_alert ()
from /usr/lib64/libssl.so.0.9.7
#5  0x00002aaaad82a084 in ssl3_shutdown () from /usr/lib64/libssl.so.0.9.7
#6  0x00002aaaaaec4095 in close_SSL () from /usr/local/lib/libpq.so.4
#7  0x00002aaaaaebd408 in pqReadData () from /usr/local/lib/libpq.so.4
#8  0x00002aaaaaebb132 in PQgetResult () from /usr/local/lib/libpq.so.4
#9  0x00002aaaaaebb20e in PQexecFinish () from /usr/local/lib/libpq.so.4
#10 0x00002aaaaabee6b4 in QPSQLResult::reset (this=0x840d10,
query=@0x7fffffcc17d0) at qsql_psql.cpp:332
#11 0x00002aaaaad4c014 in QSqlQuery::exec (this=0x7fffffcc17b0,  
query=@0x7fffffcc17d0) at qsqlquery.cpp:355
--

Before any queries send, i'm using isOpen method that looks like:
--
bool QPSQLDriver::isOpen() const
{
        return PQstatus(d->connection) == CONNECTION_OK;
}
--

if "ssl=on", PQstatus(d->connection) always return CONNECTION_OK, even if
connection is closed by postgresql

if "ssl=off" in postgresql.conf, all wroks fine.
Using sigaction impossible, because i'm wrote on c++ with Qt library as a
base. Qt library itself used libpq as interface to postgresql.

Is there a bug on libpq?

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Alexey Zayats (#1)
hackersbugs
Re: BUG #2724: Could not check connection status with "ssl=on"

"Alexey Zayats" <alexey.zayats@gmail.com> writes:

I've got next backtrace from core file:
--------------------------
Program received signal SIGPIPE, Broken pipe.
[Switching to Thread 46912549615488 (LWP 28743)]
0x00002aaaad07ae32 in __write_nocancel () from /lib64/tls/libpthread.so.0
(gdb) bt
#0  0x00002aaaad07ae32 in __write_nocancel () from
/lib64/tls/libpthread.so.0
#1  0x00002aaaad9e2187 in BIO_sock_should_retry ()
from /usr/lib64/libcrypto.so.0.9.7
#2  0x00002aaaad9e0751 in BIO_write () from /usr/lib64/libcrypto.so.0.9.7
#3  0x00002aaaad82b8a4 in ssl3_change_cipher_state ()
from /usr/lib64/libssl.so.0.9.7
#4  0x00002aaaad82b9a2 in ssl3_dispatch_alert ()
from /usr/lib64/libssl.so.0.9.7
#5  0x00002aaaad82a084 in ssl3_shutdown () from /usr/lib64/libssl.so.0.9.7
#6  0x00002aaaaaec4095 in close_SSL () from /usr/local/lib/libpq.so.4
#7  0x00002aaaaaebd408 in pqReadData () from /usr/local/lib/libpq.so.4

I would argue that this is an OpenSSL bug: it should not be trying to
write on a connection that it knows perfectly well is already dead.
(It should know that, anyway, because the only way that pqReadData would
be trying to close the connection is that it got an error indication
from OpenSSL.)

Possibly we could work around the problem by disabling SIGPIPE during
connection close, but I don't really see why we should have to do that.

Before any queries send, i'm using isOpen method that looks like:
--
bool QPSQLDriver::isOpen() const
{
        return PQstatus(d->connection) == CONNECTION_OK;
}

That's pretty much a waste of time, because all it tells you is whether
the connection was good the last time a query was done. It is *not*
intended as an active "ping".

regards, tom lane

#3Alexey Zayats
alexey.zayats@gmail.com
In reply to: Tom Lane (#2)
hackersbugs
Re: BUG #2724: Could not check connection status with "ssl=on"

Hi.

I would argue that this is an OpenSSL bug: it should not be trying to
write on a connection that it knows perfectly well is already dead.
(It should know that, anyway, because the only way that pqReadData would
be trying to close the connection is that it got an error indication
from OpenSSL.)

May be, may be...

Possibly we could work around the problem by disabling SIGPIPE during
connection close, but I don't really see why we should have to do that.

While take a look at source of libpq, i have discover following:
while reading from pipe, you are getting
case SSL_ERROR_ZERO_RETURN:
SOCK_ERRNO_SET(ECONNRESET);
but why you'r do not check
SOCK_ERRNO != ECONNRESET
while closing ssl connection ?

i was trying this and all is work fine.

In function close_SSL you are call SSL_shutdown to shutdown ssl pipe.
But if you are already get ECONNRESET (by pear?), why you call whi funtcion?

From openssl docs.
SSL_shutdown - shuts down an active TLS/SSL connection. It sends the ``close
notify'' shutdown alert to the peer.

That's why i've got SIGPIPE.

That's pretty much a waste of time, because all it tells you is whether
the connection was good the last time a query was done. It is *not*
intended as an active "ping".

Ok, i'll take it in my mind.

Alexey Zayats.

#4Bruce Momjian
bruce@momjian.us
In reply to: Alexey Zayats (#3)
hackersbugs
Re: [BUGS] BUG #2724: Could not check connection status with "ssl=on"

Based on this report, I have developed the attached patch. Is this OK?

The idea is not to call SSL_shutdown() if errno == ECONNRESET.

---------------------------------------------------------------------------

�������������� �������� wrote:

Hi.

I would argue that this is an OpenSSL bug: it should not be trying to
write on a connection that it knows perfectly well is already dead.
(It should know that, anyway, because the only way that pqReadData would
be trying to close the connection is that it got an error indication
from OpenSSL.)

May be, may be...

Possibly we could work around the problem by disabling SIGPIPE during
connection close, but I don't really see why we should have to do that.

While take a look at source of libpq, i have discover following:
while reading from pipe, you are getting
case SSL_ERROR_ZERO_RETURN:
SOCK_ERRNO_SET(ECONNRESET);
but why you'r do not check
SOCK_ERRNO != ECONNRESET
while closing ssl connection ?

i was trying this and all is work fine.

In function close_SSL you are call SSL_shutdown to shutdown ssl pipe.
But if you are already get ECONNRESET (by pear?), why you call whi funtcion?

From openssl docs.

SSL_shutdown - shuts down an active TLS/SSL connection. It sends the ``close
notify'' shutdown alert to the peer.

That's why i've got SIGPIPE.

That's pretty much a waste of time, because all it tells you is whether
the connection was good the last time a query was done. It is *not*
intended as an active "ping".

Ok, i'll take it in my mind.

Alexey Zayats.

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Attachments:

/pgpatches/ssltext/x-diffDownload+3-3
#5Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#4)
hackersbugs
Re: Re: [BUGS] BUG #2724: Could not check connection status with "ssl=on"

Bruce Momjian <bruce@momjian.us> writes:

! if (SOCK_ERRNO != ECONNRESET)
! SSL_shutdown(conn->ssl);

Ummm ... what is this supposed to fix exactly, and what are the odds
that it will introduce resource leaks?

regards, tom lane

#6Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#5)
hackersbugs
Re: Re: [BUGS] BUG #2724: Could not check connection status with "ssl=on"

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

! if (SOCK_ERRNO != ECONNRESET)
! SSL_shutdown(conn->ssl);

Ummm ... what is this supposed to fix exactly, and what are the odds

I think the user was getting SIGPIPE on SSL_shutdown() of a closed
connection.

that it will introduce resource leaks?

I don't know myself. If some SSL expert says it is OK, we apply it, if
not, we ignore the patch.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

#7Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#6)
hackersbugs
Re: Re: [BUGS] BUG #2724: Could not check connection status with "ssl=on"

Bruce Momjian <bruce@momjian.us> writes:

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

! if (SOCK_ERRNO != ECONNRESET)
! SSL_shutdown(conn->ssl);

Ummm ... what is this supposed to fix exactly, and what are the odds

I think the user was getting SIGPIPE on SSL_shutdown() of a closed
connection.

It seems moderately improbable that by the time control arrives here,
errno still has anything to do with the last operation on the SSL
socket.

regards, tom lane

#8Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#7)
hackersbugs
Re: [PATCHES] Re: [BUGS] BUG #2724: Could not check connection status with "ssl=on"

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

! if (SOCK_ERRNO != ECONNRESET)
! SSL_shutdown(conn->ssl);

Ummm ... what is this supposed to fix exactly, and what are the odds

I think the user was getting SIGPIPE on SSL_shutdown() of a closed
connection.

It seems moderately improbable that by the time control arrives here,
errno still has anything to do with the last operation on the SSL
socket.

Yep, I was wondering that too. It is called SOCK_ERRNO, but in fact it
is just errno on Unix.

I generated the patch just to try to give an example of what the user
might be suggesting. Let's see if anyone wants to research this more.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +