Supporting Windows SChannel as OpenSSL replacement
Hi,
I've been looking at Windows' native SSL implementatation, the SChannel
API. It would be nice to support that as a replacement for OpenSSL on
Windows. Currently, we bundle the OpenSSL library in the PostgreSQL,
installers, which is annoying because whenever OpenSSL puts out a new
release that fixes vulnerabilities, we need to do a security release of
PostgreSQL on Windows. I was reminded of this recently wrt. psqlODBC,
which bundles libpq and openssl as well. It's particularly annoying for
psqlODBC and other client applications, as people typically update it
less diligently than their servers.
I think that we should keep the user-visible behavior the same, i.e. the
libpq connection options, locations of the certificate files etc. would
all be the same regardless of which SSL implementation is used. Using
Windows SChannel API might make it possible to integrate better with
Windows' own certificate store etc. but I don't really know much about
that stuff, so for starters I'd like to just use it as a drop-in
replacement for OpenSSL.
Thoughts? While we're at it, we'll probably want to refactor things so
that it's easy to support other SSL implementations too, like gnutls.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Monday, June 9, 2014, Heikki Linnakangas <hlinnakangas@vmware.com> wrote:
Hi,
I've been looking at Windows' native SSL implementatation, the SChannel
API. It would be nice to support that as a replacement for OpenSSL on
Windows. Currently, we bundle the OpenSSL library in the PostgreSQL,
installers, which is annoying because whenever OpenSSL puts out a new
release that fixes vulnerabilities, we need to do a security release of
PostgreSQL on Windows. I was reminded of this recently wrt. psqlODBC, which
bundles libpq and openssl as well. It's particularly annoying for psqlODBC
and other client applications, as people typically update it less
diligently than their servers.I think that we should keep the user-visible behavior the same, i.e. the
libpq connection options, locations of the certificate files etc. would all
be the same regardless of which SSL implementation is used. Using Windows
SChannel API might make it possible to integrate better with Windows' own
certificate store etc. but I don't really know much about that stuff, so
for starters I'd like to just use it as a drop-in replacement for OpenSSL.Thoughts? While we're at it, we'll probably want to refactor things so
that it's easy to support other SSL implementations too, like gnutls.
It's a project that many have started, and nobody has finished :) I'm
definitely interested in working on such a things, but I've been unable to
carve out enough time recently.
One problem is as you say, that we're exposing openssl too much. For one
thing, we *cannot* keep the current interface, because it returns OpenSSL
internal datastructures. Those functions will need to be deprecated and
replaced with something else.
Also, my memory says that SChannel doesn't support the key file format that
we use now, which makes a much bigger break with the supported platforms.
That may have changed of course - have you researched that part?
The main other entries I've been looking at are NSS and gnutls, both of
which can speak our current file formats. I think the right thing is to
start with those and thereby make it more pluggable, and only after that
tackle schannel. But I do think it would be good to have them all.
It's also a question of if we can accept supporting a different set of
libraries on the server vs on the client. It's really on the client that
it's a bigger problem, but in the end I think we want to have "symmetrical
support". But it might be worth doing just the client side initially, and
then move to the server. I think in general, the client side is actually
likely to be *harder* than the server side..
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 06/09/2014 02:53 PM, Magnus Hagander wrote:
Also, my memory says that SChannel doesn't support the key file format that
we use now, which makes a much bigger break with the supported platforms.
That may have changed of course - have you researched that part?
A quick web search turned up a few discussion forums threads with a
recipe for this (e.g
https://stackoverflow.com/questions/1231178/load-an-x509-pem-file-into-windows-cryptoapi).
There's no direct "read this file" function, but there are low-level
functions that can decode the file format once it's read into memory. So
it seems possible to make it work.
It's also a question of if we can accept supporting a different set of
libraries on the server vs on the client. It's really on the client that
it's a bigger problem, but in the end I think we want to have "symmetrical
support". But it might be worth doing just the client side initially, and
then move to the server. I think in general, the client side is actually
likely to be *harder* than the server side..
Once we've modified the client to support multiple libraries, it's
probably not much extra effort to do the same to the server. I wouldn't
like to support different libraries in client and server, if only
because it would be more complicated to have separate ./configure
options for client and server.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 2014-06-09 13:53:15 +0200, Magnus Hagander wrote:
The main other entries I've been looking at are NSS and gnutls, both of
which can speak our current file formats. I think the right thing is to
start with those and thereby make it more pluggable, and only after that
tackle schannel. But I do think it would be good to have them all.
I think NSS makes a great deal of sense - the advantages of supporting
gnutls are much less clear to me. Maybe it's little enough additional
code that that doesn't matter much, but we imo shouldn't focus on it.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Jun 09, 2014 at 02:45:08PM +0300, Heikki Linnakangas wrote:
Thoughts? While we're at it, we'll probably want to refactor things
so that it's easy to support other SSL implementations too, like
gnutls.
One project that is proud to support several SSL implementations
is curl: http://curl.haxx.se/
Git: https://github.com/bagder/curl.git
Implementations: https://github.com/bagder/curl/tree/master/lib/vtls
List from vtls.c:
- OpenSSL
- GnuTLS
- NSS
- QsoSSL
- GSKit
- PolarSSL
- CyaSSL
- Schannel SSPI
- SecureTransport (Darwin)
We cannot reuse the code directly, but seems it's usable for
reference for various gotchas that need to be solved.
--
marko
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 06/09/2014 01:45 PM, Heikki Linnakangas wrote:
Thoughts? While we're at it, we'll probably want to refactor things so
that it's easy to support other SSL implementations too, like gnutls.
There was a patch set for this from Martijn van Oosterhout which was
quite complete.
/messages/by-id/20060504134807.GK4752@svana.org
I am interested in dropping the dependency on OpenSSL, if only to fix
the situation with Debian, libreadline and OpenSSL[1].
Notes
1. They now compile against libedit and change to using libreadline at
runtime. This does not work perfectly though since libreadline supports
some features which libedit does not which can only be checked for at
compile time.
Andreas
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Jun 9, 2014 at 3:19 PM, Andreas Karlsson <andreas@proxel.se> wrote:
On 06/09/2014 01:45 PM, Heikki Linnakangas wrote:
Thoughts? While we're at it, we'll probably want to refactor things so
that it's easy to support other SSL implementations too, like gnutls.There was a patch set for this from Martijn van Oosterhout which was quite
complete.
A lot has, unfortunately, changed since 2006. It might be a good
startingpoint. But also actively starting from the point of "let's try to
support multiple libraries" rather than "let's try to support gnutls" is
probably also important.
I am interested in dropping the dependency on OpenSSL, if only to fix the
situation with Debian, libreadline and OpenSSL[1].
That's one of the many reasons, yes :)
At some point we should design a new API, so that we can deprecate the old
one. Even if we don't hve the code ready, we need to get rid of PQgetssl(),
and replace it with something else. I'm thinking probably a functoin that
returns both a void pointer and an enum that tells you which library is
actually in use. And a boolean just saying "ssl on/off", because that's
what a lot of clients are interested in and they don't care aobut more than
that.
Obviously, we also have to do something about PQinitOpenSSL().
Unfortunately, I think it's too late to do that for 9.4 - otherwise it
would've been good to have a whole cycle of deprecation on it...
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On Mon, Jun 9, 2014 at 3:02 PM, Marko Kreen <markokr@gmail.com> wrote:
On Mon, Jun 09, 2014 at 02:45:08PM +0300, Heikki Linnakangas wrote:
Thoughts? While we're at it, we'll probably want to refactor things
so that it's easy to support other SSL implementations too, like
gnutls.One project that is proud to support several SSL implementations
is curl: http://curl.haxx.se/Git: https://github.com/bagder/curl.git
Implementations: https://github.com/bagder/curl/tree/master/lib/vtlsList from vtls.c:
- OpenSSL
- GnuTLS
- NSS
- QsoSSL
- GSKit
- PolarSSL
- CyaSSL
- Schannel SSPI
- SecureTransport (Darwin)We cannot reuse the code directly, but seems it's usable for
reference for various gotchas that need to be solved.
I did actually talk to Daniel at some point about turning that into a
generalized library, and/or getting him interested in helping out with it.
I can't remember where that ended up - I'll see if I can poke his interest
:)
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Heikki Linnakangas <hlinnakangas@vmware.com> writes:
I've been looking at Windows' native SSL implementatation, the SChannel
API. It would be nice to support that as a replacement for OpenSSL on
Windows. Currently, we bundle the OpenSSL library in the PostgreSQL,
installers, which is annoying because whenever OpenSSL puts out a new
release that fixes vulnerabilities, we need to do a security release of
PostgreSQL on Windows.
Does SChannel have a better security track record than OpenSSL? Or is
the point here just that we can define it as not our problem when a
vulnerability surfaces?
I'm doubtful that we can ignore security issues affecting PG just because
somebody else is responsible for shipping the fix, and thus am concerned
that if we support N different SSL libraries, we will need to keep track
of N sets of vulnerabilities instead of just one.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi,
On 2014-06-09 10:18:40 -0400, Tom Lane wrote:
Does SChannel have a better security track record than OpenSSL? Or is
the point here just that we can define it as not our problem when a
vulnerability surfaces?
Well, it's patched as part of the OS - so no new PG binaries have to be
released when it's buggy.
I'm doubtful that we can ignore security issues affecting PG just because
somebody else is responsible for shipping the fix, and thus am concerned
that if we support N different SSL libraries, we will need to keep track
of N sets of vulnerabilities instead of just one.
In most of the cases where such a issue exists it'll primarily affect
binary distributions that include the ssl library - and those will only
pick one anyway.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
From: "Heikki Linnakangas" <hlinnakangas@vmware.com>
Thoughts? While we're at it, we'll probably want to refactor things so
that it's easy to support other SSL implementations too, like gnutls.
That may be good because it provides users with choices. But I wonder if it
is worth the complexity and maintainability of PostgreSQL code.
* Are SChannel and other libraries more secure than OpenSSL? IIRC, recently
I read in the news that GnuTLS had a vulnerability. OpenSSL is probably the
most widely used library, and many people are getting more interested in its
quality. I expect the quality will improve thanks to the help from The
Linux foundation and other organizations/researchers.
* Do other libraries get support from commercial vendor product support?
For example, Safenet Inc., the famous HSM (hardware security module) vendor,
supports OpenSSL to access the private key stored in its HSM product. Intel
offered AES-NI implementation code to OpenSSL community. I guess OpenSSL
will continue to be the most functional and obtain the widest adoption and
support.
Regards
MauMau
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Jun 09, 2014 at 03:35:23PM +0200, Magnus Hagander wrote:
On Mon, Jun 9, 2014 at 3:19 PM, Andreas Karlsson <andreas@proxel.se> wrote:
On 06/09/2014 01:45 PM, Heikki Linnakangas wrote:
There was a patch set for this from Martijn van Oosterhout which was quite
complete.
Wow, blast from the past.
A lot has, unfortunately, changed since 2006. It might be a good
startingpoint. But also actively starting from the point of "let's try to
support multiple libraries" rather than "let's try to support gnutls" is
probably also important.
The patch did provide an API. The idea was that there were a number of
functions which would need to be defined to support an SSL library.
Each library would then have a wrapper which wrapped the library and
based on the results of configure it compiled the right file into the
backend.
These functions were:
extern void pgtls_initialize(void);
extern void pgtls_destroy(void);
extern int pgtls_open_server(Port *);
extern void pgtls_close(Port *);
extern ssize_t pgtls_read(Port *port, void *ptr, size_t len);
extern ssize_t pgtls_write(Port *port, void *ptr, size_t len);
Which should be easy enough to support for any library. These days
you'd need to add support for verifying certificates, but I don't think
that that would be difficult (unless the actual certificate formats are
different).
No switching after compile time, that would just lead to useless
overhead.
At some point we should design a new API, so that we can deprecate the old
one. Even if we don't hve the code ready, we need to get rid of PQgetssl(),
and replace it with something else. I'm thinking probably a functoin that
returns both a void pointer and an enum that tells you which library is
actually in use. And a boolean just saying "ssl on/off", because that's
what a lot of clients are interested in and they don't care aobut more than
that.Obviously, we also have to do something about PQinitOpenSSL().
Yeah, I think this was one of the more controversial parts. Support in
the backend was primarily moving code around and renaming functions,
fairly straightforward. Even error handling was not so hard (I found
the gnutls handling of errors much easier than openssl).
One tricky part is that programs like to use libpq for the
authentication, and then they hijack the connection using PGgetssl().
The way I dealt with this is defining a new state "passthrough" where
the caller would get a few function pointers to read/write/check the
connection. Then the callers would not need to know what library libpq
was compiled with. And libpq would know the connection was hijacked
and refuse to act anymore. I don't think everyone was pleased with
this, but no real alternative was presented (other than requiring
people hijacking the connection to do the hard work).
For information about which library was in use there was PQgettlsinfo()
which returned a PGresult with information about the library and
connection. I beleive since then new functions have been added to
libpq to retrive info about certificates, so that might need a rethink
also.
Basically, I think these last two points are the hard parts to get
agreement (assuming there's agreement to do anything at all about the
problem) and without nailing those down first whoever picks this up
will be in for a lot of work.
Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
He who writes carelessly confesses thereby at the very outset that he does
not attach much importance to his own thoughts.
-- Arthur Schopenhauer
On 06/09/2014 05:22 PM, Andres Freund wrote:
Hi,
On 2014-06-09 10:18:40 -0400, Tom Lane wrote:
Does SChannel have a better security track record than OpenSSL? Or is
the point here just that we can define it as not our problem when a
vulnerability surfaces?Well, it's patched as part of the OS - so no new PG binaries have to be
released when it's buggy.
Right. I have no idea what SChannel's track record is, but when there's
a vulnerability in the native SSL implementation in Windows, you better
upgrade anyway, regardless of PostgreSQL. So when we rely on that, we
don't put any extra burden on users. And we won't need to release new
binaries just to update the DLL included in it.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Jun 9, 2014 at 10:40 AM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:
Right. I have no idea what SChannel's track record is, but when there's a
vulnerability in the native SSL implementation in Windows, you better
upgrade anyway, regardless of PostgreSQL. So when we rely on that, we don't
put any extra burden on users. And we won't need to release new binaries
just to update the DLL included in it.
Right, heartily agreed. It wouldn't surprise me if there are lots of
Windows machines out there that have 4 or 5 copies of OpenSSL on them,
each provided by a different installer for some other piece of
software that happens to depend on OpenSSL. When OpenSSL then has a
security vulnerability, you're not safe until all of the people who
produce those installers produce new versions and you upgrade to all
of those new versions. In practice, I'm sure that an enormous amount
slips through the cracks here. Relying on something that is part of
the OS and updated by the OS vendor seems like less work for both
packagers (who have to prepare the updates) and users (who have to
apply them). Of course there may be cases where the OS implementation
sucks badly or otherwise can't be relied upon, and then we'll just
have to live with shipping copies of things. But avoiding it sounds
better, if someone's volunteering to do the work....
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Jun 9, 2014 at 4:39 PM, Martijn van Oosterhout <kleptog@svana.org>
wrote:
On Mon, Jun 09, 2014 at 03:35:23PM +0200, Magnus Hagander wrote:
On Mon, Jun 9, 2014 at 3:19 PM, Andreas Karlsson <andreas@proxel.se>
wrote:
On 06/09/2014 01:45 PM, Heikki Linnakangas wrote:
There was a patch set for this from Martijn van Oosterhout which wasquite
complete.
Wow, blast from the past.
That's fun, itsn't it? :)
A lot has, unfortunately, changed since 2006. It might be a good
startingpoint. But also actively starting from the point of "let's try to
support multiple libraries" rather than "let's try to support gnutls" is
probably also important.The patch did provide an API. The idea was that there were a number of
functions which would need to be defined to support an SSL library.
Each library would then have a wrapper which wrapped the library and
based on the results of configure it compiled the right file into the
backend.These functions were:
extern void pgtls_initialize(void);
extern void pgtls_destroy(void);
extern int pgtls_open_server(Port *);
extern void pgtls_close(Port *);
extern ssize_t pgtls_read(Port *port, void *ptr, size_t len);
extern ssize_t pgtls_write(Port *port, void *ptr, size_t len);Which should be easy enough to support for any library. These days
you'd need to add support for verifying certificates, but I don't think
that that would be difficult (unless the actual certificate formats are
different).
The two difficult points I think are the async support (libpq) and the
windows socket emulation support (backend). Do those really work there? In
particular the win32 stuff - though I guess that's less critical since we
can actually do hackish things there, unlike in libpq. But the example
there is that we can't have the library use recv()/send(), instead having
it work through our own functions.
No switching after compile time, that would just lead to useless
overhead.
Yes, I absolutely think we don't need to support >1 library at runtime.
At some point we should design a new API, so that we can deprecate the old
one. Even if we don't hve the code ready, we need to get rid of
PQgetssl(),
and replace it with something else. I'm thinking probably a functoin that
returns both a void pointer and an enum that tells you which library is
actually in use. And a boolean just saying "ssl on/off", because that's
what a lot of clients are interested in and they don't care aobut morethan
that.
Obviously, we also have to do something about PQinitOpenSSL().
Yeah, I think this was one of the more controversial parts. Support in
the backend was primarily moving code around and renaming functions,
fairly straightforward. Even error handling was not so hard (I found
the gnutls handling of errors much easier than openssl).
Yeah, the backend is easier, but also less important from the original
reason. For the patching reason, it's of course just as important.
One tricky part is that programs like to use libpq for the
authentication, and then they hijack the connection using PGgetssl().
Is there *anybody* other than odbc that does that? Do we actually need a
published API for that, or just a hack for pgodbc?
The way I dealt with this is defining a new state "passthrough" where
the caller would get a few function pointers to read/write/check the
connection. Then the callers would not need to know what library libpq
was compiled with. And libpq would know the connection was hijacked
and refuse to act anymore. I don't think everyone was pleased with
this, but no real alternative was presented (other than requiring
people hijacking the connection to do the hard work).For information about which library was in use there was PQgettlsinfo()
which returned a PGresult with information about the library and
connection. I beleive since then new functions have been added to
libpq to retrive info about certificates, so that might need a rethink
also.
Not really, no. we return the OpenSSL structure and have the client use
that one directly. That's quite horrible already :)
Basically, I think these last two points are the hard parts to get
agreement (assuming there's agreement to do anything at all about the
problem) and without nailing those down first whoever picks this up
will be in for a lot of work.
Agreed.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On Mon, Jun 09, 2014 at 11:39:17PM +0900, MauMau wrote:
From: "Heikki Linnakangas" <hlinnakangas@vmware.com>
Thoughts? While we're at it, we'll probably want to refactor
things so that it's easy to support other SSL implementations too,
like gnutls.That may be good because it provides users with choices. But I
wonder if it is worth the complexity and maintainability of
PostgreSQL code.
The complexity is very low. SSL is a standard protocol and so all
libraries offer the same functionality. Were not really doing anything
complex.
* Are SChannel and other libraries more secure than OpenSSL? IIRC,
recently I read in the news that GnuTLS had a vulnerability.
OpenSSL is probably the most widely used library, and many people
are getting more interested in its quality. I expect the quality
will improve thanks to the help from The Linux foundation and other
organizations/researchers.
Does that matter? What's wrong with letting people choose. OpenVPN
these days supports multiple SSL libraries, because PolarSSL (for
example) has been vetted for a higher security level than OpenSSL.
* Do other libraries get support from commercial vendor product
support? For example, Safenet Inc., the famous HSM (hardware
security module) vendor, supports OpenSSL to access the private key
stored in its HSM product. Intel offered AES-NI implementation code
to OpenSSL community. I guess OpenSSL will continue to be the most
functional and obtain the widest adoption and support.
And the crappiest license. I think it's silly for PostgreSQL dictate
what SSL library users must use, when there are so many possibilities.
We also support libedit for, in my opinion, worse reasons.
Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
He who writes carelessly confesses thereby at the very outset that he does
not attach much importance to his own thoughts.
-- Arthur Schopenhauer
On 06/09/2014 06:03 PM, Magnus Hagander wrote:
One tricky part is that programs like to use libpq for the
authentication, and then they hijack the connection using PGgetssl().
Is there*anybody* other than odbc that does that? Do we actually need a
published API for that, or just a hack for pgodbc?
I wish psqlODBC would stop doing that. It's kind of cool that it
supports compiling without libpq, but it's really quite a mess. I think
we should modify psqlODBC to use the libpq API like most people do.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Jun 9, 2014 at 7:45 PM, Heikki Linnakangas <hlinnakangas@vmware.com>
wrote:
On 06/09/2014 06:03 PM, Magnus Hagander wrote:
One tricky part is that programs like to use libpq for the
authentication, and then they hijack the connection using PGgetssl().
Is there*anybody* other than odbc that does that? Do we actually need a
published API for that, or just a hack for pgodbc?
I wish psqlODBC would stop doing that. It's kind of cool that it supports
compiling without libpq, but it's really quite a mess. I think we should
modify psqlODBC to use the libpq API like most people do.
This was, I believe, done at one point, and then reverted. I think that was
because libpq didn't actually have all the features required either for the
current or for planned featues of the ODBC driver. That situation might be
very different now though, there's more functionality available in libpq..
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 06/09/2014 05:39 PM, Martijn van Oosterhout wrote:
On Mon, Jun 09, 2014 at 03:35:23PM +0200, Magnus Hagander wrote:
On Mon, Jun 9, 2014 at 3:19 PM, Andreas Karlsson <andreas@proxel.se> wrote:
On 06/09/2014 01:45 PM, Heikki Linnakangas wrote:
There was a patch set for this from Martijn van Oosterhout which was quite
complete.Wow, blast from the past.
A lot has, unfortunately, changed since 2006. It might be a good
startingpoint. But also actively starting from the point of "let's try to
support multiple libraries" rather than "let's try to support gnutls" is
probably also important.The patch did provide an API. The idea was that there were a number of
functions which would need to be defined to support an SSL library.
Each library would then have a wrapper which wrapped the library and
based on the results of configure it compiled the right file into the
backend.These functions were:
extern void pgtls_initialize(void);
extern void pgtls_destroy(void);
extern int pgtls_open_server(Port *);
extern void pgtls_close(Port *);
extern ssize_t pgtls_read(Port *port, void *ptr, size_t len);
extern ssize_t pgtls_write(Port *port, void *ptr, size_t len);Which should be easy enough to support for any library. These days
you'd need to add support for verifying certificates, but I don't think
that that would be difficult (unless the actual certificate formats are
different).No switching after compile time, that would just lead to useless
overhead.
Yeah, that seems like a reasonable design.
I did again the refactoring you did back in 2006, patch attached. One
thing I did differently: I moved the raw, non-encrypted, read/write
functions to separate functions: pqsecure_raw_read and
pqsecure_raw_write. Those functions encapsulate the SIGPIPE handling.
The OpenSSL code implements a custom BIO, which calls to
pqsecure_raw_read/write to do the low-level I/O. Similarly in the
server-side, there are be_tls_raw_read and pg_tls_raw_write functions,
which do the prepare_for_client_read()/client_read_ended() dance, so
that the SSL implementation doesn't need to know about that.
I then implemented a quick proof-of-concept Windows SChannel
implementation of that API. It's client-side only, and there's no
support for validating server certificate or specifying non-default
ciphers or anything yet. But I did implement performing authentication
with a client certificate, as a proof of concept that it's possible to
read the OpenSSL key and certificate files - although in this proof of
concept the key and certificate are hard-coded in the sources, not read
from a file.
The SChannel implementation obviously needs a lot of work, but I'm
fairly confident that it's doable, and the new internal SSL API works
for that. Except for the user-visible PQgetssl() functions and such - I
don't know what to do with those.
At some point we should design a new API, so that we can deprecate the old
one. Even if we don't hve the code ready, we need to get rid of PQgetssl(),
and replace it with something else. I'm thinking probably a functoin that
returns both a void pointer and an enum that tells you which library is
actually in use. And a boolean just saying "ssl on/off", because that's
what a lot of clients are interested in and they don't care aobut more than
that.Obviously, we also have to do something about PQinitOpenSSL().
Yeah, I think this was one of the more controversial parts. Support in
the backend was primarily moving code around and renaming functions,
fairly straightforward. Even error handling was not so hard (I found
the gnutls handling of errors much easier than openssl).One tricky part is that programs like to use libpq for the
authentication, and then they hijack the connection using PGgetssl().
The way I dealt with this is defining a new state "passthrough" where
the caller would get a few function pointers to read/write/check the
connection. Then the callers would not need to know what library libpq
was compiled with. And libpq would know the connection was hijacked
and refuse to act anymore. I don't think everyone was pleased with
this, but no real alternative was presented (other than requiring
people hijacking the connection to do the hard work).
Sounds good. If we want to support such hijacking if the first place.
- Heikki
Attachments:
0002-Add-crude-version-of-Windows-SChannel-support.patchtext/x-diff; name=0002-Add-crude-version-of-Windows-SChannel-support.patchDownload+1008-6
0001-Invent-a-new-internal-API-for-interfacing-with-SSL.patchtext/x-diff; name=0001-Invent-a-new-internal-API-for-interfacing-with-SSL.patchDownload+2785-2310
On Wed, Jun 11, 2014 at 7:51 AM, Heikki Linnakangas <hlinnakangas@vmware.com>
wrote:
I did again the refactoring you did back in 2006, patch attached. One
thing
I did differently: I moved the raw, non-encrypted, read/write functions to
separate functions: pqsecure_raw_read and pqsecure_raw_write. Those
functions encapsulate the SIGPIPE handling. The OpenSSL code implements a
custom BIO, which calls to pqsecure_raw_read/write to do the low-level
I/O.
Similarly in the server-side, there are be_tls_raw_read and
pg_tls_raw_write
functions, which do the prepare_for_client_read()/client_read_ended()
dance,
so that the SSL implementation doesn't need to know about that.
I've tried your 0001 patch, reflecting this refactoring, on Linux and it
caused 'make check' to hang at 'starting postmaster'.
The hang seems to be in:
/tmp_check/install//home/jjanes/pgsql/test_ssl/bin/psql -X postgres
with a backtrace of:
#0 0x0000003550edf2f8 in __poll (fds=0x7fff610cbd50, nfds=1, timeout=-1)
at ../sysdeps/unix/sysv/linux/poll.c:83
#1 0x00007fcf372035b1 in pqSocketPoll (conn=0x2317770, forRead=1,
forWrite=0, end_time=-1) at fe-misc.c:1122
#2 pqSocketCheck (conn=0x2317770, forRead=1, forWrite=0, end_time=-1) at
fe-misc.c:1064
#3 0x00007fcf37203630 in pqWaitTimed (forRead=<value optimized out>,
forWrite=<value optimized out>, conn=0x2317770, finish_time=<value
optimized out>)
at fe-misc.c:996
#4 0x00007fcf371fe632 in connectDBComplete (conn=0x2317770) at
fe-connect.c:1498
#5 0x00007fcf371ff27f in PQconnectdbParams (keywords=<value optimized
out>, values=<value optimized out>, expand_dbname=<value optimized out>)
at fe-connect.c:462
#6 0x0000000000411bb5 in main (argc=<value optimized out>,
argv=0x7fff610cc038) at startup.c:219
The make check never times out, like it usually does when something gets
stalled.
That was on CentOS 6.5 patched up to date, but OpenSuSE 13.1 gives the same
hang.
Cheers,
Jeff
