WIP: Secure Transport support as OpenSSL alternative on macOS
As noted in 77FB2321-9210-4AD4-B7FC-67623A5838F0@justatheory.com, OpenSSL is
being somewhat dismantled on macOS starting with 10.11 El Capitan. To scratch
my own itch I decided to take a stab at adding support for the Secure Transport
library. The idea is to make it as much of a drop-in for the OpenSSL support
as possible, supporting the same file formats etc (Secure Transport have PKCS12
API calls while support PEM requires using lower level APIs).
Only the frontend has been implemented so far. I used a vanilla OpenSSL-
compiled backend for testing (see patch 0002 below). Currently the CRL tests
fail since Secure Transport doesn’t handle manual CRL lists like OpenSSL, CRL
handling is done automatically by the Keychain upon validation. One
alternative for supporting this could perhaps be to create a temporary Keychain
for the duration of the connection over which more control can be had (need to
research this further). For now, no Secure Transport specifics are supported
but the idea is to add support for loading certificates and keys from the
Keychain as well as from the usual locations. The engine and compression
settings will however be no-ops since there is no support in Secure Transport
for these.
Being an early WIP show-and-tell submissions and not something up for code
review (anyone wanting to take a look is of course more than welcome), it is
deliberately rough around the edges. Except the manual CRL loading, it’s
mostly featurecomplete with the OpenSSL frontend, TODO items include: removing
use of private and deprecated API calls; ensure release of all resources;
support certificates/keys in Keychain; possibly use temporary keychains for
connections; support passphrase entering (there has been lots of deprecations
around that, need to figure out the supported way forward); tidy up, and write
better, comments; documentation.
The main questions raised here are: is it of interest to support multiple SSL
libraries given the additional support burden and; is supporting Secure
Transport of any interest or is it a wasted effort to continue towards a full
frontend/backend/doc submission?
On top of the OpenSSL situation in macOS, supporting Secure Transport makes
sense if the Keychain can be fully leveraged since that would allow IT
organisations to centrally manage and provision Mac clients without passing
certificate/key files around. Should there be any interest in this I intend to
continue on one more library which is more geared towards server environments
like nss or <suggestions-welcome> once Secure Transport is done (scratching
another itch).
The attached show-and-tell patches are:
0001. The WIP frontend support for Secure Transport together with minimal
scaffolding around it such as Makefile and autoconf.
0002. Adding support to PostgresNode.pm to run the server binaries from a
defined path making it easier to test a frontend without having to a)
implement the backend at the same time or b) fiddle with paths manually.
The SSL tests are the only ones modified. There is nothing Secure Transport
specific in this patch so iff the approach is deemed interesting it could be
considered out of band. Since the only consumer is testing SSL during
development it might be too invasive however.
0003. A first little stab at tweaking the docs to mention support for
multiple SSL libraries. There is a lot left here but the idea is to contain
the library specifics in subsections with the main sections being generic
SSL support information. On top of tweaking the existing sections I intend
(once done) to write up the steps needed to implement support for an SSL
library, this should perhaps be a README in the tree though?
Thoughts, comments?
cheers ./daniel
Attachments:
0001-WIP-Add-support-for-macOS-Secure-Transport-SSL-libra.patchapplication/octet-stream; name=0001-WIP-Add-support-for-macOS-Secure-Transport-SSL-libra.patchDownload+1634-2
0002-Enable-ssl-tests-to-be-using-a-different-set-of-Post.patchapplication/octet-stream; name=0002-Enable-ssl-tests-to-be-using-a-different-set-of-Post.patchDownload+56-17
0003-A-first-stab-at-updating-the-docs-to-handle-multiple.patchapplication/octet-stream; name=0003-A-first-stab-at-updating-the-docs-to-handle-multiple.patchDownload+121-48
Daniel,
* Daniel Gustafsson (daniel@yesql.se) wrote:
The main questions raised here are: is it of interest to support multiple SSL
libraries given the additional support burden and; is supporting Secure
Transport of any interest or is it a wasted effort to continue towards a full
frontend/backend/doc submission?
We've already started working towards being able to support multiple
libraries (in particular, Heikki's work to try and contain the OpenSSL
bits) and I've been looking at what it'll take to add NSS support.
Generally speaking, I think we'd be better off looking at adding support
for multi-platform libraries like NSS rather than OS-specific APIs, but
I do understand how there can be advantages to using the OS API
(keychain integration is certainly a big part of that, as you mention;
the same is true for our Kerberos/GSS support).
On top of the OpenSSL situation in macOS, supporting Secure Transport makes
sense if the Keychain can be fully leveraged since that would allow IT
organisations to centrally manage and provision Mac clients without passing
certificate/key files around. Should there be any interest in this I intend to
continue on one more library which is more geared towards server environments
like nss or <suggestions-welcome> once Secure Transport is done (scratching
another itch).
I'd certainly be more than happy to help with the NSS side of things,
though I'd want to have both client and server support for it.
The attached show-and-tell patches are:
Haven't actually looked at the patches yet, but very happy that you're
also interested in this. :)
0003. A first little stab at tweaking the docs to mention support for
multiple SSL libraries. There is a lot left here but the idea is to contain
the library specifics in subsections with the main sections being generic
SSL support information. On top of tweaking the existing sections I intend
(once done) to write up the steps needed to implement support for an SSL
library, this should perhaps be a README in the tree though?
Agreed, this would be better as a README rather than in the formal docs,
since adding support for another SSL library isn't really user-facing.
Thanks!
Stephen
On 05 Oct 2016, at 16:45, Stephen Frost <sfrost@snowman.net> wrote:
Daniel,
* Daniel Gustafsson (daniel@yesql.se) wrote:
The main questions raised here are: is it of interest to support multiple SSL
libraries given the additional support burden and; is supporting Secure
Transport of any interest or is it a wasted effort to continue towards a full
frontend/backend/doc submission?We've already started working towards being able to support multiple
libraries (in particular, Heikki's work to try and contain the OpenSSL
bits) and I've been looking at what it'll take to add NSS support.
Building on his work it’s quite simple to keep this code contained to its own
files which is great.
Generally speaking, I think we'd be better off looking at adding support
for multi-platform libraries like NSS rather than OS-specific APIs, but
I do understand how there can be advantages to using the OS API
(keychain integration is certainly a big part of that, as you mention;
the same is true for our Kerberos/GSS support).
In general I agree with you, supporting OS specific APIs on common platforms
(for some value of) which doesn’t ship with a useable OpenSSL installation can
still be of value I think.
On top of the OpenSSL situation in macOS, supporting Secure Transport makes
sense if the Keychain can be fully leveraged since that would allow IT
organisations to centrally manage and provision Mac clients without passing
certificate/key files around. Should there be any interest in this I intend to
continue on one more library which is more geared towards server environments
like nss or <suggestions-welcome> once Secure Transport is done (scratching
another itch).I'd certainly be more than happy to help with the NSS side of things,
though I'd want to have both client and server support for it.
I think any library supported should be required to have both client and server
support. I stopped short after the frontend in this patch to get feedback but
will absolutely implement the backend as well.
NSS is on my list and I intend to get started on that somewhat shortly.
The attached show-and-tell patches are:
Haven't actually looked at the patches yet, but very happy that you're
also interested in this. :)0003. A first little stab at tweaking the docs to mention support for
multiple SSL libraries. There is a lot left here but the idea is to contain
the library specifics in subsections with the main sections being generic
SSL support information. On top of tweaking the existing sections I intend
(once done) to write up the steps needed to implement support for an SSL
library, this should perhaps be a README in the tree though?Agreed, this would be better as a README rather than in the formal docs,
since adding support for another SSL library isn't really user-facing.
cheers ./daniel
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Oct 5, 2016 at 5:36 AM, Daniel Gustafsson <daniel@yesql.se> wrote:
The main questions raised here are: is it of interest to support multiple SSL
libraries given the additional support burden and; is supporting Secure
Transport of any interest or is it a wasted effort to continue towards a full
frontend/backend/doc submission?
I think this is highly worthwhile. I wish we could interest someone
in doing the work for Windows ... but I'm a macOS user myself, so I'll
be happy to have you fix my future compile problems for me.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
On Wed, Oct 5, 2016 at 5:36 AM, Daniel Gustafsson <daniel@yesql.se> wrote:
The main questions raised here are: is it of interest to support multiple SSL
libraries given the additional support burden and; is supporting Secure
Transport of any interest or is it a wasted effort to continue towards a full
frontend/backend/doc submission?
I think this is highly worthwhile. I wish we could interest someone
in doing the work for Windows ... but I'm a macOS user myself, so I'll
be happy to have you fix my future compile problems for me.
"Future"? Apple isn't even shipping the OpenSSL headers anymore, and
I imagine soon no libraries either. We really have got little choice
on that platform but to do something with Secure Transport. I'm glad
somebody is taking up the task.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Wed, Oct 5, 2016 at 8:42 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Wed, Oct 5, 2016 at 5:36 AM, Daniel Gustafsson <daniel@yesql.se>
wrote:
The main questions raised here are: is it of interest to support
multiple SSL
libraries given the additional support burden and; is supporting Secure
Transport of any interest or is it a wasted effort to continue towardsa full
frontend/backend/doc submission?
I think this is highly worthwhile. I wish we could interest someone
in doing the work for Windows ... but I'm a macOS user myself, so I'll
be happy to have you fix my future compile problems for me."Future"? Apple isn't even shipping the OpenSSL headers anymore, and
I imagine soon no libraries either. We really have got little choice
on that platform but to do something with Secure Transport. I'm glad
somebody is taking up the task.
Sure we do. Windows doesn't ship them either, and yet somehow Postgres
manages to work just fine there, including with openssl support. There's
nothing more magic about MacOS than there is for Windows.
That said, I agree that somebody is picking up the task. I actually think
it would be a lot more useful to get Windows SChannel support (because it's
*much* more of a PITA to get OpenSSL onto Windows than it is to get it onto
macOS) or even moreso NSS (because then every platform could use that, and
they have other integrations too). But one important point is that once we
have *two* implementations (openssl + macos) then we will know a lot more
about the correct places for abstractions etc, and and adding the third one
is probably going to be easier (but not easy). But let's make sure we keep
in mind there should be more than just these two implementation when
defining any external interfaces.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/