Making sslrootcert=system work on Windows psql

Started by George MacKerron11 months ago33 messages
Jump to latest
#1George MacKerron
george@mackerron.co.uk

I was very pleased to see the sslrootcert=system connection option added in Postgres 16 (I even blogged about it: https://neon.tech/blog/avoid-mitm-attacks-with-psql-postgres-16). But sslrootcert=system has not been widely supported by psql installations, perhaps because people compiling Postgres haven’t always been aware of the requirement to point OpenSSL in the direction of the system’s root CA certificates.

I’ve recently been trying to get it more widely supported, with some success (details at end of this message).

However, psql via the EnterpriseDB Windows installer still doesn’t support sslrootcert=system, and I think a tiny patch is needed. The diff is attached, and can be seen in context here: https://github.com/postgres/postgres/compare/master...jawj:postgres:jawj-sslrootcert-system-windows

Essentially, on Windows with OpenSSL 3.2+, it replaces SSL_CTX_set_default_verify_paths(SSL_context) with SSL_CTX_load_verify_store(SSL_context, "org.openssl.winstore:”).

I’m not a Windows or OpenSSL expert, but so far the patched code seems to work in theory and in practice (sources below, and I’ve compiled and tested it working on Windows 11 x64).

# Sources

https://stackoverflow.com/a/79461864/338196
https://docs.openssl.org/master/man7/OSSL_STORE-winstore/
https://docs.openssl.org/master/man3/SSL_CTX_load_verify_locations/

# Status of sslrootcert=system in various packages providing psql

## Mac
Postgres.app — now fixed (https://github.com/PostgresApp/PostgresApp/issues/801)
MacPorts — now fixed (https://trac.macports.org/ticket/72080)
EDB installer — now fixed (https://github.com/EnterpriseDB/edb-installers/issues/264)
homebrew — was working already

## Linux
Debian/Ubuntu — now Recommends ca-certificates (https://salsa.debian.org/postgresql/postgresql/-/commit/96077ad61c36386646cdd9b5ce0e423a357ce73b)

## Windows
EDB installer — in progress
WSL1, WSL2 (Ubuntu, openSUSE) — was working already

Attachments:

sslrootcert-system-win.diffapplication/octet-stream; name=sslrootcert-system-win.diff; x-unix-mode=0644Download+4-0
#2Jacob Champion
jacob.champion@enterprisedb.com
In reply to: George MacKerron (#1)
Re: Making sslrootcert=system work on Windows psql

On Tue, Apr 1, 2025 at 2:05 PM George MacKerron <george@mackerron.co.uk> wrote:

I was very pleased to see the sslrootcert=system connection option added in Postgres 16 (I even blogged about it: https://neon.tech/blog/avoid-mitm-attacks-with-psql-postgres-16). But sslrootcert=system has not been widely supported by psql installations, perhaps because people compiling Postgres haven’t always been aware of the requirement to point OpenSSL in the direction of the system’s root CA certificates.

I’ve recently been trying to get it more widely supported, with some success (details at end of this message).

(Thank you!)

However, psql via the EnterpriseDB Windows installer still doesn’t support sslrootcert=system,

Hm. I've been in contact with Kritika recently for the EDB macOS
fixes; hopefully we can get something figured out for Windows too.

and I think a tiny patch is needed. The diff is attached, and can be seen in context here: https://github.com/postgres/postgres/compare/master...jawj:postgres:jawj-sslrootcert-system-windows

Essentially, on Windows with OpenSSL 3.2+, it replaces SSL_CTX_set_default_verify_paths(SSL_context) with SSL_CTX_load_verify_store(SSL_context, "org.openssl.winstore:”).

I’m not a Windows or OpenSSL expert, but so far the patched code seems to work in theory and in practice (sources below, and I’ve compiled and tested it working on Windows 11 x64).

While this will get things working -- if you plan to use the Windows
store! -- I worry that it's an incompatible change, and anyone who is
actually happy with the way things currently work (i.e. not using the
EDB installers) will be broken. The meaning of `sslrootcert=system` is
"do whatever OpenSSL wants to do by default." That includes
modification by the OpenSSL environment variables, which (I think)
this patch disables.

The winstore is new to me. Is there no way to get OpenSSL to switch
its default store without code changes?

--Jacob

#3Daniel Gustafsson
daniel@yesql.se
In reply to: Jacob Champion (#2)
Re: Making sslrootcert=system work on Windows psql

On 1 Apr 2025, at 23:46, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
On Tue, Apr 1, 2025 at 2:05 PM George MacKerron <george@mackerron.co.uk> wrote:

I’ve recently been trying to get it more widely supported, with some success (details at end of this message).

(Thank you!)

+many

I’m not a Windows or OpenSSL expert, but so far the patched code seems to work in theory and in practice (sources below, and I’ve compiled and tested it working on Windows 11 x64).

While this will get things working -- if you plan to use the Windows
store! -- I worry that it's an incompatible change, and anyone who is
actually happy with the way things currently work (i.e. not using the
EDB installers) will be broken. The meaning of `sslrootcert=system` is
"do whatever OpenSSL wants to do by default." That includes
modification by the OpenSSL environment variables, which (I think)
this patch disables.

Correct, this patch changes from using the defaults (directory, file and store)
and thus the env variable overrides, to hardcoding the new winstore which came
in 3.2 as the only option. While I agree that we probably should allow
winstore (and other such stores for other platforms when/if they happen) I
don't think making it the only option is the right way.

The winstore is new to me. Is there no way to get OpenSSL to switch
its default store without code changes?

AFAIK one cannot change the default store in OpenSSL short of recompiling
OpenSSL.

--
Daniel Gustafsson

#4George MacKerron
george@mackerron.co.uk
In reply to: Daniel Gustafsson (#3)
Re: Making sslrootcert=system work on Windows psql

Daniel, Jacob: thanks. My feeling is that it would be a bit odd to prioritise the preservation of a secondary behaviour (users can customise what cert store is used via environment variables) over fixing the feature’s basic reason for existing (certificates will be validated against the system CA cert store), even in the name of backward-compatibility.

But happily, I don’t think we need to choose. Can’t we just use the Windows system store if neither of the relevant environment variables is set?

I’ve updated my patch to do that. It’s attached, and also still here: https://github.com/postgres/postgres/compare/master...jawj:postgres:jawj-sslrootcert-system-windows

Attachments:

sslrootcert-system-win-2.diffapplication/octet-stream; name=sslrootcert-system-win-2.diff; x-unix-mode=0644Download+13-1
#5George MacKerron
george@mackerron.co.uk
In reply to: George MacKerron (#4)
Re: Making sslrootcert=system work on Windows psql

On 2 Apr 2025, at 14:39, George MacKerron <george@mackerron.co.uk> wrote:
But happily, I don’t think we need to choose. Can’t we just use the Windows system store if neither of the relevant environment variables is set?

Thinking about this a little more, I guess the remaining concern is about people on Windows compiling their own psql from source, using an OpenSSL build that has a meaningful OPENSSLDIR baked in.

I guess that might suggest we should make the "org.openssl.winstore:" code path something users can opt out of (or even, for maximum backwards-compatibility, opt in to) at compile-time.

My preference would be for "org.openssl.winstore:" to be the compile-time default, though, because the option is called sslrootcert=system and it’s documented as using “the system’s trusted CA roots” (not sslrootcert=openssldir or documented as using OpenSSL’s default CA roots).

#6Jacob Champion
jacob.champion@enterprisedb.com
In reply to: George MacKerron (#5)
Re: Making sslrootcert=system work on Windows psql

On Wed, Apr 2, 2025 at 7:15 AM George MacKerron <george@mackerron.co.uk> wrote:

But happily, I don’t think we need to choose. Can’t we just use the Windows system store if neither of the relevant environment variables is set?

Thinking about this a little more, I guess the remaining concern is about people on Windows compiling their own psql from source, using an OpenSSL build that has a meaningful OPENSSLDIR baked in.

Right. In a past life I shipped client stacks on Windows that looked
kind of like that; I would have been less than happy if a client
suddenly stopped using the certificate bundle I'd set up.

My preference would be for "org.openssl.winstore:" to be the compile-time default, though, because the option is called sslrootcert=system and it’s documented as using “the system’s trusted CA roots” (not sslrootcert=openssldir or documented as using OpenSSL’s default CA roots).

If we'd decided to do that from the beginning, maybe... but it looks
like the winstore URI wasn't released yet when we designed that, so
"the system" couldn't have meant anything except for OpenSSL. Maybe
the documentation needs to be more specific now that OpenSSL is
supporting more stuff.

Even if we want to change that definition sometime in the future, we'd
still have to wait at least until OpenSSL 3.1 was no longer supported;
I don't think it would be very helpful for our definition of "system"
to change abruptly when upgrading OpenSSL past the 3.2 boundary. All
this to say, I'd like to support the winstore, but I'm not convinced
it should take over the existing meaning of "system". Even just adding
it as a fallback has some risk to any packagers who have gotten it
working.

On Wed, Apr 2, 2025 at 12:33 AM Daniel Gustafsson <daniel@yesql.se> wrote:

AFAIK one cannot change the default store in OpenSSL short of recompiling
OpenSSL.

I had hoped that a `system_default` entry in openssl.cnf would be able
to override it, but no luck -- VerifyCAStore is explicitly forbidden
in the default section :(

Thanks,
--Jacob

#7Daniel Gustafsson
daniel@yesql.se
In reply to: Jacob Champion (#6)
Re: Making sslrootcert=system work on Windows psql

On 3 Apr 2025, at 03:21, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

On Wed, Apr 2, 2025 at 7:15 AM George MacKerron <george@mackerron.co.uk> wrote:

But happily, I don’t think we need to choose. Can’t we just use the Windows system store if neither of the relevant environment variables is set?

The env vars are only overrides for the default, they are not required to be
set, so their presence (or lack thereof) cannot be used to base any decision on
really.

Thinking about this a little more, I guess the remaining concern is about people on Windows compiling their own psql from source, using an OpenSSL build that has a meaningful OPENSSLDIR baked in.

Right. In a past life I shipped client stacks on Windows that looked
kind of like that; I would have been less than happy if a client
suddenly stopped using the certificate bundle I'd set up.

Agreed. I don't think it's productive to assume that OPENSSLDIR is bogus as a
general rule, OpenSSL sure doesn't. Also, remember that the API we use sets
OpenSSL to use the default dir, file or *store*, not just file/dir:

SSL_CTX_set_default_verify_paths() specifies that the default locations
from which CA certificates are loaded should be used. There is one default
directory, one default file and one default store. The default CA
certificates directory is called certs in the default OpenSSL directory,
and this is also the default store.

org.openssl.winstore isn't by OpenSSL defined as the default even on Windows,
but a future version might change that.

My preference would be for "org.openssl.winstore:" to be the compile-time default, though, because the option is called sslrootcert=system and it’s documented as using “the system’s trusted CA roots” (not sslrootcert=openssldir or documented as using OpenSSL’s default CA roots).

If we'd decided to do that from the beginning, maybe... but it looks
like the winstore URI wasn't released yet when we designed that, so
"the system" couldn't have meant anything except for OpenSSL. Maybe
the documentation needs to be more specific now that OpenSSL is
supporting more stuff.

I don't think we need to be more specific regarding what OpenSSL support, but
in hindsight I wonder if we should be more specific around that "system"
actually means. The attached (untested) small diff tries to make that more
clear. (Line reflow omitted for review ease.)

Even if we want to change that definition sometime in the future, we'd
still have to wait at least until OpenSSL 3.1 was no longer supported;
I don't think it would be very helpful for our definition of "system"
to change abruptly when upgrading OpenSSL past the 3.2 boundary. All
this to say, I'd like to support the winstore, but I'm not convinced
it should take over the existing meaning of "system". Even just adding
it as a fallback has some risk to any packagers who have gotten it
working.

Right now sslrootcert can have two different values, a filename or "system". I
don't think altering what "system" means is a good idea, but I also don't think
limiting ourselves to those two values is helpful. We either need to make a
new param. to over time replace sslrootcert with, which can handle multiple
different values; or we need to retrofit a DSL/syntax to sslrootcert for
differentiating. Both have in common that the coding task is magnitudes easier
than figuring out the user experience.

Something to consider for v19 work.

--
Daniel Gustafsson

Attachments:

rootcertsystem.diffapplication/octet-stream; name=rootcertsystem.diff; x-unix-mode=0644Download+2-2
#8Sandeep Thakkar
sandeep.thakkar@enterprisedb.com
In reply to: George MacKerron (#1)
Re: Making sslrootcert=system work on Windows psql

On Wed, Apr 2, 2025 at 2:35 AM George MacKerron <george@mackerron.co.uk>
wrote:

I was very pleased to see the sslrootcert=system connection option added
in Postgres 16 (I even blogged about it:
https://neon.tech/blog/avoid-mitm-attacks-with-psql-postgres-16). But
sslrootcert=system has not been widely supported by psql installations,
perhaps because people compiling Postgres haven’t always been aware of the
requirement to point OpenSSL in the direction of the system’s root CA
certificates.

I’ve recently been trying to get it more widely supported, with some
success (details at end of this message).

However, psql via the EnterpriseDB Windows installer still doesn’t support
sslrootcert=system, and I think a tiny patch is needed. The diff is
attached, and can be seen in context here:
https://github.com/postgres/postgres/compare/master...jawj:postgres:jawj-sslrootcert-system-windows

Essentially, on Windows with OpenSSL 3.2+, it replaces
SSL_CTX_set_default_verify_paths(SSL_context) with
SSL_CTX_load_verify_store(SSL_context, "org.openssl.winstore:”).

Please note the EDB Windows installers for PostgreSQL versions (upto v17)

use OpenSSL v3.0, which is an LTS version. PostgreSQL 18 installer may use
OpenSSL v3.5 depending on the release timeframe.

I’m not a Windows or OpenSSL expert, but so far the patched code seems to
work in theory and in practice (sources below, and I’ve compiled and tested
it working on Windows 11 x64).

# Sources

https://stackoverflow.com/a/79461864/338196
https://docs.openssl.org/master/man7/OSSL_STORE-winstore/
https://docs.openssl.org/master/man3/SSL_CTX_load_verify_locations/

# Status of sslrootcert=system in various packages providing psql

## Mac
Postgres.app — now fixed (
https://github.com/PostgresApp/PostgresApp/issues/801)
MacPorts — now fixed (https://trac.macports.org/ticket/72080)
EDB installer — now fixed (
https://github.com/EnterpriseDB/edb-installers/issues/264)
homebrew — was working already

## Linux
Debian/Ubuntu — now Recommends ca-certificates (
https://salsa.debian.org/postgresql/postgresql/-/commit/96077ad61c36386646cdd9b5ce0e423a357ce73b
)

## Windows
EDB installer — in progress
WSL1, WSL2 (Ubuntu, openSUSE) — was working already

--
Sandeep Thakkar

#9George MacKerron
george@mackerron.co.uk
In reply to: Daniel Gustafsson (#7)
Re: Making sslrootcert=system work on Windows psql

On 3 Apr 2025, at 11:41, Daniel Gustafsson <daniel@yesql.se> wrote:

org.openssl.winstore isn't by OpenSSL defined as the default even on Windows,
but a future version might change that.

Right — there’s definitely an argument that OpenSSL should in future make it possible to have this be the default via a compile-time option, at least.

I don't think we need to be more specific regarding what OpenSSL support, but
in hindsight I wonder if we should be more specific around that "system"
actually means. The attached (untested) small diff tries to make that more
clear. (Line reflow omitted for review ease.)

I guess my issue here was twofold:

(1) sslrootcert=system on Windows doesn’t do what it says on the tin. In other words, it doesn’t do (a) what it sounds like it does or (b) what it says it does in the docs.

(2) sslrootcert=system on Windows doesn’t do a thing that would be extremely useful in some common situations. Namely: connecting securely to servers that present a certificate signed by a public CA.

Your diff certainly fixes (1b), so it’s definitely an improvement. But of course it does nothing for (2). :(

Right now sslrootcert can have two different values, a filename or "system". I
don't think altering what "system" means is a good idea, but I also don't think
limiting ourselves to those two values is helpful. We either need to make a
new param. to over time replace sslrootcert with, which can handle multiple
different values; or we need to retrofit a DSL/syntax to sslrootcert for
differentiating. Both have in common that the coding task is magnitudes easier
than figuring out the user experience.

Something to consider for v19 work.

To give a bit of context here, my feeling is that the widespread use of sslmode=require is a pretty serious security problem in the Postgres community. I strongly suspect many users don’t realise that it offers no protection _at all_ against MITM attacks. I know it took me a while to figure that point out, because sslmode=require just _sounds_ reassuringly secure.

That’s why I was so pleased to read about sslrootcert=system in Postgres 16: I thought it was going to improve this situation. But sslrootcert=system (or similar) isn’t going to be widely used until Postgres providers put it in their connection strings, and Postgres providers aren’t going to put it in their connection strings until it has a damn good chance of just working.

On Linux and Mac, I would say the ‘damn good chance of just working’ bar has now been reached. But on Windows, I suspect a _lot_ of devs are using psql as installed by the EDB Installer (it’s the only option listed at https://www.postgresql.org/download/windows/, after all). So until that works, sslrootcert=system (or similar) is going to remain a no-go.

What I am saying is: it would be _really_ nice not to have to wait another whole release cycle to get a level of security on many people’s Postgres connections that’s simply on par with the security of visiting some random web page.

So: what can be done?

(1) I could ask the EDB installer guys if they’re willing to apply my patch to the Postgres source as part of their build process, so as to use the Windows store in this one case. Personally, I think that would be a clear improvement; but I don’t know if they’ll like the idea. Based on Sandeep’s comment, it seems this is also dependent on OpenSSL 3.5 (LTS) becoming available prior to the Postgres 18 release.

(2) Your idea of a new parameter, or a new value of sslrootcert, is what I was also starting to mull this morning. Is there any chance at all this could be done for Postgres 18 or, failing that, 18.1?

I quite like sslrootcert=os: it’s snappy, and it implies that the Operating System root certs are going to be used (which is what I would have liked sslrootcert=system to mean). Some possible alternatives might be sslrootcert=public-cas or sslrootcert=os-default.

The key thing is that it should work out-of-the-box basically everywhere, so my preferred behaviour for it would be:

* On Windows, use the Windows built-in cert store (per my original patch).

* On Mac and Linux, just do the exact same thing sslrootcert=system currently does.

Is there any realistic prospect of this? I appreciate that it’s not the result of a lengthy, thorough and principled UX evaluation. On the other hand, it’s a few lines of code that could enable a pretty big improvement in security for many users’ Postgres connections much sooner.

(3) Any other ideas?

--
George MacKerron

#10Christoph Berg
myon@debian.org
In reply to: George MacKerron (#9)
Re: Making sslrootcert=system work on Windows psql

Re: George MacKerron

(3) Any other ideas?

I'm not a fan of "security by adding more connection parameters".

What are the chances of making "use the system/os default CA store"
the default? "sslmode=require" would then already actually "require" a
certificate if I'm reading the docs right. This would match user
expectation for POLA.

This default could then be pointed at the correct locations (plural)
on all operating systems. (sslrootcert=system:wincert:otherlocation?)

The "default default" would still be sslmode=prefer so it wouldn't
break today's normal case. Users of sslmode=require will understand
that supplying a CA certificate is no longer optional.

Perhaps add a sslmode=require-weak could be added as a workaround.

Christoph

#11George MacKerron
george@mackerron.co.uk
In reply to: Christoph Berg (#10)
Re: Making sslrootcert=system work on Windows psql

On 3 Apr 2025, at 14:28, Christoph Berg <myon@debian.org> wrote:

What are the chances of making "use the system/os default CA store"
the default? "sslmode=require" would then already actually "require" a
certificate if I'm reading the docs right. This would match user
expectation for POLA.

Right: the issue at present is that sslmode=require does require a certificate, but IIRC basically any old certificate will do. It doesn’t need to be signed by any particular CA. It doesn’t even need to have the server’s name on it.

This default could then be pointed at the correct locations (plural)
on all operating systems. (sslrootcert=system:wincert:otherlocation?)

The "default default" would still be sslmode=prefer so it wouldn't
break today's normal case. Users of sslmode=require will understand
that supplying a CA certificate is no longer optional.

Perhaps add a sslmode=require-weak could be added as a workaround.

I would love it if sslmode=require started verifying against OS cert stores and so became secure against MITM attacks. I’d certainly support that. But I would say that’s a much bigger backwards-incompatible change than the one I was asking for. :)

--
George MacKerron

#12Daniel Gustafsson
daniel@yesql.se
In reply to: George MacKerron (#9)
Re: Making sslrootcert=system work on Windows psql

On 3 Apr 2025, at 14:41, George MacKerron <george@mackerron.co.uk> wrote:

(2) sslrootcert=system on Windows doesn’t do a thing that would be extremely useful in some common situations. Namely: connecting securely to servers that present a certificate signed by a public CA.

Just to be clear, does (2) happens when the OpenSSL installation has a bogus
OPENSSLDIR value, or does it happen regardless?

Your diff certainly fixes (1b), so it’s definitely an improvement.

Thanks, unless Jacob objects I propose to apply that backpatched down to when
sslrootcert=system went in.

(2) Your idea of a new parameter, or a new value of sslrootcert, is what I was also starting to mull this morning. Is there any chance at all this could be done for Postgres 18

Not really, a net new patch mere days before the feature freeze is not really
how we do development. Especially for security related work on a platform most
developers aren't intimately familiar with.

or, failing that, 18.1?

Minor revisions only ever get bugfixes, never features or changed behaviour of
features.

I quite like sslrootcert=os: it’s snappy, and it implies that the Operating System root certs are going to be used (which is what I would have liked sslrootcert=system to mean). Some possible alternatives might be sslrootcert=public-cas or sslrootcert=os-default.

The thing is, we don't know that sslrootcert=os will mean the operating system
root certs. We are limited to what the OpenSSL API provides, and what can ask
OpenSSL to do is use its defaults.

The key thing is that it should work out-of-the-box basically everywhere, so my preferred behaviour for it would be:

* On Windows, use the Windows built-in cert store (per my original patch).

* On Mac and Linux, just do the exact same thing sslrootcert=system currently does.

Is there any realistic prospect of this?

IMV there isn't. I can't see it being an improvement to switch the meaning of
a value based on the underlying OpenSSL version, especially since the current
meaning might be useful for some installations who would then lose that
ability. I am convinced we need to do be able to use the defaults (as we do
now) *and* use winstore and whatever new stores come, not that one replaces the
other.

I appreciate that it’s not the result of a lengthy, thorough and principled UX evaluation. On the other hand, it’s a few lines of code that could enable a pretty big improvement in security for many users’ Postgres connections much sooner.

To be clear, wanting to make postgres more secure is a Good Thing, and your
efforts are much appreciated! Don't take no's in this thread as an objection
to your idea and mission. Most likely we will support winstore in some way in
v19, we just need to make sure we develop features in a way which is
sustainable wrt our available resources and our development process.

--
Daniel Gustafsson

#13Daniel Gustafsson
daniel@yesql.se
In reply to: Daniel Gustafsson (#12)
Re: Making sslrootcert=system work on Windows psql

On 3 Apr 2025, at 16:26, Daniel Gustafsson <daniel@yesql.se> wrote:

On 3 Apr 2025, at 14:41, George MacKerron <george@mackerron.co.uk> wrote:

Your diff certainly fixes (1b), so it’s definitely an improvement.

Thanks, unless Jacob objects I propose to apply that backpatched down to when
sslrootcert=system went in.

Hearing no objections I've committed this with a backpatch down to v16.

--
Daniel Gustafsson

#14George MacKerron
george@mackerron.co.uk
In reply to: Daniel Gustafsson (#12)
Re: Making sslrootcert=system work on Windows psql

On 3 Apr 2025, at 15:26, Daniel Gustafsson <daniel@yesql.se> wrote:

I quite like sslrootcert=os: it’s snappy, and it implies that the Operating System root certs are going to be used (which is what I would have liked sslrootcert=system to mean). Some possible alternatives might be sslrootcert=public-cas or sslrootcert=os-default.

The thing is, we don't know that sslrootcert=os will mean the operating system
root certs. We are limited to what the OpenSSL API provides, and what can ask
OpenSSL to do is use its defaults.

The key thing is that it should work out-of-the-box basically everywhere, so my preferred behaviour for it would be:

* On Windows, use the Windows built-in cert store (per my original patch).

* On Mac and Linux, just do the exact same thing sslrootcert=system currently does.

Is there any realistic prospect of this?

IMV there isn't. I can't see it being an improvement to switch the meaning of
a value based on the underlying OpenSSL version, especially since the current
meaning might be useful for some installations who would then lose that
ability. I am convinced we need to do be able to use the defaults (as we do
now) *and* use winstore and whatever new stores come, not that one replaces the
other.

I appreciate that it’s not the result of a lengthy, thorough and principled UX evaluation. On the other hand, it’s a few lines of code that could enable a pretty big improvement in security for many users’ Postgres connections much sooner.

To be clear, wanting to make postgres more secure is a Good Thing, and your
efforts are much appreciated! Don't take no's in this thread as an objection
to your idea and mission. Most likely we will support winstore in some way in
v19, we just need to make sure we develop features in a way which is
sustainable wrt our available resources and our development process.

Thanks for your appreciation! It might be good to start thinking about how things might look in v19, then?

Perhaps I can start things off with one smaller idea and one bigger one.

SMALLER IDEA

I’d suggest two new special sslrootcert values:

(1) sslrootcert=openssl

This does exactly what sslrootcert=system does now, but is less confusingly named for Windows users. sslrootcert=system becomes a deprecated synonym for this option.

(2) sslrootcert=os

This does what I was proposing in my patch: it uses winstore on Windows and behaves the same as sslrootcert=openssl elsewhere, where openssl *is* the operating system SSL provider.

These changes would be fully backwards-compatible.

BIGGER IDEA

A much bigger, backwards-incompatible shake-up of libpq security parameters might incorporate the above changes, and then proceed something like this:

* Entirely remove the current default, sslmode=prefer, and make explicitly asking for sslmode=prefer an error. After all, as the docs themselves point out for sslmode=prefer: “this makes no sense from a security point of view”.

* Rename sslmode=require to sslmode=insecure, because it’s vulnerable to MITM attacks, and ideally people would get a sense of that without reading the relevant page of the docs. Make asking for sslmode=require an error (with a helpful explanation pointing out the rename to sslmode=insecure).

* Retain sslmode=verify-ca and sslmode=verify-full.

* Create a new option, sslmode=secure, which means sslmode=verify-full + sslrootcert=os. Make this the default!

In summary, you end up with these as sslmode values:

* disabled
* insecure (formerly known as require)
* verify-ca
* verify-full
* secure (the new default, meaning sslmode=verify-full + sslrootcert=os)

Obviously this would need to be well-trailed ahead of time, as some people would need to make changes to how they use psql/libpq. But it would peg the default security of a Postgres connection at the same level as the security of any random blog page (which I think is a bare minimum one might aspire to).

Please do all suggest better ideas!

All the best,
George

#15Christoph Berg
myon@debian.org
In reply to: George MacKerron (#14)
sslmode=secure by default (Re: Making sslrootcert=system work on Windows psql)

Re: George MacKerron

SMALLER IDEA

I’d suggest two new special sslrootcert values:

(1) sslrootcert=openssl

This does exactly what sslrootcert=system does now, but is less confusingly named for Windows users. sslrootcert=system becomes a deprecated synonym for this option.

(2) sslrootcert=os

This does what I was proposing in my patch: it uses winstore on Windows and behaves the same as sslrootcert=openssl elsewhere, where openssl *is* the operating system SSL provider.

These changes would be fully backwards-compatible.

On Linux/*ix, there would be 3 things that are all the same.

If the Windows Openssl store is that bad, wouldn't the smarter thing
to do for PG19 to use winstore by default? The Openssl one would still
be available when requested explicitly. This would avoid the
proliferation of default values.

BIGGER IDEA

* Entirely remove the current default, sslmode=prefer, and make explicitly asking for sslmode=prefer an error. After all, as the docs themselves point out for sslmode=prefer: “this makes no sense from a security point of view”.

(It's not really secure, but opportunistic "use SSL when available" is
still better than nothing.)

* Create a new option, sslmode=secure, which means sslmode=verify-full + sslrootcert=os. Make this the default!

I like the name.

In summary, you end up with these as sslmode values:

* disabled
* insecure (formerly known as require)
* verify-ca
* verify-full
* secure (the new default, meaning sslmode=verify-full + sslrootcert=os)

Obviously this would need to be well-trailed ahead of time, as some people would need to make changes to how they use psql/libpq. But it would peg the default security of a Postgres connection at the same level as the security of any random blog page (which I think is a bare minimum one might aspire to).

I agree that this would be a good change for SSL users, and also one
that people would likely be willing to buy.

The big problem here is that a lot of installations are not using SSL
at all (default on RPM), and another big chunk is using SSL, but
relying on the default snakeoil certificates to just work (default on
Debian), so this would not be "some people" but more like "everyone
except the few who have already configured certificates properly".

These people would have to change every single connection string to
include "sslmode=disabled" or the like. This will likely not be
received well.

Before we can make this change, I think we would have to improve the
UX. psql does not even have any --switch for it. PostgreSQL serving
non-SSL and SSL on the same port doesn't make the UX better... :-/

Christoph

#16George MacKerron
george@mackerron.co.uk
In reply to: Christoph Berg (#15)
Re: sslmode=secure by default (Re: Making sslrootcert=system work on Windows psql)

On Linux/*ix, there would be 3 things that are all the same.

If the Windows Openssl store is that bad, wouldn't the smarter thing
to do for PG19 to use winstore by default? The Openssl one would still
be available when requested explicitly. This would avoid the
proliferation of default values.

I agree ... but I think that looks rather like my most recent (rejected) patch?

However, perhaps we could extend that patch for greater backwards-compatibility, checking not only that the SSL_CERT_DIR and SSL_CERT_FILE environment variables are not set, but *also* that there is no cert.pem file and no certs/ directory inside OPENSSLDIR.

I think that should make the behaviour backwards-compatible for all scenarios *except* those that would otherwise be guaranteed to fail certificate verification because we are on Windows and there are no OpenSSL certificates configured on the system. It seems fairly safe to assume that people who are using sslrootcert=system on Windows and without any configured OpenSSL certs are not doing so with the deliberate intention that all connections should fail!

I attach a patch that would do this (side-by-side view at https://github.com/postgres/postgres/compare/master...jawj:postgres:jawj-sslrootcert-system-windows).

An advantage of this approach would be that people building Postgres who want this behaviour sooner than next year could also patch it into versions 16 – 18 without much trouble.

BIGGER IDEA

In summary, you end up with these as sslmode values:

* disabled
* insecure (formerly known as require)
* verify-ca
* verify-full
* secure (the new default, meaning sslmode=verify-full + sslrootcert=os)

Obviously this would need to be well-trailed ahead of time, as some people would need to make changes to how they use psql/libpq. But it would peg the default security of a Postgres connection at the same level as the security of any random blog page (which I think is a bare minimum one might aspire to).

I agree that this would be a good change for SSL users, and also one
that people would likely be willing to buy.

The big problem here is that a lot of installations are not using SSL
at all (default on RPM), and another big chunk is using SSL, but
relying on the default snakeoil certificates to just work (default on
Debian), so this would not be "some people" but more like "everyone
except the few who have already configured certificates properly".

These people would have to change every single connection string to
include "sslmode=disabled" or the like. This will likely not be
received well.

Before we can make this change, I think we would have to improve the
UX. psql does not even have any --switch for it. PostgreSQL serving
non-SSL and SSL on the same port doesn't make the UX better... :-/

How do you think the UX could be improved? Maybe by using a psql switch and/or an env var to opt out of (or initially even to opt into) the new sslmode treatment?

Attachments:

sslrootcert-system-windows.diffapplication/octet-stream; name=sslrootcert-system-windows.diff; x-unix-mode=0644Download+17-1
#17Christoph Berg
myon@debian.org
In reply to: George MacKerron (#16)
Re: sslmode=secure by default (Re: Making sslrootcert=system work on Windows psql)

Re: George MacKerron

Before we can make this change, I think we would have to improve the
UX. psql does not even have any --switch for it. PostgreSQL serving
non-SSL and SSL on the same port doesn't make the UX better... :-/

How do you think the UX could be improved? Maybe by using a psql switch and/or an env var to opt out of (or initially even to opt into) the new sslmode treatment?

The env var is already there (PGSSLMODE).

Now you can say `psql -h db.example.com -p 5433 dbfoo`, but for
specifying the sslmode, you have to rewrite at least the last argument
to use connection string syntax, `psql "dbname=dbfoo sslmode=verify-full`.
This needs be be less cumbersome. (And the names of the options make
me want to stay away from them, require/verify-ca/verify-full/verify-confusing.
Your sslmode=secure idea is really good.)

It should be as simple as
psql --ssl (= sslmode=secure)
psql --insecure (the old sslmode=require)
psql --no-ssl (= sslmode=disable)

psql -s and -S are unfortunately already taken :-/

For connection strings, perhaps the best action is to tell people that
always including "sslmode=something" is best practise. For libpq-style
key=value connection strings, that wouldn't even be ugly. For
postgresql://-style strings, we would ideally have something like http://
vs https://, but I am not sure how to squeeze that into the syntax.
(Appending ?sslmode= works, but meh.)

Christoph

#18Peter Eisentraut
peter_e@gmx.net
In reply to: Christoph Berg (#17)
Re: sslmode=secure by default (Re: Making sslrootcert=system work on Windows psql)

On 24.04.25 12:53, Christoph Berg wrote:

Now you can say `psql -h db.example.com -p 5433 dbfoo`, but for
specifying the sslmode, you have to rewrite at least the last argument
to use connection string syntax, `psql "dbname=dbfoo sslmode=verify-full`.
This needs be be less cumbersome. (And the names of the options make
me want to stay away from them, require/verify-ca/verify-full/verify-confusing.
Your sslmode=secure idea is really good.)

I'm generally in favor of making sslmode=verify-full the effective
default somehow.

Another detail to think about is how this affects psql -h localhost. In
principle, this should require full SSL, but you're probably not going
to have certificates that allow "localhost". And connections to
localhost are the default on Windows. We could also switch the Windows
default to Unix-domain sockets. But there are probably still other
reasons why connections to TCP/IP localhost are made. Some things to
think about.

#19Jacob Champion
jacob.champion@enterprisedb.com
In reply to: Peter Eisentraut (#18)
Re: sslmode=secure by default (Re: Making sslrootcert=system work on Windows psql)

On Thu, Apr 24, 2025 at 5:00 AM Peter Eisentraut <peter@eisentraut.org> wrote:

I'm generally in favor of making sslmode=verify-full the effective
default somehow.

+many

On Thu, Apr 24, 2025 at 3:53 AM Christoph Berg <myon@debian.org> wrote:

For
postgresql://-style strings, we would ideally have something like http://
vs https://, but I am not sure how to squeeze that into the syntax.

Not to derail things too much, but I'd also like a postgress://
scheme, and I've put a little bit of idle thought into it. I think
we'd want it to imply sslnegotiation=direct and sslrootcert=system
(modulo the Windows discussion already in progress), and potentially
make a bunch of stricter decisions about TLS settings to better match
modern practice. The intent would be to have a "browser-strength"
scheme for people who care more about security than about raw
compatibility with older systems, because they're connecting to
someone else's servers on the open Web.

The hardest part, in my opinion, is that we'd have to start following
the RFC concept of "authority". A URL of
"postgress://example.com/db?host=evil.com&hostaddr=..." is outright
dangerous, as is "postgress://example.com/db?sslmode=disable". So if
there's interest in that scheme, I think it should remain a separate
feature from "verify-full by default", because there's a lot more to
figure out.

--Jacob

#20Jacob Champion
jacob.champion@enterprisedb.com
In reply to: George MacKerron (#14)
Re: Making sslrootcert=system work on Windows psql

On Wed, Apr 23, 2025 at 8:47 AM George MacKerron <george@mackerron.co.uk> wrote:

I’d suggest two new special sslrootcert values:

(1) sslrootcert=openssl

This does exactly what sslrootcert=system does now, but is less confusingly named for Windows users. sslrootcert=system becomes a deprecated synonym for this option.

Stealing the word "system" from the existing sslrootcert domain had at
least two hazards: a) existing users might have a file named "system"
that would now be ignored, and b) users might accidentally use
sslrootcert=system on older versions of libpq, picking up an
unexpected file named "system" and doing the Wrong Thing. Problem (a)
can be worked around by saying "./system" instead, so honestly I
wasn't too concerned about that, and I considered (b) to be more of a
theoretical problem that was outweighed by the benefit of getting
OpenSSL to just Do The Thing people wanted it to do.

A couple years on, I think (b) is less theoretical than I had
originally hoped. As evidence I point to Stack Overflow questions like
[1]: https://stackoverflow.com/questions/77989772/psql-root-certificate-file-system-does-not-exist-why-sslrootcert-system-do
how connection string versioning works. If we steal more words, I
think that problem is going to get worse. So I'm leaning toward's
Daniel's earlier position that sslrootcert has kind of run its course,
and if you want to select OpenSSL stores, we need a more fully
featured syntax and probably a completely new option to be able to
pass that through safely.

(2) sslrootcert=os

This does what I was proposing in my patch: it uses winstore on Windows and behaves the same as sslrootcert=openssl elsewhere, where openssl *is* the operating system SSL provider.

Falling back to standard OpenSSL introduces the same hazard we're
running into today, though -- what if someone creates a macstore [2]https://github.com/openssl/openssl/issues/23460
for OpenSSL, so that its behavior matches Safari's or whatever, and
then everyone wonders why sslrootcert=os doesn't use that? If the
abstraction must leak the details anyway, I think we should expose
them directly instead.

(As a small soapbox, I think "application-level" fallback for a trust
chain is frequently going to lead to regret. You should ideally tell
us what you want, and either get it or fail.)

Thanks,
--Jacob

[1]: https://stackoverflow.com/questions/77989772/psql-root-certificate-file-system-does-not-exist-why-sslrootcert-system-do
[2]: https://github.com/openssl/openssl/issues/23460

#21Jelte Fennema-Nio
postgres@jeltef.nl
In reply to: George MacKerron (#14)
#22Jelte Fennema-Nio
postgres@jeltef.nl
In reply to: Jacob Champion (#19)
#23Jelte Fennema-Nio
postgres@jeltef.nl
In reply to: Jelte Fennema-Nio (#21)
#24George MacKerron
george@mackerron.co.uk
In reply to: Jelte Fennema-Nio (#23)
#25George MacKerron
george@mackerron.co.uk
In reply to: Jacob Champion (#20)
#26Jelte Fennema-Nio
postgres@jeltef.nl
In reply to: George MacKerron (#25)
#27George MacKerron
george@mackerron.co.uk
In reply to: Jelte Fennema-Nio (#26)
#28Daniel Gustafsson
daniel@yesql.se
In reply to: Daniel Gustafsson (#12)
#29Daniel Gustafsson
daniel@yesql.se
In reply to: Jelte Fennema-Nio (#22)
#30George MacKerron
george@mackerron.co.uk
In reply to: Daniel Gustafsson (#28)
#31Daniel Gustafsson
daniel@yesql.se
In reply to: George MacKerron (#30)
#32Jacob Champion
jacob.champion@enterprisedb.com
In reply to: Jelte Fennema-Nio (#22)
#33Jacob Champion
jacob.champion@enterprisedb.com
In reply to: Peter Eisentraut (#18)