Experiments with Postgres and SSL
I had a conversation a while back with Heikki where he expressed that
it was annoying that we negotiate SSL/TLS the way we do since it
introduces an extra round trip. Aside from the performance
optimization I think accepting standard TLS connections would open the
door to a number of other opportunities that would be worth it on
their own.
So I took a look into what it would take to do and I think it would
actually be quite feasible. The first byte of a standard TLS
connection can't look anything like the first byte of any flavour of
Postgres startup packet because it would be the high order bits of the
length so unless we start having multi-megabyte startup packets....
So I put together a POC patch and it's working quite well and didn't
require very much kludgery. Well, it required some but it's really not
bad. I do have a bug I'm still trying to work out and the code isn't
quite in committable form but I can send the POC patch.
Other things it would open the door to in order from least
controversial to most....
* Hiding Postgres behind a standard SSL proxy terminating SSL without
implementing the Postgres protocol.
* "Service Mesh" type tools that hide multiple services behind a
single host/port ("Service Mesh" is just a new buzzword for "proxy").
* Browser-based protocol implementations using websockets for things
like pgadmin or other tools to connect directly to postgres using
Postgres wire protocol but using native SSL implementations.
* Postgres could even implement an HTTP based version of its protocol
and enable things like queries or browser based tools using straight
up HTTP requests so they don't need to use websockets.
* Postgres could implement other protocols to serve up data like
status queries or monitoring metrics, using HTTP based standard
protocols instead of using our own protocol.
Incidentally I find the logic in ProcessStartupPacket incredibly
confusing. It took me a while before I realized it's using tail
recursion to implement the startup logic. I think it would be way more
straightforward and extensible if it used a much more common iterative
style. I think it would make it possible to keep more state than just
ssl_done and gss_done without changing the function signature every
time for example.
--
greg
On Wed, Jan 18, 2023 at 7:16 PM Greg Stark <stark@mit.edu> wrote:
So I took a look into what it would take to do and I think it would
actually be quite feasible. The first byte of a standard TLS
connection can't look anything like the first byte of any flavour of
Postgres startup packet because it would be the high order bits of the
length so unless we start having multi-megabyte startup packets....
This is a fascinating idea! I like it a lot.
But..do we have to treat any unknown start sequence of bytes as a TLS
connection? Or is there some definite subset of possible first bytes
that clearly indicates that this is a TLS connection or not?
Best regards, Andrey Borodin.
On Thu, 19 Jan 2023 at 00:45, Andrey Borodin <amborodin86@gmail.com> wrote:
But..do we have to treat any unknown start sequence of bytes as a TLS
connection? Or is there some definite subset of possible first bytes
that clearly indicates that this is a TLS connection or not?
Absolutely not, there's only one MessageType that can initiate a
connection, ClientHello, so the initial byte has to be a specific
value. (0x16)
And probably to implement HTTP/Websocket it would probably only peek
at the first byte and check for things like G(ET) and H(EAD) and so
on, possibly only over SSL but in theory it could be over any
connection if the request comes before the startup packet.
Personally I'm motivated by wanting to implement status and monitoring
data for things like Prometheus and the like. For that it would just
be simple GET queries to recognize. But tunneling pg wire protocol
over websockets sounds cool but not really something I know a lot
about. I note that Neon is doing something similar with a proxy:
https://neon.tech/blog/serverless-driver-for-postgres
--
greg
It would be great if PostgreSQL supported 'start with TLS', however, how
could clients activate the feature?
I would like to refrain users from configuring the handshake mode, and I
would like to refrain from degrading performance when a new client talks to
an old database.
What if the server that supports 'fast TLS' added an extra notification in
case client connects with a classic TLS?
Then a capable client could remember host:port and try with newer TLS
appoach the next time it connects.
It would be transparent to the clients, and the users won't need to
configure 'prefer classic or fast TLS'
The old clients could discard the notification.
Vladimir
--
Vladimir
On Thu, 19 Jan 2023 at 15:49, Vladimir Sitnikov
<sitnikov.vladimir@gmail.com> wrote:
What if the server that supports 'fast TLS' added an extra notification in case client connects with a classic TLS?
Then a capable client could remember host:port and try with newer TLS appoach the next time it connects.It would be transparent to the clients, and the users won't need to configure 'prefer classic or fast TLS'
The old clients could discard the notification.
Hm. I hadn't really thought about the case of a new client connecting
to an old server. I don't think it's worth implementing a code path in
the server like this as it would then become cruft that would be hard
to ever get rid of.
I think you can do the same thing, more or less, in the client. Like
if the driver tries to connect via SSL and gets an error it remembers
that host/port and connects using negotiation in the future.
In practice though, by the time drivers support this it'll probably be
far enough in the future that they can just enable it and you can
disable it if you're connecting to an old server. The main benefit for
the near term is going to be clients that are specifically designed to
take advantage of it because it's necessary to enable the environment
they need -- like monitoring tools and proxies.
I've attached the POC. It's not near committable, mainly because of
the lack of any proper interface to the added fields in Port. I
actually had a whole API but ripped it out while debugging because it
wasn't working out.
But here's an example of psql connecting to the same server via
negotiated SSL or through stunnel where stunnel establishes the SSL
connection and psql is just doing plain text:
stark@hatter:~/src/postgresql$ ~/pgsql-sslhacked/bin/psql
'postgresql://localhost:9432/postgres'
psql (16devel)
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384,
compression: off)
Type "help" for help.
postgres=# select * from pg_stat_ssl;
pid | ssl | version | cipher | bits | client_dn |
client_serial | issuer_dn
-------+-----+---------+------------------------+------+-----------+---------------+-----------
48771 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | |
|
(1 row)
postgres=# \q
stark@hatter:~/src/postgresql$ ~/pgsql-sslhacked/bin/psql
'postgresql://localhost:8999/postgres'
psql (16devel)
Type "help" for help.
postgres=# select * from pg_stat_ssl;
pid | ssl | version | cipher | bits | client_dn |
client_serial | issuer_dn
-------+-----+---------+------------------------+------+-----------+---------------+-----------
48797 | t | TLSv1.3 | TLS_AES_256_GCM_SHA384 | 256 | |
|
(1 row)
--
greg
Attachments:
v1-0001-Direct-SSL-Connections.patchtext/x-patch; charset=US-ASCII; name=v1-0001-Direct-SSL-Connections.patchDownload
From 4508f872720a0977cf00041a865d76a4d5f77028 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Wed, 18 Jan 2023 15:34:34 -0500
Subject: [PATCH v1] Direct SSL Connections
---
src/backend/libpq/be-secure.c | 13 +++
src/backend/libpq/pqcomm.c | 9 +-
src/backend/postmaster/postmaster.c | 140 ++++++++++++++++++++++------
src/include/libpq/libpq-be.h | 3 +
src/include/libpq/libpq.h | 2 +-
5 files changed, 134 insertions(+), 33 deletions(-)
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index a0f7084018..39366d04dd 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -235,6 +235,19 @@ secure_raw_read(Port *port, void *ptr, size_t len)
{
ssize_t n;
+ /* XXX feed the raw_buf into SSL */
+ if (port->raw_buf_remaining > 0)
+ {
+ /* consume up to len bytes from the raw_buf */
+ if (len > port->raw_buf_remaining)
+ len = port->raw_buf_remaining;
+ Assert(port->raw_buf);
+ memcpy(ptr, port->raw_buf + port->raw_buf_consumed, len);
+ port->raw_buf_consumed += len;
+ port->raw_buf_remaining -= len;
+ return len;
+ }
+
/*
* Try to read from the socket without blocking. If it succeeds we're
* done, otherwise we'll wait for the socket using the latch mechanism.
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index 864c9debe8..60fab6a52b 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -1119,13 +1119,16 @@ pq_discardbytes(size_t len)
/* --------------------------------
* pq_buffer_has_data - is any buffered data available to read?
*
- * This will *not* attempt to read more data.
+ * Actually returns the number of bytes in the buffer...
+ *
+ * This will *not* attempt to read more data. And reading up to that number of
+ * bytes should not cause reading any more data either.
* --------------------------------
*/
-bool
+size_t
pq_buffer_has_data(void)
{
- return (PqRecvPointer < PqRecvLength);
+ return (PqRecvLength - PqRecvPointer);
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 9cedc1b9f0..b1631e0830 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -412,6 +412,7 @@ static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(Port *port);
+static int ProcessSSLStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -1909,6 +1910,104 @@ ServerLoop(void)
}
}
+/*
+ * Check for a native direct SSL connection.
+ *
+ * This happens before startup packets so we are careful not to actual read
+ * any bytes from the stream if it's not a direct SSL connection.
+ */
+
+static int
+ProcessSSLStartup(Port *port)
+{
+ int firstbyte;
+
+ pq_startmsgread();
+
+ firstbyte = pq_peekbyte();
+ if (firstbyte == EOF)
+ {
+ /*
+ * If we get no data at all, don't clutter the log with a complaint;
+ * such cases often occur for legitimate reasons. An example is that
+ * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
+ * client didn't like our response, it'll probably just drop the
+ * connection. Service-monitoring software also often just opens and
+ * closes a connection without sending anything. (So do port
+ * scanners, which may be less benign, but it's not really our job to
+ * notice those.)
+ */
+ return STATUS_ERROR;
+ }
+
+ /*
+ * First byte indicates standard SSL handshake message
+ *
+ * (It can't be a Postgres startup length because in network byte order
+ * that would be a startup packet hundreds of megabytes long)
+ */
+ if (firstbyte == 0x16)
+ {
+#ifdef USE_SSL
+ ssize_t len;
+ char *buf = NULL;
+ elog(LOG, "Detected direct SSL handshake");
+
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ buf = palloc(len);
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+
+ Assert(pq_buffer_has_data() == 0);
+ if (secure_open_server(port) == -1)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("SSL Protocol Error during direct SSL connection initiation")));
+ return STATUS_ERROR;
+ }
+
+ if (port->raw_buf_remaining > 0)
+ {
+ /* This shouldn't be possible -- it would mean the client sent
+ * encrypted data before we established a session key...
+ */
+ elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ return STATUS_ERROR;
+ }
+ pfree(port->raw_buf);
+ ereport(DEBUG2,
+ (errmsg_internal("Direct native SSL connection set up")));
+
+#else
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with no SSL support")));
+ return STATUS_ERROR;
+#endif
+ }
+
+ pq_endmsgread();
+
+ if (port->ssl_in_use)
+ ereport(DEBUG2,
+ (errmsg_internal("Direct native SSL connection set up")));
+ else
+ ereport(DEBUG2,
+ (errmsg_internal("Direct native SSL connection NOT set up")));
+
+
+ return STATUS_OK;
+}
+
+
/*
* Read a client's startup packet and do something according to it.
*
@@ -1937,28 +2036,7 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
pq_startmsgread();
- /*
- * Grab the first byte of the length word separately, so that we can tell
- * whether we have no data at all or an incomplete packet. (This might
- * sound inefficient, but it's not really, because of buffering in
- * pqcomm.c.)
- */
- if (pq_getbytes((char *) &len, 1) == EOF)
- {
- /*
- * If we get no data at all, don't clutter the log with a complaint;
- * such cases often occur for legitimate reasons. An example is that
- * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
- * client didn't like our response, it'll probably just drop the
- * connection. Service-monitoring software also often just opens and
- * closes a connection without sending anything. (So do port
- * scanners, which may be less benign, but it's not really our job to
- * notice those.)
- */
- return STATUS_ERROR;
- }
-
- if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
+ if (pq_getbytes(((char *) &len), 4) == EOF)
{
/* Got a partial length word, so bleat about that */
if (!ssl_done && !gss_done)
@@ -2015,8 +2093,11 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
char SSLok;
#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ /* No SSL when disabled or on Unix sockets.
+ *
+ * Also no SSL negotiation if we already have a direct SSL
+ */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX || port->ssl_in_use)
SSLok = 'N';
else
SSLok = 'S'; /* Support for SSL */
@@ -2024,11 +2105,10 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
SSLok = 'N'; /* No support for SSL */
#endif
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
+ while (secure_write(port, &SSLok, 1) != 1)
{
if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
+ continue; /* if interrupted, just retry */
ereport(COMMERROR,
(errcode_for_socket_access(),
errmsg("failed to send SSL negotiation response: %m")));
@@ -2069,7 +2149,7 @@ retry1:
GSSok = 'G';
#endif
- while (send(port->sock, &GSSok, 1, 0) != 1)
+ while (secure_write(port, &GSSok, 1) != 1)
{
if (errno == EINTR)
continue;
@@ -4372,7 +4452,9 @@ BackendInitialize(Port *port)
* Receive the startup packet (which might turn out to be a cancel request
* packet).
*/
- status = ProcessStartupPacket(port, false, false);
+ status = ProcessSSLStartup(port);
+ if (status == STATUS_OK)
+ status = ProcessStartupPacket(port, false, false);
/*
* Disable the timeout, and prevent SIGTERM again.
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 8c70b2fd5b..c2402ea8b8 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -226,6 +226,9 @@ typedef struct Port
SSL *ssl;
X509 *peer;
#endif
+ /* XXX */
+ char *raw_buf;
+ ssize_t raw_buf_consumed, raw_buf_remaining;
} Port;
#ifdef USE_SSL
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 50fc781f47..2b02f67257 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -80,7 +80,7 @@ extern int pq_getmessage(StringInfo s, int maxlen);
extern int pq_getbyte(void);
extern int pq_peekbyte(void);
extern int pq_getbyte_if_available(unsigned char *c);
-extern bool pq_buffer_has_data(void);
+extern size_t pq_buffer_has_data(void);
extern int pq_putmessage_v2(char msgtype, const char *s, size_t len);
extern bool pq_check_connection(void);
--
2.39.0
On Wed, Jan 18, 2023 at 7:16 PM Greg Stark <stark@mit.edu> wrote:
I had a conversation a while back with Heikki where he expressed that
it was annoying that we negotiate SSL/TLS the way we do since it
introduces an extra round trip. Aside from the performance
optimization I think accepting standard TLS connections would open the
door to a number of other opportunities that would be worth it on
their own.
Nice! I want this too, but for security reasons [1]/messages/by-id/fcc3ebeb7f05775b63f3207ed52a54ea5d17fb42.camel@vmware.com -- I want to be
able to turn off negotiated (explicit) TLS, to force (implicit)
TLS-only mode.
Other things it would open the door to in order from least
controversial to most....* Hiding Postgres behind a standard SSL proxy terminating SSL without
implementing the Postgres protocol.
+1
* "Service Mesh" type tools that hide multiple services behind a
single host/port ("Service Mesh" is just a new buzzword for "proxy").
If you want to multiplex protocols on a port, now is an excellent time
to require clients to use ALPN on implicit-TLS connections. (There are
no clients that can currently connect via implicit TLS, so you'll
never have another chance to force the issue without breaking
backwards compatibility.) That should hopefully make it harder to
ALPACA yourself or others [2]https://alpaca-attack.com/.
ALPN doesn't prevent cross-port attacks though, and speaking of those...
* Browser-based protocol implementations using websockets for things
like pgadmin or other tools to connect directly to postgres using
Postgres wire protocol but using native SSL implementations.* Postgres could even implement an HTTP based version of its protocol
and enable things like queries or browser based tools using straight
up HTTP requests so they don't need to use websockets.* Postgres could implement other protocols to serve up data like
status queries or monitoring metrics, using HTTP based standard
protocols instead of using our own protocol.
I see big red warning lights going off in my head -- in a previous
life, I got to fix vulnerabilities that resulted from bolting HTTP
onto existing protocol servers. Not only do you opt into the browser
security model forever, you also gain the ability to speak for any
other web server already running on the same host.
(I know you have PG committers who are also HTTP experts, and I think
you were hacking on mod_perl well before I knew web servers existed.
Just... please be careful. ;D )
Incidentally I find the logic in ProcessStartupPacket incredibly
confusing. It took me a while before I realized it's using tail
recursion to implement the startup logic. I think it would be way more
straightforward and extensible if it used a much more common iterative
style. I think it would make it possible to keep more state than just
ssl_done and gss_done without changing the function signature every
time for example.
+1. The complexity of the startup logic, both client- and server-side,
is a big reason why I want implicit TLS in the first place. That way,
bugs in that code can't be exploited before the TLS handshake
completes.
Thanks!
--Jacob
[1]: /messages/by-id/fcc3ebeb7f05775b63f3207ed52a54ea5d17fb42.camel@vmware.com
[2]: https://alpaca-attack.com/
I don't think it's worth implementing a code path in
the server like this as it would then become cruft that would be hard
to ever get rid of.
Do you think the server can de-support the old code path soon?
I think you can do the same thing, more or less, in the client. Like
if the driver tries to connect via SSL and gets an error it remembers
that host/port and connects using negotiation in the future.
Well, I doubt everybody would instantaneously upgrade to the database that
supports fast TLS,
so there will be a timeframe when there will be a lot of old databases, and
the clients will be new.
In that case, going with "try fast, ignore exception" would degrade
performance for old databases.
I see you suggest caching, however, "degrading one of the cases" might be
more painful than
"not improving one of the cases".
I would like to refrain from implementing "parallel connect both ways
and check which is faster" in
PG clients (e.g. https://en.wikipedia.org/wiki/Happy_Eyeballs ).
Just wondering: do you consider back-porting the feature to all supported
DB versions?
In practice though, by the time drivers support this it'll probably be
far enough in the future
I think drivers release more often than the database, and we can get driver
support even before the database releases.
I'm from pgjdbc Java driver team, and I think it is unfair to suggest that
"driver support is only far enough in the future".
Vladimir
On Fri, 20 Jan 2023 at 01:41, Vladimir Sitnikov
<sitnikov.vladimir@gmail.com> wrote:
Do you think the server can de-support the old code path soon?
I don't have any intention to de-support anything. I really only
picture it being an option in environments where the client and server
are all part of a stack controlled by a single group. User tools and
general purpose tools are better served by our current more flexible
setup.
Just wondering: do you consider back-porting the feature to all supported DB versions?
I can't see that, no.
In practice though, by the time drivers support this it'll probably be
far enough in the futureI think drivers release more often than the database, and we can get driver support even before the database releases.
I'm from pgjdbc Java driver team, and I think it is unfair to suggest that "driver support is only far enough in the future".
Interesting. I didn't realize this would be so attractive to regular
driver authors. I did think of the Happy Eyeballs technique too but I
agree I wouldn't want to go that way either :)
I guess the server doesn't really have to do anything specific to do
what you want. You could just hard code that servers newer than a
specific version would have this support. Or it could be done with a
"protocol option" -- which wouldn't actually change any behaviour but
would be rejected if the server doesn't support "fast ssl" giving you
the feedback you expect without having much extra legacy complexity.
I guess a lot depends on the way the driver works and the way the
application is structured. Applications that make a single connection
or don't have shared state across connections wouldn't think this way.
And interfaces like libpq would normally just leave it up to the
application to make choices like this. But I guess JVM based
applications are more likely to have long-lived systems that make many
connections and also more likely to make it the driver's
responsibility to manage such things.
--
greg
You could just hard code that servers newer than a
specific version would have this support
Suppose PostgreSQL 21 implements "fast TLS"
Suppose pgjdbc 43 supports "fast TLS"
Suppose PgBouncer 1.17.0 does not support "fast TLS" yet
If pgjdbc connects to the DB via balancer, then the server would
respond with "server_version=21".
The balancer would forward "server_version", so the driver would
assume "fast TLS is supported".
In practice, fast TLS can't be used in that configuration since the
connection will fail when the driver attempts to ask
"fast TLS" from the PgBouncer.
Or it could be done with a "protocol option"
Would you please clarify what you mean by "protocol option"?
I guess a lot depends on the way the driver works and the way the
application is structured
There are cases when applications pre-create connections on startup,
so the faster connections are created the better.
The same case happens when the admin issues "reset connection pool",
so it discards old connections and creates new ones.
People rarely know all the knobs, so I would like to have a "fast by
default" design (e.g. server sending a notification "you may use fast
mode the next time")
rather than "keep old behaviour and require everybody to add fast=true
to their configuration" (e.g. users having to configure
"try_fast_tls_first=true")
Vladimir
On 20/01/2023 03:28, Jacob Champion wrote:
On Wed, Jan 18, 2023 at 7:16 PM Greg Stark <stark@mit.edu> wrote:
* "Service Mesh" type tools that hide multiple services behind a
single host/port ("Service Mesh" is just a new buzzword for "proxy").If you want to multiplex protocols on a port, now is an excellent time
to require clients to use ALPN on implicit-TLS connections. (There are
no clients that can currently connect via implicit TLS, so you'll
never have another chance to force the issue without breaking
backwards compatibility.) That should hopefully make it harder to
ALPACA yourself or others [2].
Good idea. Do we want to just require the protocol to be "postgres", or
perhaps "postgres/3.0"? Need to register that with IANA, I guess.
We implemented a protocol version negotiation mechanism in the libpq
protocol itself, how would this interact with it? If it's just
"postgres", then I guess we'd still negotiate the protocol version and
list of extensions after the TLS handshake.
Incidentally I find the logic in ProcessStartupPacket incredibly
confusing. It took me a while before I realized it's using tail
recursion to implement the startup logic. I think it would be way more
straightforward and extensible if it used a much more common iterative
style. I think it would make it possible to keep more state than just
ssl_done and gss_done without changing the function signature every
time for example.+1. The complexity of the startup logic, both client- and server-side,
is a big reason why I want implicit TLS in the first place. That way,
bugs in that code can't be exploited before the TLS handshake
completes.
+1. We need to support explicit TLS for a long time, so we can't
simplify by just removing it. But let's refactor the code somehow, to
make it more clear.
Looking at the patch, I think it accepts an SSLRequest packet even if
implicit TLS has already been established. That's surely wrong, and
shows how confusing the code is. (Or I'm reading it incorrectly, which
also shows how confusing it is :-) )
Regarding Vladimir's comments on how clients can migrate to this, I
don't have any great suggestions. To summarize, there are several options:
- Add an "fast_tls" option that the user can enable if they know the
server supports it
- First connect in old-fashioned way, and remember the server version.
Later, if you reconnect to the same server, use implicit TLS if the
server version was high enough. This would be most useful for connection
pools.
- Connect both ways at the same time, and continue with the fastest,
i.e. "happy eyeballs"
- Try implicit TLS first, and fall back to explicit TLS if it fails.
For libpq, we don't necessarily need to do anything right now. We can
add the implicit TLS support in a later version. Not having libpq
support makes it hard to test the server codepath, though. Maybe just
test it with 'stunnel' or 'openssl s_client'.
- Heikki
On Wed, Feb 22, 2023 at 4:26 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 20/01/2023 03:28, Jacob Champion wrote:
If you want to multiplex protocols on a port, now is an excellent time
to require clients to use ALPN on implicit-TLS connections. (There are
no clients that can currently connect via implicit TLS, so you'll
never have another chance to force the issue without breaking
backwards compatibility.) That should hopefully make it harder to
ALPACA yourself or others [2].Good idea. Do we want to just require the protocol to be "postgres", or
perhaps "postgres/3.0"? Need to register that with IANA, I guess.
Unless you plan to make the next minor protocol version fundamentally
incompatible, I don't think there's much reason to add '.0'. (And even
if that does happen, 'postgres/3.1' is still distinct from
'postgres/3'. Or 'postgres' for that matter.) The Expert Review
process might provide some additional guidance?
We implemented a protocol version negotiation mechanism in the libpq
protocol itself, how would this interact with it? If it's just
"postgres", then I guess we'd still negotiate the protocol version and
list of extensions after the TLS handshake.
Yeah. You could choose to replace major version negotiation completely
with ALPN, I suppose, but there might not be any maintenance benefit
if you still have to support plaintext negotiation. Maybe there are
performance implications to handling the negotiation earlier vs.
later?
Note that older versions of TLS will expose the ALPN in plaintext...
but that may not be a factor by the time a postgres/4 shows up, and if
the next protocol is incompatible then it may not be feasible to hide
the differences via transport encryption anyway.
Regarding Vladimir's comments on how clients can migrate to this, I
don't have any great suggestions. To summarize, there are several options:- Add an "fast_tls" option that the user can enable if they know the
server supports it
I like that such an option could eventually be leveraged for a
postgresqls:// URI scheme (which should not fall back, ever). There
would be other things we'd have to change first to make that a reality
-- postgresqls://example.com?host=evil.local is problematic, for
example -- but it'd be really nice to have an HTTPS equivalent.
--Jacob
On Wed, 22 Feb 2023 at 07:27, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 20/01/2023 03:28, Jacob Champion wrote:
On Wed, Jan 18, 2023 at 7:16 PM Greg Stark <stark@mit.edu> wrote:
* "Service Mesh" type tools that hide multiple services behind a
single host/port ("Service Mesh" is just a new buzzword for "proxy").If you want to multiplex protocols on a port, now is an excellent time
to require clients to use ALPN on implicit-TLS connections. (There are
no clients that can currently connect via implicit TLS, so you'll
never have another chance to force the issue without breaking
backwards compatibility.) That should hopefully make it harder to
ALPACA yourself or others [2].Good idea. Do we want to just require the protocol to be "postgres", or
perhaps "postgres/3.0"? Need to register that with IANA, I guess.
I had never heard of this before, it does seem useful. But if I
understand it right it's entirely independent of this patch. We can
add it to all our Client/Server exchanges whether they're the initial
direct SSL connection or the STARTTLS negotiation?
We implemented a protocol version negotiation mechanism in the libpq
protocol itself, how would this interact with it? If it's just
"postgres", then I guess we'd still negotiate the protocol version and
list of extensions after the TLS handshake.Incidentally I find the logic in ProcessStartupPacket incredibly
confusing. It took me a while before I realized it's using tail
recursion to implement the startup logic. I think it would be way more
straightforward and extensible if it used a much more common iterative
style. I think it would make it possible to keep more state than just
ssl_done and gss_done without changing the function signature every
time for example.+1. The complexity of the startup logic, both client- and server-side,
is a big reason why I want implicit TLS in the first place. That way,
bugs in that code can't be exploited before the TLS handshake
completes.+1. We need to support explicit TLS for a long time, so we can't
simplify by just removing it. But let's refactor the code somehow, to
make it more clear.Looking at the patch, I think it accepts an SSLRequest packet even if
implicit TLS has already been established. That's surely wrong, and
shows how confusing the code is. (Or I'm reading it incorrectly, which
also shows how confusing it is :-) )
I'll double check it but I think I tested that that wasn't the case. I
think it accepts the SSL request packet and sends back an N which the
client libpq just interprets as the server not supporting SSL and does
an unencrypted connection (which is tunneled over stunnel unbeknownst
to libpq).
I agree I would want to flatten this logic to an iterative approach
but having wrapped my head around it now I'm not necessarily rushing
to do it now. The main advantage of flattening it would be to make it
easy to support other protocol types which I think could be really
interesting. It would be much clearer to document the state machine if
all the state is in one place and the code just loops through
processing startup packets and going to a new state until the
connection is established. That's true now but you have to understand
how the state is passed in the function parameters and notice that all
the recursion is tail recursive (I think). And extending that state
would require extending the function signature which would get awkward
quickly.
Regarding Vladimir's comments on how clients can migrate to this, I
don't have any great suggestions. To summarize, there are several options:- Add an "fast_tls" option that the user can enable if they know the
server supports it- First connect in old-fashioned way, and remember the server version.
Later, if you reconnect to the same server, use implicit TLS if the
server version was high enough. This would be most useful for connection
pools.
Vladimir pointed out that this doesn't necessarily work. The server
may be new enough to support it but it could be behind a proxy like
pgbouncer or something. The same would be true if the server reported
a "connection option" instead of depending on version.
- Connect both ways at the same time, and continue with the fastest,
i.e. "happy eyeballs"
That seems way too complex for us to bother with imho.
- Try implicit TLS first, and fall back to explicit TLS if it fails.
For libpq, we don't necessarily need to do anything right now. We can
add the implicit TLS support in a later version. Not having libpq
support makes it hard to test the server codepath, though. Maybe just
test it with 'stunnel' or 'openssl s_client'.
I think we should have an option to explicitly enable it in psql, if
only for testing. And then wait five years and switch the default on
it then. In the meantime users can just set it based on their setup.
That's not the way to the quickest adoption but imho the main
advantages of this option are the options it gives users, not the
latency improvement, so I'm not actually super concerned about
adoption rate.
I assume we'll keep the negotiated mode indefinitely because it can
handle any other protocols we might want. For instance, it currently
handles GSSAPI -- which raises the question, are we happy with GSSAPI
having this extra round trip? Is there a similar change we could make
for it? My understanding is that GSSAPI is an abstract interface and
the actual protocol it's invoking could be anything so we can't make
any assumptions about what the first packet looks like. Perhaps we can
do something about pipelining GSSAPI messages so if the negotiation
fails the server just closes the connection but if it accepts it it
does a similar trick with unreading the buffered data and processing
it through the GSSAPI calls.
--
greg
On Tue, Feb 28, 2023 at 10:33 AM Greg Stark <stark@mit.edu> wrote:
On Wed, 22 Feb 2023 at 07:27, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
Good idea. Do we want to just require the protocol to be "postgres", or
perhaps "postgres/3.0"? Need to register that with IANA, I guess.I had never heard of this before, it does seem useful. But if I
understand it right it's entirely independent of this patch.
It can be. If you want to use it in the strongest possible way,
though, you'd have to require its use by clients. Introducing that
requirement later would break existing ones, so I think it makes sense
to do it at the same time as the initial implementation, if there's
interest.
We can
add it to all our Client/Server exchanges whether they're the initial
direct SSL connection or the STARTTLS negotiation?
I'm not sure it would buy you anything during the STARTTLS-style
opening. You already know what protocol you're speaking in that case.
(So with the ALPACA example, the damage is already done.)
Thanks,
--Jacob
Here's an updated patch for direct SSL connections.
I've added libpq client support with a new connection parameter. This
allows testing it easily with psql. It's still a bit hard to see
what's going on though. I'm thinking it would be good to have libpq
keep a string which describes what negotiations were attempted and
failed and what was eventually accepted which psql could print with
the SSL message or expose some other way.
In the end I didn't see how adding an API for this really helped any
more than just saying the API is to stuff the unread data into the
Port structure. So I just documented that. If anyone has any better
idea...
I added documentation for the libpq connection setting.
One thing, I *think* it's ok to replace the send(2) call with
secure_write() in the negotiation. It does mean it's possible for the
connection to fail with FATAL at that point instead of COMMERROR but I
don't think that's a problem.
I haven't added tests. I'm not sure how to test this since to test it
properly means running the server with every permutation of ssl and
gssapi configurations.
Incidentally, some of the configuration combinations -- namely
sslnegotiation=direct and default gssencmode and sslmode results in a
counter-intuitive behaviour. But I don't see a better option that
doesn't mean making the defaults less useful.
Attachments:
v2-0003-Direct-SSL-connections-documentation.patchtext/x-patch; charset=US-ASCII; name=v2-0003-Direct-SSL-connections-documentation.patchDownload
From b07e19223bee52b7bb9b50afea39e4baaa0a46f3 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 15:10:15 -0400
Subject: [PATCH v2 3/3] Direct SSL connections documentation
---
doc/src/sgml/libpq.sgml | 102 ++++++++++++++++++++++++++++++++++++----
1 file changed, 93 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 3706d349ab..e2f0891ea5 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1701,10 +1701,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
Note that if <acronym>GSSAPI</acronym> encryption is possible,
that will be used in preference to <acronym>SSL</acronym>
encryption, regardless of the value of <literal>sslmode</literal>.
- To force use of <acronym>SSL</acronym> encryption in an
- environment that has working <acronym>GSSAPI</acronym>
- infrastructure (such as a Kerberos server), also
- set <literal>gssencmode</literal> to <literal>disable</literal>.
+ To negotiate <acronym>SSL</acronym> encryption in an environment that
+ has working <acronym>GSSAPI</acronym> infrastructure (such as a
+ Kerberos server), also set <literal>gssencmode</literal>
+ to <literal>disable</literal>.
+ Use of non-default values of <literal>sslnegotiation</literal> can
+ also cause <acronym>SSL</acronym> to be used instead of
+ negotiating <acronym>GSSAPI</acronym> encryption.
</para>
</listitem>
</varlistentry>
@@ -1731,6 +1734,75 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
</listitem>
</varlistentry>
+ <varlistentry id="libpq-connect-sslnegotiation" xreflabel="sslnegotiation">
+ <term><literal>sslnegotiation</literal></term>
+ <listitem>
+ <para>
+ This option controls whether <productname>PostgreSQL</productname>
+ will perform its protocol negotiation to request encryption from the
+ server or will just directly make a standard <acronym>SSL</acronym>
+ connection. Traditional <productname>PostgreSQL</productname>
+ protocol negotiation is the default and the most flexible with
+ different server configurations. If the server is known to support
+ direct <acronym>SSL</acronym> connections then the latter requires one
+ fewer round trip reducing connection latency and also allows the use
+ of protocol agnostic SSL network tools.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>postgres</literal></term>
+ <listitem>
+ <para>
+ perform <productname>PostgreSQL</productname> protocol
+ negotiation. This is the default if the option is not provided.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>direct</literal></term>
+ <listitem>
+ <para>
+ first attempt to establish a standard SSL connection and if that
+ fails reconnect and perform the negotiation. This fallback
+ process adds significant latency if the initial SSL connection
+ fails.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>requiredirect</literal></term>
+ <listitem>
+ <para>
+ attempt to establish a standard SSL connection and if that fails
+ return a connection failure immediately.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ If <literal>sslmode</literal> set to <literal>disable</literal>
+ or <literal>allow</literal> then <literal>sslnegotiation</literal> is
+ ignored. If <literal>gssencmode</literal> is set
+ to <literal>require</literal> then <literal>sslnegotiation</literal>
+ must be the default <literal>postgres</literal> value.
+ </para>
+
+ <para>
+ Moreover, note if <literal>gssencmode</literal> is set
+ to <literal>prefer</literal> and <literal>sslnegotiation</literal>
+ to <literal>direct</literal> then the effective preference will be
+ direct <acronym>SSL</acronym> connections, followed by
+ negotiated <acronym>GSS</acronym> connections, followed by
+ negotiated <acronym>SSL</acronym> connections, possibly followed
+ lastly by unencrypted connections.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-connect-sslcompression" xreflabel="sslcompression">
<term><literal>sslcompression</literal></term>
<listitem>
@@ -1884,11 +1956,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
<para>
The Server Name Indication can be used by SSL-aware proxies to route
- connections without having to decrypt the SSL stream. (Note that this
- requires a proxy that is aware of the PostgreSQL protocol handshake,
- not just any SSL proxy.) However, <acronym>SNI</acronym> makes the
- destination host name appear in cleartext in the network traffic, so
- it might be undesirable in some cases.
+ connections without having to decrypt the SSL stream. (Note that
+ unless the proxy is aware of the PostgreSQL protocol handshake this
+ would require setting <literal>sslnegotiation</literal>
+ to <literal>direct</literal> or <literal>requiredirect</literal>.)
+ However, <acronym>SNI</acronym> makes the destination host name appear
+ in cleartext in the network traffic, so it might be undesirable in
+ some cases.
</para>
</listitem>
</varlistentry>
@@ -7966,6 +8040,16 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
</para>
</listitem>
+ <listitem>
+ <para>
+ <indexterm>
+ <primary><envar>PGSSLNEGOTIATION</envar></primary>
+ </indexterm>
+ <envar>PGSSLNEGOTIATION</envar> behaves the same as the <xref
+ linkend="libpq-connect-sslnegotiation"/> connection parameter.
+ </para>
+ </listitem>
+
<listitem>
<para>
<indexterm>
--
2.39.2
v2-0001-Direct-SSL-connections-postmaster-support.patchtext/x-patch; charset=US-ASCII; name=v2-0001-Direct-SSL-connections-postmaster-support.patchDownload
From bcdabe6b1bb33daca31f1ba37e1b0901871b7119 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Wed, 15 Mar 2023 15:51:02 -0400
Subject: [PATCH v2 1/3] Direct SSL connections postmaster support
---
src/backend/libpq/be-secure.c | 13 +++
src/backend/libpq/pqcomm.c | 9 +-
src/backend/postmaster/postmaster.c | 133 ++++++++++++++++++++++------
src/include/libpq/libpq-be.h | 10 +++
src/include/libpq/libpq.h | 2 +-
5 files changed, 134 insertions(+), 33 deletions(-)
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index a0f7084018..1020b3adb0 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -235,6 +235,19 @@ secure_raw_read(Port *port, void *ptr, size_t len)
{
ssize_t n;
+ /* Read from the "unread" buffered data first. c.f. libpq-be.h */
+ if (port->raw_buf_remaining > 0)
+ {
+ /* consume up to len bytes from the raw_buf */
+ if (len > port->raw_buf_remaining)
+ len = port->raw_buf_remaining;
+ Assert(port->raw_buf);
+ memcpy(ptr, port->raw_buf + port->raw_buf_consumed, len);
+ port->raw_buf_consumed += len;
+ port->raw_buf_remaining -= len;
+ return len;
+ }
+
/*
* Try to read from the socket without blocking. If it succeeds we're
* done, otherwise we'll wait for the socket using the latch mechanism.
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index da5bb5fc5d..17d025bb8e 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -1126,13 +1126,16 @@ pq_discardbytes(size_t len)
/* --------------------------------
* pq_buffer_has_data - is any buffered data available to read?
*
- * This will *not* attempt to read more data.
+ * Actually returns the number of bytes in the buffer...
+ *
+ * This will *not* attempt to read more data. And reading up to that number of
+ * bytes should not cause reading any more data either.
* --------------------------------
*/
-bool
+size_t
pq_buffer_has_data(void)
{
- return (PqRecvPointer < PqRecvLength);
+ return (PqRecvLength - PqRecvPointer);
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 71198b72c8..4cb267527e 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -422,6 +422,7 @@ static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(Port *port);
+static int ProcessSSLStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -1926,6 +1927,97 @@ ServerLoop(void)
}
}
+/*
+ * Check for a native direct SSL connection.
+ *
+ * This happens before startup packets so we are careful not to actual read
+ * any bytes from the stream if it's not a direct SSL connection.
+ */
+
+static int
+ProcessSSLStartup(Port *port)
+{
+ int firstbyte;
+
+ pq_startmsgread();
+
+ firstbyte = pq_peekbyte();
+ if (firstbyte == EOF)
+ {
+ /*
+ * If we get no data at all, don't clutter the log with a complaint;
+ * such cases often occur for legitimate reasons. An example is that
+ * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
+ * client didn't like our response, it'll probably just drop the
+ * connection. Service-monitoring software also often just opens and
+ * closes a connection without sending anything. (So do port
+ * scanners, which may be less benign, but it's not really our job to
+ * notice those.)
+ */
+ return STATUS_ERROR;
+ }
+
+ /*
+ * First byte indicates standard SSL handshake message
+ *
+ * (It can't be a Postgres startup length because in network byte order
+ * that would be a startup packet hundreds of megabytes long)
+ */
+ if (firstbyte == 0x16)
+ {
+#ifdef USE_SSL
+ ssize_t len;
+ char *buf = NULL;
+ elog(LOG, "Detected direct SSL handshake");
+
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ buf = palloc(len);
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+
+ Assert(pq_buffer_has_data() == 0);
+ if (secure_open_server(port) == -1)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("SSL Protocol Error during direct SSL connection initiation")));
+ return STATUS_ERROR;
+ }
+
+ if (port->raw_buf_remaining > 0)
+ {
+ /* This shouldn't be possible -- it would mean the client sent
+ * encrypted data before we established a session key...
+ */
+ elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ return STATUS_ERROR;
+ }
+ pfree(port->raw_buf);
+#else
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with no SSL support")));
+ return STATUS_ERROR;
+#endif
+ }
+
+ pq_endmsgread();
+
+ if (port->ssl_in_use)
+ ereport(DEBUG2,
+ (errmsg_internal("Direct SSL connection established")));
+
+ return STATUS_OK;
+}
+
+
/*
* Read a client's startup packet and do something according to it.
*
@@ -1954,28 +2046,7 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
pq_startmsgread();
- /*
- * Grab the first byte of the length word separately, so that we can tell
- * whether we have no data at all or an incomplete packet. (This might
- * sound inefficient, but it's not really, because of buffering in
- * pqcomm.c.)
- */
- if (pq_getbytes((char *) &len, 1) == EOF)
- {
- /*
- * If we get no data at all, don't clutter the log with a complaint;
- * such cases often occur for legitimate reasons. An example is that
- * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
- * client didn't like our response, it'll probably just drop the
- * connection. Service-monitoring software also often just opens and
- * closes a connection without sending anything. (So do port
- * scanners, which may be less benign, but it's not really our job to
- * notice those.)
- */
- return STATUS_ERROR;
- }
-
- if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
+ if (pq_getbytes(((char *) &len), 4) == EOF)
{
/* Got a partial length word, so bleat about that */
if (!ssl_done && !gss_done)
@@ -2039,8 +2110,11 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
char SSLok;
#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ /* No SSL when disabled or on Unix sockets.
+ *
+ * Also no SSL negotiation if we already have a direct SSL connection
+ */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX || port->ssl_in_use)
SSLok = 'N';
else
SSLok = 'S'; /* Support for SSL */
@@ -2048,11 +2122,10 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
SSLok = 'N'; /* No support for SSL */
#endif
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
+ while (secure_write(port, &SSLok, 1) != 1)
{
if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
+ continue; /* if interrupted, just retry */
ereport(COMMERROR,
(errcode_for_socket_access(),
errmsg("failed to send SSL negotiation response: %m")));
@@ -2093,7 +2166,7 @@ retry1:
GSSok = 'G';
#endif
- while (send(port->sock, &GSSok, 1, 0) != 1)
+ while (secure_write(port, &GSSok, 1) != 1)
{
if (errno == EINTR)
continue;
@@ -4396,7 +4469,9 @@ BackendInitialize(Port *port)
* Receive the startup packet (which might turn out to be a cancel request
* packet).
*/
- status = ProcessStartupPacket(port, false, false);
+ status = ProcessSSLStartup(port);
+ if (status == STATUS_OK)
+ status = ProcessStartupPacket(port, false, false);
/*
* Disable the timeout, and prevent SIGTERM again.
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index ac6407e9f6..824b28e824 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -226,6 +226,16 @@ typedef struct Port
SSL *ssl;
X509 *peer;
#endif
+ /* This is a bit of a hack. The raw_buf is data that was previously read
+ * and buffered in a higher layer but then "unread" and needs to be read
+ * again while establishing an SSL connection via the SSL library layer.
+ *
+ * There's no API to "unread", the upper layer just places the data in the
+ * Port structure in raw_buf and sets raw_buf_remaining to the amount of
+ * bytes unread and raw_buf_consumed to 0.
+ */
+ char *raw_buf;
+ ssize_t raw_buf_consumed, raw_buf_remaining;
} Port;
#ifdef USE_SSL
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 50fc781f47..2b02f67257 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -80,7 +80,7 @@ extern int pq_getmessage(StringInfo s, int maxlen);
extern int pq_getbyte(void);
extern int pq_peekbyte(void);
extern int pq_getbyte_if_available(unsigned char *c);
-extern bool pq_buffer_has_data(void);
+extern size_t pq_buffer_has_data(void);
extern int pq_putmessage_v2(char msgtype, const char *s, size_t len);
extern bool pq_check_connection(void);
--
2.39.2
v2-0002-Direct-SSL-connections-client-support.patchtext/x-patch; charset=US-ASCII; name=v2-0002-Direct-SSL-connections-client-support.patchDownload
From c064333877672198c0dd7dd7644cf17bdd8ee3e8 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 11:55:16 -0400
Subject: [PATCH v2 2/3] Direct SSL connections client support
---
src/interfaces/libpq/fe-connect.c | 90 ++++++++++++++++++++++++++++---
src/interfaces/libpq/libpq-fe.h | 1 +
src/interfaces/libpq/libpq-int.h | 3 ++
3 files changed, 88 insertions(+), 6 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index dd4b98e099..0f6637b108 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -271,6 +271,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-Mode", "", 12, /* sizeof("verify-full") == 12 */
offsetof(struct pg_conn, sslmode)},
+ {"sslnegotiation", "PGSSLNEGOTIATION", "postgres", NULL,
+ "SSL-Negotiation", "", 14, /* strlen("requiredirect") == 14 */
+ offsetof(struct pg_conn, sslnegotiation)},
+
{"sslcompression", "PGSSLCOMPRESSION", "0", NULL,
"SSL-Compression", "", 1,
offsetof(struct pg_conn, sslcompression)},
@@ -1467,11 +1471,36 @@ connectOptions2(PGconn *conn)
}
#endif
}
+
+ /*
+ * validate sslnegotiation option, default is "postgres" for the postgres
+ * style negotiated connection with an extra round trip but more options.
+ */
+ if (conn->sslnegotiation)
+ {
+ if (strcmp(conn->sslnegotiation, "postgres") != 0
+ && strcmp(conn->sslnegotiation, "direct") != 0
+ && strcmp(conn->sslnegotiation, "requiredirect") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
+ "sslnegotiation", conn->sslnegotiation);
+ return false;
+ }
+
+#ifndef USE_SSL
+ if (conn->negotiation[0] != 'p') {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "sslnegotiation value \"%s\" invalid when SSL support is not compiled in",
+ conn->sslnegotiation);
+ return false;
+ }
+#endif
+ }
else
{
- conn->sslmode = strdup(DefaultSSLMode);
- if (!conn->sslmode)
- goto oom_error;
+ libpq_append_conn_error(conn, "sslnegotiation missing?");
+ return false;
}
/*
@@ -1531,6 +1560,18 @@ connectOptions2(PGconn *conn)
conn->gssencmode);
return false;
}
+#endif
+#ifdef USE_SSL
+ /* GSS is incompatible with direct SSL connections so it requires the
+ * default postgres style connection ssl negotiation */
+ if (strcmp(conn->gssencmode, "require") == 0 &&
+ strcmp(conn->sslnegotiation, "postgres") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when Direct SSL negotiation is enabled",
+ conn->gssencmode);
+ return false;
+ }
#endif
}
else
@@ -2587,11 +2628,12 @@ keep_going: /* We will come back to here until there is
/* initialize these values based on SSL mode */
conn->allow_ssl_try = (conn->sslmode[0] != 'd'); /* "disable" */
conn->wait_ssl_try = (conn->sslmode[0] == 'a'); /* "allow" */
+ /* direct ssl is incompatible with "allow" or "disabled" ssl */
+ conn->allow_direct_ssl_try = conn->allow_ssl_try && !conn->wait_ssl_try && (conn->sslnegotiation[0] != 'p');
#endif
#ifdef ENABLE_GSS
conn->try_gss = (conn->gssencmode[0] != 'd'); /* "disable" */
#endif
-
reset_connection_state_machine = false;
need_new_connection = true;
}
@@ -3026,6 +3068,28 @@ keep_going: /* We will come back to here until there is
if (pqsecure_initialize(conn, false, true) < 0)
goto error_return;
+ /* If SSL is enabled and direct SSL connections are enabled
+ * and we haven't already established an SSL connection (or
+ * already tried a direct connection and failed or succeeded)
+ * then try just enabling SSL directly.
+ *
+ * If we fail then we'll either fail the connection (if
+ * sslnegotiation is set to requiredirect or turn
+ * allow_direct_ssl_try to false
+ */
+ if (conn->allow_ssl_try
+ && !conn->wait_ssl_try
+ && conn->allow_direct_ssl_try
+ && !conn->ssl_in_use
+#ifdef ENABLE_GSS
+ && !conn->gssenc
+#endif
+ )
+ {
+ conn->status = CONNECTION_SSL_STARTUP;
+ return PGRES_POLLING_WRITING;
+ }
+
/*
* If SSL is enabled and we haven't already got encryption of
* some sort running, request SSL instead of sending the
@@ -3102,9 +3166,11 @@ keep_going: /* We will come back to here until there is
/*
* On first time through, get the postmaster's response to our
- * SSL negotiation packet.
+ * SSL negotiation packet. If we are trying a direct ssl
+ * connection skip reading the negotiation packet and go
+ * straight to initiating an ssl connection.
*/
- if (!conn->ssl_in_use)
+ if (!conn->ssl_in_use && !conn->allow_direct_ssl_try)
{
/*
* We use pqReadData here since it has the logic to
@@ -3210,6 +3276,18 @@ keep_going: /* We will come back to here until there is
}
if (pollres == PGRES_POLLING_FAILED)
{
+ /* Failed direct ssl connection, possibly try a new connection with postgres negotiation */
+ if (conn->allow_direct_ssl_try)
+ {
+ /* if it's requiredirect then it's a hard failure */
+ if (conn->sslnegotiation[0] == 'r')
+ goto error_return;
+ /* otherwise only retry using postgres connection */
+ conn->allow_direct_ssl_try = false;
+ need_new_connection = true;
+ goto keep_going;
+ }
+
/*
* Failed ... if sslmode is "prefer" then do a non-SSL
* retry
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index f3d9220496..05821b8473 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -72,6 +72,7 @@ typedef enum
CONNECTION_AUTH_OK, /* Received authentication; waiting for
* backend startup. */
CONNECTION_SETENV, /* This state is no longer used. */
+ CONNECTION_DIRECT_SSL_STARTUP, /* Starting SSL without PG Negotiation. */
CONNECTION_SSL_STARTUP, /* Negotiating SSL. */
CONNECTION_NEEDED, /* Internal state: connect() needed */
CONNECTION_CHECK_WRITABLE, /* Checking if session is read-write. */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 1dc264fe54..8d8964d835 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -380,6 +380,7 @@ struct pg_conn
char *keepalives_count; /* maximum number of TCP keepalive
* retransmits */
char *sslmode; /* SSL mode (require,prefer,allow,disable) */
+ char *sslnegotiation; /* SSL initiation style (postgres,direct,requiredirect) */
char *sslcompression; /* SSL compression (0 or 1) */
char *sslkey; /* client key filename */
char *sslcert; /* client certificate filename */
@@ -529,6 +530,8 @@ struct pg_conn
bool ssl_in_use;
#ifdef USE_SSL
+ bool allow_direct_ssl_try; /* Try to make a direct SSL connection
+ * without an "SSL negotiation packet" */
bool allow_ssl_try; /* Allowed to try SSL negotiation */
bool wait_ssl_try; /* Delay SSL negotiation until after
* attempting normal connection */
--
2.39.2
Here's a first cut at ALPN support.
Currently it's using a hard coded "Postgres/3.0" protocol (hard coded
both in the client and the server...). And it's hard coded to be
required for direct connections and supported but not required for
regular connections.
IIRC I put a variable labeled a "GUC" but forgot to actually make it a
GUC. But I'm thinking of maybe removing that variable since I don't
see much of a use case for controlling this manually. I *think* ALPN
is supported by all the versions of OpenSSL we support.
The other patches are unchanged (modulo a free() that I missed in the
client before). They still have the semi-open issues I mentioned in
the previous email.
--
greg
Attachments:
v3-0001-Direct-SSL-connections-postmaster-support.patchtext/x-patch; charset=US-ASCII; name=v3-0001-Direct-SSL-connections-postmaster-support.patchDownload
From 53cd555f73d91150db7bc316bc3dca831ee98ce3 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Wed, 15 Mar 2023 15:51:02 -0400
Subject: [PATCH v3 1/4] Direct SSL connections postmaster support
---
src/backend/libpq/be-secure.c | 13 +++
src/backend/libpq/pqcomm.c | 9 +-
src/backend/postmaster/postmaster.c | 133 ++++++++++++++++++++++------
src/include/libpq/libpq-be.h | 10 +++
src/include/libpq/libpq.h | 2 +-
5 files changed, 134 insertions(+), 33 deletions(-)
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index a0f7084018..1020b3adb0 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -235,6 +235,19 @@ secure_raw_read(Port *port, void *ptr, size_t len)
{
ssize_t n;
+ /* Read from the "unread" buffered data first. c.f. libpq-be.h */
+ if (port->raw_buf_remaining > 0)
+ {
+ /* consume up to len bytes from the raw_buf */
+ if (len > port->raw_buf_remaining)
+ len = port->raw_buf_remaining;
+ Assert(port->raw_buf);
+ memcpy(ptr, port->raw_buf + port->raw_buf_consumed, len);
+ port->raw_buf_consumed += len;
+ port->raw_buf_remaining -= len;
+ return len;
+ }
+
/*
* Try to read from the socket without blocking. If it succeeds we're
* done, otherwise we'll wait for the socket using the latch mechanism.
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index da5bb5fc5d..17d025bb8e 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -1126,13 +1126,16 @@ pq_discardbytes(size_t len)
/* --------------------------------
* pq_buffer_has_data - is any buffered data available to read?
*
- * This will *not* attempt to read more data.
+ * Actually returns the number of bytes in the buffer...
+ *
+ * This will *not* attempt to read more data. And reading up to that number of
+ * bytes should not cause reading any more data either.
* --------------------------------
*/
-bool
+size_t
pq_buffer_has_data(void)
{
- return (PqRecvPointer < PqRecvLength);
+ return (PqRecvLength - PqRecvPointer);
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 4c49393fc5..ec1d895a23 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -422,6 +422,7 @@ static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(Port *port);
+static int ProcessSSLStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -1926,6 +1927,97 @@ ServerLoop(void)
}
}
+/*
+ * Check for a native direct SSL connection.
+ *
+ * This happens before startup packets so we are careful not to actual read
+ * any bytes from the stream if it's not a direct SSL connection.
+ */
+
+static int
+ProcessSSLStartup(Port *port)
+{
+ int firstbyte;
+
+ pq_startmsgread();
+
+ firstbyte = pq_peekbyte();
+ if (firstbyte == EOF)
+ {
+ /*
+ * If we get no data at all, don't clutter the log with a complaint;
+ * such cases often occur for legitimate reasons. An example is that
+ * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
+ * client didn't like our response, it'll probably just drop the
+ * connection. Service-monitoring software also often just opens and
+ * closes a connection without sending anything. (So do port
+ * scanners, which may be less benign, but it's not really our job to
+ * notice those.)
+ */
+ return STATUS_ERROR;
+ }
+
+ /*
+ * First byte indicates standard SSL handshake message
+ *
+ * (It can't be a Postgres startup length because in network byte order
+ * that would be a startup packet hundreds of megabytes long)
+ */
+ if (firstbyte == 0x16)
+ {
+#ifdef USE_SSL
+ ssize_t len;
+ char *buf = NULL;
+ elog(LOG, "Detected direct SSL handshake");
+
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ buf = palloc(len);
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+
+ Assert(pq_buffer_has_data() == 0);
+ if (secure_open_server(port) == -1)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("SSL Protocol Error during direct SSL connection initiation")));
+ return STATUS_ERROR;
+ }
+
+ if (port->raw_buf_remaining > 0)
+ {
+ /* This shouldn't be possible -- it would mean the client sent
+ * encrypted data before we established a session key...
+ */
+ elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ return STATUS_ERROR;
+ }
+ pfree(port->raw_buf);
+#else
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with no SSL support")));
+ return STATUS_ERROR;
+#endif
+ }
+
+ pq_endmsgread();
+
+ if (port->ssl_in_use)
+ ereport(DEBUG2,
+ (errmsg_internal("Direct SSL connection established")));
+
+ return STATUS_OK;
+}
+
+
/*
* Read a client's startup packet and do something according to it.
*
@@ -1954,28 +2046,7 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
pq_startmsgread();
- /*
- * Grab the first byte of the length word separately, so that we can tell
- * whether we have no data at all or an incomplete packet. (This might
- * sound inefficient, but it's not really, because of buffering in
- * pqcomm.c.)
- */
- if (pq_getbytes((char *) &len, 1) == EOF)
- {
- /*
- * If we get no data at all, don't clutter the log with a complaint;
- * such cases often occur for legitimate reasons. An example is that
- * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
- * client didn't like our response, it'll probably just drop the
- * connection. Service-monitoring software also often just opens and
- * closes a connection without sending anything. (So do port
- * scanners, which may be less benign, but it's not really our job to
- * notice those.)
- */
- return STATUS_ERROR;
- }
-
- if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
+ if (pq_getbytes(((char *) &len), 4) == EOF)
{
/* Got a partial length word, so bleat about that */
if (!ssl_done && !gss_done)
@@ -2039,8 +2110,11 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
char SSLok;
#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ /* No SSL when disabled or on Unix sockets.
+ *
+ * Also no SSL negotiation if we already have a direct SSL connection
+ */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX || port->ssl_in_use)
SSLok = 'N';
else
SSLok = 'S'; /* Support for SSL */
@@ -2048,11 +2122,10 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
SSLok = 'N'; /* No support for SSL */
#endif
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
+ while (secure_write(port, &SSLok, 1) != 1)
{
if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
+ continue; /* if interrupted, just retry */
ereport(COMMERROR,
(errcode_for_socket_access(),
errmsg("failed to send SSL negotiation response: %m")));
@@ -2093,7 +2166,7 @@ retry1:
GSSok = 'G';
#endif
- while (send(port->sock, &GSSok, 1, 0) != 1)
+ while (secure_write(port, &GSSok, 1) != 1)
{
if (errno == EINTR)
continue;
@@ -4396,7 +4469,9 @@ BackendInitialize(Port *port)
* Receive the startup packet (which might turn out to be a cancel request
* packet).
*/
- status = ProcessStartupPacket(port, false, false);
+ status = ProcessSSLStartup(port);
+ if (status == STATUS_OK)
+ status = ProcessStartupPacket(port, false, false);
/*
* Disable the timeout, and prevent SIGTERM again.
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index ac6407e9f6..824b28e824 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -226,6 +226,16 @@ typedef struct Port
SSL *ssl;
X509 *peer;
#endif
+ /* This is a bit of a hack. The raw_buf is data that was previously read
+ * and buffered in a higher layer but then "unread" and needs to be read
+ * again while establishing an SSL connection via the SSL library layer.
+ *
+ * There's no API to "unread", the upper layer just places the data in the
+ * Port structure in raw_buf and sets raw_buf_remaining to the amount of
+ * bytes unread and raw_buf_consumed to 0.
+ */
+ char *raw_buf;
+ ssize_t raw_buf_consumed, raw_buf_remaining;
} Port;
#ifdef USE_SSL
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 50fc781f47..2b02f67257 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -80,7 +80,7 @@ extern int pq_getmessage(StringInfo s, int maxlen);
extern int pq_getbyte(void);
extern int pq_peekbyte(void);
extern int pq_getbyte_if_available(unsigned char *c);
-extern bool pq_buffer_has_data(void);
+extern size_t pq_buffer_has_data(void);
extern int pq_putmessage_v2(char msgtype, const char *s, size_t len);
extern bool pq_check_connection(void);
--
2.39.2
v3-0002-Direct-SSL-connections-client-support.patchtext/x-patch; charset=US-ASCII; name=v3-0002-Direct-SSL-connections-client-support.patchDownload
From 058e215f801b3e7ade5cacfc7e1889966430624c Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 11:55:16 -0400
Subject: [PATCH v3 2/4] Direct SSL connections client support
---
src/interfaces/libpq/fe-connect.c | 91 +++++++++++++++++++++++++++++--
src/interfaces/libpq/libpq-fe.h | 1 +
src/interfaces/libpq/libpq-int.h | 3 +
3 files changed, 89 insertions(+), 6 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index b9f899c552..80d3e9169b 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -271,6 +271,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-Mode", "", 12, /* sizeof("verify-full") == 12 */
offsetof(struct pg_conn, sslmode)},
+ {"sslnegotiation", "PGSSLNEGOTIATION", "postgres", NULL,
+ "SSL-Negotiation", "", 14, /* strlen("requiredirect") == 14 */
+ offsetof(struct pg_conn, sslnegotiation)},
+
{"sslcompression", "PGSSLCOMPRESSION", "0", NULL,
"SSL-Compression", "", 1,
offsetof(struct pg_conn, sslcompression)},
@@ -1463,11 +1467,36 @@ connectOptions2(PGconn *conn)
}
#endif
}
+
+ /*
+ * validate sslnegotiation option, default is "postgres" for the postgres
+ * style negotiated connection with an extra round trip but more options.
+ */
+ if (conn->sslnegotiation)
+ {
+ if (strcmp(conn->sslnegotiation, "postgres") != 0
+ && strcmp(conn->sslnegotiation, "direct") != 0
+ && strcmp(conn->sslnegotiation, "requiredirect") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
+ "sslnegotiation", conn->sslnegotiation);
+ return false;
+ }
+
+#ifndef USE_SSL
+ if (conn->negotiation[0] != 'p') {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "sslnegotiation value \"%s\" invalid when SSL support is not compiled in",
+ conn->sslnegotiation);
+ return false;
+ }
+#endif
+ }
else
{
- conn->sslmode = strdup(DefaultSSLMode);
- if (!conn->sslmode)
- goto oom_error;
+ libpq_append_conn_error(conn, "sslnegotiation missing?");
+ return false;
}
/*
@@ -1527,6 +1556,18 @@ connectOptions2(PGconn *conn)
conn->gssencmode);
return false;
}
+#endif
+#ifdef USE_SSL
+ /* GSS is incompatible with direct SSL connections so it requires the
+ * default postgres style connection ssl negotiation */
+ if (strcmp(conn->gssencmode, "require") == 0 &&
+ strcmp(conn->sslnegotiation, "postgres") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when Direct SSL negotiation is enabled",
+ conn->gssencmode);
+ return false;
+ }
#endif
}
else
@@ -2583,11 +2624,12 @@ keep_going: /* We will come back to here until there is
/* initialize these values based on SSL mode */
conn->allow_ssl_try = (conn->sslmode[0] != 'd'); /* "disable" */
conn->wait_ssl_try = (conn->sslmode[0] == 'a'); /* "allow" */
+ /* direct ssl is incompatible with "allow" or "disabled" ssl */
+ conn->allow_direct_ssl_try = conn->allow_ssl_try && !conn->wait_ssl_try && (conn->sslnegotiation[0] != 'p');
#endif
#ifdef ENABLE_GSS
conn->try_gss = (conn->gssencmode[0] != 'd'); /* "disable" */
#endif
-
reset_connection_state_machine = false;
need_new_connection = true;
}
@@ -3046,6 +3088,28 @@ keep_going: /* We will come back to here until there is
if (pqsecure_initialize(conn, false, true) < 0)
goto error_return;
+ /* If SSL is enabled and direct SSL connections are enabled
+ * and we haven't already established an SSL connection (or
+ * already tried a direct connection and failed or succeeded)
+ * then try just enabling SSL directly.
+ *
+ * If we fail then we'll either fail the connection (if
+ * sslnegotiation is set to requiredirect or turn
+ * allow_direct_ssl_try to false
+ */
+ if (conn->allow_ssl_try
+ && !conn->wait_ssl_try
+ && conn->allow_direct_ssl_try
+ && !conn->ssl_in_use
+#ifdef ENABLE_GSS
+ && !conn->gssenc
+#endif
+ )
+ {
+ conn->status = CONNECTION_SSL_STARTUP;
+ return PGRES_POLLING_WRITING;
+ }
+
/*
* If SSL is enabled and we haven't already got encryption of
* some sort running, request SSL instead of sending the
@@ -3122,9 +3186,11 @@ keep_going: /* We will come back to here until there is
/*
* On first time through, get the postmaster's response to our
- * SSL negotiation packet.
+ * SSL negotiation packet. If we are trying a direct ssl
+ * connection skip reading the negotiation packet and go
+ * straight to initiating an ssl connection.
*/
- if (!conn->ssl_in_use)
+ if (!conn->ssl_in_use && !conn->allow_direct_ssl_try)
{
/*
* We use pqReadData here since it has the logic to
@@ -3230,6 +3296,18 @@ keep_going: /* We will come back to here until there is
}
if (pollres == PGRES_POLLING_FAILED)
{
+ /* Failed direct ssl connection, possibly try a new connection with postgres negotiation */
+ if (conn->allow_direct_ssl_try)
+ {
+ /* if it's requiredirect then it's a hard failure */
+ if (conn->sslnegotiation[0] == 'r')
+ goto error_return;
+ /* otherwise only retry using postgres connection */
+ conn->allow_direct_ssl_try = false;
+ need_new_connection = true;
+ goto keep_going;
+ }
+
/*
* Failed ... if sslmode is "prefer" then do a non-SSL
* retry
@@ -4231,6 +4309,7 @@ freePGconn(PGconn *conn)
free(conn->keepalives_interval);
free(conn->keepalives_count);
free(conn->sslmode);
+ free(conn->sslnegotiation);
free(conn->sslcert);
free(conn->sslkey);
if (conn->sslpassword)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index f3d9220496..05821b8473 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -72,6 +72,7 @@ typedef enum
CONNECTION_AUTH_OK, /* Received authentication; waiting for
* backend startup. */
CONNECTION_SETENV, /* This state is no longer used. */
+ CONNECTION_DIRECT_SSL_STARTUP, /* Starting SSL without PG Negotiation. */
CONNECTION_SSL_STARTUP, /* Negotiating SSL. */
CONNECTION_NEEDED, /* Internal state: connect() needed */
CONNECTION_CHECK_WRITABLE, /* Checking if session is read-write. */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 1dc264fe54..8d8964d835 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -380,6 +380,7 @@ struct pg_conn
char *keepalives_count; /* maximum number of TCP keepalive
* retransmits */
char *sslmode; /* SSL mode (require,prefer,allow,disable) */
+ char *sslnegotiation; /* SSL initiation style (postgres,direct,requiredirect) */
char *sslcompression; /* SSL compression (0 or 1) */
char *sslkey; /* client key filename */
char *sslcert; /* client certificate filename */
@@ -529,6 +530,8 @@ struct pg_conn
bool ssl_in_use;
#ifdef USE_SSL
+ bool allow_direct_ssl_try; /* Try to make a direct SSL connection
+ * without an "SSL negotiation packet" */
bool allow_ssl_try; /* Allowed to try SSL negotiation */
bool wait_ssl_try; /* Delay SSL negotiation until after
* attempting normal connection */
--
2.39.2
v3-0003-Direct-SSL-connections-documentation.patchtext/x-patch; charset=US-ASCII; name=v3-0003-Direct-SSL-connections-documentation.patchDownload
From 15b5d799d013d6e8a242b1e13f425b6e02025cde Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 15:10:15 -0400
Subject: [PATCH v3 3/4] Direct SSL connections documentation
---
doc/src/sgml/libpq.sgml | 102 ++++++++++++++++++++++++++++++++++++----
1 file changed, 93 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 9ee5532c07..0b89b224ba 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1691,10 +1691,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
Note that if <acronym>GSSAPI</acronym> encryption is possible,
that will be used in preference to <acronym>SSL</acronym>
encryption, regardless of the value of <literal>sslmode</literal>.
- To force use of <acronym>SSL</acronym> encryption in an
- environment that has working <acronym>GSSAPI</acronym>
- infrastructure (such as a Kerberos server), also
- set <literal>gssencmode</literal> to <literal>disable</literal>.
+ To negotiate <acronym>SSL</acronym> encryption in an environment that
+ has working <acronym>GSSAPI</acronym> infrastructure (such as a
+ Kerberos server), also set <literal>gssencmode</literal>
+ to <literal>disable</literal>.
+ Use of non-default values of <literal>sslnegotiation</literal> can
+ also cause <acronym>SSL</acronym> to be used instead of
+ negotiating <acronym>GSSAPI</acronym> encryption.
</para>
</listitem>
</varlistentry>
@@ -1721,6 +1724,75 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
</listitem>
</varlistentry>
+ <varlistentry id="libpq-connect-sslnegotiation" xreflabel="sslnegotiation">
+ <term><literal>sslnegotiation</literal></term>
+ <listitem>
+ <para>
+ This option controls whether <productname>PostgreSQL</productname>
+ will perform its protocol negotiation to request encryption from the
+ server or will just directly make a standard <acronym>SSL</acronym>
+ connection. Traditional <productname>PostgreSQL</productname>
+ protocol negotiation is the default and the most flexible with
+ different server configurations. If the server is known to support
+ direct <acronym>SSL</acronym> connections then the latter requires one
+ fewer round trip reducing connection latency and also allows the use
+ of protocol agnostic SSL network tools.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>postgres</literal></term>
+ <listitem>
+ <para>
+ perform <productname>PostgreSQL</productname> protocol
+ negotiation. This is the default if the option is not provided.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>direct</literal></term>
+ <listitem>
+ <para>
+ first attempt to establish a standard SSL connection and if that
+ fails reconnect and perform the negotiation. This fallback
+ process adds significant latency if the initial SSL connection
+ fails.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>requiredirect</literal></term>
+ <listitem>
+ <para>
+ attempt to establish a standard SSL connection and if that fails
+ return a connection failure immediately.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ If <literal>sslmode</literal> set to <literal>disable</literal>
+ or <literal>allow</literal> then <literal>sslnegotiation</literal> is
+ ignored. If <literal>gssencmode</literal> is set
+ to <literal>require</literal> then <literal>sslnegotiation</literal>
+ must be the default <literal>postgres</literal> value.
+ </para>
+
+ <para>
+ Moreover, note if <literal>gssencmode</literal> is set
+ to <literal>prefer</literal> and <literal>sslnegotiation</literal>
+ to <literal>direct</literal> then the effective preference will be
+ direct <acronym>SSL</acronym> connections, followed by
+ negotiated <acronym>GSS</acronym> connections, followed by
+ negotiated <acronym>SSL</acronym> connections, possibly followed
+ lastly by unencrypted connections.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-connect-sslcompression" xreflabel="sslcompression">
<term><literal>sslcompression</literal></term>
<listitem>
@@ -1874,11 +1946,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
<para>
The Server Name Indication can be used by SSL-aware proxies to route
- connections without having to decrypt the SSL stream. (Note that this
- requires a proxy that is aware of the PostgreSQL protocol handshake,
- not just any SSL proxy.) However, <acronym>SNI</acronym> makes the
- destination host name appear in cleartext in the network traffic, so
- it might be undesirable in some cases.
+ connections without having to decrypt the SSL stream. (Note that
+ unless the proxy is aware of the PostgreSQL protocol handshake this
+ would require setting <literal>sslnegotiation</literal>
+ to <literal>direct</literal> or <literal>requiredirect</literal>.)
+ However, <acronym>SNI</acronym> makes the destination host name appear
+ in cleartext in the network traffic, so it might be undesirable in
+ some cases.
</para>
</listitem>
</varlistentry>
@@ -7956,6 +8030,16 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
</para>
</listitem>
+ <listitem>
+ <para>
+ <indexterm>
+ <primary><envar>PGSSLNEGOTIATION</envar></primary>
+ </indexterm>
+ <envar>PGSSLNEGOTIATION</envar> behaves the same as the <xref
+ linkend="libpq-connect-sslnegotiation"/> connection parameter.
+ </para>
+ </listitem>
+
<listitem>
<para>
<indexterm>
--
2.39.2
v3-0004-alpn-support.patchtext/x-patch; charset=US-ASCII; name=v3-0004-alpn-support.patchDownload
From 32185020927824c4b57af900100a37f92ae6a040 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Mon, 20 Mar 2023 14:09:30 -0400
Subject: [PATCH v3 4/4] alpn support
---
src/backend/libpq/be-secure-openssl.c | 65 ++++++++++++++++++++++++
src/backend/libpq/be-secure.c | 3 ++
src/backend/postmaster/postmaster.c | 23 +++++++++
src/bin/psql/command.c | 7 ++-
src/include/libpq/libpq-be.h | 1 +
src/include/libpq/libpq.h | 1 +
src/interfaces/libpq/fe-connect.c | 5 ++
src/interfaces/libpq/fe-secure-openssl.c | 25 +++++++++
src/interfaces/libpq/libpq-int.h | 1 +
9 files changed, 129 insertions(+), 2 deletions(-)
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
index 685aa2ed69..034e1cf2ec 100644
--- a/src/backend/libpq/be-secure-openssl.c
+++ b/src/backend/libpq/be-secure-openssl.c
@@ -67,6 +67,12 @@ static int ssl_external_passwd_cb(char *buf, int size, int rwflag, void *userdat
static int dummy_ssl_passwd_cb(char *buf, int size, int rwflag, void *userdata);
static int verify_cb(int ok, X509_STORE_CTX *ctx);
static void info_cb(const SSL *ssl, int type, int args);
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata);
static bool initialize_dh(SSL_CTX *context, bool isServerStart);
static bool initialize_ecdh(SSL_CTX *context, bool isServerStart);
static const char *SSLerrmessage(unsigned long ecode);
@@ -432,6 +438,11 @@ be_tls_open_server(Port *port)
/* set up debugging/info callback */
SSL_CTX_set_info_callback(SSL_context, info_cb);
+ if (ssl_enable_alpn) {
+ elog(WARNING, "Enabling ALPN Callback");
+ SSL_CTX_set_alpn_select_cb(SSL_context, alpn_cb, port);
+ }
+
if (!(port->ssl = SSL_new(SSL_context)))
{
ereport(COMMERROR,
@@ -1250,6 +1261,60 @@ info_cb(const SSL *ssl, int type, int args)
}
}
+static unsigned char alpn_protos[] = {
+ 12, 'P','o','s','t','g','r','e','s','/','3','.','0'
+};
+static unsigned int alpn_protos_len = sizeof(alpn_protos);
+
+/*
+ * Server callback for ALPN negotiation. We use use the standard "helper"
+ * function even though currently we only accept one value. We store the
+ * negotiated protocol in Port->ssl_alpn_protocol and rely on higher level
+ * logic (in postmaster.c) to decide what to do with that info.
+ *
+ * XXX Not sure if it's kosher to use palloc and elog() inside OpenSSL
+ * callbacks. If we throw an error from here is there a risk of messing up
+ * OpenSSL data structures? It's possible it's ok becuase this is only called
+ * during connection negotiation which we don't try to recover from.
+ */
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata)
+{
+ /* why does OpenSSL provide a helper function that requires a nonconst
+ * vector when the callback is declared to take a const vector? What are
+ * we to do with that?*/
+ int retval;
+ Assert(userdata != NULL);
+ Assert(out != NULL);
+ Assert(outlen != NULL);
+ Assert(in != NULL);
+
+ retval = SSL_select_next_proto((unsigned char **)out, outlen,
+ alpn_protos, alpn_protos_len,
+ in, inlen);
+ if (*out == NULL || *outlen > alpn_protos_len || outlen <= 0)
+ elog(ERROR, "SSL_select_next_proto returned nonsensical results");
+
+ if (retval == OPENSSL_NPN_NEGOTIATED)
+ {
+ struct Port *port = (struct Port *)userdata;
+
+ port->ssl_alpn_protocol = palloc0(*outlen+1);
+ /* SSL uses unsigned chars but Postgres uses host signedness of chars */
+ strncpy(port->ssl_alpn_protocol, (char*)*out, *outlen);
+ return SSL_TLSEXT_ERR_OK;
+ } else if (retval == OPENSSL_NPN_NO_OVERLAP) {
+ return SSL_TLSEXT_ERR_NOACK;
+ } else {
+ return SSL_TLSEXT_ERR_NOACK;
+ }
+}
+
+
/*
* Set DH parameters for generating ephemeral DH keys. The
* DH parameters can take a long time to compute, so they must be
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 1020b3adb0..79a61900ba 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -61,6 +61,9 @@ bool SSLPreferServerCiphers;
int ssl_min_protocol_version = PG_TLS1_2_VERSION;
int ssl_max_protocol_version = PG_TLS_ANY;
+/* GUC variable: if false ignore ALPN negotiation */
+bool ssl_enable_alpn = true;
+
/* ------------------------------------------------------------ */
/* Procedures common to all secure sessions */
/* ------------------------------------------------------------ */
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index ec1d895a23..2640b69fed 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1934,6 +1934,8 @@ ServerLoop(void)
* any bytes from the stream if it's not a direct SSL connection.
*/
+static const char *expected_alpn_protocol = "Postgres/3.0";
+
static int
ProcessSSLStartup(Port *port)
{
@@ -1970,6 +1972,10 @@ ProcessSSLStartup(Port *port)
char *buf = NULL;
elog(LOG, "Detected direct SSL handshake");
+ if (!ssl_enable_alpn) {
+ elog(WARNING, "Received direct SSL connection without ssl_enable_alpn enabled");
+ }
+
/* push unencrypted buffered data back through SSL setup */
len = pq_buffer_has_data();
if (len > 0)
@@ -2000,6 +2006,23 @@ ProcessSSLStartup(Port *port)
return STATUS_ERROR;
}
pfree(port->raw_buf);
+
+ if (port->ssl_alpn_protocol == NULL)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request without required ALPN protocol negotiation extension")));
+ return STATUS_ERROR;
+ }
+ if (strcmp(port->ssl_alpn_protocol, expected_alpn_protocol) != 0)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with unexpected ALPN protocol \"%s\" expected \"%s\"",
+ port->ssl_alpn_protocol,
+ expected_alpn_protocol)));
+ return STATUS_ERROR;
+ }
#else
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 61ec049f05..a219962ea1 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -3715,6 +3715,7 @@ printSSLInfo(void)
const char *protocol;
const char *cipher;
const char *compression;
+ const char *alpn;
if (!PQsslInUse(pset.db))
return; /* no SSL */
@@ -3722,11 +3723,13 @@ printSSLInfo(void)
protocol = PQsslAttribute(pset.db, "protocol");
cipher = PQsslAttribute(pset.db, "cipher");
compression = PQsslAttribute(pset.db, "compression");
+ alpn = PQsslAttribute(pset.db, "alpn");
- printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s)\n"),
+ printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s, ALPN: %s)\n"),
protocol ? protocol : _("unknown"),
cipher ? cipher : _("unknown"),
- (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"));
+ (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"),
+ alpn ? alpn : _("none"));
}
/*
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 824b28e824..2258241770 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -217,6 +217,7 @@ typedef struct Port
char *peer_cn;
char *peer_dn;
bool peer_cert_valid;
+ char *ssl_alpn_protocol;
/*
* OpenSSL structures. (Keep these last so that the locations of other
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 2b02f67257..e7adf4a989 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -123,6 +123,7 @@ extern PGDLLIMPORT char *SSLECDHCurve;
extern PGDLLIMPORT bool SSLPreferServerCiphers;
extern PGDLLIMPORT int ssl_min_protocol_version;
extern PGDLLIMPORT int ssl_max_protocol_version;
+extern PGDLLIMPORT bool ssl_enable_alpn;
enum ssl_protocol_versions
{
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 80d3e9169b..d2aa0f3941 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -307,6 +307,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-SNI", "", 1,
offsetof(struct pg_conn, sslsni)},
+ {"sslalpn", "PGSSLALPN", "1", NULL,
+ "SSL-ALPN", "", 1,
+ offsetof(struct pg_conn, sslalpn)},
+
{"requirepeer", "PGREQUIREPEER", NULL, NULL,
"Require-Peer", "", 10,
offsetof(struct pg_conn, requirepeer)},
@@ -4322,6 +4326,7 @@ freePGconn(PGconn *conn)
free(conn->sslcrldir);
free(conn->sslcompression);
free(conn->sslsni);
+ free(conn->sslalpn);
free(conn->requirepeer);
free(conn->require_auth);
free(conn->ssl_min_protocol_version);
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 6a4431ddfe..1b15542894 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -884,6 +884,13 @@ destroy_ssl_system(void)
#endif
}
+/* XXX This should move somewhere I guess */
+static unsigned char alpn_protos[] = {
+ 12, 'P','o','s','t','g','r','e','s','/','3','.','0'
+};
+static unsigned int alpn_protos_len = sizeof(alpn_protos);
+
+
/*
* Create per-connection SSL object, and load the client certificate,
* private key, and trusted CA certs.
@@ -1202,6 +1209,11 @@ initialize_SSL(PGconn *conn)
}
}
+ if (conn->sslalpn && conn->sslalpn[0] == '1')
+ {
+ SSL_set_alpn_protos(conn->ssl, alpn_protos, alpn_protos_len);
+ }
+
/*
* Read the SSL key. If a key is specified, treat it as an engine:key
* combination if there is colon present - we don't support files with
@@ -1739,6 +1751,19 @@ PQsslAttribute(PGconn *conn, const char *attribute_name)
if (strcmp(attribute_name, "protocol") == 0)
return SSL_get_version(conn->ssl);
+ if (strcmp(attribute_name, "alpn") == 0)
+ {
+ const unsigned char *data;
+ unsigned int len;
+ static char alpn_str[256]; /* alpn doesn't support longer than 255 bytes */
+ SSL_get0_alpn_selected(conn->ssl, &data, &len);
+ if (data == NULL || len==0 || len > sizeof(alpn_str)-1)
+ return NULL;
+ memcpy(alpn_str, data, len);
+ alpn_str[len] = 0;
+ return alpn_str;
+ }
+
return NULL; /* unknown attribute */
}
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 8d8964d835..6cdb5d22b3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -389,6 +389,7 @@ struct pg_conn
char *sslcrl; /* certificate revocation list filename */
char *sslcrldir; /* certificate revocation list directory name */
char *sslsni; /* use SSL SNI extension (0 or 1) */
+ char *sslalpn; /* use SSL ALPN extension (0 or 1) */
char *requirepeer; /* required peer credentials for local sockets */
char *gssencmode; /* GSS mode (require,prefer,disable) */
char *krbsrvname; /* Kerberos service name */
--
2.39.2
Sorry, checking the cfbot apparently I had a typo in the #ifndef USE_SSL case.
Attachments:
v4-0004-alpn-support.patchtext/x-patch; charset=US-ASCII; name=v4-0004-alpn-support.patchDownload
From 4b6e01c7f569a919d660cd80ce64cb913bc9f220 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Mon, 20 Mar 2023 14:09:30 -0400
Subject: [PATCH v4 4/4] alpn support
---
src/backend/libpq/be-secure-openssl.c | 65 ++++++++++++++++++++++++
src/backend/libpq/be-secure.c | 3 ++
src/backend/postmaster/postmaster.c | 23 +++++++++
src/bin/psql/command.c | 7 ++-
src/include/libpq/libpq-be.h | 1 +
src/include/libpq/libpq.h | 1 +
src/interfaces/libpq/fe-connect.c | 5 ++
src/interfaces/libpq/fe-secure-openssl.c | 25 +++++++++
src/interfaces/libpq/libpq-int.h | 1 +
9 files changed, 129 insertions(+), 2 deletions(-)
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
index 685aa2ed69..034e1cf2ec 100644
--- a/src/backend/libpq/be-secure-openssl.c
+++ b/src/backend/libpq/be-secure-openssl.c
@@ -67,6 +67,12 @@ static int ssl_external_passwd_cb(char *buf, int size, int rwflag, void *userdat
static int dummy_ssl_passwd_cb(char *buf, int size, int rwflag, void *userdata);
static int verify_cb(int ok, X509_STORE_CTX *ctx);
static void info_cb(const SSL *ssl, int type, int args);
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata);
static bool initialize_dh(SSL_CTX *context, bool isServerStart);
static bool initialize_ecdh(SSL_CTX *context, bool isServerStart);
static const char *SSLerrmessage(unsigned long ecode);
@@ -432,6 +438,11 @@ be_tls_open_server(Port *port)
/* set up debugging/info callback */
SSL_CTX_set_info_callback(SSL_context, info_cb);
+ if (ssl_enable_alpn) {
+ elog(WARNING, "Enabling ALPN Callback");
+ SSL_CTX_set_alpn_select_cb(SSL_context, alpn_cb, port);
+ }
+
if (!(port->ssl = SSL_new(SSL_context)))
{
ereport(COMMERROR,
@@ -1250,6 +1261,60 @@ info_cb(const SSL *ssl, int type, int args)
}
}
+static unsigned char alpn_protos[] = {
+ 12, 'P','o','s','t','g','r','e','s','/','3','.','0'
+};
+static unsigned int alpn_protos_len = sizeof(alpn_protos);
+
+/*
+ * Server callback for ALPN negotiation. We use use the standard "helper"
+ * function even though currently we only accept one value. We store the
+ * negotiated protocol in Port->ssl_alpn_protocol and rely on higher level
+ * logic (in postmaster.c) to decide what to do with that info.
+ *
+ * XXX Not sure if it's kosher to use palloc and elog() inside OpenSSL
+ * callbacks. If we throw an error from here is there a risk of messing up
+ * OpenSSL data structures? It's possible it's ok becuase this is only called
+ * during connection negotiation which we don't try to recover from.
+ */
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata)
+{
+ /* why does OpenSSL provide a helper function that requires a nonconst
+ * vector when the callback is declared to take a const vector? What are
+ * we to do with that?*/
+ int retval;
+ Assert(userdata != NULL);
+ Assert(out != NULL);
+ Assert(outlen != NULL);
+ Assert(in != NULL);
+
+ retval = SSL_select_next_proto((unsigned char **)out, outlen,
+ alpn_protos, alpn_protos_len,
+ in, inlen);
+ if (*out == NULL || *outlen > alpn_protos_len || outlen <= 0)
+ elog(ERROR, "SSL_select_next_proto returned nonsensical results");
+
+ if (retval == OPENSSL_NPN_NEGOTIATED)
+ {
+ struct Port *port = (struct Port *)userdata;
+
+ port->ssl_alpn_protocol = palloc0(*outlen+1);
+ /* SSL uses unsigned chars but Postgres uses host signedness of chars */
+ strncpy(port->ssl_alpn_protocol, (char*)*out, *outlen);
+ return SSL_TLSEXT_ERR_OK;
+ } else if (retval == OPENSSL_NPN_NO_OVERLAP) {
+ return SSL_TLSEXT_ERR_NOACK;
+ } else {
+ return SSL_TLSEXT_ERR_NOACK;
+ }
+}
+
+
/*
* Set DH parameters for generating ephemeral DH keys. The
* DH parameters can take a long time to compute, so they must be
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 1020b3adb0..79a61900ba 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -61,6 +61,9 @@ bool SSLPreferServerCiphers;
int ssl_min_protocol_version = PG_TLS1_2_VERSION;
int ssl_max_protocol_version = PG_TLS_ANY;
+/* GUC variable: if false ignore ALPN negotiation */
+bool ssl_enable_alpn = true;
+
/* ------------------------------------------------------------ */
/* Procedures common to all secure sessions */
/* ------------------------------------------------------------ */
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index ec1d895a23..2640b69fed 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1934,6 +1934,8 @@ ServerLoop(void)
* any bytes from the stream if it's not a direct SSL connection.
*/
+static const char *expected_alpn_protocol = "Postgres/3.0";
+
static int
ProcessSSLStartup(Port *port)
{
@@ -1970,6 +1972,10 @@ ProcessSSLStartup(Port *port)
char *buf = NULL;
elog(LOG, "Detected direct SSL handshake");
+ if (!ssl_enable_alpn) {
+ elog(WARNING, "Received direct SSL connection without ssl_enable_alpn enabled");
+ }
+
/* push unencrypted buffered data back through SSL setup */
len = pq_buffer_has_data();
if (len > 0)
@@ -2000,6 +2006,23 @@ ProcessSSLStartup(Port *port)
return STATUS_ERROR;
}
pfree(port->raw_buf);
+
+ if (port->ssl_alpn_protocol == NULL)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request without required ALPN protocol negotiation extension")));
+ return STATUS_ERROR;
+ }
+ if (strcmp(port->ssl_alpn_protocol, expected_alpn_protocol) != 0)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with unexpected ALPN protocol \"%s\" expected \"%s\"",
+ port->ssl_alpn_protocol,
+ expected_alpn_protocol)));
+ return STATUS_ERROR;
+ }
#else
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 61ec049f05..a219962ea1 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -3715,6 +3715,7 @@ printSSLInfo(void)
const char *protocol;
const char *cipher;
const char *compression;
+ const char *alpn;
if (!PQsslInUse(pset.db))
return; /* no SSL */
@@ -3722,11 +3723,13 @@ printSSLInfo(void)
protocol = PQsslAttribute(pset.db, "protocol");
cipher = PQsslAttribute(pset.db, "cipher");
compression = PQsslAttribute(pset.db, "compression");
+ alpn = PQsslAttribute(pset.db, "alpn");
- printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s)\n"),
+ printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s, ALPN: %s)\n"),
protocol ? protocol : _("unknown"),
cipher ? cipher : _("unknown"),
- (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"));
+ (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"),
+ alpn ? alpn : _("none"));
}
/*
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 824b28e824..2258241770 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -217,6 +217,7 @@ typedef struct Port
char *peer_cn;
char *peer_dn;
bool peer_cert_valid;
+ char *ssl_alpn_protocol;
/*
* OpenSSL structures. (Keep these last so that the locations of other
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 2b02f67257..e7adf4a989 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -123,6 +123,7 @@ extern PGDLLIMPORT char *SSLECDHCurve;
extern PGDLLIMPORT bool SSLPreferServerCiphers;
extern PGDLLIMPORT int ssl_min_protocol_version;
extern PGDLLIMPORT int ssl_max_protocol_version;
+extern PGDLLIMPORT bool ssl_enable_alpn;
enum ssl_protocol_versions
{
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index eb4cfdee17..f050257752 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -307,6 +307,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-SNI", "", 1,
offsetof(struct pg_conn, sslsni)},
+ {"sslalpn", "PGSSLALPN", "1", NULL,
+ "SSL-ALPN", "", 1,
+ offsetof(struct pg_conn, sslalpn)},
+
{"requirepeer", "PGREQUIREPEER", NULL, NULL,
"Require-Peer", "", 10,
offsetof(struct pg_conn, requirepeer)},
@@ -4322,6 +4326,7 @@ freePGconn(PGconn *conn)
free(conn->sslcrldir);
free(conn->sslcompression);
free(conn->sslsni);
+ free(conn->sslalpn);
free(conn->requirepeer);
free(conn->require_auth);
free(conn->ssl_min_protocol_version);
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 6a4431ddfe..1b15542894 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -884,6 +884,13 @@ destroy_ssl_system(void)
#endif
}
+/* XXX This should move somewhere I guess */
+static unsigned char alpn_protos[] = {
+ 12, 'P','o','s','t','g','r','e','s','/','3','.','0'
+};
+static unsigned int alpn_protos_len = sizeof(alpn_protos);
+
+
/*
* Create per-connection SSL object, and load the client certificate,
* private key, and trusted CA certs.
@@ -1202,6 +1209,11 @@ initialize_SSL(PGconn *conn)
}
}
+ if (conn->sslalpn && conn->sslalpn[0] == '1')
+ {
+ SSL_set_alpn_protos(conn->ssl, alpn_protos, alpn_protos_len);
+ }
+
/*
* Read the SSL key. If a key is specified, treat it as an engine:key
* combination if there is colon present - we don't support files with
@@ -1739,6 +1751,19 @@ PQsslAttribute(PGconn *conn, const char *attribute_name)
if (strcmp(attribute_name, "protocol") == 0)
return SSL_get_version(conn->ssl);
+ if (strcmp(attribute_name, "alpn") == 0)
+ {
+ const unsigned char *data;
+ unsigned int len;
+ static char alpn_str[256]; /* alpn doesn't support longer than 255 bytes */
+ SSL_get0_alpn_selected(conn->ssl, &data, &len);
+ if (data == NULL || len==0 || len > sizeof(alpn_str)-1)
+ return NULL;
+ memcpy(alpn_str, data, len);
+ alpn_str[len] = 0;
+ return alpn_str;
+ }
+
return NULL; /* unknown attribute */
}
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 8d8964d835..6cdb5d22b3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -389,6 +389,7 @@ struct pg_conn
char *sslcrl; /* certificate revocation list filename */
char *sslcrldir; /* certificate revocation list directory name */
char *sslsni; /* use SSL SNI extension (0 or 1) */
+ char *sslalpn; /* use SSL ALPN extension (0 or 1) */
char *requirepeer; /* required peer credentials for local sockets */
char *gssencmode; /* GSS mode (require,prefer,disable) */
char *krbsrvname; /* Kerberos service name */
--
2.39.2
v4-0003-Direct-SSL-connections-documentation.patchtext/x-patch; charset=US-ASCII; name=v4-0003-Direct-SSL-connections-documentation.patchDownload
From 4dd508fad6f9088894054d37ab3d49dc5fdcd701 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 15:10:15 -0400
Subject: [PATCH v4 3/4] Direct SSL connections documentation
---
doc/src/sgml/libpq.sgml | 102 ++++++++++++++++++++++++++++++++++++----
1 file changed, 93 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 9ee5532c07..0b89b224ba 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1691,10 +1691,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
Note that if <acronym>GSSAPI</acronym> encryption is possible,
that will be used in preference to <acronym>SSL</acronym>
encryption, regardless of the value of <literal>sslmode</literal>.
- To force use of <acronym>SSL</acronym> encryption in an
- environment that has working <acronym>GSSAPI</acronym>
- infrastructure (such as a Kerberos server), also
- set <literal>gssencmode</literal> to <literal>disable</literal>.
+ To negotiate <acronym>SSL</acronym> encryption in an environment that
+ has working <acronym>GSSAPI</acronym> infrastructure (such as a
+ Kerberos server), also set <literal>gssencmode</literal>
+ to <literal>disable</literal>.
+ Use of non-default values of <literal>sslnegotiation</literal> can
+ also cause <acronym>SSL</acronym> to be used instead of
+ negotiating <acronym>GSSAPI</acronym> encryption.
</para>
</listitem>
</varlistentry>
@@ -1721,6 +1724,75 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
</listitem>
</varlistentry>
+ <varlistentry id="libpq-connect-sslnegotiation" xreflabel="sslnegotiation">
+ <term><literal>sslnegotiation</literal></term>
+ <listitem>
+ <para>
+ This option controls whether <productname>PostgreSQL</productname>
+ will perform its protocol negotiation to request encryption from the
+ server or will just directly make a standard <acronym>SSL</acronym>
+ connection. Traditional <productname>PostgreSQL</productname>
+ protocol negotiation is the default and the most flexible with
+ different server configurations. If the server is known to support
+ direct <acronym>SSL</acronym> connections then the latter requires one
+ fewer round trip reducing connection latency and also allows the use
+ of protocol agnostic SSL network tools.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>postgres</literal></term>
+ <listitem>
+ <para>
+ perform <productname>PostgreSQL</productname> protocol
+ negotiation. This is the default if the option is not provided.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>direct</literal></term>
+ <listitem>
+ <para>
+ first attempt to establish a standard SSL connection and if that
+ fails reconnect and perform the negotiation. This fallback
+ process adds significant latency if the initial SSL connection
+ fails.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>requiredirect</literal></term>
+ <listitem>
+ <para>
+ attempt to establish a standard SSL connection and if that fails
+ return a connection failure immediately.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ If <literal>sslmode</literal> set to <literal>disable</literal>
+ or <literal>allow</literal> then <literal>sslnegotiation</literal> is
+ ignored. If <literal>gssencmode</literal> is set
+ to <literal>require</literal> then <literal>sslnegotiation</literal>
+ must be the default <literal>postgres</literal> value.
+ </para>
+
+ <para>
+ Moreover, note if <literal>gssencmode</literal> is set
+ to <literal>prefer</literal> and <literal>sslnegotiation</literal>
+ to <literal>direct</literal> then the effective preference will be
+ direct <acronym>SSL</acronym> connections, followed by
+ negotiated <acronym>GSS</acronym> connections, followed by
+ negotiated <acronym>SSL</acronym> connections, possibly followed
+ lastly by unencrypted connections.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-connect-sslcompression" xreflabel="sslcompression">
<term><literal>sslcompression</literal></term>
<listitem>
@@ -1874,11 +1946,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
<para>
The Server Name Indication can be used by SSL-aware proxies to route
- connections without having to decrypt the SSL stream. (Note that this
- requires a proxy that is aware of the PostgreSQL protocol handshake,
- not just any SSL proxy.) However, <acronym>SNI</acronym> makes the
- destination host name appear in cleartext in the network traffic, so
- it might be undesirable in some cases.
+ connections without having to decrypt the SSL stream. (Note that
+ unless the proxy is aware of the PostgreSQL protocol handshake this
+ would require setting <literal>sslnegotiation</literal>
+ to <literal>direct</literal> or <literal>requiredirect</literal>.)
+ However, <acronym>SNI</acronym> makes the destination host name appear
+ in cleartext in the network traffic, so it might be undesirable in
+ some cases.
</para>
</listitem>
</varlistentry>
@@ -7956,6 +8030,16 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
</para>
</listitem>
+ <listitem>
+ <para>
+ <indexterm>
+ <primary><envar>PGSSLNEGOTIATION</envar></primary>
+ </indexterm>
+ <envar>PGSSLNEGOTIATION</envar> behaves the same as the <xref
+ linkend="libpq-connect-sslnegotiation"/> connection parameter.
+ </para>
+ </listitem>
+
<listitem>
<para>
<indexterm>
--
2.39.2
v4-0002-Direct-SSL-connections-client-support.patchtext/x-patch; charset=US-ASCII; name=v4-0002-Direct-SSL-connections-client-support.patchDownload
From 964fcb2ded5b54ce95a2a52b54d006a69d41e118 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 11:55:16 -0400
Subject: [PATCH v4 2/4] Direct SSL connections client support
---
src/interfaces/libpq/fe-connect.c | 91 +++++++++++++++++++++++++++++--
src/interfaces/libpq/libpq-fe.h | 1 +
src/interfaces/libpq/libpq-int.h | 3 +
3 files changed, 89 insertions(+), 6 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index b9f899c552..eb4cfdee17 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -271,6 +271,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-Mode", "", 12, /* sizeof("verify-full") == 12 */
offsetof(struct pg_conn, sslmode)},
+ {"sslnegotiation", "PGSSLNEGOTIATION", "postgres", NULL,
+ "SSL-Negotiation", "", 14, /* strlen("requiredirect") == 14 */
+ offsetof(struct pg_conn, sslnegotiation)},
+
{"sslcompression", "PGSSLCOMPRESSION", "0", NULL,
"SSL-Compression", "", 1,
offsetof(struct pg_conn, sslcompression)},
@@ -1463,11 +1467,36 @@ connectOptions2(PGconn *conn)
}
#endif
}
+
+ /*
+ * validate sslnegotiation option, default is "postgres" for the postgres
+ * style negotiated connection with an extra round trip but more options.
+ */
+ if (conn->sslnegotiation)
+ {
+ if (strcmp(conn->sslnegotiation, "postgres") != 0
+ && strcmp(conn->sslnegotiation, "direct") != 0
+ && strcmp(conn->sslnegotiation, "requiredirect") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
+ "sslnegotiation", conn->sslnegotiation);
+ return false;
+ }
+
+#ifndef USE_SSL
+ if (conn->sslnegotiation[0] != 'p') {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "sslnegotiation value \"%s\" invalid when SSL support is not compiled in",
+ conn->sslnegotiation);
+ return false;
+ }
+#endif
+ }
else
{
- conn->sslmode = strdup(DefaultSSLMode);
- if (!conn->sslmode)
- goto oom_error;
+ libpq_append_conn_error(conn, "sslnegotiation missing?");
+ return false;
}
/*
@@ -1527,6 +1556,18 @@ connectOptions2(PGconn *conn)
conn->gssencmode);
return false;
}
+#endif
+#ifdef USE_SSL
+ /* GSS is incompatible with direct SSL connections so it requires the
+ * default postgres style connection ssl negotiation */
+ if (strcmp(conn->gssencmode, "require") == 0 &&
+ strcmp(conn->sslnegotiation, "postgres") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when Direct SSL negotiation is enabled",
+ conn->gssencmode);
+ return false;
+ }
#endif
}
else
@@ -2583,11 +2624,12 @@ keep_going: /* We will come back to here until there is
/* initialize these values based on SSL mode */
conn->allow_ssl_try = (conn->sslmode[0] != 'd'); /* "disable" */
conn->wait_ssl_try = (conn->sslmode[0] == 'a'); /* "allow" */
+ /* direct ssl is incompatible with "allow" or "disabled" ssl */
+ conn->allow_direct_ssl_try = conn->allow_ssl_try && !conn->wait_ssl_try && (conn->sslnegotiation[0] != 'p');
#endif
#ifdef ENABLE_GSS
conn->try_gss = (conn->gssencmode[0] != 'd'); /* "disable" */
#endif
-
reset_connection_state_machine = false;
need_new_connection = true;
}
@@ -3046,6 +3088,28 @@ keep_going: /* We will come back to here until there is
if (pqsecure_initialize(conn, false, true) < 0)
goto error_return;
+ /* If SSL is enabled and direct SSL connections are enabled
+ * and we haven't already established an SSL connection (or
+ * already tried a direct connection and failed or succeeded)
+ * then try just enabling SSL directly.
+ *
+ * If we fail then we'll either fail the connection (if
+ * sslnegotiation is set to requiredirect or turn
+ * allow_direct_ssl_try to false
+ */
+ if (conn->allow_ssl_try
+ && !conn->wait_ssl_try
+ && conn->allow_direct_ssl_try
+ && !conn->ssl_in_use
+#ifdef ENABLE_GSS
+ && !conn->gssenc
+#endif
+ )
+ {
+ conn->status = CONNECTION_SSL_STARTUP;
+ return PGRES_POLLING_WRITING;
+ }
+
/*
* If SSL is enabled and we haven't already got encryption of
* some sort running, request SSL instead of sending the
@@ -3122,9 +3186,11 @@ keep_going: /* We will come back to here until there is
/*
* On first time through, get the postmaster's response to our
- * SSL negotiation packet.
+ * SSL negotiation packet. If we are trying a direct ssl
+ * connection skip reading the negotiation packet and go
+ * straight to initiating an ssl connection.
*/
- if (!conn->ssl_in_use)
+ if (!conn->ssl_in_use && !conn->allow_direct_ssl_try)
{
/*
* We use pqReadData here since it has the logic to
@@ -3230,6 +3296,18 @@ keep_going: /* We will come back to here until there is
}
if (pollres == PGRES_POLLING_FAILED)
{
+ /* Failed direct ssl connection, possibly try a new connection with postgres negotiation */
+ if (conn->allow_direct_ssl_try)
+ {
+ /* if it's requiredirect then it's a hard failure */
+ if (conn->sslnegotiation[0] == 'r')
+ goto error_return;
+ /* otherwise only retry using postgres connection */
+ conn->allow_direct_ssl_try = false;
+ need_new_connection = true;
+ goto keep_going;
+ }
+
/*
* Failed ... if sslmode is "prefer" then do a non-SSL
* retry
@@ -4231,6 +4309,7 @@ freePGconn(PGconn *conn)
free(conn->keepalives_interval);
free(conn->keepalives_count);
free(conn->sslmode);
+ free(conn->sslnegotiation);
free(conn->sslcert);
free(conn->sslkey);
if (conn->sslpassword)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index f3d9220496..05821b8473 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -72,6 +72,7 @@ typedef enum
CONNECTION_AUTH_OK, /* Received authentication; waiting for
* backend startup. */
CONNECTION_SETENV, /* This state is no longer used. */
+ CONNECTION_DIRECT_SSL_STARTUP, /* Starting SSL without PG Negotiation. */
CONNECTION_SSL_STARTUP, /* Negotiating SSL. */
CONNECTION_NEEDED, /* Internal state: connect() needed */
CONNECTION_CHECK_WRITABLE, /* Checking if session is read-write. */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 1dc264fe54..8d8964d835 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -380,6 +380,7 @@ struct pg_conn
char *keepalives_count; /* maximum number of TCP keepalive
* retransmits */
char *sslmode; /* SSL mode (require,prefer,allow,disable) */
+ char *sslnegotiation; /* SSL initiation style (postgres,direct,requiredirect) */
char *sslcompression; /* SSL compression (0 or 1) */
char *sslkey; /* client key filename */
char *sslcert; /* client certificate filename */
@@ -529,6 +530,8 @@ struct pg_conn
bool ssl_in_use;
#ifdef USE_SSL
+ bool allow_direct_ssl_try; /* Try to make a direct SSL connection
+ * without an "SSL negotiation packet" */
bool allow_ssl_try; /* Allowed to try SSL negotiation */
bool wait_ssl_try; /* Delay SSL negotiation until after
* attempting normal connection */
--
2.39.2
v4-0001-Direct-SSL-connections-postmaster-support.patchtext/x-patch; charset=US-ASCII; name=v4-0001-Direct-SSL-connections-postmaster-support.patchDownload
From 53cd555f73d91150db7bc316bc3dca831ee98ce3 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Wed, 15 Mar 2023 15:51:02 -0400
Subject: [PATCH v4 1/4] Direct SSL connections postmaster support
---
src/backend/libpq/be-secure.c | 13 +++
src/backend/libpq/pqcomm.c | 9 +-
src/backend/postmaster/postmaster.c | 133 ++++++++++++++++++++++------
src/include/libpq/libpq-be.h | 10 +++
src/include/libpq/libpq.h | 2 +-
5 files changed, 134 insertions(+), 33 deletions(-)
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index a0f7084018..1020b3adb0 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -235,6 +235,19 @@ secure_raw_read(Port *port, void *ptr, size_t len)
{
ssize_t n;
+ /* Read from the "unread" buffered data first. c.f. libpq-be.h */
+ if (port->raw_buf_remaining > 0)
+ {
+ /* consume up to len bytes from the raw_buf */
+ if (len > port->raw_buf_remaining)
+ len = port->raw_buf_remaining;
+ Assert(port->raw_buf);
+ memcpy(ptr, port->raw_buf + port->raw_buf_consumed, len);
+ port->raw_buf_consumed += len;
+ port->raw_buf_remaining -= len;
+ return len;
+ }
+
/*
* Try to read from the socket without blocking. If it succeeds we're
* done, otherwise we'll wait for the socket using the latch mechanism.
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index da5bb5fc5d..17d025bb8e 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -1126,13 +1126,16 @@ pq_discardbytes(size_t len)
/* --------------------------------
* pq_buffer_has_data - is any buffered data available to read?
*
- * This will *not* attempt to read more data.
+ * Actually returns the number of bytes in the buffer...
+ *
+ * This will *not* attempt to read more data. And reading up to that number of
+ * bytes should not cause reading any more data either.
* --------------------------------
*/
-bool
+size_t
pq_buffer_has_data(void)
{
- return (PqRecvPointer < PqRecvLength);
+ return (PqRecvLength - PqRecvPointer);
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 4c49393fc5..ec1d895a23 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -422,6 +422,7 @@ static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(Port *port);
+static int ProcessSSLStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -1926,6 +1927,97 @@ ServerLoop(void)
}
}
+/*
+ * Check for a native direct SSL connection.
+ *
+ * This happens before startup packets so we are careful not to actual read
+ * any bytes from the stream if it's not a direct SSL connection.
+ */
+
+static int
+ProcessSSLStartup(Port *port)
+{
+ int firstbyte;
+
+ pq_startmsgread();
+
+ firstbyte = pq_peekbyte();
+ if (firstbyte == EOF)
+ {
+ /*
+ * If we get no data at all, don't clutter the log with a complaint;
+ * such cases often occur for legitimate reasons. An example is that
+ * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
+ * client didn't like our response, it'll probably just drop the
+ * connection. Service-monitoring software also often just opens and
+ * closes a connection without sending anything. (So do port
+ * scanners, which may be less benign, but it's not really our job to
+ * notice those.)
+ */
+ return STATUS_ERROR;
+ }
+
+ /*
+ * First byte indicates standard SSL handshake message
+ *
+ * (It can't be a Postgres startup length because in network byte order
+ * that would be a startup packet hundreds of megabytes long)
+ */
+ if (firstbyte == 0x16)
+ {
+#ifdef USE_SSL
+ ssize_t len;
+ char *buf = NULL;
+ elog(LOG, "Detected direct SSL handshake");
+
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ buf = palloc(len);
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+
+ Assert(pq_buffer_has_data() == 0);
+ if (secure_open_server(port) == -1)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("SSL Protocol Error during direct SSL connection initiation")));
+ return STATUS_ERROR;
+ }
+
+ if (port->raw_buf_remaining > 0)
+ {
+ /* This shouldn't be possible -- it would mean the client sent
+ * encrypted data before we established a session key...
+ */
+ elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ return STATUS_ERROR;
+ }
+ pfree(port->raw_buf);
+#else
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with no SSL support")));
+ return STATUS_ERROR;
+#endif
+ }
+
+ pq_endmsgread();
+
+ if (port->ssl_in_use)
+ ereport(DEBUG2,
+ (errmsg_internal("Direct SSL connection established")));
+
+ return STATUS_OK;
+}
+
+
/*
* Read a client's startup packet and do something according to it.
*
@@ -1954,28 +2046,7 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
pq_startmsgread();
- /*
- * Grab the first byte of the length word separately, so that we can tell
- * whether we have no data at all or an incomplete packet. (This might
- * sound inefficient, but it's not really, because of buffering in
- * pqcomm.c.)
- */
- if (pq_getbytes((char *) &len, 1) == EOF)
- {
- /*
- * If we get no data at all, don't clutter the log with a complaint;
- * such cases often occur for legitimate reasons. An example is that
- * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
- * client didn't like our response, it'll probably just drop the
- * connection. Service-monitoring software also often just opens and
- * closes a connection without sending anything. (So do port
- * scanners, which may be less benign, but it's not really our job to
- * notice those.)
- */
- return STATUS_ERROR;
- }
-
- if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
+ if (pq_getbytes(((char *) &len), 4) == EOF)
{
/* Got a partial length word, so bleat about that */
if (!ssl_done && !gss_done)
@@ -2039,8 +2110,11 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
char SSLok;
#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ /* No SSL when disabled or on Unix sockets.
+ *
+ * Also no SSL negotiation if we already have a direct SSL connection
+ */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX || port->ssl_in_use)
SSLok = 'N';
else
SSLok = 'S'; /* Support for SSL */
@@ -2048,11 +2122,10 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
SSLok = 'N'; /* No support for SSL */
#endif
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
+ while (secure_write(port, &SSLok, 1) != 1)
{
if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
+ continue; /* if interrupted, just retry */
ereport(COMMERROR,
(errcode_for_socket_access(),
errmsg("failed to send SSL negotiation response: %m")));
@@ -2093,7 +2166,7 @@ retry1:
GSSok = 'G';
#endif
- while (send(port->sock, &GSSok, 1, 0) != 1)
+ while (secure_write(port, &GSSok, 1) != 1)
{
if (errno == EINTR)
continue;
@@ -4396,7 +4469,9 @@ BackendInitialize(Port *port)
* Receive the startup packet (which might turn out to be a cancel request
* packet).
*/
- status = ProcessStartupPacket(port, false, false);
+ status = ProcessSSLStartup(port);
+ if (status == STATUS_OK)
+ status = ProcessStartupPacket(port, false, false);
/*
* Disable the timeout, and prevent SIGTERM again.
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index ac6407e9f6..824b28e824 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -226,6 +226,16 @@ typedef struct Port
SSL *ssl;
X509 *peer;
#endif
+ /* This is a bit of a hack. The raw_buf is data that was previously read
+ * and buffered in a higher layer but then "unread" and needs to be read
+ * again while establishing an SSL connection via the SSL library layer.
+ *
+ * There's no API to "unread", the upper layer just places the data in the
+ * Port structure in raw_buf and sets raw_buf_remaining to the amount of
+ * bytes unread and raw_buf_consumed to 0.
+ */
+ char *raw_buf;
+ ssize_t raw_buf_consumed, raw_buf_remaining;
} Port;
#ifdef USE_SSL
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 50fc781f47..2b02f67257 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -80,7 +80,7 @@ extern int pq_getmessage(StringInfo s, int maxlen);
extern int pq_getbyte(void);
extern int pq_peekbyte(void);
extern int pq_getbyte_if_available(unsigned char *c);
-extern bool pq_buffer_has_data(void);
+extern size_t pq_buffer_has_data(void);
extern int pq_putmessage_v2(char msgtype, const char *s, size_t len);
extern bool pq_check_connection(void);
--
2.39.2
On Mon, 20 Mar 2023 at 16:31, Greg Stark <stark@mit.edu> wrote:
Here's a first cut at ALPN support.
Currently it's using a hard coded "Postgres/3.0" protocol
Apparently that is explicitly disrecommended by the IETF folk. They
want something like "TBD" so people don't start using a string until
it's been added to the registry. So I've changed this for now (to
"TBD-pgsql")
Ok, I think this has pretty much everything I was hoping to do.
The one thing I'm not sure of is it seems some codepaths in postmaster
have ereport(COMMERROR) followed by returning an error whereas other
codepaths just have ereport(FATAL). And I don't actually see much
logic in which do which. (I get the principle behind COMMERR it just
seems like it doesn't really match the code).
I realized I had exactly the infrastructure needed to allow pipelining
the SSL ClientHello like Neon wanted to do so I added that too. It's
kind of redundant with direct SSL connections but seems like there may
be reasons to use that instead.
--
greg
Attachments:
v5-0002-Direct-SSL-connections-client-support.patchtext/x-patch; charset=US-ASCII; name=v5-0002-Direct-SSL-connections-client-support.patchDownload
From 083df15eff52f025064e2879a404270e405f7dde Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 11:55:16 -0400
Subject: [PATCH v5 2/6] Direct SSL connections client support
---
src/interfaces/libpq/fe-connect.c | 91 +++++++++++++++++++++++++++++--
src/interfaces/libpq/libpq-fe.h | 1 +
src/interfaces/libpq/libpq-int.h | 3 +
3 files changed, 89 insertions(+), 6 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index bb7347cb0c..7cd0eb261f 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -274,6 +274,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-Mode", "", 12, /* sizeof("verify-full") == 12 */
offsetof(struct pg_conn, sslmode)},
+ {"sslnegotiation", "PGSSLNEGOTIATION", "postgres", NULL,
+ "SSL-Negotiation", "", 14, /* strlen("requiredirect") == 14 */
+ offsetof(struct pg_conn, sslnegotiation)},
+
{"sslcompression", "PGSSLCOMPRESSION", "0", NULL,
"SSL-Compression", "", 1,
offsetof(struct pg_conn, sslcompression)},
@@ -1504,11 +1508,36 @@ connectOptions2(PGconn *conn)
}
#endif
}
+
+ /*
+ * validate sslnegotiation option, default is "postgres" for the postgres
+ * style negotiated connection with an extra round trip but more options.
+ */
+ if (conn->sslnegotiation)
+ {
+ if (strcmp(conn->sslnegotiation, "postgres") != 0
+ && strcmp(conn->sslnegotiation, "direct") != 0
+ && strcmp(conn->sslnegotiation, "requiredirect") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
+ "sslnegotiation", conn->sslnegotiation);
+ return false;
+ }
+
+#ifndef USE_SSL
+ if (conn->sslnegotiation[0] != 'p') {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "sslnegotiation value \"%s\" invalid when SSL support is not compiled in",
+ conn->sslnegotiation);
+ return false;
+ }
+#endif
+ }
else
{
- conn->sslmode = strdup(DefaultSSLMode);
- if (!conn->sslmode)
- goto oom_error;
+ libpq_append_conn_error(conn, "sslnegotiation missing?");
+ return false;
}
/*
@@ -1614,6 +1643,18 @@ connectOptions2(PGconn *conn)
conn->gssencmode);
return false;
}
+#endif
+#ifdef USE_SSL
+ /* GSS is incompatible with direct SSL connections so it requires the
+ * default postgres style connection ssl negotiation */
+ if (strcmp(conn->gssencmode, "require") == 0 &&
+ strcmp(conn->sslnegotiation, "postgres") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when Direct SSL negotiation is enabled",
+ conn->gssencmode);
+ return false;
+ }
#endif
}
else
@@ -2747,11 +2788,12 @@ keep_going: /* We will come back to here until there is
/* initialize these values based on SSL mode */
conn->allow_ssl_try = (conn->sslmode[0] != 'd'); /* "disable" */
conn->wait_ssl_try = (conn->sslmode[0] == 'a'); /* "allow" */
+ /* direct ssl is incompatible with "allow" or "disabled" ssl */
+ conn->allow_direct_ssl_try = conn->allow_ssl_try && !conn->wait_ssl_try && (conn->sslnegotiation[0] != 'p');
#endif
#ifdef ENABLE_GSS
conn->try_gss = (conn->gssencmode[0] != 'd'); /* "disable" */
#endif
-
reset_connection_state_machine = false;
need_new_connection = true;
}
@@ -3209,6 +3251,28 @@ keep_going: /* We will come back to here until there is
if (pqsecure_initialize(conn, false, true) < 0)
goto error_return;
+ /* If SSL is enabled and direct SSL connections are enabled
+ * and we haven't already established an SSL connection (or
+ * already tried a direct connection and failed or succeeded)
+ * then try just enabling SSL directly.
+ *
+ * If we fail then we'll either fail the connection (if
+ * sslnegotiation is set to requiredirect or turn
+ * allow_direct_ssl_try to false
+ */
+ if (conn->allow_ssl_try
+ && !conn->wait_ssl_try
+ && conn->allow_direct_ssl_try
+ && !conn->ssl_in_use
+#ifdef ENABLE_GSS
+ && !conn->gssenc
+#endif
+ )
+ {
+ conn->status = CONNECTION_SSL_STARTUP;
+ return PGRES_POLLING_WRITING;
+ }
+
/*
* If SSL is enabled and we haven't already got encryption of
* some sort running, request SSL instead of sending the
@@ -3285,9 +3349,11 @@ keep_going: /* We will come back to here until there is
/*
* On first time through, get the postmaster's response to our
- * SSL negotiation packet.
+ * SSL negotiation packet. If we are trying a direct ssl
+ * connection skip reading the negotiation packet and go
+ * straight to initiating an ssl connection.
*/
- if (!conn->ssl_in_use)
+ if (!conn->ssl_in_use && !conn->allow_direct_ssl_try)
{
/*
* We use pqReadData here since it has the logic to
@@ -3393,6 +3459,18 @@ keep_going: /* We will come back to here until there is
}
if (pollres == PGRES_POLLING_FAILED)
{
+ /* Failed direct ssl connection, possibly try a new connection with postgres negotiation */
+ if (conn->allow_direct_ssl_try)
+ {
+ /* if it's requiredirect then it's a hard failure */
+ if (conn->sslnegotiation[0] == 'r')
+ goto error_return;
+ /* otherwise only retry using postgres connection */
+ conn->allow_direct_ssl_try = false;
+ need_new_connection = true;
+ goto keep_going;
+ }
+
/*
* Failed ... if sslmode is "prefer" then do a non-SSL
* retry
@@ -4395,6 +4473,7 @@ freePGconn(PGconn *conn)
free(conn->keepalives_interval);
free(conn->keepalives_count);
free(conn->sslmode);
+ free(conn->sslnegotiation);
free(conn->sslcert);
free(conn->sslkey);
if (conn->sslpassword)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index f3d9220496..05821b8473 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -72,6 +72,7 @@ typedef enum
CONNECTION_AUTH_OK, /* Received authentication; waiting for
* backend startup. */
CONNECTION_SETENV, /* This state is no longer used. */
+ CONNECTION_DIRECT_SSL_STARTUP, /* Starting SSL without PG Negotiation. */
CONNECTION_SSL_STARTUP, /* Negotiating SSL. */
CONNECTION_NEEDED, /* Internal state: connect() needed */
CONNECTION_CHECK_WRITABLE, /* Checking if session is read-write. */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index d93e976ca5..db63afd786 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -390,6 +390,7 @@ struct pg_conn
char *keepalives_count; /* maximum number of TCP keepalive
* retransmits */
char *sslmode; /* SSL mode (require,prefer,allow,disable) */
+ char *sslnegotiation; /* SSL initiation style (postgres,direct,requiredirect) */
char *sslcompression; /* SSL compression (0 or 1) */
char *sslkey; /* client key filename */
char *sslcert; /* client certificate filename */
@@ -549,6 +550,8 @@ struct pg_conn
bool ssl_cert_sent; /* Did we send one in reply? */
#ifdef USE_SSL
+ bool allow_direct_ssl_try; /* Try to make a direct SSL connection
+ * without an "SSL negotiation packet" */
bool allow_ssl_try; /* Allowed to try SSL negotiation */
bool wait_ssl_try; /* Delay SSL negotiation until after
* attempting normal connection */
--
2.40.0
v5-0006-Some-added-docs.patchtext/x-patch; charset=US-ASCII; name=v5-0006-Some-added-docs.patchDownload
From 9357d177f07f20f21e896958cec9992a60515195 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Fri, 31 Mar 2023 03:01:35 -0400
Subject: [PATCH v5 6/6] Some added docs
---
doc/src/sgml/protocol.sgml | 37 +++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 8b5e7b1ad7..dd363656fd 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -1532,17 +1532,54 @@ SELCT 1/0;<!-- this typo is intentional -->
bytes.
</para>
+ <para>
+ Likewise the server expects the client to not begin
+ the <acronym>SSL</acronym> negotiation until it receives the server's
+ single byte response to the <acronym>SSL</acronym> request. If the
+ client begins the <acronym>SSL</acronym> negotiation immediately without
+ waiting for the server response to be received it can reduce connection
+ latency by one round-trip. However this comes at the cost of not being
+ able to handle the case where the server sends a negative response to the
+ <acronym>SSL</acronym> request. In that case instead of continuing with either GSSAPI or an
+ unencrypted connection or a protocol error the server will simply
+ disconnect.
+ </para>
+
<para>
An initial SSLRequest can also be used in a connection that is being
opened to send a CancelRequest message.
</para>
+ <para>
+ A second alternate way to initiate <acronym>SSL</acronym> encryption is
+ available. The server will recognize connections which immediately
+ begin <acronym>SSL</acronym> negotiation without any previous SSLRequest
+ packets. Once the <acronym>SSL</acronym> connection is established the
+ server will expect a normal startup-request packet and continue
+ negotiation over the encrypted channel. In this case any other requests
+ for encryption will be refused. This method is not preferred for general
+ purpose tools as it cannot negotiate the best connection encryption
+ available or handle unencrypted connections. However it is useful for
+ environments where both the server and client are controlled together.
+ In that case it avoids one round trip of latency and allows the use of
+ network tools that depend on standard <acronym>SSL</acronym> connections.
+ When using <acronym>SSL</acronym> connections in this style the client is
+ required to use the ALPN extension defined
+ by <ulink url="https://tools.ietf.org/html/rfc7301">RFC 7301</ulink> to
+ protect against protocol confusion attacks.
+ The <productname>PostgreSQL</productname> protocol is "TBD-pgsql" as
+ registered
+ at <ulink url="https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids">IANA
+ TLS ALPN Protocol IDs</ulink> registry.
+ </para>
+
<para>
While the protocol itself does not provide a way for the server to
force <acronym>SSL</acronym> encryption, the administrator can
configure the server to reject unencrypted sessions as a byproduct
of authentication checking.
</para>
+
</sect2>
<sect2 id="protocol-flow-gssapi">
--
2.40.0
v5-0004-alpn-support.patchtext/x-patch; charset=US-ASCII; name=v5-0004-alpn-support.patchDownload
From 9c3e0c5d3457322b9fa4aa618ffdeb88978e9109 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Mon, 20 Mar 2023 14:09:30 -0400
Subject: [PATCH v5 4/6] alpn support
---
src/backend/libpq/be-secure-openssl.c | 66 ++++++++++++++++++++++++
src/backend/libpq/be-secure.c | 3 ++
src/backend/postmaster/postmaster.c | 25 +++++++++
src/backend/utils/misc/guc_tables.c | 9 ++++
src/bin/psql/command.c | 7 ++-
src/include/libpq/libpq-be.h | 1 +
src/include/libpq/libpq.h | 1 +
src/include/libpq/pqcomm.h | 19 +++++++
src/interfaces/libpq/fe-connect.c | 5 ++
src/interfaces/libpq/fe-secure-openssl.c | 31 +++++++++++
src/interfaces/libpq/libpq-int.h | 1 +
11 files changed, 166 insertions(+), 2 deletions(-)
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
index 685aa2ed69..620ffafb0b 100644
--- a/src/backend/libpq/be-secure-openssl.c
+++ b/src/backend/libpq/be-secure-openssl.c
@@ -67,6 +67,12 @@ static int ssl_external_passwd_cb(char *buf, int size, int rwflag, void *userdat
static int dummy_ssl_passwd_cb(char *buf, int size, int rwflag, void *userdata);
static int verify_cb(int ok, X509_STORE_CTX *ctx);
static void info_cb(const SSL *ssl, int type, int args);
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata);
static bool initialize_dh(SSL_CTX *context, bool isServerStart);
static bool initialize_ecdh(SSL_CTX *context, bool isServerStart);
static const char *SSLerrmessage(unsigned long ecode);
@@ -432,6 +438,13 @@ be_tls_open_server(Port *port)
/* set up debugging/info callback */
SSL_CTX_set_info_callback(SSL_context, info_cb);
+ if (ssl_enable_alpn) {
+ elog(DEBUG2, "Enabling OpenSSL ALPN callback");
+ SSL_CTX_set_alpn_select_cb(SSL_context, alpn_cb, port);
+ } else {
+ elog(DEBUG2, "OpenSSL ALPN is disabled, not setting callback");
+ }
+
if (!(port->ssl = SSL_new(SSL_context)))
{
ereport(COMMERROR,
@@ -692,6 +705,12 @@ be_tls_close(Port *port)
pfree(port->peer_dn);
port->peer_dn = NULL;
}
+
+ if (port->ssl_alpn_protocol)
+ {
+ pfree(port->ssl_alpn_protocol);
+ port->ssl_alpn_protocol = NULL;
+ }
}
ssize_t
@@ -1250,6 +1269,53 @@ info_cb(const SSL *ssl, int type, int args)
}
}
+/* See pqcomm.h comments on OpenSSL implementation of ALPN (RFC 7301) */
+static unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
+
+/*
+ * Server callback for ALPN negotiation. We use use the standard "helper"
+ * function even though currently we only accept one value. We store the
+ * negotiated protocol in Port->ssl_alpn_protocol and rely on higher level
+ * logic (in postmaster.c) to decide what to do with that info.
+ */
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata)
+{
+ /* Why does OpenSSL provide a helper function that requires a nonconst
+ * vector when the callback is declared to take a const vector? What are
+ * we to do with that?
+ */
+ int retval;
+ Assert(userdata != NULL);
+ Assert(out != NULL);
+ Assert(outlen != NULL);
+ Assert(in != NULL);
+
+ retval = SSL_select_next_proto((unsigned char **)out, outlen,
+ alpn_protos, sizeof(alpn_protos),
+ in, inlen);
+ if (*out == NULL || *outlen > sizeof(alpn_protos) || outlen <= 0)
+ return SSL_TLSEXT_ERR_NOACK; /* can't happen */
+
+ if (retval == OPENSSL_NPN_NEGOTIATED)
+ {
+ struct Port *port = (struct Port *)userdata;
+ char *alpn_protocol = MemoryContextAllocZero(TopMemoryContext, *outlen+1);
+ memcpy(alpn_protocol, *out, *outlen);
+ port->ssl_alpn_protocol = alpn_protocol;
+ return SSL_TLSEXT_ERR_OK;
+ } else if (retval == OPENSSL_NPN_NO_OVERLAP) {
+ return SSL_TLSEXT_ERR_NOACK;
+ } else {
+ return SSL_TLSEXT_ERR_NOACK;
+ }
+}
+
+
/*
* Set DH parameters for generating ephemeral DH keys. The
* DH parameters can take a long time to compute, so they must be
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 1020b3adb0..79a61900ba 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -61,6 +61,9 @@ bool SSLPreferServerCiphers;
int ssl_min_protocol_version = PG_TLS1_2_VERSION;
int ssl_max_protocol_version = PG_TLS_ANY;
+/* GUC variable: if false ignore ALPN negotiation */
+bool ssl_enable_alpn = true;
+
/* ------------------------------------------------------------ */
/* Procedures common to all secure sessions */
/* ------------------------------------------------------------ */
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index ec1d895a23..9b4b37b997 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1934,6 +1934,10 @@ ServerLoop(void)
* any bytes from the stream if it's not a direct SSL connection.
*/
+#ifdef USE_SSL
+static const char *expected_alpn_protocol = PG_ALPN_PROTOCOL;
+#endif
+
static int
ProcessSSLStartup(Port *port)
{
@@ -1970,6 +1974,10 @@ ProcessSSLStartup(Port *port)
char *buf = NULL;
elog(LOG, "Detected direct SSL handshake");
+ if (!ssl_enable_alpn) {
+ elog(WARNING, "Received direct SSL connection without ssl_enable_alpn enabled");
+ }
+
/* push unencrypted buffered data back through SSL setup */
len = pq_buffer_has_data();
if (len > 0)
@@ -2000,6 +2008,23 @@ ProcessSSLStartup(Port *port)
return STATUS_ERROR;
}
pfree(port->raw_buf);
+
+ if (port->ssl_alpn_protocol == NULL)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request without required ALPN protocol negotiation extension")));
+ return STATUS_ERROR;
+ }
+ if (strcmp(port->ssl_alpn_protocol, expected_alpn_protocol) != 0)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with unexpected ALPN protocol \"%s\" expected \"%s\"",
+ port->ssl_alpn_protocol,
+ expected_alpn_protocol)));
+ return STATUS_ERROR;
+ }
#else
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 8062589efd..79a1d0a100 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1088,6 +1088,15 @@ struct config_bool ConfigureNamesBool[] =
true,
NULL, NULL, NULL
},
+ {
+ {"ssl_enable_alpn", PGC_SIGHUP, CONN_AUTH_SSL,
+ gettext_noop("Respond to TLS ALPN Extension Requests."),
+ NULL,
+ },
+ &ssl_enable_alpn,
+ true,
+ NULL, NULL, NULL
+ },
{
{"fsync", PGC_SIGHUP, WAL_SETTINGS,
gettext_noop("Forces synchronization of updates to disk."),
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index d7731234b6..816b336973 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -3715,6 +3715,7 @@ printSSLInfo(void)
const char *protocol;
const char *cipher;
const char *compression;
+ const char *alpn;
if (!PQsslInUse(pset.db))
return; /* no SSL */
@@ -3722,11 +3723,13 @@ printSSLInfo(void)
protocol = PQsslAttribute(pset.db, "protocol");
cipher = PQsslAttribute(pset.db, "cipher");
compression = PQsslAttribute(pset.db, "compression");
+ alpn = PQsslAttribute(pset.db, "alpn");
- printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s)\n"),
+ printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s, ALPN: %s)\n"),
protocol ? protocol : _("unknown"),
cipher ? cipher : _("unknown"),
- (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"));
+ (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"),
+ alpn ? alpn : _("none"));
}
/*
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 824b28e824..2258241770 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -217,6 +217,7 @@ typedef struct Port
char *peer_cn;
char *peer_dn;
bool peer_cert_valid;
+ char *ssl_alpn_protocol;
/*
* OpenSSL structures. (Keep these last so that the locations of other
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 2b02f67257..e7adf4a989 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -123,6 +123,7 @@ extern PGDLLIMPORT char *SSLECDHCurve;
extern PGDLLIMPORT bool SSLPreferServerCiphers;
extern PGDLLIMPORT int ssl_min_protocol_version;
extern PGDLLIMPORT int ssl_max_protocol_version;
+extern PGDLLIMPORT bool ssl_enable_alpn;
enum ssl_protocol_versions
{
diff --git a/src/include/libpq/pqcomm.h b/src/include/libpq/pqcomm.h
index c85090259d..d9fc02adfc 100644
--- a/src/include/libpq/pqcomm.h
+++ b/src/include/libpq/pqcomm.h
@@ -152,6 +152,25 @@ typedef struct CancelRequestPacket
uint32 cancelAuthCode; /* secret key to authorize cancel */
} CancelRequestPacket;
+/* Application-Layer Protocol Negotiation is required for direct connections
+ * to avoid protocol confusion attacks (e.g https://alpaca-attack.com/).
+ *
+ * ALPN is specified in RFC 7301
+ *
+ * This string should be registered at:
+ * https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids
+ *
+ * OpenSSL uses this wire-format for the list of alpn protocols even in the
+ * API. Both server and client take the same format parameter but the client
+ * actually sends it to the server as-is and the server it specifies the
+ * preference order to use to choose the one selected to send back.
+ *
+ * c.f. https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_alpn_select_cb.html
+ *
+ * The #define can be used to initialize a char[] vector to use directly in the API
+ */
+#define PG_ALPN_PROTOCOL "TBD-pgsql"
+#define PG_ALPN_PROTOCOL_VECTOR { 9, 'T','B','D','-','p','g','s','q','l' }
/*
* A client can also start by sending a SSL or GSSAPI negotiation request to
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 7cd0eb261f..3118c19edd 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -314,6 +314,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-SNI", "", 1,
offsetof(struct pg_conn, sslsni)},
+ {"sslalpn", "PGSSLALPN", "1", NULL,
+ "SSL-ALPN", "", 1,
+ offsetof(struct pg_conn, sslalpn)},
+
{"requirepeer", "PGREQUIREPEER", NULL, NULL,
"Require-Peer", "", 10,
offsetof(struct pg_conn, requirepeer)},
@@ -4487,6 +4491,7 @@ freePGconn(PGconn *conn)
free(conn->sslcrldir);
free(conn->sslcompression);
free(conn->sslsni);
+ free(conn->sslalpn);
free(conn->requirepeer);
free(conn->require_auth);
free(conn->ssl_min_protocol_version);
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 4d1e4009ef..02af7bf571 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -912,6 +912,9 @@ destroy_ssl_system(void)
#endif
}
+/* See pqcomm.h comments on OpenSSL implementation of ALPN (RFC 7301) */
+static unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
+
/*
* Create per-connection SSL object, and load the client certificate,
* private key, and trusted CA certs.
@@ -1240,6 +1243,20 @@ initialize_SSL(PGconn *conn)
}
}
+ if (conn->sslalpn && conn->sslalpn[0] == '1')
+ {
+ int retval;
+ retval = SSL_set_alpn_protos(conn->ssl, alpn_protos, sizeof(alpn_protos));
+
+ if (retval != 0)
+ {
+ char *err = SSLerrmessage(ERR_get_error());
+ libpq_append_conn_error(conn, "could not set ssl alpn extension: %s", err);
+ SSLerrfree(err);
+ return -1;
+ }
+ }
+
/*
* Read the SSL key. If a key is specified, treat it as an engine:key
* combination if there is colon present - we don't support files with
@@ -1723,6 +1740,7 @@ PQsslAttributeNames(PGconn *conn)
"cipher",
"compression",
"protocol",
+ "alpn",
NULL
};
static const char *const empty_attrs[] = {NULL};
@@ -1777,6 +1795,19 @@ PQsslAttribute(PGconn *conn, const char *attribute_name)
if (strcmp(attribute_name, "protocol") == 0)
return SSL_get_version(conn->ssl);
+ if (strcmp(attribute_name, "alpn") == 0)
+ {
+ const unsigned char *data;
+ unsigned int len;
+ static char alpn_str[256]; /* alpn doesn't support longer than 255 bytes */
+ SSL_get0_alpn_selected(conn->ssl, &data, &len);
+ if (data == NULL || len==0 || len > sizeof(alpn_str)-1)
+ return NULL;
+ memcpy(alpn_str, data, len);
+ alpn_str[len] = 0;
+ return alpn_str;
+ }
+
return NULL; /* unknown attribute */
}
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index db63afd786..17e5b2528f 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -400,6 +400,7 @@ struct pg_conn
char *sslcrl; /* certificate revocation list filename */
char *sslcrldir; /* certificate revocation list directory name */
char *sslsni; /* use SSL SNI extension (0 or 1) */
+ char *sslalpn; /* use SSL ALPN extension (0 or 1) */
char *requirepeer; /* required peer credentials for local sockets */
char *gssencmode; /* GSS mode (require,prefer,disable) */
char *krbsrvname; /* Kerberos service name */
--
2.40.0
v5-0005-Allow-pipelining-data-after-ssl-request.patchtext/x-patch; charset=US-ASCII; name=v5-0005-Allow-pipelining-data-after-ssl-request.patchDownload
From 07197e9aba2715e6d864a95d09b06c3486e4cbca Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Fri, 31 Mar 2023 02:31:31 -0400
Subject: [PATCH v5 5/6] Allow pipelining data after ssl request
---
src/backend/postmaster/postmaster.c | 54 ++++++++++++++++++++++-------
1 file changed, 42 insertions(+), 12 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 9b4b37b997..33b317f98b 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2001,13 +2001,14 @@ ProcessSSLStartup(Port *port)
if (port->raw_buf_remaining > 0)
{
- /* This shouldn't be possible -- it would mean the client sent
- * encrypted data before we established a session key...
- */
- elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after SSL request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
return STATUS_ERROR;
}
- pfree(port->raw_buf);
+ if (port->raw_buf)
+ pfree(port->raw_buf);
if (port->ssl_alpn_protocol == NULL)
{
@@ -2158,15 +2159,44 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
}
#ifdef USE_SSL
- if (SSLok == 'S' && secure_open_server(port) == -1)
- return STATUS_ERROR;
+ if (SSLok == 'S')
+ {
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ buf = palloc(len);
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+ Assert(pq_buffer_has_data() == 0);
+
+ if (secure_open_server(port) == -1)
+ return STATUS_ERROR;
+
+ /*
+ * At this point we should have no data already buffered. If we do,
+ * it was received before we performed the SSL handshake, so it wasn't
+ * encrypted and indeed may have been injected by a man-in-the-middle.
+ * We report this case to the client.
+ */
+ if (port->raw_buf_remaining > 0)
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after SSL request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
+ if (port->raw_buf)
+ pfree(port->raw_buf);
+ }
#endif
- /*
- * At this point we should have no data already buffered. If we do,
- * it was received before we performed the SSL handshake, so it wasn't
- * encrypted and indeed may have been injected by a man-in-the-middle.
- * We report this case to the client.
+ /* This can only really occur now if there was data pipelined after
+ * the SSL Request but we have refused to do SSL. In that case we need
+ * to give up because the client has presumably assumed the SSL
+ * request would be accepted.
*/
if (pq_buffer_has_data())
ereport(FATAL,
--
2.40.0
v5-0003-Direct-SSL-connections-documentation.patchtext/x-patch; charset=US-ASCII; name=v5-0003-Direct-SSL-connections-documentation.patchDownload
From 8d0abefa640fab9d9be16ae52fe65b86b272a537 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 15:10:15 -0400
Subject: [PATCH v5 3/6] Direct SSL connections documentation
---
doc/src/sgml/libpq.sgml | 102 ++++++++++++++++++++++++++++++++++++----
1 file changed, 93 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 9f72dd29d8..6efc70f801 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1691,10 +1691,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
Note that if <acronym>GSSAPI</acronym> encryption is possible,
that will be used in preference to <acronym>SSL</acronym>
encryption, regardless of the value of <literal>sslmode</literal>.
- To force use of <acronym>SSL</acronym> encryption in an
- environment that has working <acronym>GSSAPI</acronym>
- infrastructure (such as a Kerberos server), also
- set <literal>gssencmode</literal> to <literal>disable</literal>.
+ To negotiate <acronym>SSL</acronym> encryption in an environment that
+ has working <acronym>GSSAPI</acronym> infrastructure (such as a
+ Kerberos server), also set <literal>gssencmode</literal>
+ to <literal>disable</literal>.
+ Use of non-default values of <literal>sslnegotiation</literal> can
+ also cause <acronym>SSL</acronym> to be used instead of
+ negotiating <acronym>GSSAPI</acronym> encryption.
</para>
</listitem>
</varlistentry>
@@ -1721,6 +1724,75 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
</listitem>
</varlistentry>
+ <varlistentry id="libpq-connect-sslnegotiation" xreflabel="sslnegotiation">
+ <term><literal>sslnegotiation</literal></term>
+ <listitem>
+ <para>
+ This option controls whether <productname>PostgreSQL</productname>
+ will perform its protocol negotiation to request encryption from the
+ server or will just directly make a standard <acronym>SSL</acronym>
+ connection. Traditional <productname>PostgreSQL</productname>
+ protocol negotiation is the default and the most flexible with
+ different server configurations. If the server is known to support
+ direct <acronym>SSL</acronym> connections then the latter requires one
+ fewer round trip reducing connection latency and also allows the use
+ of protocol agnostic SSL network tools.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>postgres</literal></term>
+ <listitem>
+ <para>
+ perform <productname>PostgreSQL</productname> protocol
+ negotiation. This is the default if the option is not provided.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>direct</literal></term>
+ <listitem>
+ <para>
+ first attempt to establish a standard SSL connection and if that
+ fails reconnect and perform the negotiation. This fallback
+ process adds significant latency if the initial SSL connection
+ fails.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>requiredirect</literal></term>
+ <listitem>
+ <para>
+ attempt to establish a standard SSL connection and if that fails
+ return a connection failure immediately.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ If <literal>sslmode</literal> set to <literal>disable</literal>
+ or <literal>allow</literal> then <literal>sslnegotiation</literal> is
+ ignored. If <literal>gssencmode</literal> is set
+ to <literal>require</literal> then <literal>sslnegotiation</literal>
+ must be the default <literal>postgres</literal> value.
+ </para>
+
+ <para>
+ Moreover, note if <literal>gssencmode</literal> is set
+ to <literal>prefer</literal> and <literal>sslnegotiation</literal>
+ to <literal>direct</literal> then the effective preference will be
+ direct <acronym>SSL</acronym> connections, followed by
+ negotiated <acronym>GSS</acronym> connections, followed by
+ negotiated <acronym>SSL</acronym> connections, possibly followed
+ lastly by unencrypted connections.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-connect-sslcompression" xreflabel="sslcompression">
<term><literal>sslcompression</literal></term>
<listitem>
@@ -1930,11 +2002,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
<para>
The Server Name Indication can be used by SSL-aware proxies to route
- connections without having to decrypt the SSL stream. (Note that this
- requires a proxy that is aware of the PostgreSQL protocol handshake,
- not just any SSL proxy.) However, <acronym>SNI</acronym> makes the
- destination host name appear in cleartext in the network traffic, so
- it might be undesirable in some cases.
+ connections without having to decrypt the SSL stream. (Note that
+ unless the proxy is aware of the PostgreSQL protocol handshake this
+ would require setting <literal>sslnegotiation</literal>
+ to <literal>direct</literal> or <literal>requiredirect</literal>.)
+ However, <acronym>SNI</acronym> makes the destination host name appear
+ in cleartext in the network traffic, so it might be undesirable in
+ some cases.
</para>
</listitem>
</varlistentry>
@@ -8073,6 +8147,16 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
</para>
</listitem>
+ <listitem>
+ <para>
+ <indexterm>
+ <primary><envar>PGSSLNEGOTIATION</envar></primary>
+ </indexterm>
+ <envar>PGSSLNEGOTIATION</envar> behaves the same as the <xref
+ linkend="libpq-connect-sslnegotiation"/> connection parameter.
+ </para>
+ </listitem>
+
<listitem>
<para>
<indexterm>
--
2.40.0
v5-0001-Direct-SSL-connections-postmaster-support.patchtext/x-patch; charset=US-ASCII; name=v5-0001-Direct-SSL-connections-postmaster-support.patchDownload
From 6829925a144ac2acc605bbdebed7653b333e7e04 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Wed, 15 Mar 2023 15:51:02 -0400
Subject: [PATCH v5 1/6] Direct SSL connections postmaster support
---
src/backend/libpq/be-secure.c | 13 +++
src/backend/libpq/pqcomm.c | 9 +-
src/backend/postmaster/postmaster.c | 133 ++++++++++++++++++++++------
src/include/libpq/libpq-be.h | 10 +++
src/include/libpq/libpq.h | 2 +-
5 files changed, 134 insertions(+), 33 deletions(-)
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index a0f7084018..1020b3adb0 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -235,6 +235,19 @@ secure_raw_read(Port *port, void *ptr, size_t len)
{
ssize_t n;
+ /* Read from the "unread" buffered data first. c.f. libpq-be.h */
+ if (port->raw_buf_remaining > 0)
+ {
+ /* consume up to len bytes from the raw_buf */
+ if (len > port->raw_buf_remaining)
+ len = port->raw_buf_remaining;
+ Assert(port->raw_buf);
+ memcpy(ptr, port->raw_buf + port->raw_buf_consumed, len);
+ port->raw_buf_consumed += len;
+ port->raw_buf_remaining -= len;
+ return len;
+ }
+
/*
* Try to read from the socket without blocking. If it succeeds we're
* done, otherwise we'll wait for the socket using the latch mechanism.
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index da5bb5fc5d..17d025bb8e 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -1126,13 +1126,16 @@ pq_discardbytes(size_t len)
/* --------------------------------
* pq_buffer_has_data - is any buffered data available to read?
*
- * This will *not* attempt to read more data.
+ * Actually returns the number of bytes in the buffer...
+ *
+ * This will *not* attempt to read more data. And reading up to that number of
+ * bytes should not cause reading any more data either.
* --------------------------------
*/
-bool
+size_t
pq_buffer_has_data(void)
{
- return (PqRecvPointer < PqRecvLength);
+ return (PqRecvLength - PqRecvPointer);
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 4c49393fc5..ec1d895a23 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -422,6 +422,7 @@ static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(Port *port);
+static int ProcessSSLStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -1926,6 +1927,97 @@ ServerLoop(void)
}
}
+/*
+ * Check for a native direct SSL connection.
+ *
+ * This happens before startup packets so we are careful not to actual read
+ * any bytes from the stream if it's not a direct SSL connection.
+ */
+
+static int
+ProcessSSLStartup(Port *port)
+{
+ int firstbyte;
+
+ pq_startmsgread();
+
+ firstbyte = pq_peekbyte();
+ if (firstbyte == EOF)
+ {
+ /*
+ * If we get no data at all, don't clutter the log with a complaint;
+ * such cases often occur for legitimate reasons. An example is that
+ * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
+ * client didn't like our response, it'll probably just drop the
+ * connection. Service-monitoring software also often just opens and
+ * closes a connection without sending anything. (So do port
+ * scanners, which may be less benign, but it's not really our job to
+ * notice those.)
+ */
+ return STATUS_ERROR;
+ }
+
+ /*
+ * First byte indicates standard SSL handshake message
+ *
+ * (It can't be a Postgres startup length because in network byte order
+ * that would be a startup packet hundreds of megabytes long)
+ */
+ if (firstbyte == 0x16)
+ {
+#ifdef USE_SSL
+ ssize_t len;
+ char *buf = NULL;
+ elog(LOG, "Detected direct SSL handshake");
+
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ buf = palloc(len);
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+
+ Assert(pq_buffer_has_data() == 0);
+ if (secure_open_server(port) == -1)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("SSL Protocol Error during direct SSL connection initiation")));
+ return STATUS_ERROR;
+ }
+
+ if (port->raw_buf_remaining > 0)
+ {
+ /* This shouldn't be possible -- it would mean the client sent
+ * encrypted data before we established a session key...
+ */
+ elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ return STATUS_ERROR;
+ }
+ pfree(port->raw_buf);
+#else
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with no SSL support")));
+ return STATUS_ERROR;
+#endif
+ }
+
+ pq_endmsgread();
+
+ if (port->ssl_in_use)
+ ereport(DEBUG2,
+ (errmsg_internal("Direct SSL connection established")));
+
+ return STATUS_OK;
+}
+
+
/*
* Read a client's startup packet and do something according to it.
*
@@ -1954,28 +2046,7 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
pq_startmsgread();
- /*
- * Grab the first byte of the length word separately, so that we can tell
- * whether we have no data at all or an incomplete packet. (This might
- * sound inefficient, but it's not really, because of buffering in
- * pqcomm.c.)
- */
- if (pq_getbytes((char *) &len, 1) == EOF)
- {
- /*
- * If we get no data at all, don't clutter the log with a complaint;
- * such cases often occur for legitimate reasons. An example is that
- * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
- * client didn't like our response, it'll probably just drop the
- * connection. Service-monitoring software also often just opens and
- * closes a connection without sending anything. (So do port
- * scanners, which may be less benign, but it's not really our job to
- * notice those.)
- */
- return STATUS_ERROR;
- }
-
- if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
+ if (pq_getbytes(((char *) &len), 4) == EOF)
{
/* Got a partial length word, so bleat about that */
if (!ssl_done && !gss_done)
@@ -2039,8 +2110,11 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
char SSLok;
#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ /* No SSL when disabled or on Unix sockets.
+ *
+ * Also no SSL negotiation if we already have a direct SSL connection
+ */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX || port->ssl_in_use)
SSLok = 'N';
else
SSLok = 'S'; /* Support for SSL */
@@ -2048,11 +2122,10 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
SSLok = 'N'; /* No support for SSL */
#endif
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
+ while (secure_write(port, &SSLok, 1) != 1)
{
if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
+ continue; /* if interrupted, just retry */
ereport(COMMERROR,
(errcode_for_socket_access(),
errmsg("failed to send SSL negotiation response: %m")));
@@ -2093,7 +2166,7 @@ retry1:
GSSok = 'G';
#endif
- while (send(port->sock, &GSSok, 1, 0) != 1)
+ while (secure_write(port, &GSSok, 1) != 1)
{
if (errno == EINTR)
continue;
@@ -4396,7 +4469,9 @@ BackendInitialize(Port *port)
* Receive the startup packet (which might turn out to be a cancel request
* packet).
*/
- status = ProcessStartupPacket(port, false, false);
+ status = ProcessSSLStartup(port);
+ if (status == STATUS_OK)
+ status = ProcessStartupPacket(port, false, false);
/*
* Disable the timeout, and prevent SIGTERM again.
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index ac6407e9f6..824b28e824 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -226,6 +226,16 @@ typedef struct Port
SSL *ssl;
X509 *peer;
#endif
+ /* This is a bit of a hack. The raw_buf is data that was previously read
+ * and buffered in a higher layer but then "unread" and needs to be read
+ * again while establishing an SSL connection via the SSL library layer.
+ *
+ * There's no API to "unread", the upper layer just places the data in the
+ * Port structure in raw_buf and sets raw_buf_remaining to the amount of
+ * bytes unread and raw_buf_consumed to 0.
+ */
+ char *raw_buf;
+ ssize_t raw_buf_consumed, raw_buf_remaining;
} Port;
#ifdef USE_SSL
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 50fc781f47..2b02f67257 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -80,7 +80,7 @@ extern int pq_getmessage(StringInfo s, int maxlen);
extern int pq_getbyte(void);
extern int pq_peekbyte(void);
extern int pq_getbyte_if_available(unsigned char *c);
-extern bool pq_buffer_has_data(void);
+extern size_t pq_buffer_has_data(void);
extern int pq_putmessage_v2(char msgtype, const char *s, size_t len);
extern bool pq_check_connection(void);
--
2.40.0
And the cfbot wants a small tweak
Attachments:
v6-0005-Allow-pipelining-data-after-ssl-request.patchtext/x-patch; charset=US-ASCII; name=v6-0005-Allow-pipelining-data-after-ssl-request.patchDownload
From 3d0a502c25504da32b7a362831c700b4e891f79b Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Fri, 31 Mar 2023 02:31:31 -0400
Subject: [PATCH v6 5/6] Allow pipelining data after ssl request
---
src/backend/postmaster/postmaster.c | 54 ++++++++++++++++++++++-------
1 file changed, 42 insertions(+), 12 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 9b4b37b997..33b317f98b 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2001,13 +2001,14 @@ ProcessSSLStartup(Port *port)
if (port->raw_buf_remaining > 0)
{
- /* This shouldn't be possible -- it would mean the client sent
- * encrypted data before we established a session key...
- */
- elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after SSL request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
return STATUS_ERROR;
}
- pfree(port->raw_buf);
+ if (port->raw_buf)
+ pfree(port->raw_buf);
if (port->ssl_alpn_protocol == NULL)
{
@@ -2158,15 +2159,44 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
}
#ifdef USE_SSL
- if (SSLok == 'S' && secure_open_server(port) == -1)
- return STATUS_ERROR;
+ if (SSLok == 'S')
+ {
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ buf = palloc(len);
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+ Assert(pq_buffer_has_data() == 0);
+
+ if (secure_open_server(port) == -1)
+ return STATUS_ERROR;
+
+ /*
+ * At this point we should have no data already buffered. If we do,
+ * it was received before we performed the SSL handshake, so it wasn't
+ * encrypted and indeed may have been injected by a man-in-the-middle.
+ * We report this case to the client.
+ */
+ if (port->raw_buf_remaining > 0)
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after SSL request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
+ if (port->raw_buf)
+ pfree(port->raw_buf);
+ }
#endif
- /*
- * At this point we should have no data already buffered. If we do,
- * it was received before we performed the SSL handshake, so it wasn't
- * encrypted and indeed may have been injected by a man-in-the-middle.
- * We report this case to the client.
+ /* This can only really occur now if there was data pipelined after
+ * the SSL Request but we have refused to do SSL. In that case we need
+ * to give up because the client has presumably assumed the SSL
+ * request would be accepted.
*/
if (pq_buffer_has_data())
ereport(FATAL,
--
2.40.0
v6-0004-Direct-SSL-connections-ALPN-support.patchtext/x-patch; charset=US-ASCII; name=v6-0004-Direct-SSL-connections-ALPN-support.patchDownload
From 5413f1a1ee897640bd3bb99bae226eec7e2e9f50 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Mon, 20 Mar 2023 14:09:30 -0400
Subject: [PATCH v6 4/6] Direct SSL connections ALPN support
---
src/backend/libpq/be-secure-openssl.c | 66 +++++++++++++++++++
src/backend/libpq/be-secure.c | 3 +
src/backend/postmaster/postmaster.c | 25 +++++++
src/backend/utils/misc/guc_tables.c | 9 +++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/bin/psql/command.c | 7 +-
src/include/libpq/libpq-be.h | 1 +
src/include/libpq/libpq.h | 1 +
src/include/libpq/pqcomm.h | 19 ++++++
src/interfaces/libpq/fe-connect.c | 5 ++
src/interfaces/libpq/fe-secure-openssl.c | 31 +++++++++
src/interfaces/libpq/libpq-int.h | 1 +
12 files changed, 167 insertions(+), 2 deletions(-)
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
index 685aa2ed69..620ffafb0b 100644
--- a/src/backend/libpq/be-secure-openssl.c
+++ b/src/backend/libpq/be-secure-openssl.c
@@ -67,6 +67,12 @@ static int ssl_external_passwd_cb(char *buf, int size, int rwflag, void *userdat
static int dummy_ssl_passwd_cb(char *buf, int size, int rwflag, void *userdata);
static int verify_cb(int ok, X509_STORE_CTX *ctx);
static void info_cb(const SSL *ssl, int type, int args);
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata);
static bool initialize_dh(SSL_CTX *context, bool isServerStart);
static bool initialize_ecdh(SSL_CTX *context, bool isServerStart);
static const char *SSLerrmessage(unsigned long ecode);
@@ -432,6 +438,13 @@ be_tls_open_server(Port *port)
/* set up debugging/info callback */
SSL_CTX_set_info_callback(SSL_context, info_cb);
+ if (ssl_enable_alpn) {
+ elog(DEBUG2, "Enabling OpenSSL ALPN callback");
+ SSL_CTX_set_alpn_select_cb(SSL_context, alpn_cb, port);
+ } else {
+ elog(DEBUG2, "OpenSSL ALPN is disabled, not setting callback");
+ }
+
if (!(port->ssl = SSL_new(SSL_context)))
{
ereport(COMMERROR,
@@ -692,6 +705,12 @@ be_tls_close(Port *port)
pfree(port->peer_dn);
port->peer_dn = NULL;
}
+
+ if (port->ssl_alpn_protocol)
+ {
+ pfree(port->ssl_alpn_protocol);
+ port->ssl_alpn_protocol = NULL;
+ }
}
ssize_t
@@ -1250,6 +1269,53 @@ info_cb(const SSL *ssl, int type, int args)
}
}
+/* See pqcomm.h comments on OpenSSL implementation of ALPN (RFC 7301) */
+static unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
+
+/*
+ * Server callback for ALPN negotiation. We use use the standard "helper"
+ * function even though currently we only accept one value. We store the
+ * negotiated protocol in Port->ssl_alpn_protocol and rely on higher level
+ * logic (in postmaster.c) to decide what to do with that info.
+ */
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata)
+{
+ /* Why does OpenSSL provide a helper function that requires a nonconst
+ * vector when the callback is declared to take a const vector? What are
+ * we to do with that?
+ */
+ int retval;
+ Assert(userdata != NULL);
+ Assert(out != NULL);
+ Assert(outlen != NULL);
+ Assert(in != NULL);
+
+ retval = SSL_select_next_proto((unsigned char **)out, outlen,
+ alpn_protos, sizeof(alpn_protos),
+ in, inlen);
+ if (*out == NULL || *outlen > sizeof(alpn_protos) || outlen <= 0)
+ return SSL_TLSEXT_ERR_NOACK; /* can't happen */
+
+ if (retval == OPENSSL_NPN_NEGOTIATED)
+ {
+ struct Port *port = (struct Port *)userdata;
+ char *alpn_protocol = MemoryContextAllocZero(TopMemoryContext, *outlen+1);
+ memcpy(alpn_protocol, *out, *outlen);
+ port->ssl_alpn_protocol = alpn_protocol;
+ return SSL_TLSEXT_ERR_OK;
+ } else if (retval == OPENSSL_NPN_NO_OVERLAP) {
+ return SSL_TLSEXT_ERR_NOACK;
+ } else {
+ return SSL_TLSEXT_ERR_NOACK;
+ }
+}
+
+
/*
* Set DH parameters for generating ephemeral DH keys. The
* DH parameters can take a long time to compute, so they must be
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 1020b3adb0..79a61900ba 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -61,6 +61,9 @@ bool SSLPreferServerCiphers;
int ssl_min_protocol_version = PG_TLS1_2_VERSION;
int ssl_max_protocol_version = PG_TLS_ANY;
+/* GUC variable: if false ignore ALPN negotiation */
+bool ssl_enable_alpn = true;
+
/* ------------------------------------------------------------ */
/* Procedures common to all secure sessions */
/* ------------------------------------------------------------ */
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index ec1d895a23..9b4b37b997 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1934,6 +1934,10 @@ ServerLoop(void)
* any bytes from the stream if it's not a direct SSL connection.
*/
+#ifdef USE_SSL
+static const char *expected_alpn_protocol = PG_ALPN_PROTOCOL;
+#endif
+
static int
ProcessSSLStartup(Port *port)
{
@@ -1970,6 +1974,10 @@ ProcessSSLStartup(Port *port)
char *buf = NULL;
elog(LOG, "Detected direct SSL handshake");
+ if (!ssl_enable_alpn) {
+ elog(WARNING, "Received direct SSL connection without ssl_enable_alpn enabled");
+ }
+
/* push unencrypted buffered data back through SSL setup */
len = pq_buffer_has_data();
if (len > 0)
@@ -2000,6 +2008,23 @@ ProcessSSLStartup(Port *port)
return STATUS_ERROR;
}
pfree(port->raw_buf);
+
+ if (port->ssl_alpn_protocol == NULL)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request without required ALPN protocol negotiation extension")));
+ return STATUS_ERROR;
+ }
+ if (strcmp(port->ssl_alpn_protocol, expected_alpn_protocol) != 0)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with unexpected ALPN protocol \"%s\" expected \"%s\"",
+ port->ssl_alpn_protocol,
+ expected_alpn_protocol)));
+ return STATUS_ERROR;
+ }
#else
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 8062589efd..79a1d0a100 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1088,6 +1088,15 @@ struct config_bool ConfigureNamesBool[] =
true,
NULL, NULL, NULL
},
+ {
+ {"ssl_enable_alpn", PGC_SIGHUP, CONN_AUTH_SSL,
+ gettext_noop("Respond to TLS ALPN Extension Requests."),
+ NULL,
+ },
+ &ssl_enable_alpn,
+ true,
+ NULL, NULL, NULL
+ },
{
{"fsync", PGC_SIGHUP, WAL_SETTINGS,
gettext_noop("Forces synchronization of updates to disk."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ee49ca3937..f93bcc23d0 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -118,6 +118,7 @@
#ssl_dh_params_file = ''
#ssl_passphrase_command = ''
#ssl_passphrase_command_supports_reload = off
+#ssl_enable_alpn = on
#------------------------------------------------------------------------------
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index d7731234b6..816b336973 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -3715,6 +3715,7 @@ printSSLInfo(void)
const char *protocol;
const char *cipher;
const char *compression;
+ const char *alpn;
if (!PQsslInUse(pset.db))
return; /* no SSL */
@@ -3722,11 +3723,13 @@ printSSLInfo(void)
protocol = PQsslAttribute(pset.db, "protocol");
cipher = PQsslAttribute(pset.db, "cipher");
compression = PQsslAttribute(pset.db, "compression");
+ alpn = PQsslAttribute(pset.db, "alpn");
- printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s)\n"),
+ printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s, ALPN: %s)\n"),
protocol ? protocol : _("unknown"),
cipher ? cipher : _("unknown"),
- (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"));
+ (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"),
+ alpn ? alpn : _("none"));
}
/*
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 824b28e824..2258241770 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -217,6 +217,7 @@ typedef struct Port
char *peer_cn;
char *peer_dn;
bool peer_cert_valid;
+ char *ssl_alpn_protocol;
/*
* OpenSSL structures. (Keep these last so that the locations of other
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 2b02f67257..e7adf4a989 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -123,6 +123,7 @@ extern PGDLLIMPORT char *SSLECDHCurve;
extern PGDLLIMPORT bool SSLPreferServerCiphers;
extern PGDLLIMPORT int ssl_min_protocol_version;
extern PGDLLIMPORT int ssl_max_protocol_version;
+extern PGDLLIMPORT bool ssl_enable_alpn;
enum ssl_protocol_versions
{
diff --git a/src/include/libpq/pqcomm.h b/src/include/libpq/pqcomm.h
index c85090259d..d9fc02adfc 100644
--- a/src/include/libpq/pqcomm.h
+++ b/src/include/libpq/pqcomm.h
@@ -152,6 +152,25 @@ typedef struct CancelRequestPacket
uint32 cancelAuthCode; /* secret key to authorize cancel */
} CancelRequestPacket;
+/* Application-Layer Protocol Negotiation is required for direct connections
+ * to avoid protocol confusion attacks (e.g https://alpaca-attack.com/).
+ *
+ * ALPN is specified in RFC 7301
+ *
+ * This string should be registered at:
+ * https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids
+ *
+ * OpenSSL uses this wire-format for the list of alpn protocols even in the
+ * API. Both server and client take the same format parameter but the client
+ * actually sends it to the server as-is and the server it specifies the
+ * preference order to use to choose the one selected to send back.
+ *
+ * c.f. https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_alpn_select_cb.html
+ *
+ * The #define can be used to initialize a char[] vector to use directly in the API
+ */
+#define PG_ALPN_PROTOCOL "TBD-pgsql"
+#define PG_ALPN_PROTOCOL_VECTOR { 9, 'T','B','D','-','p','g','s','q','l' }
/*
* A client can also start by sending a SSL or GSSAPI negotiation request to
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 7cd0eb261f..3118c19edd 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -314,6 +314,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-SNI", "", 1,
offsetof(struct pg_conn, sslsni)},
+ {"sslalpn", "PGSSLALPN", "1", NULL,
+ "SSL-ALPN", "", 1,
+ offsetof(struct pg_conn, sslalpn)},
+
{"requirepeer", "PGREQUIREPEER", NULL, NULL,
"Require-Peer", "", 10,
offsetof(struct pg_conn, requirepeer)},
@@ -4487,6 +4491,7 @@ freePGconn(PGconn *conn)
free(conn->sslcrldir);
free(conn->sslcompression);
free(conn->sslsni);
+ free(conn->sslalpn);
free(conn->requirepeer);
free(conn->require_auth);
free(conn->ssl_min_protocol_version);
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 4d1e4009ef..02af7bf571 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -912,6 +912,9 @@ destroy_ssl_system(void)
#endif
}
+/* See pqcomm.h comments on OpenSSL implementation of ALPN (RFC 7301) */
+static unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
+
/*
* Create per-connection SSL object, and load the client certificate,
* private key, and trusted CA certs.
@@ -1240,6 +1243,20 @@ initialize_SSL(PGconn *conn)
}
}
+ if (conn->sslalpn && conn->sslalpn[0] == '1')
+ {
+ int retval;
+ retval = SSL_set_alpn_protos(conn->ssl, alpn_protos, sizeof(alpn_protos));
+
+ if (retval != 0)
+ {
+ char *err = SSLerrmessage(ERR_get_error());
+ libpq_append_conn_error(conn, "could not set ssl alpn extension: %s", err);
+ SSLerrfree(err);
+ return -1;
+ }
+ }
+
/*
* Read the SSL key. If a key is specified, treat it as an engine:key
* combination if there is colon present - we don't support files with
@@ -1723,6 +1740,7 @@ PQsslAttributeNames(PGconn *conn)
"cipher",
"compression",
"protocol",
+ "alpn",
NULL
};
static const char *const empty_attrs[] = {NULL};
@@ -1777,6 +1795,19 @@ PQsslAttribute(PGconn *conn, const char *attribute_name)
if (strcmp(attribute_name, "protocol") == 0)
return SSL_get_version(conn->ssl);
+ if (strcmp(attribute_name, "alpn") == 0)
+ {
+ const unsigned char *data;
+ unsigned int len;
+ static char alpn_str[256]; /* alpn doesn't support longer than 255 bytes */
+ SSL_get0_alpn_selected(conn->ssl, &data, &len);
+ if (data == NULL || len==0 || len > sizeof(alpn_str)-1)
+ return NULL;
+ memcpy(alpn_str, data, len);
+ alpn_str[len] = 0;
+ return alpn_str;
+ }
+
return NULL; /* unknown attribute */
}
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index db63afd786..17e5b2528f 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -400,6 +400,7 @@ struct pg_conn
char *sslcrl; /* certificate revocation list filename */
char *sslcrldir; /* certificate revocation list directory name */
char *sslsni; /* use SSL SNI extension (0 or 1) */
+ char *sslalpn; /* use SSL ALPN extension (0 or 1) */
char *requirepeer; /* required peer credentials for local sockets */
char *gssencmode; /* GSS mode (require,prefer,disable) */
char *krbsrvname; /* Kerberos service name */
--
2.40.0
v6-0002-Direct-SSL-connections-client-support.patchtext/x-patch; charset=US-ASCII; name=v6-0002-Direct-SSL-connections-client-support.patchDownload
From 083df15eff52f025064e2879a404270e405f7dde Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 11:55:16 -0400
Subject: [PATCH v6 2/6] Direct SSL connections client support
---
src/interfaces/libpq/fe-connect.c | 91 +++++++++++++++++++++++++++++--
src/interfaces/libpq/libpq-fe.h | 1 +
src/interfaces/libpq/libpq-int.h | 3 +
3 files changed, 89 insertions(+), 6 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index bb7347cb0c..7cd0eb261f 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -274,6 +274,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-Mode", "", 12, /* sizeof("verify-full") == 12 */
offsetof(struct pg_conn, sslmode)},
+ {"sslnegotiation", "PGSSLNEGOTIATION", "postgres", NULL,
+ "SSL-Negotiation", "", 14, /* strlen("requiredirect") == 14 */
+ offsetof(struct pg_conn, sslnegotiation)},
+
{"sslcompression", "PGSSLCOMPRESSION", "0", NULL,
"SSL-Compression", "", 1,
offsetof(struct pg_conn, sslcompression)},
@@ -1504,11 +1508,36 @@ connectOptions2(PGconn *conn)
}
#endif
}
+
+ /*
+ * validate sslnegotiation option, default is "postgres" for the postgres
+ * style negotiated connection with an extra round trip but more options.
+ */
+ if (conn->sslnegotiation)
+ {
+ if (strcmp(conn->sslnegotiation, "postgres") != 0
+ && strcmp(conn->sslnegotiation, "direct") != 0
+ && strcmp(conn->sslnegotiation, "requiredirect") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
+ "sslnegotiation", conn->sslnegotiation);
+ return false;
+ }
+
+#ifndef USE_SSL
+ if (conn->sslnegotiation[0] != 'p') {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "sslnegotiation value \"%s\" invalid when SSL support is not compiled in",
+ conn->sslnegotiation);
+ return false;
+ }
+#endif
+ }
else
{
- conn->sslmode = strdup(DefaultSSLMode);
- if (!conn->sslmode)
- goto oom_error;
+ libpq_append_conn_error(conn, "sslnegotiation missing?");
+ return false;
}
/*
@@ -1614,6 +1643,18 @@ connectOptions2(PGconn *conn)
conn->gssencmode);
return false;
}
+#endif
+#ifdef USE_SSL
+ /* GSS is incompatible with direct SSL connections so it requires the
+ * default postgres style connection ssl negotiation */
+ if (strcmp(conn->gssencmode, "require") == 0 &&
+ strcmp(conn->sslnegotiation, "postgres") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when Direct SSL negotiation is enabled",
+ conn->gssencmode);
+ return false;
+ }
#endif
}
else
@@ -2747,11 +2788,12 @@ keep_going: /* We will come back to here until there is
/* initialize these values based on SSL mode */
conn->allow_ssl_try = (conn->sslmode[0] != 'd'); /* "disable" */
conn->wait_ssl_try = (conn->sslmode[0] == 'a'); /* "allow" */
+ /* direct ssl is incompatible with "allow" or "disabled" ssl */
+ conn->allow_direct_ssl_try = conn->allow_ssl_try && !conn->wait_ssl_try && (conn->sslnegotiation[0] != 'p');
#endif
#ifdef ENABLE_GSS
conn->try_gss = (conn->gssencmode[0] != 'd'); /* "disable" */
#endif
-
reset_connection_state_machine = false;
need_new_connection = true;
}
@@ -3209,6 +3251,28 @@ keep_going: /* We will come back to here until there is
if (pqsecure_initialize(conn, false, true) < 0)
goto error_return;
+ /* If SSL is enabled and direct SSL connections are enabled
+ * and we haven't already established an SSL connection (or
+ * already tried a direct connection and failed or succeeded)
+ * then try just enabling SSL directly.
+ *
+ * If we fail then we'll either fail the connection (if
+ * sslnegotiation is set to requiredirect or turn
+ * allow_direct_ssl_try to false
+ */
+ if (conn->allow_ssl_try
+ && !conn->wait_ssl_try
+ && conn->allow_direct_ssl_try
+ && !conn->ssl_in_use
+#ifdef ENABLE_GSS
+ && !conn->gssenc
+#endif
+ )
+ {
+ conn->status = CONNECTION_SSL_STARTUP;
+ return PGRES_POLLING_WRITING;
+ }
+
/*
* If SSL is enabled and we haven't already got encryption of
* some sort running, request SSL instead of sending the
@@ -3285,9 +3349,11 @@ keep_going: /* We will come back to here until there is
/*
* On first time through, get the postmaster's response to our
- * SSL negotiation packet.
+ * SSL negotiation packet. If we are trying a direct ssl
+ * connection skip reading the negotiation packet and go
+ * straight to initiating an ssl connection.
*/
- if (!conn->ssl_in_use)
+ if (!conn->ssl_in_use && !conn->allow_direct_ssl_try)
{
/*
* We use pqReadData here since it has the logic to
@@ -3393,6 +3459,18 @@ keep_going: /* We will come back to here until there is
}
if (pollres == PGRES_POLLING_FAILED)
{
+ /* Failed direct ssl connection, possibly try a new connection with postgres negotiation */
+ if (conn->allow_direct_ssl_try)
+ {
+ /* if it's requiredirect then it's a hard failure */
+ if (conn->sslnegotiation[0] == 'r')
+ goto error_return;
+ /* otherwise only retry using postgres connection */
+ conn->allow_direct_ssl_try = false;
+ need_new_connection = true;
+ goto keep_going;
+ }
+
/*
* Failed ... if sslmode is "prefer" then do a non-SSL
* retry
@@ -4395,6 +4473,7 @@ freePGconn(PGconn *conn)
free(conn->keepalives_interval);
free(conn->keepalives_count);
free(conn->sslmode);
+ free(conn->sslnegotiation);
free(conn->sslcert);
free(conn->sslkey);
if (conn->sslpassword)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index f3d9220496..05821b8473 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -72,6 +72,7 @@ typedef enum
CONNECTION_AUTH_OK, /* Received authentication; waiting for
* backend startup. */
CONNECTION_SETENV, /* This state is no longer used. */
+ CONNECTION_DIRECT_SSL_STARTUP, /* Starting SSL without PG Negotiation. */
CONNECTION_SSL_STARTUP, /* Negotiating SSL. */
CONNECTION_NEEDED, /* Internal state: connect() needed */
CONNECTION_CHECK_WRITABLE, /* Checking if session is read-write. */
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index d93e976ca5..db63afd786 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -390,6 +390,7 @@ struct pg_conn
char *keepalives_count; /* maximum number of TCP keepalive
* retransmits */
char *sslmode; /* SSL mode (require,prefer,allow,disable) */
+ char *sslnegotiation; /* SSL initiation style (postgres,direct,requiredirect) */
char *sslcompression; /* SSL compression (0 or 1) */
char *sslkey; /* client key filename */
char *sslcert; /* client certificate filename */
@@ -549,6 +550,8 @@ struct pg_conn
bool ssl_cert_sent; /* Did we send one in reply? */
#ifdef USE_SSL
+ bool allow_direct_ssl_try; /* Try to make a direct SSL connection
+ * without an "SSL negotiation packet" */
bool allow_ssl_try; /* Allowed to try SSL negotiation */
bool wait_ssl_try; /* Delay SSL negotiation until after
* attempting normal connection */
--
2.40.0
v6-0003-Direct-SSL-connections-documentation.patchtext/x-patch; charset=US-ASCII; name=v6-0003-Direct-SSL-connections-documentation.patchDownload
From 8d0abefa640fab9d9be16ae52fe65b86b272a537 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 15:10:15 -0400
Subject: [PATCH v6 3/6] Direct SSL connections documentation
---
doc/src/sgml/libpq.sgml | 102 ++++++++++++++++++++++++++++++++++++----
1 file changed, 93 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 9f72dd29d8..6efc70f801 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1691,10 +1691,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
Note that if <acronym>GSSAPI</acronym> encryption is possible,
that will be used in preference to <acronym>SSL</acronym>
encryption, regardless of the value of <literal>sslmode</literal>.
- To force use of <acronym>SSL</acronym> encryption in an
- environment that has working <acronym>GSSAPI</acronym>
- infrastructure (such as a Kerberos server), also
- set <literal>gssencmode</literal> to <literal>disable</literal>.
+ To negotiate <acronym>SSL</acronym> encryption in an environment that
+ has working <acronym>GSSAPI</acronym> infrastructure (such as a
+ Kerberos server), also set <literal>gssencmode</literal>
+ to <literal>disable</literal>.
+ Use of non-default values of <literal>sslnegotiation</literal> can
+ also cause <acronym>SSL</acronym> to be used instead of
+ negotiating <acronym>GSSAPI</acronym> encryption.
</para>
</listitem>
</varlistentry>
@@ -1721,6 +1724,75 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
</listitem>
</varlistentry>
+ <varlistentry id="libpq-connect-sslnegotiation" xreflabel="sslnegotiation">
+ <term><literal>sslnegotiation</literal></term>
+ <listitem>
+ <para>
+ This option controls whether <productname>PostgreSQL</productname>
+ will perform its protocol negotiation to request encryption from the
+ server or will just directly make a standard <acronym>SSL</acronym>
+ connection. Traditional <productname>PostgreSQL</productname>
+ protocol negotiation is the default and the most flexible with
+ different server configurations. If the server is known to support
+ direct <acronym>SSL</acronym> connections then the latter requires one
+ fewer round trip reducing connection latency and also allows the use
+ of protocol agnostic SSL network tools.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>postgres</literal></term>
+ <listitem>
+ <para>
+ perform <productname>PostgreSQL</productname> protocol
+ negotiation. This is the default if the option is not provided.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>direct</literal></term>
+ <listitem>
+ <para>
+ first attempt to establish a standard SSL connection and if that
+ fails reconnect and perform the negotiation. This fallback
+ process adds significant latency if the initial SSL connection
+ fails.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>requiredirect</literal></term>
+ <listitem>
+ <para>
+ attempt to establish a standard SSL connection and if that fails
+ return a connection failure immediately.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ If <literal>sslmode</literal> set to <literal>disable</literal>
+ or <literal>allow</literal> then <literal>sslnegotiation</literal> is
+ ignored. If <literal>gssencmode</literal> is set
+ to <literal>require</literal> then <literal>sslnegotiation</literal>
+ must be the default <literal>postgres</literal> value.
+ </para>
+
+ <para>
+ Moreover, note if <literal>gssencmode</literal> is set
+ to <literal>prefer</literal> and <literal>sslnegotiation</literal>
+ to <literal>direct</literal> then the effective preference will be
+ direct <acronym>SSL</acronym> connections, followed by
+ negotiated <acronym>GSS</acronym> connections, followed by
+ negotiated <acronym>SSL</acronym> connections, possibly followed
+ lastly by unencrypted connections.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-connect-sslcompression" xreflabel="sslcompression">
<term><literal>sslcompression</literal></term>
<listitem>
@@ -1930,11 +2002,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
<para>
The Server Name Indication can be used by SSL-aware proxies to route
- connections without having to decrypt the SSL stream. (Note that this
- requires a proxy that is aware of the PostgreSQL protocol handshake,
- not just any SSL proxy.) However, <acronym>SNI</acronym> makes the
- destination host name appear in cleartext in the network traffic, so
- it might be undesirable in some cases.
+ connections without having to decrypt the SSL stream. (Note that
+ unless the proxy is aware of the PostgreSQL protocol handshake this
+ would require setting <literal>sslnegotiation</literal>
+ to <literal>direct</literal> or <literal>requiredirect</literal>.)
+ However, <acronym>SNI</acronym> makes the destination host name appear
+ in cleartext in the network traffic, so it might be undesirable in
+ some cases.
</para>
</listitem>
</varlistentry>
@@ -8073,6 +8147,16 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
</para>
</listitem>
+ <listitem>
+ <para>
+ <indexterm>
+ <primary><envar>PGSSLNEGOTIATION</envar></primary>
+ </indexterm>
+ <envar>PGSSLNEGOTIATION</envar> behaves the same as the <xref
+ linkend="libpq-connect-sslnegotiation"/> connection parameter.
+ </para>
+ </listitem>
+
<listitem>
<para>
<indexterm>
--
2.40.0
v6-0006-Direct-SSL-connections-some-additional-docs.patchtext/x-patch; charset=US-ASCII; name=v6-0006-Direct-SSL-connections-some-additional-docs.patchDownload
From af1646f62f0486a86be2afb771c44ed42e430dd1 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Fri, 31 Mar 2023 03:01:35 -0400
Subject: [PATCH v6 6/6] Direct SSL connections some additional docs
---
doc/src/sgml/protocol.sgml | 37 +++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 8b5e7b1ad7..dd363656fd 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -1532,17 +1532,54 @@ SELCT 1/0;<!-- this typo is intentional -->
bytes.
</para>
+ <para>
+ Likewise the server expects the client to not begin
+ the <acronym>SSL</acronym> negotiation until it receives the server's
+ single byte response to the <acronym>SSL</acronym> request. If the
+ client begins the <acronym>SSL</acronym> negotiation immediately without
+ waiting for the server response to be received it can reduce connection
+ latency by one round-trip. However this comes at the cost of not being
+ able to handle the case where the server sends a negative response to the
+ <acronym>SSL</acronym> request. In that case instead of continuing with either GSSAPI or an
+ unencrypted connection or a protocol error the server will simply
+ disconnect.
+ </para>
+
<para>
An initial SSLRequest can also be used in a connection that is being
opened to send a CancelRequest message.
</para>
+ <para>
+ A second alternate way to initiate <acronym>SSL</acronym> encryption is
+ available. The server will recognize connections which immediately
+ begin <acronym>SSL</acronym> negotiation without any previous SSLRequest
+ packets. Once the <acronym>SSL</acronym> connection is established the
+ server will expect a normal startup-request packet and continue
+ negotiation over the encrypted channel. In this case any other requests
+ for encryption will be refused. This method is not preferred for general
+ purpose tools as it cannot negotiate the best connection encryption
+ available or handle unencrypted connections. However it is useful for
+ environments where both the server and client are controlled together.
+ In that case it avoids one round trip of latency and allows the use of
+ network tools that depend on standard <acronym>SSL</acronym> connections.
+ When using <acronym>SSL</acronym> connections in this style the client is
+ required to use the ALPN extension defined
+ by <ulink url="https://tools.ietf.org/html/rfc7301">RFC 7301</ulink> to
+ protect against protocol confusion attacks.
+ The <productname>PostgreSQL</productname> protocol is "TBD-pgsql" as
+ registered
+ at <ulink url="https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids">IANA
+ TLS ALPN Protocol IDs</ulink> registry.
+ </para>
+
<para>
While the protocol itself does not provide a way for the server to
force <acronym>SSL</acronym> encryption, the administrator can
configure the server to reject unencrypted sessions as a byproduct
of authentication checking.
</para>
+
</sect2>
<sect2 id="protocol-flow-gssapi">
--
2.40.0
v6-0001-Direct-SSL-connections-postmaster-support.patchtext/x-patch; charset=US-ASCII; name=v6-0001-Direct-SSL-connections-postmaster-support.patchDownload
From 6829925a144ac2acc605bbdebed7653b333e7e04 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Wed, 15 Mar 2023 15:51:02 -0400
Subject: [PATCH v6 1/6] Direct SSL connections postmaster support
---
src/backend/libpq/be-secure.c | 13 +++
src/backend/libpq/pqcomm.c | 9 +-
src/backend/postmaster/postmaster.c | 133 ++++++++++++++++++++++------
src/include/libpq/libpq-be.h | 10 +++
src/include/libpq/libpq.h | 2 +-
5 files changed, 134 insertions(+), 33 deletions(-)
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index a0f7084018..1020b3adb0 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -235,6 +235,19 @@ secure_raw_read(Port *port, void *ptr, size_t len)
{
ssize_t n;
+ /* Read from the "unread" buffered data first. c.f. libpq-be.h */
+ if (port->raw_buf_remaining > 0)
+ {
+ /* consume up to len bytes from the raw_buf */
+ if (len > port->raw_buf_remaining)
+ len = port->raw_buf_remaining;
+ Assert(port->raw_buf);
+ memcpy(ptr, port->raw_buf + port->raw_buf_consumed, len);
+ port->raw_buf_consumed += len;
+ port->raw_buf_remaining -= len;
+ return len;
+ }
+
/*
* Try to read from the socket without blocking. If it succeeds we're
* done, otherwise we'll wait for the socket using the latch mechanism.
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index da5bb5fc5d..17d025bb8e 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -1126,13 +1126,16 @@ pq_discardbytes(size_t len)
/* --------------------------------
* pq_buffer_has_data - is any buffered data available to read?
*
- * This will *not* attempt to read more data.
+ * Actually returns the number of bytes in the buffer...
+ *
+ * This will *not* attempt to read more data. And reading up to that number of
+ * bytes should not cause reading any more data either.
* --------------------------------
*/
-bool
+size_t
pq_buffer_has_data(void)
{
- return (PqRecvPointer < PqRecvLength);
+ return (PqRecvLength - PqRecvPointer);
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 4c49393fc5..ec1d895a23 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -422,6 +422,7 @@ static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(Port *port);
+static int ProcessSSLStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -1926,6 +1927,97 @@ ServerLoop(void)
}
}
+/*
+ * Check for a native direct SSL connection.
+ *
+ * This happens before startup packets so we are careful not to actual read
+ * any bytes from the stream if it's not a direct SSL connection.
+ */
+
+static int
+ProcessSSLStartup(Port *port)
+{
+ int firstbyte;
+
+ pq_startmsgread();
+
+ firstbyte = pq_peekbyte();
+ if (firstbyte == EOF)
+ {
+ /*
+ * If we get no data at all, don't clutter the log with a complaint;
+ * such cases often occur for legitimate reasons. An example is that
+ * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
+ * client didn't like our response, it'll probably just drop the
+ * connection. Service-monitoring software also often just opens and
+ * closes a connection without sending anything. (So do port
+ * scanners, which may be less benign, but it's not really our job to
+ * notice those.)
+ */
+ return STATUS_ERROR;
+ }
+
+ /*
+ * First byte indicates standard SSL handshake message
+ *
+ * (It can't be a Postgres startup length because in network byte order
+ * that would be a startup packet hundreds of megabytes long)
+ */
+ if (firstbyte == 0x16)
+ {
+#ifdef USE_SSL
+ ssize_t len;
+ char *buf = NULL;
+ elog(LOG, "Detected direct SSL handshake");
+
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ buf = palloc(len);
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+
+ Assert(pq_buffer_has_data() == 0);
+ if (secure_open_server(port) == -1)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("SSL Protocol Error during direct SSL connection initiation")));
+ return STATUS_ERROR;
+ }
+
+ if (port->raw_buf_remaining > 0)
+ {
+ /* This shouldn't be possible -- it would mean the client sent
+ * encrypted data before we established a session key...
+ */
+ elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ return STATUS_ERROR;
+ }
+ pfree(port->raw_buf);
+#else
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with no SSL support")));
+ return STATUS_ERROR;
+#endif
+ }
+
+ pq_endmsgread();
+
+ if (port->ssl_in_use)
+ ereport(DEBUG2,
+ (errmsg_internal("Direct SSL connection established")));
+
+ return STATUS_OK;
+}
+
+
/*
* Read a client's startup packet and do something according to it.
*
@@ -1954,28 +2046,7 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
pq_startmsgread();
- /*
- * Grab the first byte of the length word separately, so that we can tell
- * whether we have no data at all or an incomplete packet. (This might
- * sound inefficient, but it's not really, because of buffering in
- * pqcomm.c.)
- */
- if (pq_getbytes((char *) &len, 1) == EOF)
- {
- /*
- * If we get no data at all, don't clutter the log with a complaint;
- * such cases often occur for legitimate reasons. An example is that
- * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
- * client didn't like our response, it'll probably just drop the
- * connection. Service-monitoring software also often just opens and
- * closes a connection without sending anything. (So do port
- * scanners, which may be less benign, but it's not really our job to
- * notice those.)
- */
- return STATUS_ERROR;
- }
-
- if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
+ if (pq_getbytes(((char *) &len), 4) == EOF)
{
/* Got a partial length word, so bleat about that */
if (!ssl_done && !gss_done)
@@ -2039,8 +2110,11 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
char SSLok;
#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ /* No SSL when disabled or on Unix sockets.
+ *
+ * Also no SSL negotiation if we already have a direct SSL connection
+ */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX || port->ssl_in_use)
SSLok = 'N';
else
SSLok = 'S'; /* Support for SSL */
@@ -2048,11 +2122,10 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
SSLok = 'N'; /* No support for SSL */
#endif
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
+ while (secure_write(port, &SSLok, 1) != 1)
{
if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
+ continue; /* if interrupted, just retry */
ereport(COMMERROR,
(errcode_for_socket_access(),
errmsg("failed to send SSL negotiation response: %m")));
@@ -2093,7 +2166,7 @@ retry1:
GSSok = 'G';
#endif
- while (send(port->sock, &GSSok, 1, 0) != 1)
+ while (secure_write(port, &GSSok, 1) != 1)
{
if (errno == EINTR)
continue;
@@ -4396,7 +4469,9 @@ BackendInitialize(Port *port)
* Receive the startup packet (which might turn out to be a cancel request
* packet).
*/
- status = ProcessStartupPacket(port, false, false);
+ status = ProcessSSLStartup(port);
+ if (status == STATUS_OK)
+ status = ProcessStartupPacket(port, false, false);
/*
* Disable the timeout, and prevent SIGTERM again.
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index ac6407e9f6..824b28e824 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -226,6 +226,16 @@ typedef struct Port
SSL *ssl;
X509 *peer;
#endif
+ /* This is a bit of a hack. The raw_buf is data that was previously read
+ * and buffered in a higher layer but then "unread" and needs to be read
+ * again while establishing an SSL connection via the SSL library layer.
+ *
+ * There's no API to "unread", the upper layer just places the data in the
+ * Port structure in raw_buf and sets raw_buf_remaining to the amount of
+ * bytes unread and raw_buf_consumed to 0.
+ */
+ char *raw_buf;
+ ssize_t raw_buf_consumed, raw_buf_remaining;
} Port;
#ifdef USE_SSL
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 50fc781f47..2b02f67257 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -80,7 +80,7 @@ extern int pq_getmessage(StringInfo s, int maxlen);
extern int pq_getbyte(void);
extern int pq_peekbyte(void);
extern int pq_getbyte_if_available(unsigned char *c);
-extern bool pq_buffer_has_data(void);
+extern size_t pq_buffer_has_data(void);
extern int pq_putmessage_v2(char msgtype, const char *s, size_t len);
extern bool pq_check_connection(void);
--
2.40.0
On 31/03/2023 10:59, Greg Stark wrote:
IIRC I put a variable labeled a "GUC" but forgot to actually make it a
GUC. But I'm thinking of maybe removing that variable since I don't
see much of a use case for controlling this manually. I *think* ALPN
is supported by all the versions of OpenSSL we support.
+1 on removing the variable. Let's make ALPN mandatory for direct SSL
connections, like Jacob suggested. And for old-style handshakes, accept
and check ALPN if it's given.
I don't see the point of the libpq 'sslalpn' option either. Let's send
ALPN always.
Admittedly having the options make testing different of combinations of
old and new clients and servers a little easier. But I don't think we
should add options for the sake of backwards compatibility tests.
--- a/src/backend/libpq/pqcomm.c +++ b/src/backend/libpq/pqcomm.c @@ -1126,13 +1126,16 @@ pq_discardbytes(size_t len) /* -------------------------------- * pq_buffer_has_data - is any buffered data available to read? * - * This will *not* attempt to read more data. + * Actually returns the number of bytes in the buffer... + * + * This will *not* attempt to read more data. And reading up to that number of + * bytes should not cause reading any more data either. * -------------------------------- */ -bool +size_t pq_buffer_has_data(void) { - return (PqRecvPointer < PqRecvLength); + return (PqRecvLength - PqRecvPointer); }
Let's rename the function.
/* push unencrypted buffered data back through SSL setup */
len = pq_buffer_has_data();
if (len > 0)
{
buf = palloc(len);
if (pq_getbytes(buf, len) == EOF)
return STATUS_ERROR; /* shouldn't be possible */
port->raw_buf = buf;
port->raw_buf_remaining = len;
port->raw_buf_consumed = 0;
}Assert(pq_buffer_has_data() == 0);
if (secure_open_server(port) == -1)
{
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("SSL Protocol Error during direct SSL connection initiation")));
return STATUS_ERROR;
}if (port->raw_buf_remaining > 0)
{
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("received unencrypted data after SSL request"),
errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
return STATUS_ERROR;
}
if (port->raw_buf)
pfree(port->raw_buf);
This pattern is repeated in both callers of secure_open_server(). Could
we move this into secure_open_server() itself? That would feel pretty
natural, be-secure.c already contains the secure_raw_read() function
that reads the 'raw_buf' field.
const char *
PQsslAttribute(PGconn *conn, const char *attribute_name)
{
...if (strcmp(attribute_name, "alpn") == 0)
{
const unsigned char *data;
unsigned int len;
static char alpn_str[256]; /* alpn doesn't support longer than 255 bytes */
SSL_get0_alpn_selected(conn->ssl, &data, &len);
if (data == NULL || len==0 || len > sizeof(alpn_str)-1)
return NULL;
memcpy(alpn_str, data, len);
alpn_str[len] = 0;
return alpn_str;
}
Using a static buffer doesn't look right. If you call PQsslAttribute on
two different connections from two different threads concurrently, they
will write to the same buffer. I see that you copied it from the
"key_bits" handlng, but it has the same issue.
--
Heikki Linnakangas
Neon (https://neon.tech)
On Tue, Jul 04, 2023 at 05:15:49PM +0300, Heikki Linnakangas wrote:
I don't see the point of the libpq 'sslalpn' option either. Let's send ALPN
always.Admittedly having the options make testing different of combinations of old
and new clients and servers a little easier. But I don't think we should add
options for the sake of backwards compatibility tests.
Hmm. I would actually argue in favor of having these with tests in
core to stress the previous SSL hanshake protocol, as not having these
parameters would mean that we rely only on major version upgrades in
the buildfarm to test the backward-compatible code path, making issues
much harder to catch. And we still need to maintain the
backward-compatible path for 10 years based on what pg_dump and
pg_upgrade need to support.
--
Michael
On 05/07/2023 02:33, Michael Paquier wrote:
On Tue, Jul 04, 2023 at 05:15:49PM +0300, Heikki Linnakangas wrote:
I don't see the point of the libpq 'sslalpn' option either. Let's send ALPN
always.Admittedly having the options make testing different of combinations of old
and new clients and servers a little easier. But I don't think we should add
options for the sake of backwards compatibility tests.Hmm. I would actually argue in favor of having these with tests in
core to stress the previous SSL hanshake protocol, as not having these
parameters would mean that we rely only on major version upgrades in
the buildfarm to test the backward-compatible code path, making issues
much harder to catch. And we still need to maintain the
backward-compatible path for 10 years based on what pg_dump and
pg_upgrade need to support.
Ok, let's keep it.
I started to review this again. There's a lot of little things to fix
before this is ready for commit, but overall this looks pretty good. A
few notes / questions on the first two patches (in addition to the few
comments I made earlier):
If the client sends TLS HelloClient directly, but the server does not
support TLS, it just closes the connection. It would be nice to still
send some kind of an error to the client. Maybe a TLS alert packet? I
don't want to start implementing TLS, but I think a TLS alert packet
with a suitable error code would be just a constant.
The new CONNECTION_DIRECT_SSL_STARTUP state needs to be moved to end of
the enum. We cannot change the integer values of existing of enum
values, or clients compiled with old libpq version would mix up the states.
/*
* validate sslnegotiation option, default is "postgres" for the postgres
* style negotiated connection with an extra round trip but more options.
*/
What "more options" does the negotiated connection provide?
if (conn->sslnegotiation)
{
if (strcmp(conn->sslnegotiation, "postgres") != 0
&& strcmp(conn->sslnegotiation, "direct") != 0
&& strcmp(conn->sslnegotiation, "requiredirect") != 0)
{
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
"sslnegotiation", conn->sslnegotiation);
return false;
}#ifndef USE_SSL
if (conn->sslnegotiation[0] != 'p') {
conn->status = CONNECTION_BAD;
libpq_append_conn_error(conn, "sslnegotiation value \"%s\" invalid when SSL support is not compiled in",
conn->sslnegotiation);
return false;
}
#endif
}
At the same time, the patch allows the combination of "sslmode=disable"
and "sslnegotiation=requiredirect". Seems inconsistent to error out if
compiled without SSL support.
else
{
libpq_append_conn_error(conn, "sslnegotiation missing?");
return false;
}
In the other similar settings, like 'channel_binding' and 'sslcertmode',
we strdup() the compiled-in default if the option is NULL. I'm not sure
if that's necessary, I think the compiled-in defaults should get filled
in conninfo_add_defaults(). If so, then those other places could be
turned into errors like this too. This seems to be a bit of a mess even
before this patch.
In pg_conn struct:
+ bool allow_direct_ssl_try; /* Try to make a direct SSL connection + * without an "SSL negotiation packet" */ bool allow_ssl_try; /* Allowed to try SSL negotiation */ bool wait_ssl_try; /* Delay SSL negotiation until after * attempting normal connection */
It's getting hard to follow what combinations of these booleans are
valid and what they're set to at different stages. I think it's time to
turn all these into one enum, or something like that.
I intend to continue reviewing this after Jan 8th. I'd still like to get
this into v17.
--
Heikki Linnakangas
Neon (https://neon.tech)
Some more comments on this:
1. It feels weird that the combination of "gssencmode=require
sslnegotiation=direct" combination is forbidden. Sure, the ssl
negotiation will never happen with gssencmode=require, so the
sslnegotiation option has no effect. But by that token, should we also
forbid the combination "sslmode=disable sslnegotiation=direct"? I think
not. The sslnegotiation option should mean "if we are going to try SSL,
should we try it in direct or negotiated mode?"
2. Should we allow direct SSL only at the very beginning of a TCP
connection, or should we also allow it after we have requested GSS and
the server said no? Like this:
Client: GSSENCRequest
Server: 'N' (gss not supported)
Client: TLS client Hello
On one hand, why not? It saves you a round-trip in this case too. If we
don't allow it, the client will have to send SSLRequest and wait for
response, or reconnect to try direct SSL. On the other hand, flexibility
is not necessarily a good thing in security-critical code like this.
The patch set is confused on whether that's allowed or not. The server
rejects it. But if you use "gssencmode=prefer
sslnegotiation=requiredrect", libpq will attempt to do it, and fail.
3. With "sslmode=verify-full sslnegotiation=direct", if the direct SSL
connection fails because of a problem with the certificate, libpq will
try again in negotiated SSL mode. That seems pointless. If the server
responded to the direct TLS Client Hello message with a valid
ServerHello, that indicates that the server supports direct SSL. If
anything goes wrong after that, retrying in negotiated mode is not going
to help.
4. The number of combinations of sslmode, gssencmode and sslnegotiation
settings is scary. And we have very few tests for them.
Attached patch set addresses the above, but is very much WIP. I
refactored the state machine in libpq, to make the states and
transitions more clear. I think that helps, but it's still pretty
complex. I'm all ears for ideas on how to simplify it further.
I added a new test suite to test the different libpq options. See
src/test/libpq_encryption. I think this is very much needed, but I'm
still not very happy with the implementation. Some combinations are
still impossible to test, like connecting to an older server that
doesn't support direct SSL, or having the server respond with 'N' to
GSSEncRequest. I'd also like to check more details of each connection
attempt, like how many TCP connections are established, to check for
things like 3. above. Maybe we need to add more logging to libpq or the
server and check the logs after each test.
I'm tempted to implement a mock server from scratch that could easily be
instructed to accept/reject the connection at just the right places. But
that's a lot of work.
I'm going to put this down for now. The attached patch set is even more
raw than v6, but I'm including it here to "save the work".
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v7-0001-Move-Kerberos-module.patchtext/x-patch; charset=UTF-8; name=v7-0001-Move-Kerberos-module.patchDownload
From 14493f5c25141893f75d577919940e62948a43df Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 9 Jan 2024 13:16:44 +0200
Subject: [PATCH v7 01/12] Move Kerberos module
---
src/test/kerberos/t/001_auth.pl | 174 ++----------------
src/test/perl/PostgreSQL/Test/Kerberos.pm | 207 ++++++++++++++++++++++
2 files changed, 218 insertions(+), 163 deletions(-)
create mode 100644 src/test/perl/PostgreSQL/Test/Kerberos.pm
diff --git a/src/test/kerberos/t/001_auth.pl b/src/test/kerberos/t/001_auth.pl
index 2a81ce8834b..efd5f1f2c0c 100644
--- a/src/test/kerberos/t/001_auth.pl
+++ b/src/test/kerberos/t/001_auth.pl
@@ -21,6 +21,7 @@ use strict;
use warnings FATAL => 'all';
use PostgreSQL::Test::Utils;
use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Kerberos;
use Test::More;
use Time::HiRes qw(usleep);
@@ -34,177 +35,27 @@ elsif (!$ENV{PG_TEST_EXTRA} || $ENV{PG_TEST_EXTRA} !~ /\bkerberos\b/)
'Potentially unsafe test GSSAPI/Kerberos not enabled in PG_TEST_EXTRA';
}
-my ($krb5_bin_dir, $krb5_sbin_dir);
-
-if ($^O eq 'darwin' && -d "/opt/homebrew")
-{
- # typical paths for Homebrew on ARM
- $krb5_bin_dir = '/opt/homebrew/opt/krb5/bin';
- $krb5_sbin_dir = '/opt/homebrew/opt/krb5/sbin';
-}
-elsif ($^O eq 'darwin')
-{
- # typical paths for Homebrew on Intel
- $krb5_bin_dir = '/usr/local/opt/krb5/bin';
- $krb5_sbin_dir = '/usr/local/opt/krb5/sbin';
-}
-elsif ($^O eq 'freebsd')
-{
- $krb5_bin_dir = '/usr/local/bin';
- $krb5_sbin_dir = '/usr/local/sbin';
-}
-elsif ($^O eq 'linux')
-{
- $krb5_sbin_dir = '/usr/sbin';
-}
-
-my $krb5_config = 'krb5-config';
-my $kinit = 'kinit';
-my $klist = 'klist';
-my $kdb5_util = 'kdb5_util';
-my $kadmin_local = 'kadmin.local';
-my $krb5kdc = 'krb5kdc';
-
-if ($krb5_bin_dir && -d $krb5_bin_dir)
-{
- $krb5_config = $krb5_bin_dir . '/' . $krb5_config;
- $kinit = $krb5_bin_dir . '/' . $kinit;
- $klist = $krb5_bin_dir . '/' . $klist;
-}
-if ($krb5_sbin_dir && -d $krb5_sbin_dir)
-{
- $kdb5_util = $krb5_sbin_dir . '/' . $kdb5_util;
- $kadmin_local = $krb5_sbin_dir . '/' . $kadmin_local;
- $krb5kdc = $krb5_sbin_dir . '/' . $krb5kdc;
-}
-
-my $host = 'auth-test-localhost.postgresql.example.com';
-my $hostaddr = '127.0.0.1';
-my $realm = 'EXAMPLE.COM';
-
-my $krb5_conf = "${PostgreSQL::Test::Utils::tmp_check}/krb5.conf";
-my $kdc_conf = "${PostgreSQL::Test::Utils::tmp_check}/kdc.conf";
-my $krb5_cache = "${PostgreSQL::Test::Utils::tmp_check}/krb5cc";
-my $krb5_log = "${PostgreSQL::Test::Utils::log_path}/krb5libs.log";
-my $kdc_log = "${PostgreSQL::Test::Utils::log_path}/krb5kdc.log";
-my $kdc_port = PostgreSQL::Test::Cluster::get_free_port();
-my $kdc_datadir = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc";
-my $kdc_pidfile = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc.pid";
-my $keytab = "${PostgreSQL::Test::Utils::tmp_check}/krb5.keytab";
-
my $pgpass = "${PostgreSQL::Test::Utils::tmp_check}/.pgpass";
my $dbname = 'postgres';
my $username = 'test1';
my $application = '001_auth.pl';
-note "setting up Kerberos";
-
-my ($stdout, $krb5_version);
-run_log [ $krb5_config, '--version' ], '>', \$stdout
- or BAIL_OUT("could not execute krb5-config");
-BAIL_OUT("Heimdal is not supported") if $stdout =~ m/heimdal/;
-$stdout =~ m/Kerberos 5 release ([0-9]+\.[0-9]+)/
- or BAIL_OUT("could not get Kerberos version");
-$krb5_version = $1;
-
# Construct a pgpass file to make sure we don't use it
append_to_file($pgpass, '*:*:*:*:abc123');
chmod 0600, $pgpass;
-# Build the krb5.conf to use.
-#
-# Explicitly specify the default (test) realm and the KDC for
-# that realm to avoid the Kerberos library trying to look up
-# that information in DNS, and also because we're using a
-# non-standard KDC port.
-#
-# Also explicitly disable DNS lookups since this isn't really
-# our domain and we shouldn't be causing random DNS requests
-# to be sent out (not to mention that broken DNS environments
-# can cause the tests to take an extra long time and timeout).
-#
-# Reverse DNS is explicitly disabled to avoid any issue with a
-# captive portal or other cases where the reverse DNS succeeds
-# and the Kerberos library uses that as the canonical name of
-# the host and then tries to acquire a cross-realm ticket.
-append_to_file(
- $krb5_conf,
- qq![logging]
-default = FILE:$krb5_log
-kdc = FILE:$kdc_log
-
-[libdefaults]
-dns_lookup_realm = false
-dns_lookup_kdc = false
-default_realm = $realm
-forwardable = false
-rdns = false
-
-[realms]
-$realm = {
- kdc = $hostaddr:$kdc_port
-}
-!);
-
-append_to_file(
- $kdc_conf,
- qq![kdcdefaults]
-!);
-
-# For new-enough versions of krb5, use the _listen settings rather
-# than the _ports settings so that we can bind to localhost only.
-if ($krb5_version >= 1.15)
-{
- append_to_file(
- $kdc_conf,
- qq!kdc_listen = $hostaddr:$kdc_port
-kdc_tcp_listen = $hostaddr:$kdc_port
-!);
-}
-else
-{
- append_to_file(
- $kdc_conf,
- qq!kdc_ports = $kdc_port
-kdc_tcp_ports = $kdc_port
-!);
-}
-append_to_file(
- $kdc_conf,
- qq!
-[realms]
-$realm = {
- database_name = $kdc_datadir/principal
- admin_keytab = FILE:$kdc_datadir/kadm5.keytab
- acl_file = $kdc_datadir/kadm5.acl
- key_stash_file = $kdc_datadir/_k5.$realm
-}!);
-
-mkdir $kdc_datadir or die;
-
-# Ensure that we use test's config and cache files, not global ones.
-$ENV{'KRB5_CONFIG'} = $krb5_conf;
-$ENV{'KRB5_KDC_PROFILE'} = $kdc_conf;
-$ENV{'KRB5CCNAME'} = $krb5_cache;
+note "setting up Kerberos";
-my $service_principal = "$ENV{with_krb_srvnam}/$host";
+my $host = 'auth-test-localhost.postgresql.example.com';
+my $hostaddr = '127.0.0.1';
+my $realm = 'EXAMPLE.COM';
-system_or_bail $kdb5_util, 'create', '-s', '-P', 'secret0';
+my $krb = PostgreSQL::Test::Kerberos->new($host, $hostaddr, $realm);
my $test1_password = 'secret1';
-system_or_bail $kadmin_local, '-q', "addprinc -pw $test1_password test1";
-
-system_or_bail $kadmin_local, '-q', "addprinc -randkey $service_principal";
-system_or_bail $kadmin_local, '-q', "ktadd -k $keytab $service_principal";
-
-system_or_bail $krb5kdc, '-P', $kdc_pidfile;
-
-END
-{
- kill 'INT', `cat $kdc_pidfile` if defined($kdc_pidfile) && -f $kdc_pidfile;
-}
+$krb->create_principle('test1', $test1_password);
note "setting up PostgreSQL instance";
@@ -213,7 +64,7 @@ $node->init;
$node->append_conf(
'postgresql.conf', qq{
listen_addresses = '$hostaddr'
-krb_server_keyfile = '$keytab'
+krb_server_keyfile = '$krb->{keytab}'
log_connections = on
lc_messages = 'C'
});
@@ -327,8 +178,7 @@ $node->restart;
test_access($node, 'test1', 'SELECT true', 2, '', 'fails without ticket');
-run_log [ $kinit, 'test1' ], \$test1_password or BAIL_OUT($?);
-run_log [ $klist, '-f' ] or BAIL_OUT($?);
+$krb->create_ticket('test1', $test1_password);
test_access(
$node,
@@ -470,10 +320,8 @@ $node->append_conf(
hostgssenc all all $hostaddr/32 gss map=mymap
});
-string_replace_file($krb5_conf, "forwardable = false", "forwardable = true");
-
-run_log [ $kinit, 'test1' ], \$test1_password or BAIL_OUT($?);
-run_log [ $klist, '-f' ] or BAIL_OUT($?);
+# Re-create the ticket, with the forwardable flag set
+$krb->create_ticket('test1', $test1_password, forwardable => 1);
test_access(
$node,
diff --git a/src/test/perl/PostgreSQL/Test/Kerberos.pm b/src/test/perl/PostgreSQL/Test/Kerberos.pm
new file mode 100644
index 00000000000..382a38ec9d2
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/Kerberos.pm
@@ -0,0 +1,207 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+# Sets up a KDC XXX
+#
+# Since this requires setting up a full KDC, it doesn't make much sense
+# to have multiple test scripts (since they'd have to also create their
+# own KDC and that could cause race conditions or other problems)- so
+# just add whatever other tests are needed to here. XXX
+
+package PostgreSQL::Test::Kerberos;
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+
+our ($krb5_bin_dir, $krb5_sbin_dir, $krb5_config, $kinit, $klist,
+ $kdb5_util, $kadmin_local, $krb5kdc,
+ $krb5_conf, $kdc_conf, $krb5_cache, $krb5_log, $kdc_log,
+ $kdc_port, $kdc_datadir, $kdc_pidfile, $keytab);
+
+INIT
+{
+ if ($^O eq 'darwin' && -d "/opt/homebrew")
+ {
+ # typical paths for Homebrew on ARM
+ $krb5_bin_dir = '/opt/homebrew/opt/krb5/bin';
+ $krb5_sbin_dir = '/opt/homebrew/opt/krb5/sbin';
+ }
+ elsif ($^O eq 'darwin')
+ {
+ # typical paths for Homebrew on Intel
+ $krb5_bin_dir = '/usr/local/opt/krb5/bin';
+ $krb5_sbin_dir = '/usr/local/opt/krb5/sbin';
+ }
+ elsif ($^O eq 'freebsd')
+ {
+ $krb5_bin_dir = '/usr/local/bin';
+ $krb5_sbin_dir = '/usr/local/sbin';
+ }
+ elsif ($^O eq 'linux')
+ {
+ $krb5_sbin_dir = '/usr/sbin';
+ }
+
+ $krb5_config = 'krb5-config';
+ $kinit = 'kinit';
+ $klist = 'klist';
+ $kdb5_util = 'kdb5_util';
+ $kadmin_local = 'kadmin.local';
+ $krb5kdc = 'krb5kdc';
+
+ if ($krb5_bin_dir && -d $krb5_bin_dir)
+ {
+ $krb5_config = $krb5_bin_dir . '/' . $krb5_config;
+ $kinit = $krb5_bin_dir . '/' . $kinit;
+ $klist = $krb5_bin_dir . '/' . $klist;
+ }
+ if ($krb5_sbin_dir && -d $krb5_sbin_dir)
+ {
+ $kdb5_util = $krb5_sbin_dir . '/' . $kdb5_util;
+ $kadmin_local = $krb5_sbin_dir . '/' . $kadmin_local;
+ $krb5kdc = $krb5_sbin_dir . '/' . $krb5kdc;
+ }
+
+ $krb5_conf = "${PostgreSQL::Test::Utils::tmp_check}/krb5.conf";
+ $kdc_conf = "${PostgreSQL::Test::Utils::tmp_check}/kdc.conf";
+ $krb5_cache = "${PostgreSQL::Test::Utils::tmp_check}/krb5cc";
+ $krb5_log = "${PostgreSQL::Test::Utils::log_path}/krb5libs.log";
+ $kdc_log = "${PostgreSQL::Test::Utils::log_path}/krb5kdc.log";
+ $kdc_port = PostgreSQL::Test::Cluster::get_free_port();
+ $kdc_datadir = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc";
+ $kdc_pidfile = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc.pid";
+ $keytab = "${PostgreSQL::Test::Utils::tmp_check}/krb5.keytab";
+}
+
+sub new
+{
+ my $class = shift;
+ my ($host, $hostaddr, $realm, %params) = @_;
+
+ my ($stdout, $krb5_version);
+ run_log [ $krb5_config, '--version' ], '>', \$stdout
+ or BAIL_OUT("could not execute krb5-config");
+ BAIL_OUT("Heimdal is not supported") if $stdout =~ m/heimdal/;
+ $stdout =~ m/Kerberos 5 release ([0-9]+\.[0-9]+)/
+ or BAIL_OUT("could not get Kerberos version");
+ $krb5_version = $1;
+
+ # Build the krb5.conf to use.
+ #
+ # Explicitly specify the default (test) realm and the KDC for
+ # that realm to avoid the Kerberos library trying to look up
+ # that information in DNS, and also because we're using a
+ # non-standard KDC port.
+ #
+ # Also explicitly disable DNS lookups since this isn't really
+ # our domain and we shouldn't be causing random DNS requests
+ # to be sent out (not to mention that broken DNS environments
+ # can cause the tests to take an extra long time and timeout).
+ #
+ # Reverse DNS is explicitly disabled to avoid any issue with a
+ # captive portal or other cases where the reverse DNS succeeds
+ # and the Kerberos library uses that as the canonical name of
+ # the host and then tries to acquire a cross-realm ticket.
+ append_to_file(
+ $krb5_conf,
+ qq![logging]
+default = FILE:$krb5_log
+kdc = FILE:$kdc_log
+
+[libdefaults]
+dns_lookup_realm = false
+dns_lookup_kdc = false
+default_realm = $realm
+forwardable = false
+rdns = false
+
+[realms]
+$realm = {
+ kdc = $hostaddr:$kdc_port
+}
+!);
+
+ append_to_file(
+ $kdc_conf,
+ qq![kdcdefaults]
+!);
+
+ # For new-enough versions of krb5, use the _listen settings rather
+ # than the _ports settings so that we can bind to localhost only.
+ if ($krb5_version >= 1.15)
+ {
+ append_to_file(
+ $kdc_conf,
+ qq!kdc_listen = $hostaddr:$kdc_port
+kdc_tcp_listen = $hostaddr:$kdc_port
+!);
+ }
+ else
+ {
+ append_to_file(
+ $kdc_conf,
+ qq!kdc_ports = $kdc_port
+kdc_tcp_ports = $kdc_port
+!);
+ }
+ append_to_file(
+ $kdc_conf,
+ qq!
+[realms]
+$realm = {
+ database_name = $kdc_datadir/principal
+ admin_keytab = FILE:$kdc_datadir/kadm5.keytab
+ acl_file = $kdc_datadir/kadm5.acl
+ key_stash_file = $kdc_datadir/_k5.$realm
+}!);
+
+ mkdir $kdc_datadir or die;
+
+ # Ensure that we use test's config and cache files, not global ones.
+ $ENV{'KRB5_CONFIG'} = $krb5_conf;
+ $ENV{'KRB5_KDC_PROFILE'} = $kdc_conf;
+ $ENV{'KRB5CCNAME'} = $krb5_cache;
+
+ my $service_principal = "$ENV{with_krb_srvnam}/$host";
+
+ system_or_bail $kdb5_util, 'create', '-s', '-P', 'secret0';
+
+ system_or_bail $kadmin_local, '-q', "addprinc -randkey $service_principal";
+ system_or_bail $kadmin_local, '-q', "ktadd -k $keytab $service_principal";
+
+ system_or_bail $krb5kdc, '-P', $kdc_pidfile;
+
+ my $self = {};
+ $self->{keytab} = $keytab;
+
+ bless $self, $class;
+
+ return $self;
+}
+
+sub create_principle
+{
+ my ($self, $principal, $password) = @_;
+
+ system_or_bail $kadmin_local, '-q', "addprinc -pw $password $principal";
+}
+
+sub create_ticket
+{
+ my ($self, $principal, $password, %params) = @_;
+
+ my @cmd = ($kinit, $principal);
+
+ push @cmd, '-f' if ($params{forwardable});
+
+ run_log [@cmd], \$password or BAIL_OUT($?);
+ run_log [ $klist, '-f' ] or BAIL_OUT($?);
+}
+
+END
+{
+ kill 'INT', `cat $kdc_pidfile` if defined($kdc_pidfile) && -f $kdc_pidfile;
+}
+
+1;
--
2.39.2
v7-0002-Add-enc-tests.patchtext/x-patch; charset=UTF-8; name=v7-0002-Add-enc-tests.patchDownload
From 9df81d1607ad9a94ddfa48132bfd3f7b7c49f120 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 09:47:46 +0200
Subject: [PATCH v7 02/12] Add enc tests
---
src/test/libpq_encryption/Makefile | 25 ++
src/test/libpq_encryption/README | 23 ++
src/test/libpq_encryption/meson.build | 19 ++
.../t/001_negotiate_encryption.pl | 294 ++++++++++++++++++
src/test/perl/PostgreSQL/Test/Kerberos.pm | 6 +-
5 files changed, 364 insertions(+), 3 deletions(-)
create mode 100644 src/test/libpq_encryption/Makefile
create mode 100644 src/test/libpq_encryption/README
create mode 100644 src/test/libpq_encryption/meson.build
create mode 100644 src/test/libpq_encryption/t/001_negotiate_encryption.pl
diff --git a/src/test/libpq_encryption/Makefile b/src/test/libpq_encryption/Makefile
new file mode 100644
index 00000000000..710929c4cce
--- /dev/null
+++ b/src/test/libpq_encryption/Makefile
@@ -0,0 +1,25 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for src/test/libpq_encryption
+#
+# Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/test/ldap/libpq_encryption
+#
+#-------------------------------------------------------------------------
+
+subdir = src/test/libpq_encryption
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+export OPENSSL with_ssl with_gssapi with_krb_srvnam
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
+
+clean distclean:
+ rm -rf tmp_check
diff --git a/src/test/libpq_encryption/README b/src/test/libpq_encryption/README
new file mode 100644
index 00000000000..66430b54316
--- /dev/null
+++ b/src/test/libpq_encryption/README
@@ -0,0 +1,23 @@
+src/test/libpq_encryption/README
+
+Tests for negotiating network encryption method
+===============================================
+
+
+Running the tests
+=================
+
+NOTE: You must have given the --enable-tap-tests argument to configure.
+
+Run
+ make check PG_TEST_EXTRA=libpq_encryption
+
+XXX You can use "make installcheck" if you previously did "make install".
+In that case, the code in the installation tree is tested. With
+"make check", a temporary installation tree is built from the current
+sources and then tested.
+
+XXX Either way, this test initializes, starts, and stops a test Postgres
+cluster, as well as a test LDAP server.
+
+See src/test/perl/README for more info about running these tests.
diff --git a/src/test/libpq_encryption/meson.build b/src/test/libpq_encryption/meson.build
new file mode 100644
index 00000000000..04f479e9fe7
--- /dev/null
+++ b/src/test/libpq_encryption/meson.build
@@ -0,0 +1,19 @@
+# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+
+tests += {
+ 'name': 'ldap',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'tap': {
+ 'tests': [
+ 't/001_auth.pl',
+ 't/002_bindpasswd.pl',
+ ],
+ 'env': {
+ 'with_ssl': ssl_library,
+ 'OPENSSL': openssl.found() ? openssl.path() : '',
+ 'with_gssapi': gssapi.found() ? 'yes' : 'no',
+ 'with_krb_srvnam': 'postgres',
+ },
+ },
+}
diff --git a/src/test/libpq_encryption/t/001_negotiate_encryption.pl b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
new file mode 100644
index 00000000000..7a80fe90f56
--- /dev/null
+++ b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
@@ -0,0 +1,294 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+# Test negotiation of SSL and GSSAPI encryption
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Kerberos;
+use File::Basename;
+use File::Copy;
+use Test::More;
+
+if (!$ENV{PG_TEST_EXTRA} || $ENV{PG_TEST_EXTRA} !~ /\blibpq_encryption\b/)
+{
+ plan skip_all =>
+ 'Potentially unsafe test libpq_encryption not enabled in PG_TEST_EXTRA';
+}
+
+my $host = 'enc-test-localhost.postgresql.example.com';
+my $hostaddr = '127.0.0.1';
+my $servercidr = '127.0.0.1/32';
+
+note "setting up PostgreSQL instance";
+
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init;
+$node->append_conf(
+ 'postgresql.conf', qq{
+listen_addresses = '$hostaddr'
+log_connections = on
+lc_messages = 'C'
+});
+my $pgdata = $node->data_dir;
+
+my $dbname = 'postgres';
+my $username = 'enctest';
+my $application = '001_negotiate_encryption.pl';
+
+my $gssuser_password = 'secret1';
+
+my $krb;
+
+my $ssl_supported = $ENV{with_ssl} eq 'openssl';
+my $gss_supported = $ENV{with_gssapi} eq 'yes';
+
+if ($ENV{with_gssapi} eq 'yes')
+{
+ note "setting up Kerberos";
+
+ my $realm = 'EXAMPLE.COM';
+ $krb = PostgreSQL::Test::Kerberos->new($host, $hostaddr, $realm);
+ $node->append_conf('postgresql.conf', "krb_server_keyfile = '$krb->{keytab}'\n");
+}
+
+if ($ENV{with_ssl} eq 'openssl')
+{
+ my $certdir = dirname(__FILE__) . "/../../ssl/ssl";
+
+ copy "$certdir/server-cn-only.crt", "$pgdata/server.crt"
+ || die "copying server.crt: $!";
+ copy "$certdir/server-cn-only.key", "$pgdata/server.key"
+ || die "copying server.key: $!";
+ chmod(0600, "$pgdata/server.key");
+
+ # Start with SSL disabled.
+ $node->append_conf('postgresql.conf', "ssl = off\n");
+}
+
+$node->start;
+
+$node->safe_psql('postgres', 'CREATE USER localuser;');
+$node->safe_psql('postgres', 'CREATE USER testuser;');
+$node->safe_psql('postgres', 'CREATE USER ssluser;');
+$node->safe_psql('postgres', 'CREATE USER nossluser;');
+$node->safe_psql('postgres', 'CREATE USER gssuser;');
+$node->safe_psql('postgres', 'CREATE USER nogssuser;');
+
+my $unixdir = $node->safe_psql('postgres', 'SHOW unix_socket_directories;');
+chomp($unixdir);
+
+$node->safe_psql('postgres', q{
+CREATE FUNCTION current_enc() RETURNS text LANGUAGE plpgsql AS $$
+DECLARE
+ ssl_in_use bool;
+ gss_in_use bool;
+BEGIN
+ ssl_in_use = (SELECT ssl FROM pg_stat_ssl WHERE pid = pg_backend_pid());
+ gss_in_use = (SELECT encrypted FROM pg_stat_gssapi WHERE pid = pg_backend_pid());
+
+ raise log 'ssl % gss %', ssl_in_use, gss_in_use;
+
+ IF ssl_in_use AND gss_in_use THEN
+ RETURN 'ssl+gss'; -- shouldn't happen
+ ELSIF ssl_in_use THEN
+ RETURN 'ssl';
+ ELSIF gss_in_use THEN
+ RETURN 'gss';
+ ELSE
+ RETURN 'plain';
+ END IF;
+END;
+$$;
+});
+
+# Only accept SSL connections from $servercidr. Our tests don't depend on this
+# but seems best to keep it as narrow as possible for security reasons.
+#
+# When connecting to certdb, also check the client certificate.
+open my $hba, '>', "$pgdata/pg_hba.conf";
+print $hba qq{
+# TYPE DATABASE USER ADDRESS METHOD OPTIONS
+local postgres localuser trust
+host postgres testuser $servercidr trust
+hostnossl postgres nossluser $servercidr trust
+hostnogssenc postgres nogssuser $servercidr trust
+};
+
+print $hba qq{
+hostssl postgres ssluser $servercidr trust
+} if ($ENV{with_ssl} eq 'openssl');
+
+print $hba qq{
+hostgssenc postgres gssuser $servercidr trust
+} if ($ENV{with_gssapi} eq 'yes');
+close $hba;
+$node->reload;
+
+note "running tests";
+
+sub connect_test
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $connstr, $expected_enc, @expect_log_msgs)
+ = @_;
+
+ my %params = ();
+
+ if (@expect_log_msgs)
+ {
+ # Match every message literally.
+ my @regexes = map { qr/\Q$_\E/ } @expect_log_msgs;
+
+ $params{log_like} = \@regexes;
+ }
+
+ my $test_name = "'$connstr' -> $expected_enc";
+
+ my $connstr_full = "";
+ $connstr_full .= "dbname=postgres " unless $connstr =~ m/dbname=/;
+ $connstr_full .= "host=$host hostaddr=$hostaddr " unless $connstr =~ m/host=/;
+ $connstr_full .= $connstr;
+
+ if ($expected_enc eq "fail")
+ {
+ $node->connect_fails($connstr_full, $test_name, %params);
+ }
+ else
+ {
+ $params{sql} = "SELECT current_enc()";
+ $params{expected_stdout} = qr/^$expected_enc$/;;
+
+ $node->connect_ok($connstr_full, $test_name, %params);
+ }
+}
+
+
+# First test with SSL disabled in the server
+connect_test($node, 'user=testuser sslmode=disable', 'plain');
+connect_test($node, 'user=testuser sslmode=allow', 'plain');
+connect_test($node, 'user=testuser sslmode=prefer', 'plain');
+connect_test($node, 'user=testuser sslmode=require', 'fail');
+
+# Test GSSAPI
+SKIP:
+{
+ skip "GSSAPI/Kerberos not supported by this build" unless $ENV{with_gssapi} eq 'yes';
+
+ # No ticket
+ connect_test($node, 'user=testuser sslmode=disable gssencmode=require', 'fail');
+
+ $krb->create_principle('gssuser', $gssuser_password);
+ $krb->create_ticket('gssuser', $gssuser_password);
+
+ connect_test($node, 'user=testuser sslmode=disable gssencmode=disable', 'plain');
+ connect_test($node, 'user=testuser sslmode=disable gssencmode=prefer', 'gss');
+ connect_test($node, 'user=testuser sslmode=disable gssencmode=require', 'gss');
+
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=disable', 'plain');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=require', 'gss');
+
+ connect_test($node, 'user=testuser sslmode=require gssencmode=disable', 'fail');
+ connect_test($node, 'user=testuser sslmode=require gssencmode=prefer', 'gss');
+
+ # If you set both sslmode and gssencmode to 'require', 'gssencmode=require' takes
+ # precedence.
+ connect_test($node, 'user=testuser sslmode=require gssencmode=require', 'gss');
+
+ # Test case that server supports GSSAPI, but it's not allowed for
+ # this user.
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=require', 'fail',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+
+ # With 'gssencmode=prefer', libpq will first negotiate GSSAPI
+ # encryption, but the connection will fail because pg_hba.conf
+ # forbids GSSAPI encryption for this user. It will then reconnect
+ # with SSL, but the server doesn't support it, so it will continue
+ # with no encryption.
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer', 'plain',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+}
+
+# Enable SSL in the server
+SKIP:
+{
+ skip "SSL not supported by this build" unless $ssl_supported;
+
+ my $certdir = dirname(__FILE__) . "/../../ssl/ssl";
+
+ copy "$certdir/server-cn-only.crt", "$pgdata/server.crt"
+ || die "copying server.crt: $!";
+ copy "$certdir/server-cn-only.key", "$pgdata/server.key"
+ || die "copying server.key: $!";
+ chmod(0600, "$pgdata/server.key");
+
+ $node->adjust_conf('postgresql.conf', 'ssl', 'on');
+ $node->reload;
+
+ connect_test($node, 'user=testuser sslmode=disable gssencmode=disable', 'plain');
+ connect_test($node, 'user=testuser sslmode=allow gssencmode=disable', 'plain');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=disable', 'ssl');
+ connect_test($node, 'user=testuser sslmode=require gssencmode=disable', 'ssl');
+
+ connect_test($node, 'user=ssluser sslmode=disable gssencmode=disable', 'fail');
+ connect_test($node, 'user=ssluser sslmode=allow gssencmode=disable', 'ssl');
+ connect_test($node, 'user=ssluser sslmode=prefer gssencmode=disable', 'ssl');
+ connect_test($node, 'user=ssluser sslmode=require gssencmode=disable', 'ssl');
+
+ connect_test($node, 'user=nossluser sslmode=disable gssencmode=disable', 'plain');
+ connect_test($node, 'user=nossluser sslmode=allow gssencmode=disable', 'plain');
+ connect_test($node, 'user=nossluser sslmode=prefer gssencmode=disable', 'plain');
+ connect_test($node, 'user=nossluser sslmode=require gssencmode=disable', 'fail');
+}
+
+# Server supports SSL and GSSAPI
+SKIP:
+{
+ skip "GSSAPI/Kerberos or SSL not supported by this build" unless ($ssl_supported && $gss_supported);
+
+ connect_test($node, 'user=testuser sslmode=disable gssencmode=disable', 'plain');
+ connect_test($node, 'user=testuser sslmode=disable gssencmode=prefer', 'gss');
+ connect_test($node, 'user=testuser sslmode=disable gssencmode=require', 'gss');
+
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=disable', 'ssl');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=require', 'gss');
+
+ connect_test($node, 'user=testuser sslmode=require gssencmode=disable', 'ssl');
+ connect_test($node, 'user=testuser sslmode=require gssencmode=prefer', 'gss');
+
+ # If you set both sslmode and gssencmode to 'require', 'gssencmode=require' takes
+ # precedence.
+ connect_test($node, 'user=testuser sslmode=require gssencmode=require', 'gss');
+
+ # Test case that server supports GSSAPI, but it's not allowed for
+ # this user.
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=require', 'fail',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+
+ # with 'gssencmode=prefer', libpq will first negotiate GSSAPI
+ # encryption, but the connection will fail because pg_hba.conf
+ # forbids GSSAPI encryption for this user. It will then reconnect
+ # with SSL.
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer', 'ssl',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+
+ # Setting both sslmode=require and gssencmode=require fails if GSSAPI is not
+ # available.
+ connect_test($node, 'user=nogssuser sslmode=require gssencmode=require', 'fail');
+}
+
+# Test negotiation over unix domain sockets.
+SKIP:
+{
+ skip "Unix domain sockets not supported" unless ($unixdir ne "");
+
+ connect_test($node, "user=localuser sslmode=require gssencmode=prefer host=$unixdir", 'plain');
+ connect_test($node, "user=localuser sslmode=prefer gssencmode=require host=$unixdir", 'fail');
+}
+
+done_testing();
diff --git a/src/test/perl/PostgreSQL/Test/Kerberos.pm b/src/test/perl/PostgreSQL/Test/Kerberos.pm
index 382a38ec9d2..b9c5c388085 100644
--- a/src/test/perl/PostgreSQL/Test/Kerberos.pm
+++ b/src/test/perl/PostgreSQL/Test/Kerberos.pm
@@ -81,10 +81,10 @@ sub new
my ($stdout, $krb5_version);
run_log [ $krb5_config, '--version' ], '>', \$stdout
- or BAIL_OUT("could not execute krb5-config");
- BAIL_OUT("Heimdal is not supported") if $stdout =~ m/heimdal/;
+ or die("could not execute krb5-config");
+ die("Heimdal is not supported") if $stdout =~ m/heimdal/;
$stdout =~ m/Kerberos 5 release ([0-9]+\.[0-9]+)/
- or BAIL_OUT("could not get Kerberos version");
+ or die("could not get Kerberos version");
$krb5_version = $1;
# Build the krb5.conf to use.
--
2.39.2
v7-0003-Direct-SSL-connections-postmaster-support.patchtext/x-patch; charset=UTF-8; name=v7-0003-Direct-SSL-connections-postmaster-support.patchDownload
From 17ee70be1279807b3cff0f496127f6fc64894f73 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Wed, 15 Mar 2023 15:51:02 -0400
Subject: [PATCH v7 03/12] Direct SSL connections postmaster support
---
src/backend/libpq/be-secure.c | 13 +++
src/backend/libpq/pqcomm.c | 9 +-
src/backend/postmaster/postmaster.c | 135 ++++++++++++++++++++++------
src/include/libpq/libpq-be.h | 10 +++
src/include/libpq/libpq.h | 2 +-
5 files changed, 136 insertions(+), 33 deletions(-)
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 6923c241b99..0e4786cb2b6 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -235,6 +235,19 @@ secure_raw_read(Port *port, void *ptr, size_t len)
{
ssize_t n;
+ /* Read from the "unread" buffered data first. c.f. libpq-be.h */
+ if (port->raw_buf_remaining > 0)
+ {
+ /* consume up to len bytes from the raw_buf */
+ if (len > port->raw_buf_remaining)
+ len = port->raw_buf_remaining;
+ Assert(port->raw_buf);
+ memcpy(ptr, port->raw_buf + port->raw_buf_consumed, len);
+ port->raw_buf_consumed += len;
+ port->raw_buf_remaining -= len;
+ return len;
+ }
+
/*
* Try to read from the socket without blocking. If it succeeds we're
* done, otherwise we'll wait for the socket using the latch mechanism.
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index c606bf34473..ff3a03a2428 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -1136,13 +1136,16 @@ pq_discardbytes(size_t len)
/* --------------------------------
* pq_buffer_has_data - is any buffered data available to read?
*
- * This will *not* attempt to read more data.
+ * Actually returns the number of bytes in the buffer...
+ *
+ * This will *not* attempt to read more data. And reading up to that number of
+ * bytes should not cause reading any more data either.
* --------------------------------
*/
-bool
+size_t
pq_buffer_has_data(void)
{
- return (PqRecvPointer < PqRecvLength);
+ return (PqRecvLength - PqRecvPointer);
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index feb471dd1df..085c940aec5 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -423,6 +423,7 @@ static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(Port *port);
+static int ProcessSSLStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -1926,6 +1927,99 @@ ServerLoop(void)
}
}
+/*
+ * Check for a native direct SSL connection.
+ *
+ * This happens before startup packets so we are careful not to actual read
+ * any bytes from the stream if it's not a direct SSL connection.
+ */
+
+static int
+ProcessSSLStartup(Port *port)
+{
+ int firstbyte;
+
+ pq_startmsgread();
+
+ pq_startmsgread();
+ firstbyte = pq_peekbyte();
+ pq_endmsgread();
+ if (firstbyte == EOF)
+ {
+ /*
+ * If we get no data at all, don't clutter the log with a complaint;
+ * such cases often occur for legitimate reasons. An example is that
+ * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
+ * client didn't like our response, it'll probably just drop the
+ * connection. Service-monitoring software also often just opens and
+ * closes a connection without sending anything. (So do port
+ * scanners, which may be less benign, but it's not really our job to
+ * notice those.)
+ */
+ return STATUS_ERROR;
+ }
+
+ /*
+ * First byte indicates standard SSL handshake message
+ *
+ * (It can't be a Postgres startup length because in network byte order
+ * that would be a startup packet hundreds of megabytes long)
+ */
+ if (firstbyte == 0x16)
+ {
+#ifdef USE_SSL
+ ssize_t len;
+ char *buf = NULL;
+ elog(LOG, "Detected direct SSL handshake");
+
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ buf = palloc(len);
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+
+ Assert(pq_buffer_has_data() == 0);
+ if (secure_open_server(port) == -1)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("SSL Protocol Error during direct SSL connection initiation")));
+ return STATUS_ERROR;
+ }
+
+ if (port->raw_buf_remaining > 0)
+ {
+ /* This shouldn't be possible -- it would mean the client sent
+ * encrypted data before we established a session key...
+ */
+ elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ return STATUS_ERROR;
+ }
+ pfree(port->raw_buf);
+#else
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with no SSL support")));
+ return STATUS_ERROR;
+#endif
+ }
+
+ pq_endmsgread();
+
+ if (port->ssl_in_use)
+ ereport(DEBUG2,
+ (errmsg_internal("Direct SSL connection established")));
+
+ return STATUS_OK;
+}
+
+
/*
* Read a client's startup packet and do something according to it.
*
@@ -1954,28 +2048,7 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
pq_startmsgread();
- /*
- * Grab the first byte of the length word separately, so that we can tell
- * whether we have no data at all or an incomplete packet. (This might
- * sound inefficient, but it's not really, because of buffering in
- * pqcomm.c.)
- */
- if (pq_getbytes((char *) &len, 1) == EOF)
- {
- /*
- * If we get no data at all, don't clutter the log with a complaint;
- * such cases often occur for legitimate reasons. An example is that
- * we might be here after responding to NEGOTIATE_SSL_CODE, and if the
- * client didn't like our response, it'll probably just drop the
- * connection. Service-monitoring software also often just opens and
- * closes a connection without sending anything. (So do port
- * scanners, which may be less benign, but it's not really our job to
- * notice those.)
- */
- return STATUS_ERROR;
- }
-
- if (pq_getbytes(((char *) &len) + 1, 3) == EOF)
+ if (pq_getbytes(((char *) &len), 4) == EOF)
{
/* Got a partial length word, so bleat about that */
if (!ssl_done && !gss_done)
@@ -2039,8 +2112,11 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
char SSLok;
#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ /* No SSL when disabled or on Unix sockets.
+ *
+ * Also no SSL negotiation if we already have a direct SSL connection
+ */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX || port->ssl_in_use)
SSLok = 'N';
else
SSLok = 'S'; /* Support for SSL */
@@ -2048,11 +2124,10 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
SSLok = 'N'; /* No support for SSL */
#endif
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
+ while (secure_write(port, &SSLok, 1) != 1)
{
if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
+ continue; /* if interrupted, just retry */
ereport(COMMERROR,
(errcode_for_socket_access(),
errmsg("failed to send SSL negotiation response: %m")));
@@ -2093,7 +2168,7 @@ retry1:
GSSok = 'G';
#endif
- while (send(port->sock, &GSSok, 1, 0) != 1)
+ while (secure_write(port, &GSSok, 1) != 1)
{
if (errno == EINTR)
continue;
@@ -4347,7 +4422,9 @@ BackendInitialize(Port *port)
* Receive the startup packet (which might turn out to be a cancel request
* packet).
*/
- status = ProcessStartupPacket(port, false, false);
+ status = ProcessSSLStartup(port);
+ if (status == STATUS_OK)
+ status = ProcessStartupPacket(port, false, false);
/*
* If we're going to reject the connection due to database state, say so
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 47d66d55241..3bb80ae073d 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -227,6 +227,16 @@ typedef struct Port
SSL *ssl;
X509 *peer;
#endif
+ /* This is a bit of a hack. The raw_buf is data that was previously read
+ * and buffered in a higher layer but then "unread" and needs to be read
+ * again while establishing an SSL connection via the SSL library layer.
+ *
+ * There's no API to "unread", the upper layer just places the data in the
+ * Port structure in raw_buf and sets raw_buf_remaining to the amount of
+ * bytes unread and raw_buf_consumed to 0.
+ */
+ char *raw_buf;
+ ssize_t raw_buf_consumed, raw_buf_remaining;
} Port;
#ifdef USE_SSL
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 6171a0d17a5..79c5f6b754b 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -80,7 +80,7 @@ extern int pq_getmessage(StringInfo s, int maxlen);
extern int pq_getbyte(void);
extern int pq_peekbyte(void);
extern int pq_getbyte_if_available(unsigned char *c);
-extern bool pq_buffer_has_data(void);
+extern size_t pq_buffer_has_data(void);
extern int pq_putmessage_v2(char msgtype, const char *s, size_t len);
extern bool pq_check_connection(void);
--
2.39.2
v7-0004-WIP-refactorings-to-backend-support.patchtext/x-patch; charset=UTF-8; name=v7-0004-WIP-refactorings-to-backend-support.patchDownload
From 7aec324c254c834008bbcd94b23fc4eba4977512 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 09:04:45 +0200
Subject: [PATCH v7 04/12] WIP: refactorings to backend support
---
src/backend/libpq/be-secure.c | 38 +++++++++++++++++++++--
src/backend/libpq/pqcomm.c | 4 +--
src/backend/postmaster/postmaster.c | 47 ++++++-----------------------
3 files changed, 46 insertions(+), 43 deletions(-)
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 0e4786cb2b6..8c770ba5afb 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -112,18 +112,50 @@ secure_loaded_verify_locations(void)
int
secure_open_server(Port *port)
{
+#ifdef USE_SSL
int r = 0;
+ ssize_t len;
+
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ char *buf = palloc(len);
+
+ pq_startmsgread();
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ pq_endmsgread();
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+ Assert(pq_buffer_has_data() == 0);
-#ifdef USE_SSL
r = be_tls_open_server(port);
+ if (port->raw_buf_remaining > 0)
+ {
+ /* This shouldn't be possible -- it would mean the client sent
+ * encrypted data before we established a session key...
+ */
+ elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ return STATUS_ERROR;
+ }
+ if (port->raw_buf != NULL)
+ {
+ pfree(port->raw_buf);
+ port->raw_buf = NULL;
+ }
+
ereport(DEBUG2,
(errmsg_internal("SSL connection from DN:\"%s\" CN:\"%s\"",
port->peer_dn ? port->peer_dn : "(anonymous)",
port->peer_cn ? port->peer_cn : "(anonymous)")));
-#endif
-
return r;
+#else
+ return 0;
+#endif
}
/*
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index ff3a03a2428..caa502b6373 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -1134,9 +1134,7 @@ pq_discardbytes(size_t len)
}
/* --------------------------------
- * pq_buffer_has_data - is any buffered data available to read?
- *
- * Actually returns the number of bytes in the buffer...
+ * pq_buffer_has_data - return number of bytes in receive buffer
*
* This will *not* attempt to read more data. And reading up to that number of
* bytes should not cause reading any more data either.
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 085c940aec5..b70e41c9e26 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1933,13 +1933,10 @@ ServerLoop(void)
* This happens before startup packets so we are careful not to actual read
* any bytes from the stream if it's not a direct SSL connection.
*/
-
static int
ProcessSSLStartup(Port *port)
{
- int firstbyte;
-
- pq_startmsgread();
+ int firstbyte;
pq_startmsgread();
firstbyte = pq_peekbyte();
@@ -1956,7 +1953,9 @@ ProcessSSLStartup(Port *port)
* scanners, which may be less benign, but it's not really our job to
* notice those.)
*/
- return STATUS_ERROR;
+ // XXX: this is OK as far as this function is concerned. Let ProcessStartupPacket
+ // handle it
+ return STATUS_OK;
}
/*
@@ -1967,51 +1966,25 @@ ProcessSSLStartup(Port *port)
*/
if (firstbyte == 0x16)
{
-#ifdef USE_SSL
- ssize_t len;
- char *buf = NULL;
elog(LOG, "Detected direct SSL handshake");
- /* push unencrypted buffered data back through SSL setup */
- len = pq_buffer_has_data();
- if (len > 0)
- {
- buf = palloc(len);
- if (pq_getbytes(buf, len) == EOF)
- return STATUS_ERROR; /* shouldn't be possible */
- port->raw_buf = buf;
- port->raw_buf_remaining = len;
- port->raw_buf_consumed = 0;
- }
-
- Assert(pq_buffer_has_data() == 0);
- if (secure_open_server(port) == -1)
+#ifdef USE_SSL
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX || port->ssl_in_use)
{
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("SSL Protocol Error during direct SSL connection initiation")));
+ // XXX: Send TLS alert ?
return STATUS_ERROR;
}
-
- if (port->raw_buf_remaining > 0)
+ else if (secure_open_server(port) == -1)
{
- /* This shouldn't be possible -- it would mean the client sent
- * encrypted data before we established a session key...
- */
- elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ /* we assume secure_open_server() sent an appropriate TLS alert already */
return STATUS_ERROR;
}
- pfree(port->raw_buf);
#else
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("Received direct SSL connection request with no SSL support")));
+ // XXX: Send TLS alert ?
return STATUS_ERROR;
#endif
}
- pq_endmsgread();
-
if (port->ssl_in_use)
ereport(DEBUG2,
(errmsg_internal("Direct SSL connection established")));
--
2.39.2
v7-0005-Direct-SSL-connections-client-support.patchtext/x-patch; charset=UTF-8; name=v7-0005-Direct-SSL-connections-client-support.patchDownload
From b643458f2d8fd6fdd5f5356512f1ad3866e8e5e0 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 11:55:16 -0400
Subject: [PATCH v7 05/12] Direct SSL connections client support
---
src/interfaces/libpq/fe-connect.c | 92 +++++++++++++++++++++++++++++--
src/interfaces/libpq/libpq-fe.h | 3 +-
src/interfaces/libpq/libpq-int.h | 3 +
3 files changed, 92 insertions(+), 6 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 79e0b73d618..2c446c15343 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -129,6 +129,7 @@ static int ldapServiceLookup(const char *purl, PQconninfoOption *options,
#define DefaultSSLMode "disable"
#define DefaultSSLCertMode "disable"
#endif
+#define DefaultSSLNegotiation "postgres"
#ifdef ENABLE_GSS
#include "fe-gssapi-common.h"
#define DefaultGSSMode "prefer"
@@ -272,6 +273,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-Mode", "", 12, /* sizeof("verify-full") == 12 */
offsetof(struct pg_conn, sslmode)},
+ {"sslnegotiation", "PGSSLNEGOTIATION", DefaultSSLNegotiation, NULL,
+ "SSL-Negotiation", "", 14, /* strlen("requiredirect") == 14 */
+ offsetof(struct pg_conn, sslnegotiation)},
+
{"sslcompression", "PGSSLCOMPRESSION", "0", NULL,
"SSL-Compression", "", 1,
offsetof(struct pg_conn, sslcompression)},
@@ -1524,10 +1529,36 @@ connectOptions2(PGconn *conn)
}
#endif
}
+
+ /*
+ * validate sslnegotiation option, default is "postgres" for the postgres
+ * style negotiated connection with an extra round trip but more options.
+ */
+ if (conn->sslnegotiation)
+ {
+ if (strcmp(conn->sslnegotiation, "postgres") != 0
+ && strcmp(conn->sslnegotiation, "direct") != 0
+ && strcmp(conn->sslnegotiation, "requiredirect") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
+ "sslnegotiation", conn->sslnegotiation);
+ return false;
+ }
+
+#ifndef USE_SSL
+ if (conn->sslnegotiation[0] != 'p') {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "sslnegotiation value \"%s\" invalid when SSL support is not compiled in",
+ conn->sslnegotiation);
+ return false;
+ }
+#endif
+ }
else
{
- conn->sslmode = strdup(DefaultSSLMode);
- if (!conn->sslmode)
+ conn->sslnegotiation = strdup(DefaultSSLNegotiation);
+ if (!conn->sslnegotiation)
goto oom_error;
}
@@ -1650,6 +1681,18 @@ connectOptions2(PGconn *conn)
conn->gssencmode);
return false;
}
+#endif
+#ifdef USE_SSL
+ /* GSS is incompatible with direct SSL connections so it requires the
+ * default postgres style connection ssl negotiation */
+ if (strcmp(conn->gssencmode, "require") == 0 &&
+ strcmp(conn->sslnegotiation, "postgres") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when Direct SSL negotiation is enabled",
+ conn->gssencmode);
+ return false;
+ }
#endif
}
else
@@ -2783,11 +2826,12 @@ keep_going: /* We will come back to here until there is
/* initialize these values based on SSL mode */
conn->allow_ssl_try = (conn->sslmode[0] != 'd'); /* "disable" */
conn->wait_ssl_try = (conn->sslmode[0] == 'a'); /* "allow" */
+ /* direct ssl is incompatible with "allow" or "disabled" ssl */
+ conn->allow_direct_ssl_try = conn->allow_ssl_try && !conn->wait_ssl_try && (conn->sslnegotiation[0] != 'p');
#endif
#ifdef ENABLE_GSS
conn->try_gss = (conn->gssencmode[0] != 'd'); /* "disable" */
#endif
-
reset_connection_state_machine = false;
need_new_connection = true;
}
@@ -3245,6 +3289,29 @@ keep_going: /* We will come back to here until there is
if (pqsecure_initialize(conn, false, true) < 0)
goto error_return;
+ /*
+ * If SSL is enabled and direct SSL connections are enabled
+ * and we haven't already established an SSL connection (or
+ * already tried a direct connection and failed or succeeded)
+ * then try just enabling SSL directly.
+ *
+ * If we fail then we'll either fail the connection (if
+ * sslnegotiation is set to requiredirect or turn
+ * allow_direct_ssl_try to false
+ */
+ if (conn->allow_ssl_try
+ && !conn->wait_ssl_try
+ && conn->allow_direct_ssl_try
+ && !conn->ssl_in_use
+#ifdef ENABLE_GSS
+ && !conn->gssenc
+#endif
+ )
+ {
+ conn->status = CONNECTION_SSL_STARTUP;
+ return PGRES_POLLING_WRITING;
+ }
+
/*
* If SSL is enabled and we haven't already got encryption of
* some sort running, request SSL instead of sending the
@@ -3321,9 +3388,11 @@ keep_going: /* We will come back to here until there is
/*
* On first time through, get the postmaster's response to our
- * SSL negotiation packet.
+ * SSL negotiation packet. If we are trying a direct ssl
+ * connection skip reading the negotiation packet and go
+ * straight to initiating an ssl connection.
*/
- if (!conn->ssl_in_use)
+ if (!conn->ssl_in_use && !conn->allow_direct_ssl_try)
{
/*
* We use pqReadData here since it has the logic to
@@ -3429,6 +3498,18 @@ keep_going: /* We will come back to here until there is
}
if (pollres == PGRES_POLLING_FAILED)
{
+ /* Failed direct ssl connection, possibly try a new connection with postgres negotiation */
+ if (conn->allow_direct_ssl_try)
+ {
+ /* if it's requiredirect then it's a hard failure */
+ if (conn->sslnegotiation[0] == 'r')
+ goto error_return;
+ /* otherwise only retry using postgres connection */
+ conn->allow_direct_ssl_try = false;
+ need_new_connection = true;
+ goto keep_going;
+ }
+
/*
* Failed ... if sslmode is "prefer" then do a non-SSL
* retry
@@ -4436,6 +4517,7 @@ freePGconn(PGconn *conn)
free(conn->keepalives_interval);
free(conn->keepalives_count);
free(conn->sslmode);
+ free(conn->sslnegotiation);
free(conn->sslcert);
free(conn->sslkey);
if (conn->sslpassword)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index f0ec660cb69..3a4e8ae4080 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,8 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_DIRECT_SSL_STARTUP /* Starting SSL without PG Negotiation. */
} ConnStatusType;
typedef enum
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index f0143726bbc..c5391bd926c 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -388,6 +388,7 @@ struct pg_conn
char *keepalives_count; /* maximum number of TCP keepalive
* retransmits */
char *sslmode; /* SSL mode (require,prefer,allow,disable) */
+ char *sslnegotiation; /* SSL initiation style (postgres,direct,requiredirect) */
char *sslcompression; /* SSL compression (0 or 1) */
char *sslkey; /* client key filename */
char *sslcert; /* client certificate filename */
@@ -549,6 +550,8 @@ struct pg_conn
bool ssl_cert_sent; /* Did we send one in reply? */
#ifdef USE_SSL
+ bool allow_direct_ssl_try; /* Try to make a direct SSL connection
+ * without an "SSL negotiation packet" */
bool allow_ssl_try; /* Allowed to try SSL negotiation */
bool wait_ssl_try; /* Delay SSL negotiation until after
* attempting normal connection */
--
2.39.2
v7-0006-Direct-SSL-connections-documentation.patchtext/x-patch; charset=UTF-8; name=v7-0006-Direct-SSL-connections-documentation.patchDownload
From 88a1ee68da9de3cfc48650f9c67d454335272c53 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Thu, 16 Mar 2023 15:10:15 -0400
Subject: [PATCH v7 06/12] Direct SSL connections documentation
---
doc/src/sgml/libpq.sgml | 102 ++++++++++++++++++++++++++++++++++++----
1 file changed, 93 insertions(+), 9 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 21195e0e728..3861045f1a9 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1691,10 +1691,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
Note that if <acronym>GSSAPI</acronym> encryption is possible,
that will be used in preference to <acronym>SSL</acronym>
encryption, regardless of the value of <literal>sslmode</literal>.
- To force use of <acronym>SSL</acronym> encryption in an
- environment that has working <acronym>GSSAPI</acronym>
- infrastructure (such as a Kerberos server), also
- set <literal>gssencmode</literal> to <literal>disable</literal>.
+ To negotiate <acronym>SSL</acronym> encryption in an environment that
+ has working <acronym>GSSAPI</acronym> infrastructure (such as a
+ Kerberos server), also set <literal>gssencmode</literal>
+ to <literal>disable</literal>.
+ Use of non-default values of <literal>sslnegotiation</literal> can
+ also cause <acronym>SSL</acronym> to be used instead of
+ negotiating <acronym>GSSAPI</acronym> encryption.
</para>
</listitem>
</varlistentry>
@@ -1721,6 +1724,75 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
</listitem>
</varlistentry>
+ <varlistentry id="libpq-connect-sslnegotiation" xreflabel="sslnegotiation">
+ <term><literal>sslnegotiation</literal></term>
+ <listitem>
+ <para>
+ This option controls whether <productname>PostgreSQL</productname>
+ will perform its protocol negotiation to request encryption from the
+ server or will just directly make a standard <acronym>SSL</acronym>
+ connection. Traditional <productname>PostgreSQL</productname>
+ protocol negotiation is the default and the most flexible with
+ different server configurations. If the server is known to support
+ direct <acronym>SSL</acronym> connections then the latter requires one
+ fewer round trip reducing connection latency and also allows the use
+ of protocol agnostic SSL network tools.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>postgres</literal></term>
+ <listitem>
+ <para>
+ perform <productname>PostgreSQL</productname> protocol
+ negotiation. This is the default if the option is not provided.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>direct</literal></term>
+ <listitem>
+ <para>
+ first attempt to establish a standard SSL connection and if that
+ fails reconnect and perform the negotiation. This fallback
+ process adds significant latency if the initial SSL connection
+ fails.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>requiredirect</literal></term>
+ <listitem>
+ <para>
+ attempt to establish a standard SSL connection and if that fails
+ return a connection failure immediately.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ If <literal>sslmode</literal> set to <literal>disable</literal>
+ or <literal>allow</literal> then <literal>sslnegotiation</literal> is
+ ignored. If <literal>gssencmode</literal> is set
+ to <literal>require</literal> then <literal>sslnegotiation</literal>
+ must be the default <literal>postgres</literal> value.
+ </para>
+
+ <para>
+ Moreover, note if <literal>gssencmode</literal> is set
+ to <literal>prefer</literal> and <literal>sslnegotiation</literal>
+ to <literal>direct</literal> then the effective preference will be
+ direct <acronym>SSL</acronym> connections, followed by
+ negotiated <acronym>GSS</acronym> connections, followed by
+ negotiated <acronym>SSL</acronym> connections, possibly followed
+ lastly by unencrypted connections.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-connect-sslcompression" xreflabel="sslcompression">
<term><literal>sslcompression</literal></term>
<listitem>
@@ -1954,11 +2026,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
<para>
The Server Name Indication can be used by SSL-aware proxies to route
- connections without having to decrypt the SSL stream. (Note that this
- requires a proxy that is aware of the PostgreSQL protocol handshake,
- not just any SSL proxy.) However, <acronym>SNI</acronym> makes the
- destination host name appear in cleartext in the network traffic, so
- it might be undesirable in some cases.
+ connections without having to decrypt the SSL stream. (Note that
+ unless the proxy is aware of the PostgreSQL protocol handshake this
+ would require setting <literal>sslnegotiation</literal>
+ to <literal>direct</literal> or <literal>requiredirect</literal>.)
+ However, <acronym>SNI</acronym> makes the destination host name appear
+ in cleartext in the network traffic, so it might be undesirable in
+ some cases.
</para>
</listitem>
</varlistentry>
@@ -8121,6 +8195,16 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
</para>
</listitem>
+ <listitem>
+ <para>
+ <indexterm>
+ <primary><envar>PGSSLNEGOTIATION</envar></primary>
+ </indexterm>
+ <envar>PGSSLNEGOTIATION</envar> behaves the same as the <xref
+ linkend="libpq-connect-sslnegotiation"/> connection parameter.
+ </para>
+ </listitem>
+
<listitem>
<para>
<indexterm>
--
2.39.2
v7-0007-Direct-SSL-connections-ALPN-support.patchtext/x-patch; charset=UTF-8; name=v7-0007-Direct-SSL-connections-ALPN-support.patchDownload
From 206352b38a31bbc29e79c2ad79fe564a5e9f5ff0 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Mon, 20 Mar 2023 14:09:30 -0400
Subject: [PATCH v7 07/12] Direct SSL connections ALPN support
---
src/backend/libpq/be-secure-openssl.c | 66 +++++++++++++++++++
src/backend/libpq/be-secure.c | 3 +
src/backend/postmaster/postmaster.c | 26 ++++++++
src/backend/utils/misc/guc_tables.c | 9 +++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/bin/psql/command.c | 7 +-
src/include/libpq/libpq-be.h | 1 +
src/include/libpq/libpq.h | 1 +
src/include/libpq/pqcomm.h | 19 ++++++
src/interfaces/libpq/fe-connect.c | 5 ++
src/interfaces/libpq/fe-secure-openssl.c | 31 +++++++++
src/interfaces/libpq/libpq-int.h | 1 +
12 files changed, 168 insertions(+), 2 deletions(-)
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
index e12b1cc9e3b..d3c8b4117fb 100644
--- a/src/backend/libpq/be-secure-openssl.c
+++ b/src/backend/libpq/be-secure-openssl.c
@@ -67,6 +67,12 @@ static int ssl_external_passwd_cb(char *buf, int size, int rwflag, void *userdat
static int dummy_ssl_passwd_cb(char *buf, int size, int rwflag, void *userdata);
static int verify_cb(int ok, X509_STORE_CTX *ctx);
static void info_cb(const SSL *ssl, int type, int args);
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata);
static bool initialize_dh(SSL_CTX *context, bool isServerStart);
static bool initialize_ecdh(SSL_CTX *context, bool isServerStart);
static const char *SSLerrmessage(unsigned long ecode);
@@ -432,6 +438,13 @@ be_tls_open_server(Port *port)
/* set up debugging/info callback */
SSL_CTX_set_info_callback(SSL_context, info_cb);
+ if (ssl_enable_alpn) {
+ elog(DEBUG2, "Enabling OpenSSL ALPN callback");
+ SSL_CTX_set_alpn_select_cb(SSL_context, alpn_cb, port);
+ } else {
+ elog(DEBUG2, "OpenSSL ALPN is disabled, not setting callback");
+ }
+
if (!(port->ssl = SSL_new(SSL_context)))
{
ereport(COMMERROR,
@@ -702,6 +715,12 @@ be_tls_close(Port *port)
pfree(port->peer_dn);
port->peer_dn = NULL;
}
+
+ if (port->ssl_alpn_protocol)
+ {
+ pfree(port->ssl_alpn_protocol);
+ port->ssl_alpn_protocol = NULL;
+ }
}
ssize_t
@@ -1259,6 +1278,53 @@ info_cb(const SSL *ssl, int type, int args)
}
}
+/* See pqcomm.h comments on OpenSSL implementation of ALPN (RFC 7301) */
+static unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
+
+/*
+ * Server callback for ALPN negotiation. We use use the standard "helper"
+ * function even though currently we only accept one value. We store the
+ * negotiated protocol in Port->ssl_alpn_protocol and rely on higher level
+ * logic (in postmaster.c) to decide what to do with that info.
+ */
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata)
+{
+ /* Why does OpenSSL provide a helper function that requires a nonconst
+ * vector when the callback is declared to take a const vector? What are
+ * we to do with that?
+ */
+ int retval;
+ Assert(userdata != NULL);
+ Assert(out != NULL);
+ Assert(outlen != NULL);
+ Assert(in != NULL);
+
+ retval = SSL_select_next_proto((unsigned char **)out, outlen,
+ alpn_protos, sizeof(alpn_protos),
+ in, inlen);
+ if (*out == NULL || *outlen > sizeof(alpn_protos) || outlen <= 0)
+ return SSL_TLSEXT_ERR_NOACK; /* can't happen */
+
+ if (retval == OPENSSL_NPN_NEGOTIATED)
+ {
+ struct Port *port = (struct Port *)userdata;
+ char *alpn_protocol = MemoryContextAllocZero(TopMemoryContext, *outlen+1);
+ memcpy(alpn_protocol, *out, *outlen);
+ port->ssl_alpn_protocol = alpn_protocol;
+ return SSL_TLSEXT_ERR_OK;
+ } else if (retval == OPENSSL_NPN_NO_OVERLAP) {
+ return SSL_TLSEXT_ERR_NOACK;
+ } else {
+ return SSL_TLSEXT_ERR_NOACK;
+ }
+}
+
+
/*
* Set DH parameters for generating ephemeral DH keys. The
* DH parameters can take a long time to compute, so they must be
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 8c770ba5afb..9f2c1fce19c 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -61,6 +61,9 @@ bool SSLPreferServerCiphers;
int ssl_min_protocol_version = PG_TLS1_2_VERSION;
int ssl_max_protocol_version = PG_TLS_ANY;
+/* GUC variable: if false ignore ALPN negotiation */
+bool ssl_enable_alpn = true;
+
/* ------------------------------------------------------------ */
/* Procedures common to all secure sessions */
/* ------------------------------------------------------------ */
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index b70e41c9e26..3c47bfb7e54 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1933,6 +1933,11 @@ ServerLoop(void)
* This happens before startup packets so we are careful not to actual read
* any bytes from the stream if it's not a direct SSL connection.
*/
+
+#ifdef USE_SSL
+static const char *expected_alpn_protocol = PG_ALPN_PROTOCOL;
+#endif
+
static int
ProcessSSLStartup(Port *port)
{
@@ -1968,6 +1973,10 @@ ProcessSSLStartup(Port *port)
{
elog(LOG, "Detected direct SSL handshake");
+ if (!ssl_enable_alpn) {
+ elog(WARNING, "Received direct SSL connection without ssl_enable_alpn enabled");
+ }
+
#ifdef USE_SSL
if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX || port->ssl_in_use)
{
@@ -1979,6 +1988,23 @@ ProcessSSLStartup(Port *port)
/* we assume secure_open_server() sent an appropriate TLS alert already */
return STATUS_ERROR;
}
+
+ if (port->ssl_alpn_protocol == NULL)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request without required ALPN protocol negotiation extension")));
+ return STATUS_ERROR;
+ }
+ if (strcmp(port->ssl_alpn_protocol, expected_alpn_protocol) != 0)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request with unexpected ALPN protocol \"%s\" expected \"%s\"",
+ port->ssl_alpn_protocol,
+ expected_alpn_protocol)));
+ return STATUS_ERROR;
+ }
#else
// XXX: Send TLS alert ?
return STATUS_ERROR;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index e53ebc6dc2b..ad95b064b31 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1120,6 +1120,15 @@ struct config_bool ConfigureNamesBool[] =
true,
NULL, NULL, NULL
},
+ {
+ {"ssl_enable_alpn", PGC_SIGHUP, CONN_AUTH_SSL,
+ gettext_noop("Respond to TLS ALPN Extension Requests."),
+ NULL,
+ },
+ &ssl_enable_alpn,
+ true,
+ NULL, NULL, NULL
+ },
{
{"fsync", PGC_SIGHUP, WAL_SETTINGS,
gettext_noop("Forces synchronization of updates to disk."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index b2809c711a1..32e764e1daa 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -118,6 +118,7 @@
#ssl_dh_params_file = ''
#ssl_passphrase_command = ''
#ssl_passphrase_command_supports_reload = off
+#ssl_enable_alpn = on
#------------------------------------------------------------------------------
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index eb216b7c09e..31802f23e97 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -3819,6 +3819,7 @@ printSSLInfo(void)
const char *protocol;
const char *cipher;
const char *compression;
+ const char *alpn;
if (!PQsslInUse(pset.db))
return; /* no SSL */
@@ -3826,11 +3827,13 @@ printSSLInfo(void)
protocol = PQsslAttribute(pset.db, "protocol");
cipher = PQsslAttribute(pset.db, "cipher");
compression = PQsslAttribute(pset.db, "compression");
+ alpn = PQsslAttribute(pset.db, "alpn");
- printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s)\n"),
+ printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s, ALPN: %s)\n"),
protocol ? protocol : _("unknown"),
cipher ? cipher : _("unknown"),
- (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"));
+ (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"),
+ alpn ? alpn : _("none"));
}
/*
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 3bb80ae073d..e947e6f7cf0 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -218,6 +218,7 @@ typedef struct Port
char *peer_cn;
char *peer_dn;
bool peer_cert_valid;
+ char *ssl_alpn_protocol;
/*
* OpenSSL structures. (Keep these last so that the locations of other
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 79c5f6b754b..b65b32f2093 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -123,6 +123,7 @@ extern PGDLLIMPORT char *SSLECDHCurve;
extern PGDLLIMPORT bool SSLPreferServerCiphers;
extern PGDLLIMPORT int ssl_min_protocol_version;
extern PGDLLIMPORT int ssl_max_protocol_version;
+extern PGDLLIMPORT bool ssl_enable_alpn;
enum ssl_protocol_versions
{
diff --git a/src/include/libpq/pqcomm.h b/src/include/libpq/pqcomm.h
index 9ae469c86c4..a8fb6a043cc 100644
--- a/src/include/libpq/pqcomm.h
+++ b/src/include/libpq/pqcomm.h
@@ -139,6 +139,25 @@ typedef struct CancelRequestPacket
uint32 cancelAuthCode; /* secret key to authorize cancel */
} CancelRequestPacket;
+/* Application-Layer Protocol Negotiation is required for direct connections
+ * to avoid protocol confusion attacks (e.g https://alpaca-attack.com/).
+ *
+ * ALPN is specified in RFC 7301
+ *
+ * This string should be registered at:
+ * https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids
+ *
+ * OpenSSL uses this wire-format for the list of alpn protocols even in the
+ * API. Both server and client take the same format parameter but the client
+ * actually sends it to the server as-is and the server it specifies the
+ * preference order to use to choose the one selected to send back.
+ *
+ * c.f. https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_alpn_select_cb.html
+ *
+ * The #define can be used to initialize a char[] vector to use directly in the API
+ */
+#define PG_ALPN_PROTOCOL "TBD-pgsql"
+#define PG_ALPN_PROTOCOL_VECTOR { 9, 'T','B','D','-','p','g','s','q','l' }
/*
* A client can also start by sending a SSL or GSSAPI negotiation request to
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 2c446c15343..2025cace2fd 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -313,6 +313,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-SNI", "", 1,
offsetof(struct pg_conn, sslsni)},
+ {"sslalpn", "PGSSLALPN", "1", NULL,
+ "SSL-ALPN", "", 1,
+ offsetof(struct pg_conn, sslalpn)},
+
{"requirepeer", "PGREQUIREPEER", NULL, NULL,
"Require-Peer", "", 10,
offsetof(struct pg_conn, requirepeer)},
@@ -4531,6 +4535,7 @@ freePGconn(PGconn *conn)
free(conn->sslcrldir);
free(conn->sslcompression);
free(conn->sslsni);
+ free(conn->sslalpn);
free(conn->requirepeer);
free(conn->require_auth);
free(conn->ssl_min_protocol_version);
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 6bc216956d1..fcd402dc6b7 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -905,6 +905,9 @@ destroy_ssl_system(void)
#endif
}
+/* See pqcomm.h comments on OpenSSL implementation of ALPN (RFC 7301) */
+static unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
+
/*
* Create per-connection SSL object, and load the client certificate,
* private key, and trusted CA certs.
@@ -1254,6 +1257,20 @@ initialize_SSL(PGconn *conn)
}
}
+ if (conn->sslalpn && conn->sslalpn[0] == '1')
+ {
+ int retval;
+ retval = SSL_set_alpn_protos(conn->ssl, alpn_protos, sizeof(alpn_protos));
+
+ if (retval != 0)
+ {
+ char *err = SSLerrmessage(ERR_get_error());
+ libpq_append_conn_error(conn, "could not set ssl alpn extension: %s", err);
+ SSLerrfree(err);
+ return -1;
+ }
+ }
+
/*
* Read the SSL key. If a key is specified, treat it as an engine:key
* combination if there is colon present - we don't support files with
@@ -1756,6 +1773,7 @@ PQsslAttributeNames(PGconn *conn)
"cipher",
"compression",
"protocol",
+ "alpn",
NULL
};
static const char *const empty_attrs[] = {NULL};
@@ -1810,6 +1828,19 @@ PQsslAttribute(PGconn *conn, const char *attribute_name)
if (strcmp(attribute_name, "protocol") == 0)
return SSL_get_version(conn->ssl);
+ if (strcmp(attribute_name, "alpn") == 0)
+ {
+ const unsigned char *data;
+ unsigned int len;
+ static char alpn_str[256]; /* alpn doesn't support longer than 255 bytes */
+ SSL_get0_alpn_selected(conn->ssl, &data, &len);
+ if (data == NULL || len==0 || len > sizeof(alpn_str)-1)
+ return NULL;
+ memcpy(alpn_str, data, len);
+ alpn_str[len] = 0;
+ return alpn_str;
+ }
+
return NULL; /* unknown attribute */
}
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index c5391bd926c..6daf34b9afe 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -398,6 +398,7 @@ struct pg_conn
char *sslcrl; /* certificate revocation list filename */
char *sslcrldir; /* certificate revocation list directory name */
char *sslsni; /* use SSL SNI extension (0 or 1) */
+ char *sslalpn; /* use SSL ALPN extension (0 or 1) */
char *requirepeer; /* required peer credentials for local sockets */
char *gssencmode; /* GSS mode (require,prefer,disable) */
char *krbsrvname; /* Kerberos service name */
--
2.39.2
v7-0008-Allow-pipelining-data-after-ssl-request.patchtext/x-patch; charset=UTF-8; name=v7-0008-Allow-pipelining-data-after-ssl-request.patchDownload
From 682c7cdb1cd7c02b3376126e44dbfa753074a3e7 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Fri, 31 Mar 2023 02:31:31 -0400
Subject: [PATCH v7 08/12] Allow pipelining data after ssl request
---
src/backend/libpq/be-secure.c | 8 +++---
src/backend/postmaster/postmaster.c | 43 ++++++++++++++++++++++++-----
2 files changed, 40 insertions(+), 11 deletions(-)
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 9f2c1fce19c..45ae514d9ab 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -139,10 +139,10 @@ secure_open_server(Port *port)
if (port->raw_buf_remaining > 0)
{
- /* This shouldn't be possible -- it would mean the client sent
- * encrypted data before we established a session key...
- */
- elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after SSL request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
return STATUS_ERROR;
}
if (port->raw_buf != NULL)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 3c47bfb7e54..872bbd79f09 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2134,15 +2134,44 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
}
#ifdef USE_SSL
- if (SSLok == 'S' && secure_open_server(port) == -1)
- return STATUS_ERROR;
+ if (SSLok == 'S')
+ {
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ buf = palloc(len);
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+ Assert(pq_buffer_has_data() == 0);
+
+ if (secure_open_server(port) == -1)
+ return STATUS_ERROR;
+
+ /*
+ * At this point we should have no data already buffered. If we do,
+ * it was received before we performed the SSL handshake, so it wasn't
+ * encrypted and indeed may have been injected by a man-in-the-middle.
+ * We report this case to the client.
+ */
+ if (port->raw_buf_remaining > 0)
+ ereport(FATAL,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("received unencrypted data after SSL request"),
+ errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
+ if (port->raw_buf)
+ pfree(port->raw_buf);
+ }
#endif
- /*
- * At this point we should have no data already buffered. If we do,
- * it was received before we performed the SSL handshake, so it wasn't
- * encrypted and indeed may have been injected by a man-in-the-middle.
- * We report this case to the client.
+ /* This can only really occur now if there was data pipelined after
+ * the SSL Request but we have refused to do SSL. In that case we need
+ * to give up because the client has presumably assumed the SSL
+ * request would be accepted.
*/
if (pq_buffer_has_data())
ereport(FATAL,
--
2.39.2
v7-0009-Direct-SSL-connections-some-additional-docs.patchtext/x-patch; charset=UTF-8; name=v7-0009-Direct-SSL-connections-some-additional-docs.patchDownload
From ace7b10e775cba2df69ced99e0c680ac3edfa990 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Fri, 31 Mar 2023 03:01:35 -0400
Subject: [PATCH v7 09/12] Direct SSL connections some additional docs
---
doc/src/sgml/protocol.sgml | 37 +++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 6c3e8a631d7..7ab95987414 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -1532,17 +1532,54 @@ SELCT 1/0;<!-- this typo is intentional -->
bytes.
</para>
+ <para>
+ Likewise the server expects the client to not begin
+ the <acronym>SSL</acronym> negotiation until it receives the server's
+ single byte response to the <acronym>SSL</acronym> request. If the
+ client begins the <acronym>SSL</acronym> negotiation immediately without
+ waiting for the server response to be received it can reduce connection
+ latency by one round-trip. However this comes at the cost of not being
+ able to handle the case where the server sends a negative response to the
+ <acronym>SSL</acronym> request. In that case instead of continuing with either GSSAPI or an
+ unencrypted connection or a protocol error the server will simply
+ disconnect.
+ </para>
+
<para>
An initial SSLRequest can also be used in a connection that is being
opened to send a CancelRequest message.
</para>
+ <para>
+ A second alternate way to initiate <acronym>SSL</acronym> encryption is
+ available. The server will recognize connections which immediately
+ begin <acronym>SSL</acronym> negotiation without any previous SSLRequest
+ packets. Once the <acronym>SSL</acronym> connection is established the
+ server will expect a normal startup-request packet and continue
+ negotiation over the encrypted channel. In this case any other requests
+ for encryption will be refused. This method is not preferred for general
+ purpose tools as it cannot negotiate the best connection encryption
+ available or handle unencrypted connections. However it is useful for
+ environments where both the server and client are controlled together.
+ In that case it avoids one round trip of latency and allows the use of
+ network tools that depend on standard <acronym>SSL</acronym> connections.
+ When using <acronym>SSL</acronym> connections in this style the client is
+ required to use the ALPN extension defined
+ by <ulink url="https://tools.ietf.org/html/rfc7301">RFC 7301</ulink> to
+ protect against protocol confusion attacks.
+ The <productname>PostgreSQL</productname> protocol is "TBD-pgsql" as
+ registered
+ at <ulink url="https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids">IANA
+ TLS ALPN Protocol IDs</ulink> registry.
+ </para>
+
<para>
While the protocol itself does not provide a way for the server to
force <acronym>SSL</acronym> encryption, the administrator can
configure the server to reject unencrypted sessions as a byproduct
of authentication checking.
</para>
+
</sect2>
<sect2 id="protocol-flow-gssapi">
--
2.39.2
v7-0010-Move-code-to-check-for-alpn.patchtext/x-patch; charset=UTF-8; name=v7-0010-Move-code-to-check-for-alpn.patchDownload
From 3b1955bae494593dd9e7479ee64d854c4b5dfda7 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 09:05:29 +0200
Subject: [PATCH v7 10/12] Move code to check for alpn
- if non-direct SSL is used, and client sends an unexpected ALPN
protocol, that's now an error ?
---
src/backend/libpq/be-secure-openssl.c | 49 +++++++++++++++++----------
src/backend/postmaster/postmaster.c | 18 ++--------
src/include/libpq/libpq-be.h | 2 +-
3 files changed, 35 insertions(+), 34 deletions(-)
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
index d3c8b4117fb..efe59f672d2 100644
--- a/src/backend/libpq/be-secure-openssl.c
+++ b/src/backend/libpq/be-secure-openssl.c
@@ -584,6 +584,32 @@ aloop:
return -1;
}
+ /* Get protocol selected by ALPN */
+ port->alpn_used = false;
+ {
+ const unsigned char *selected;
+ unsigned int len;
+
+ SSL_get0_alpn_selected(port->ssl, &selected, &len);
+
+ /* If ALPN is used, check that we negotiated the expected protocol */
+ if (selected != NULL)
+ {
+ if (len == strlen(PG_ALPN_PROTOCOL) &&
+ memcmp(selected, PG_ALPN_PROTOCOL, strlen(PG_ALPN_PROTOCOL)) == 0)
+ {
+ port->alpn_used = true;
+ }
+ else
+ {
+ /* shouldn't happen */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received SSL connection request with unexpected ALPN protocol")));
+ }
+ }
+ }
+
/* Get client certificate, if available. */
port->peer = SSL_get_peer_certificate(port->ssl);
@@ -715,12 +741,6 @@ be_tls_close(Port *port)
pfree(port->peer_dn);
port->peer_dn = NULL;
}
-
- if (port->ssl_alpn_protocol)
- {
- pfree(port->ssl_alpn_protocol);
- port->ssl_alpn_protocol = NULL;
- }
}
ssize_t
@@ -1279,7 +1299,7 @@ info_cb(const SSL *ssl, int type, int args)
}
/* See pqcomm.h comments on OpenSSL implementation of ALPN (RFC 7301) */
-static unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
+static const unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
/*
* Server callback for ALPN negotiation. We use use the standard "helper"
@@ -1298,30 +1318,25 @@ static int alpn_cb(SSL *ssl,
* vector when the callback is declared to take a const vector? What are
* we to do with that?
*/
- int retval;
+ int retval;
+
Assert(userdata != NULL);
Assert(out != NULL);
Assert(outlen != NULL);
Assert(in != NULL);
- retval = SSL_select_next_proto((unsigned char **)out, outlen,
+ retval = SSL_select_next_proto((unsigned char **) out, outlen,
alpn_protos, sizeof(alpn_protos),
in, inlen);
if (*out == NULL || *outlen > sizeof(alpn_protos) || outlen <= 0)
return SSL_TLSEXT_ERR_NOACK; /* can't happen */
if (retval == OPENSSL_NPN_NEGOTIATED)
- {
- struct Port *port = (struct Port *)userdata;
- char *alpn_protocol = MemoryContextAllocZero(TopMemoryContext, *outlen+1);
- memcpy(alpn_protocol, *out, *outlen);
- port->ssl_alpn_protocol = alpn_protocol;
return SSL_TLSEXT_ERR_OK;
- } else if (retval == OPENSSL_NPN_NO_OVERLAP) {
+ else if (retval == OPENSSL_NPN_NO_OVERLAP)
return SSL_TLSEXT_ERR_NOACK;
- } else {
+ else
return SSL_TLSEXT_ERR_NOACK;
- }
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 872bbd79f09..3ed6134941e 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1933,11 +1933,6 @@ ServerLoop(void)
* This happens before startup packets so we are careful not to actual read
* any bytes from the stream if it's not a direct SSL connection.
*/
-
-#ifdef USE_SSL
-static const char *expected_alpn_protocol = PG_ALPN_PROTOCOL;
-#endif
-
static int
ProcessSSLStartup(Port *port)
{
@@ -1989,22 +1984,13 @@ ProcessSSLStartup(Port *port)
return STATUS_ERROR;
}
- if (port->ssl_alpn_protocol == NULL)
+ if (!port->alpn_used)
{
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("Received direct SSL connection request without required ALPN protocol negotiation extension")));
return STATUS_ERROR;
}
- if (strcmp(port->ssl_alpn_protocol, expected_alpn_protocol) != 0)
- {
- ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("Received direct SSL connection request with unexpected ALPN protocol \"%s\" expected \"%s\"",
- port->ssl_alpn_protocol,
- expected_alpn_protocol)));
- return STATUS_ERROR;
- }
#else
// XXX: Send TLS alert ?
return STATUS_ERROR;
@@ -2151,7 +2137,7 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
if (secure_open_server(port) == -1)
return STATUS_ERROR;
-
+
/*
* At this point we should have no data already buffered. If we do,
* it was received before we performed the SSL handshake, so it wasn't
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index e947e6f7cf0..44ad2a0848f 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -218,7 +218,7 @@ typedef struct Port
char *peer_cn;
char *peer_dn;
bool peer_cert_valid;
- char *ssl_alpn_protocol;
+ bool alpn_used;
/*
* OpenSSL structures. (Keep these last so that the locations of other
--
2.39.2
v7-0011-WIP-refactor-state-machine-in-libpq.patchtext/x-patch; charset=UTF-8; name=v7-0011-WIP-refactor-state-machine-in-libpq.patchDownload
From af74ca811d0677f2efe85a1760e0714a1a9a28e2 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 09:45:34 +0200
Subject: [PATCH v7 11/12] WIP: refactor state machine in libpq
---
src/interfaces/libpq/fe-connect.c | 447 +++++++++++++----------
src/interfaces/libpq/fe-secure-openssl.c | 12 +-
src/interfaces/libpq/libpq-fe.h | 3 +-
src/interfaces/libpq/libpq-int.h | 18 +-
4 files changed, 268 insertions(+), 212 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 2025cace2fd..e9c538f3ec8 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -399,6 +399,10 @@ static bool connectOptions1(PGconn *conn, const char *conninfo);
static bool connectOptions2(PGconn *conn);
static int connectDBStart(PGconn *conn);
static int connectDBComplete(PGconn *conn);
+static bool init_allowed_encryption_methods(PGconn *conn);
+static int encryption_negotiation_failed(PGconn *conn);
+static bool connection_failed(PGconn *conn);
+static bool select_next_encryption_method(PGconn *conn, bool negotiation_failure);
static PGPing internal_ping(PGconn *conn);
static PGconn *makeEmptyPGconn(void);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
@@ -1685,18 +1689,6 @@ connectOptions2(PGconn *conn)
conn->gssencmode);
return false;
}
-#endif
-#ifdef USE_SSL
- /* GSS is incompatible with direct SSL connections so it requires the
- * default postgres style connection ssl negotiation */
- if (strcmp(conn->gssencmode, "require") == 0 &&
- strcmp(conn->sslnegotiation, "postgres") != 0)
- {
- conn->status = CONNECTION_BAD;
- libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when Direct SSL negotiation is enabled",
- conn->gssencmode);
- return false;
- }
#endif
}
else
@@ -2826,16 +2818,9 @@ keep_going: /* We will come back to here until there is
*/
conn->pversion = PG_PROTOCOL(3, 0);
conn->send_appname = true;
-#ifdef USE_SSL
- /* initialize these values based on SSL mode */
- conn->allow_ssl_try = (conn->sslmode[0] != 'd'); /* "disable" */
- conn->wait_ssl_try = (conn->sslmode[0] == 'a'); /* "allow" */
- /* direct ssl is incompatible with "allow" or "disabled" ssl */
- conn->allow_direct_ssl_try = conn->allow_ssl_try && !conn->wait_ssl_try && (conn->sslnegotiation[0] != 'p');
-#endif
-#ifdef ENABLE_GSS
- conn->try_gss = (conn->gssencmode[0] != 'd'); /* "disable" */
-#endif
+ conn->failed_enc_methods = 0;
+ conn->current_enc_method = 0;
+ conn->allowed_enc_methods = 0;
reset_connection_state_machine = false;
need_new_connection = true;
}
@@ -2861,6 +2846,32 @@ keep_going: /* We will come back to here until there is
need_new_connection = false;
}
+#define ENCRYPTION_NEGOTIATION_FAILED() \
+ do { \
+ switch (encryption_negotiation_failed(conn)) \
+ { \
+ case 0: \
+ goto error_return; \
+ case 1: \
+ conn->status = CONNECTION_MADE; \
+ return PGRES_POLLING_WRITING; \
+ case 2: \
+ need_new_connection = true; \
+ goto keep_going; \
+ } \
+ } while(0);
+
+#define CONNECTION_FAILED() \
+ do { \
+ if (connection_failed(conn)) \
+ { \
+ need_new_connection = true; \
+ goto keep_going; \
+ } \
+ else \
+ goto error_return; \
+ } while(0);
+
/* Now try to advance the state machine for this connection */
switch (conn->status)
{
@@ -3175,18 +3186,6 @@ keep_going: /* We will come back to here until there is
goto error_return;
}
- /*
- * Make sure we can write before advancing to next step.
- */
- conn->status = CONNECTION_MADE;
- return PGRES_POLLING_WRITING;
- }
-
- case CONNECTION_MADE:
- {
- char *startpacket;
- int packetlen;
-
/*
* Implement requirepeer check, if requested and it's a
* Unix-domain socket.
@@ -3235,30 +3234,31 @@ keep_going: /* We will come back to here until there is
#endif /* WIN32 */
}
- if (conn->raddr.addr.ss_family == AF_UNIX)
- {
- /* Don't request SSL or GSSAPI over Unix sockets */
-#ifdef USE_SSL
- conn->allow_ssl_try = false;
-#endif
-#ifdef ENABLE_GSS
- conn->try_gss = false;
-#endif
- }
+ /* Choose encryption method to try first */
+ if (!init_allowed_encryption_methods(conn))
+ goto error_return;
+
+ /*
+ * Make sure we can write before advancing to next step.
+ */
+ conn->status = CONNECTION_MADE;
+ return PGRES_POLLING_WRITING;
+ }
+
+ case CONNECTION_MADE:
+ {
+ char *startpacket;
+ int packetlen;
#ifdef ENABLE_GSS
/*
- * If GSSAPI encryption is enabled, then call
- * pg_GSS_have_cred_cache() which will return true if we can
- * acquire credentials (and give us a handle to use in
- * conn->gcred), and then send a packet to the server asking
- * for GSSAPI Encryption (and skip past SSL negotiation and
- * regular startup below).
+ * If GSSAPI encryption is enabled, send a packet to the
+ * server asking for GSSAPI Encryption and proceed with GSSAPI
+ * handshake. We will come back here after GSSAPI encryption
+ * has been established, with conn->gctx set.
*/
- if (conn->try_gss && !conn->gctx)
- conn->try_gss = pg_GSS_have_cred_cache(&conn->gcred);
- if (conn->try_gss && !conn->gctx)
+ if (conn->current_enc_method == ENC_GSSAPI && !conn->gctx)
{
ProtocolVersion pv = pg_hton32(NEGOTIATE_GSS_CODE);
@@ -3273,12 +3273,6 @@ keep_going: /* We will come back to here until there is
conn->status = CONNECTION_GSS_STARTUP;
return PGRES_POLLING_READING;
}
- else if (!conn->gctx && conn->gssencmode[0] == 'r')
- {
- libpq_append_conn_error(conn,
- "GSSAPI encryption required but was impossible (possibly no credential cache, no server support, or using a local socket)");
- goto error_return;
- }
#endif
#ifdef USE_SSL
@@ -3294,39 +3288,22 @@ keep_going: /* We will come back to here until there is
goto error_return;
/*
- * If SSL is enabled and direct SSL connections are enabled
- * and we haven't already established an SSL connection (or
- * already tried a direct connection and failed or succeeded)
- * then try just enabling SSL directly.
- *
- * If we fail then we'll either fail the connection (if
- * sslnegotiation is set to requiredirect or turn
- * allow_direct_ssl_try to false
+ * If direct SSL is enabled, jump right into SSL handshake.
+ * We will come back here after SSL encryption has been established,
+ * with ssl_in_use set.
*/
- if (conn->allow_ssl_try
- && !conn->wait_ssl_try
- && conn->allow_direct_ssl_try
- && !conn->ssl_in_use
-#ifdef ENABLE_GSS
- && !conn->gssenc
-#endif
- )
+ if (conn->current_enc_method == ENC_DIRECT_SSL && !conn->ssl_in_use)
{
conn->status = CONNECTION_SSL_STARTUP;
return PGRES_POLLING_WRITING;
}
/*
- * If SSL is enabled and we haven't already got encryption of
- * some sort running, request SSL instead of sending the
- * startup message.
+ * If negotiated SSL is enabled, request SSL and proceed with
+ * SSL handshake. We will come back here after SSL encryption
+ * has been established, with ssl_in_use set.
*/
- if (conn->allow_ssl_try && !conn->wait_ssl_try &&
- !conn->ssl_in_use
-#ifdef ENABLE_GSS
- && !conn->gssenc
-#endif
- )
+ if (conn->current_enc_method == ENC_NEGOTIATED_SSL && !conn->ssl_in_use)
{
ProtocolVersion pv;
@@ -3351,8 +3328,11 @@ keep_going: /* We will come back to here until there is
#endif /* USE_SSL */
/*
- * Build the startup packet.
+ * We have now established encryption, or we are happy to
+ * proceed without.
*/
+
+ /* Build the startup packet. */
startpacket = pqBuildStartupPacket3(conn, &packetlen,
EnvironmentOptions);
if (!startpacket)
@@ -3393,10 +3373,9 @@ keep_going: /* We will come back to here until there is
/*
* On first time through, get the postmaster's response to our
* SSL negotiation packet. If we are trying a direct ssl
- * connection skip reading the negotiation packet and go
- * straight to initiating an ssl connection.
+ * connection, go straight to initiating ssl.
*/
- if (!conn->ssl_in_use && !conn->allow_direct_ssl_try)
+ if (!conn->ssl_in_use && conn->current_enc_method == ENC_NEGOTIATED_SSL)
{
/*
* We use pqReadData here since it has the logic to
@@ -3441,19 +3420,8 @@ keep_going: /* We will come back to here until there is
/* mark byte consumed */
conn->inStart = conn->inCursor;
/* OK to do without SSL? */
- if (conn->sslmode[0] == 'r' || /* "require" */
- conn->sslmode[0] == 'v') /* "verify-ca" or
- * "verify-full" */
- {
- /* Require SSL, but server does not want it */
- libpq_append_conn_error(conn, "server does not support SSL, but SSL was required");
- goto error_return;
- }
- /* Otherwise, proceed with normal startup */
- conn->allow_ssl_try = false;
/* We can proceed using this connection */
- conn->status = CONNECTION_MADE;
- return PGRES_POLLING_WRITING;
+ ENCRYPTION_NEGOTIATION_FAILED();
}
else if (SSLok == 'E')
{
@@ -3503,32 +3471,7 @@ keep_going: /* We will come back to here until there is
if (pollres == PGRES_POLLING_FAILED)
{
/* Failed direct ssl connection, possibly try a new connection with postgres negotiation */
- if (conn->allow_direct_ssl_try)
- {
- /* if it's requiredirect then it's a hard failure */
- if (conn->sslnegotiation[0] == 'r')
- goto error_return;
- /* otherwise only retry using postgres connection */
- conn->allow_direct_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
-
- /*
- * Failed ... if sslmode is "prefer" then do a non-SSL
- * retry
- */
- if (conn->sslmode[0] == 'p' /* "prefer" */
- && conn->allow_ssl_try /* redundant? */
- && !conn->wait_ssl_try) /* redundant? */
- {
- /* only retry once */
- conn->allow_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
- /* Else it's a hard failure */
- goto error_return;
+ CONNECTION_FAILED();
}
/* Else, return POLLING_READING or POLLING_WRITING status */
return pollres;
@@ -3547,7 +3490,7 @@ keep_going: /* We will come back to here until there is
* If we haven't yet, get the postmaster's response to our
* negotiation packet
*/
- if (conn->try_gss && !conn->gctx)
+ if (!conn->gctx)
{
char gss_ok;
int rdresult = pqReadData(conn);
@@ -3571,9 +3514,7 @@ keep_going: /* We will come back to here until there is
* error message on retry). Server gets fussy if we
* don't hang up the socket, though.
*/
- conn->try_gss = false;
- need_new_connection = true;
- goto keep_going;
+ CONNECTION_FAILED();
}
/* mark byte consumed */
@@ -3581,17 +3522,8 @@ keep_going: /* We will come back to here until there is
if (gss_ok == 'N')
{
- /* Server doesn't want GSSAPI; fall back if we can */
- if (conn->gssencmode[0] == 'r')
- {
- libpq_append_conn_error(conn, "server doesn't support GSSAPI encryption, but it was required");
- goto error_return;
- }
-
- conn->try_gss = false;
/* We can proceed using this connection */
- conn->status = CONNECTION_MADE;
- return PGRES_POLLING_WRITING;
+ ENCRYPTION_NEGOTIATION_FAILED();
}
else if (gss_ok != 'G')
{
@@ -3623,18 +3555,7 @@ keep_going: /* We will come back to here until there is
}
else if (pollres == PGRES_POLLING_FAILED)
{
- if (conn->gssencmode[0] == 'p')
- {
- /*
- * We failed, but we can retry on "prefer". Have to
- * drop the current connection to do so, though.
- */
- conn->try_gss = false;
- need_new_connection = true;
- goto keep_going;
- }
- /* Else it's a hard failure */
- goto error_return;
+ CONNECTION_FAILED();
}
/* Else, return POLLING_READING or POLLING_WRITING status */
return pollres;
@@ -3810,55 +3731,7 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
-#ifdef ENABLE_GSS
-
- /*
- * If gssencmode is "prefer" and we're using GSSAPI, retry
- * without it.
- */
- if (conn->gssenc && conn->gssencmode[0] == 'p')
- {
- /* only retry once */
- conn->try_gss = false;
- need_new_connection = true;
- goto keep_going;
- }
-#endif
-
-#ifdef USE_SSL
-
- /*
- * if sslmode is "allow" and we haven't tried an SSL
- * connection already, then retry with an SSL connection
- */
- if (conn->sslmode[0] == 'a' /* "allow" */
- && !conn->ssl_in_use
- && conn->allow_ssl_try
- && conn->wait_ssl_try)
- {
- /* only retry once */
- conn->wait_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
-
- /*
- * if sslmode is "prefer" and we're in an SSL connection,
- * then do a non-SSL retry
- */
- if (conn->sslmode[0] == 'p' /* "prefer" */
- && conn->ssl_in_use
- && conn->allow_ssl_try /* redundant? */
- && !conn->wait_ssl_try) /* redundant? */
- {
- /* only retry once */
- conn->allow_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
-#endif
-
- goto error_return;
+ CONNECTION_FAILED();
}
else if (beresp == PqMsg_NegotiateProtocolVersion)
{
@@ -4298,6 +4171,178 @@ error_return:
return PGRES_POLLING_FAILED;
}
+static bool
+init_allowed_encryption_methods(PGconn *conn)
+{
+ if (conn->raddr.addr.ss_family == AF_UNIX)
+ {
+ /* Don't request SSL or GSSAPI over Unix sockets */
+ conn->allowed_enc_methods &= ~(ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL | ENC_GSSAPI);
+
+ /* to give a better error message */
+ /* XXX: we probably should not do this. sslmode=require works differently */
+ if (conn->gssencmode[0] == 'r')
+ {
+ libpq_append_conn_error(conn,
+ "GSSAPI encryption required but it is not supported over a local socket)");
+ conn->allowed_enc_methods = 0;
+ conn->current_enc_method = ENC_ERROR;
+ return false;
+ }
+
+ conn->allowed_enc_methods = ENC_PLAINTEXT;
+ conn->current_enc_method = ENC_PLAINTEXT;
+ return true;
+ }
+
+ /* initialize these values based on sslmode and gssencmode */
+ conn->allowed_enc_methods = 0;
+
+#ifdef USE_SSL
+ /* sslmode anything but 'disable', and GSSAPI not required */
+ if (conn->sslmode[0] != 'd' && conn->gssencmode[0] != 'r')
+ {
+ if (conn->sslnegotiation[0] == 'p')
+ conn->allowed_enc_methods |= ENC_NEGOTIATED_SSL;
+ else if (conn->sslnegotiation[0] == 'd')
+ conn->allowed_enc_methods |= ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL;
+ else if (conn->sslnegotiation[0] == 'r')
+ conn->allowed_enc_methods |= ENC_DIRECT_SSL;
+ }
+#endif
+
+#ifdef ENABLE_GSS
+ if (conn->gssencmode[0] != 'd')
+ conn->allowed_enc_methods |= ENC_GSSAPI;
+#endif
+
+ if ((conn->sslmode[0] == 'd' || conn->sslmode[0] == 'p' || conn->sslmode[0] == 'a') &&
+ (conn->gssencmode[0] == 'd' || conn->gssencmode[0] == 'p'))
+ {
+ conn->allowed_enc_methods |= ENC_PLAINTEXT;
+ }
+
+ return select_next_encryption_method(conn, false);
+}
+
+static int
+encryption_negotiation_failed(PGconn *conn)
+{
+ Assert((conn->failed_enc_methods & conn->current_enc_method) == 0);
+ conn->failed_enc_methods |= conn->current_enc_method;
+
+ if (select_next_encryption_method(conn, true))
+ {
+ if (conn->current_enc_method == ENC_DIRECT_SSL)
+ return 2;
+ else
+ return 1;
+ }
+ else
+ return 0;
+}
+
+static bool
+connection_failed(PGconn *conn)
+{
+ Assert((conn->failed_enc_methods & conn->current_enc_method) == 0);
+ conn->failed_enc_methods |= conn->current_enc_method;
+
+ /* If the server reported an error after the SSL handshake, no point in retrying with negotiated vs direct SSL */
+ if ((conn->current_enc_method & (ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL)) != 0 && conn->ssl_handshake_started)
+ conn->failed_enc_methods |= (ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL) & conn->allowed_enc_methods;
+ else
+ conn->failed_enc_methods |= conn->current_enc_method;
+
+ return select_next_encryption_method(conn, false);
+}
+
+static bool
+select_next_encryption_method(PGconn *conn, bool negotiation_failure)
+{
+ int remaining_methods;
+
+ remaining_methods = conn->allowed_enc_methods & ~conn->failed_enc_methods;
+
+ /*
+ * Try GSSAPI before SSL
+ */
+#ifdef ENABLE_GSS
+ if ((remaining_methods & ENC_GSSAPI) != 0)
+ {
+ /*
+ * If GSSAPI encryption is enabled, then call
+ * pg_GSS_have_cred_cache() which will return true if we can
+ * acquire credentials (and give us a handle to use in
+ * conn->gcred), and then send a packet to the server asking
+ * for GSSAPI Encryption (and skip past SSL negotiation and
+ * regular startup below).
+ */
+ if (!conn->gctx)
+ {
+ if (!pg_GSS_have_cred_cache(&conn->gcred))
+ {
+ conn->allowed_enc_methods &= ~ENC_GSSAPI;
+ remaining_methods &= ~ENC_GSSAPI;
+
+ if (conn->gssencmode[0] == 'r')
+ {
+ libpq_append_conn_error(conn,
+ "GSSAPI encryption required but no credential cache");
+ }
+ }
+ }
+ if ((remaining_methods & ENC_GSSAPI) != 0)
+ {
+ conn->current_enc_method = ENC_GSSAPI;
+ return true;
+ }
+ }
+#endif
+
+ /* With sslmode=allow, try plaintext connection before SSL. */
+ if (conn->sslmode[0] == 'a' && (remaining_methods & ENC_PLAINTEXT) != 0)
+ {
+ conn->current_enc_method = ENC_PLAINTEXT;
+ return true;
+ }
+
+ /*
+ * Try SSL. If enabled, try direct SSL. Unless we have a valid TCP
+ * connection that failed negotiating GSSAPI encryption; in that case we
+ * prefer to reuse the connection with negotiated SSL, instead of
+ * reconnecting to do direct SSL. The point of direct SSL is to avoid the
+ * roundtrip from the negotiation, but reconnecting would also incur a
+ * roundtrip.
+ */
+ if (negotiation_failure && (remaining_methods & ENC_NEGOTIATED_SSL) != 0)
+ {
+ conn->current_enc_method = ENC_NEGOTIATED_SSL;
+ return true;
+ }
+
+ if ((remaining_methods & ENC_DIRECT_SSL) != 0)
+ {
+ conn->current_enc_method = ENC_DIRECT_SSL;
+ return true;
+ }
+
+ if ((remaining_methods & ENC_NEGOTIATED_SSL) != 0)
+ {
+ conn->current_enc_method = ENC_NEGOTIATED_SSL;
+ return true;
+ }
+
+ if (conn->sslmode[0] != 'a' && (remaining_methods & ENC_PLAINTEXT) != 0)
+ {
+ conn->current_enc_method = ENC_PLAINTEXT;
+ return true;
+ }
+
+ /* No more options */
+ conn->current_enc_method = ENC_ERROR;
+ return false;
+}
/*
* internal_ping
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index fcd402dc6b7..4a12c279350 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -1502,6 +1502,7 @@ open_client_SSL(PGconn *conn)
SOCK_ERRNO_SET(0);
ERR_clear_error();
r = SSL_connect(conn->ssl);
+
if (r <= 0)
{
int save_errno = SOCK_ERRNO;
@@ -1605,7 +1606,7 @@ open_client_SSL(PGconn *conn)
/*
* We already checked the server certificate in initialize_SSL() using
- * SSL_CTX_set_verify(), if root.crt exists.
+ * SSL_set_verify(), if root.crt exists.
*/
/* get server certificate */
@@ -1649,6 +1650,7 @@ pgtls_close(PGconn *conn)
SSL_free(conn->ssl);
conn->ssl = NULL;
conn->ssl_in_use = false;
+ conn->ssl_handshake_started = false;
destroy_needed = true;
}
@@ -1672,7 +1674,7 @@ pgtls_close(PGconn *conn)
{
/*
* In the non-SSL case, just remove the crypto callbacks if the
- * connection has then loaded. This code path has no dependency on
+ * connection has loaded them. This code path has no dependency on
* any pending SSL calls.
*/
if (conn->crypto_loaded)
@@ -1859,9 +1861,10 @@ static BIO_METHOD *my_bio_methods;
static int
my_sock_read(BIO *h, char *buf, int size)
{
+ PGconn *conn = (PGconn *) BIO_get_app_data(h);
int res;
- res = pqsecure_raw_read((PGconn *) BIO_get_app_data(h), buf, size);
+ res = pqsecure_raw_read(conn, buf, size);
BIO_clear_retry_flags(h);
if (res < 0)
{
@@ -1883,6 +1886,9 @@ my_sock_read(BIO *h, char *buf, int size)
}
}
+ if (res > 0)
+ conn->ssl_handshake_started = true;
+
return res;
}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 3a4e8ae4080..69632a48e26 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -72,14 +72,13 @@ typedef enum
CONNECTION_AUTH_OK, /* Received authentication; waiting for
* backend startup. */
CONNECTION_SETENV, /* This state is no longer used. */
- CONNECTION_SSL_STARTUP, /* Negotiating SSL. */
+ CONNECTION_SSL_STARTUP, /* Performing SSL handshake. */
CONNECTION_NEEDED, /* Internal state: connect() needed */
CONNECTION_CHECK_WRITABLE, /* Checking if session is read-write. */
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
- CONNECTION_DIRECT_SSL_STARTUP /* Starting SSL without PG Negotiation. */
} ConnStatusType;
typedef enum
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 6daf34b9afe..05a7256410f 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -76,6 +76,7 @@ typedef struct
#include <openssl/ssl.h>
#include <openssl/err.h>
+
#ifndef OPENSSL_NO_ENGINE
#define USE_SSL_ENGINE
#endif
@@ -231,6 +232,12 @@ typedef enum
PGASYNC_PIPELINE_IDLE, /* "Idle" between commands in pipeline mode */
} PGAsyncStatusType;
+#define ENC_ERROR 0
+#define ENC_DIRECT_SSL 0x01
+#define ENC_GSSAPI 0x02
+#define ENC_NEGOTIATED_SSL 0x04
+#define ENC_PLAINTEXT 0x08
+
/* Target server type (decoded value of target_session_attrs) */
typedef enum
{
@@ -545,17 +552,17 @@ struct pg_conn
void *sasl_state;
int scram_sha_256_iterations;
+ uint8 allowed_enc_methods;
+ uint8 failed_enc_methods;
+ uint8 current_enc_method;
+
/* SSL structures */
bool ssl_in_use;
+ bool ssl_handshake_started;
bool ssl_cert_requested; /* Did the server ask us for a cert? */
bool ssl_cert_sent; /* Did we send one in reply? */
#ifdef USE_SSL
- bool allow_direct_ssl_try; /* Try to make a direct SSL connection
- * without an "SSL negotiation packet" */
- bool allow_ssl_try; /* Allowed to try SSL negotiation */
- bool wait_ssl_try; /* Delay SSL negotiation until after
- * attempting normal connection */
#ifdef USE_OPENSSL
SSL *ssl; /* SSL status, if have SSL connection */
X509 *peer; /* X509 cert of server */
@@ -578,7 +585,6 @@ struct pg_conn
gss_name_t gtarg_nam; /* GSS target name */
/* The following are encryption-only */
- bool try_gss; /* GSS attempting permitted */
bool gssenc; /* GSS encryption is usable */
gss_cred_id_t gcred; /* GSS credential temp storage. */
--
2.39.2
v7-0012-Add-tests-for-sslnegotiation.patchtext/x-patch; charset=UTF-8; name=v7-0012-Add-tests-for-sslnegotiation.patchDownload
From 8b618dd8f03ea9d6dc18bd1dc9c858d795428c95 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 10:25:08 +0200
Subject: [PATCH v7 12/12] Add tests for sslnegotiation
---
.../t/001_negotiate_encryption.pl | 110 +++++++++++++-----
1 file changed, 78 insertions(+), 32 deletions(-)
diff --git a/src/test/libpq_encryption/t/001_negotiate_encryption.pl b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
index 7a80fe90f56..85f55e9c07f 100644
--- a/src/test/libpq_encryption/t/001_negotiate_encryption.pl
+++ b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
@@ -173,6 +173,63 @@ connect_test($node, 'user=testuser sslmode=allow', 'plain');
connect_test($node, 'user=testuser sslmode=prefer', 'plain');
connect_test($node, 'user=testuser sslmode=require', 'fail');
+connect_test($node, 'user=testuser sslmode=disable sslnegotiation=direct', 'plain');
+connect_test($node, 'user=testuser sslmode=allow sslnegotiation=direct', 'plain');
+connect_test($node, 'user=testuser sslmode=prefer sslnegotiation=direct', 'plain');
+connect_test($node, 'user=testuser sslmode=require sslnegotiation=direct', 'fail');
+
+connect_test($node, 'user=testuser sslmode=disable sslnegotiation=requiredirect', 'plain');
+connect_test($node, 'user=testuser sslmode=allow sslnegotiation=requiredirect', 'plain');
+connect_test($node, 'user=testuser sslmode=prefer sslnegotiation=requiredirect', 'plain');
+connect_test($node, 'user=testuser sslmode=require sslnegotiation=requiredirect', 'fail');
+
+# Enable SSL in the server
+SKIP:
+{
+ skip "SSL not supported by this build" unless $ssl_supported;
+
+ $node->adjust_conf('postgresql.conf', 'ssl', 'on');
+ $node->restart;
+
+ connect_test($node, 'user=testuser sslmode=disable', 'plain');
+ connect_test($node, 'user=testuser sslmode=allow', 'plain');
+ connect_test($node, 'user=testuser sslmode=prefer', 'ssl');
+ connect_test($node, 'user=testuser sslmode=require', 'ssl');
+
+ connect_test($node, 'user=testuser sslmode=disable sslnegotiation=direct', 'plain');
+ connect_test($node, 'user=testuser sslmode=allow sslnegotiation=direct', 'plain');
+ connect_test($node, 'user=testuser sslmode=prefer sslnegotiation=direct', 'ssl');
+ connect_test($node, 'user=testuser sslmode=require sslnegotiation=direct', 'ssl');
+
+ connect_test($node, 'user=testuser sslmode=disable sslnegotiation=requiredirect', 'plain');
+ connect_test($node, 'user=testuser sslmode=allow sslnegotiation=requiredirect', 'plain');
+ connect_test($node, 'user=testuser sslmode=prefer sslnegotiation=requiredirect', 'ssl');
+ connect_test($node, 'user=testuser sslmode=require sslnegotiation=requiredirect', 'ssl');
+
+ connect_test($node, 'user=ssluser sslmode=disable', 'fail');
+ connect_test($node, 'user=ssluser sslmode=allow', 'ssl');
+ connect_test($node, 'user=ssluser sslmode=prefer', 'ssl');
+ connect_test($node, 'user=ssluser sslmode=require', 'ssl');
+
+ connect_test($node, 'user=ssluser sslmode=disable sslnegotiation=direct', 'fail');
+ connect_test($node, 'user=ssluser sslmode=allow sslnegotiation=direct', 'ssl');
+ connect_test($node, 'user=ssluser sslmode=prefer sslnegotiation=direct', 'ssl');
+ connect_test($node, 'user=ssluser sslmode=require sslnegotiation=direct', 'ssl');
+
+ connect_test($node, 'user=ssluser sslmode=disable sslnegotiation=requiredirect', 'fail');
+ connect_test($node, 'user=ssluser sslmode=allow sslnegotiation=requiredirect', 'ssl');
+ connect_test($node, 'user=ssluser sslmode=prefer sslnegotiation=requiredirect', 'ssl');
+ connect_test($node, 'user=ssluser sslmode=require sslnegotiation=requiredirect', 'ssl');
+
+ connect_test($node, 'user=nossluser sslmode=disable', 'plain');
+ connect_test($node, 'user=nossluser sslmode=allow', 'plain');
+ connect_test($node, 'user=nossluser sslmode=prefer', 'plain');
+ connect_test($node, 'user=nossluser sslmode=require', 'fail');
+
+ $node->adjust_conf('postgresql.conf', 'ssl', 'off');
+ $node->reload;
+}
+
# Test GSSAPI
SKIP:
{
@@ -192,8 +249,13 @@ SKIP:
connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer', 'gss');
connect_test($node, 'user=testuser sslmode=prefer gssencmode=require', 'gss');
- connect_test($node, 'user=testuser sslmode=require gssencmode=disable', 'fail');
- connect_test($node, 'user=testuser sslmode=require gssencmode=prefer', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer sslnegotiation=direct gssencmode=disable', 'plain');
+ connect_test($node, 'user=testuser sslmode=prefer sslnegotiation=direct gssencmode=prefer', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer sslnegotiation=direct gssencmode=require', 'gss');
+
+ connect_test($node, 'user=testuser sslmode=require sslnegotiation=direct gssencmode=disable', 'fail');
+ connect_test($node, 'user=testuser sslmode=require sslnegotiation=direct gssencmode=prefer', 'gss');
+ connect_test($node, 'user=testuser sslmode=require sslnegotiation=requiredirect gssencmode=prefer', 'gss');
# If you set both sslmode and gssencmode to 'require', 'gssencmode=require' takes
# precedence.
@@ -211,45 +273,26 @@ SKIP:
# with no encryption.
connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer', 'plain',
'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer sslnegotiation=direct', 'plain',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer sslnegotiation=requiredirect', 'plain',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
}
-# Enable SSL in the server
+# Server supports both SSL and GSSAPI
SKIP:
{
- skip "SSL not supported by this build" unless $ssl_supported;
-
- my $certdir = dirname(__FILE__) . "/../../ssl/ssl";
+ skip "GSSAPI/Kerberos or SSL not supported by this build" unless ($ssl_supported && $gss_supported);
- copy "$certdir/server-cn-only.crt", "$pgdata/server.crt"
- || die "copying server.crt: $!";
- copy "$certdir/server-cn-only.key", "$pgdata/server.key"
- || die "copying server.key: $!";
- chmod(0600, "$pgdata/server.key");
+ # SSL is still disabled
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer sslnegotiation=direct', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer sslnegotiation=requiredirect', 'gss');
+ # Enable SSL
$node->adjust_conf('postgresql.conf', 'ssl', 'on');
$node->reload;
- connect_test($node, 'user=testuser sslmode=disable gssencmode=disable', 'plain');
- connect_test($node, 'user=testuser sslmode=allow gssencmode=disable', 'plain');
- connect_test($node, 'user=testuser sslmode=prefer gssencmode=disable', 'ssl');
- connect_test($node, 'user=testuser sslmode=require gssencmode=disable', 'ssl');
-
- connect_test($node, 'user=ssluser sslmode=disable gssencmode=disable', 'fail');
- connect_test($node, 'user=ssluser sslmode=allow gssencmode=disable', 'ssl');
- connect_test($node, 'user=ssluser sslmode=prefer gssencmode=disable', 'ssl');
- connect_test($node, 'user=ssluser sslmode=require gssencmode=disable', 'ssl');
-
- connect_test($node, 'user=nossluser sslmode=disable gssencmode=disable', 'plain');
- connect_test($node, 'user=nossluser sslmode=allow gssencmode=disable', 'plain');
- connect_test($node, 'user=nossluser sslmode=prefer gssencmode=disable', 'plain');
- connect_test($node, 'user=nossluser sslmode=require gssencmode=disable', 'fail');
-}
-
-# Server supports SSL and GSSAPI
-SKIP:
-{
- skip "GSSAPI/Kerberos or SSL not supported by this build" unless ($ssl_supported && $gss_supported);
-
connect_test($node, 'user=testuser sslmode=disable gssencmode=disable', 'plain');
connect_test($node, 'user=testuser sslmode=disable gssencmode=prefer', 'gss');
connect_test($node, 'user=testuser sslmode=disable gssencmode=require', 'gss');
@@ -261,6 +304,9 @@ SKIP:
connect_test($node, 'user=testuser sslmode=require gssencmode=disable', 'ssl');
connect_test($node, 'user=testuser sslmode=require gssencmode=prefer', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer sslnegotiation=direct', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer sslnegotiation=requiredirect', 'gss');
+
# If you set both sslmode and gssencmode to 'require', 'gssencmode=require' takes
# precedence.
connect_test($node, 'user=testuser sslmode=require gssencmode=require', 'gss');
--
2.39.2
I've been asked to take a look at this thread and review some patches,
and the subject looks interesting enough, so here I am.
On Thu, 19 Jan 2023 at 04:16, Greg Stark <stark@mit.edu> wrote:
I had a conversation a while back with Heikki where he expressed that
it was annoying that we negotiate SSL/TLS the way we do since it
introduces an extra round trip. Aside from the performance
optimization I think accepting standard TLS connections would open the
door to a number of other opportunities that would be worth it on
their own.
I agree that this would be very nice.
Other things it would open the door to in order from least
controversial to most....* Hiding Postgres behind a standard SSL proxy terminating SSL without
implementing the Postgres protocol.
I think there is also the option "hiding Postgres behind a standard
SNI-based SSL router that does not terminate SSL", as that's arguably
a more secure way to deploy any SSL service than SSL-terminating
proxies.
* "Service Mesh" type tools that hide multiple services behind a
single host/port ("Service Mesh" is just a new buzzword for "proxy").
People proxying PostgreSQL seems fine, and enabling better proxying
seems reasonable.
* Browser-based protocol implementations using websockets for things
like pgadmin or other tools to connect directly to postgres using
Postgres wire protocol but using native SSL implementations.* Postgres could even implement an HTTP based version of its protocol
and enable things like queries or browser based tools using straight
up HTTP requests so they don't need to use websockets.* Postgres could implement other protocols to serve up data like
status queries or monitoring metrics, using HTTP based standard
protocols instead of using our own protocol.
I don't think we should be trying to serve anything HTTP-like, even
with a ten-foot pole, on a port that we serve the PostgreSQL wire
protocol on.
If someone wants to multiplex the PostgreSQL wire protocol on the same
port that serves HTTPS traffic, they're welcome to do so with their
own proxy, but I'd rather we keep the PostgreSQL server's socket
handling fundamentaly incapable of servicng protocols primarily used
in web browsers on the same socket that handles normal psql data
connections.
PostgreSQL may have its own host-based authentication with HBA, but
I'd rather not have to depend on it to filter incoming connections
between valid psql connections and people trying to grab the latest
monitoring statistics at some http endpoint - I'd rather use my trusty
firewall that can already limit access to specific ports very
efficiently without causing undue load on the database server.
Matthias van de Meent
Neon (https://neon.tech)
On Wed, 10 Jan 2024 at 09:31, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
Some more comments on this:
1. It feels weird that the combination of "gssencmode=require
sslnegotiation=direct" combination is forbidden. Sure, the ssl
negotiation will never happen with gssencmode=require, so the
sslnegotiation option has no effect. But by that token, should we also
forbid the combination "sslmode=disable sslnegotiation=direct"? I think
not. The sslnegotiation option should mean "if we are going to try SSL,
should we try it in direct or negotiated mode?"
I'm not sure about this either. The 'gssencmode' option is already
quite weird in that it seems to override the "require"d priority of
"sslmode=require", which it IMO really shouldn't.
2. Should we allow direct SSL only at the very beginning of a TCP
connection, or should we also allow it after we have requested GSS and
the server said no? Like this:Client: GSSENCRequest
Server: 'N' (gss not supported)
Client: TLS client HelloOn one hand, why not? It saves you a round-trip in this case too. If we
don't allow it, the client will have to send SSLRequest and wait for
response, or reconnect to try direct SSL. On the other hand, flexibility
is not necessarily a good thing in security-critical code like this.
I think this should be "no".
Once we start accepting PostgreSQL protocol packets (such as the
GSSENCRequest packet) I don't think we should start treating data
stream corruption as attempted SSL connections.
The patch set is confused on whether that's allowed or not. The server
rejects it. But if you use "gssencmode=prefer
sslnegotiation=requiredrect", libpq will attempt to do it, and fail.
That should then be detected as an incorrect combination of flags in
psql: you can't have direct-to-ssl and put something in front of it.
3. With "sslmode=verify-full sslnegotiation=direct", if the direct SSL
connection fails because of a problem with the certificate, libpq will
try again in negotiated SSL mode. That seems pointless. If the server
responded to the direct TLS Client Hello message with a valid
ServerHello, that indicates that the server supports direct SSL. If
anything goes wrong after that, retrying in negotiated mode is not going
to help.
This makes sense.
4. The number of combinations of sslmode, gssencmode and sslnegotiation
settings is scary. And we have very few tests for them.
Yeah, it's not great. We could easily automate this better though. I
mean, can't we run the tests using a "cube" configuration, i.e. test
every combination of parameters? We would use a mapping function of
(psql connection parameter values -> expectations), which would be
along the lines of the attached pl testfile. I feel it's a bit more
approachable than the lists of manual option configurations, and makes
it a bit easier to program the logic of which connection security
option we should have used to connect.
The attached file would be a drop-in replacement; it's tested to work
with SSL only - without GSS - because I've been having issues getting
GSS working on my machine.
I'm going to put this down for now. The attached patch set is even more
raw than v6, but I'm including it here to "save the work".
v6 doesn't apply cleanly anymore after 774bcffe, but here are some notes:
Several patches are still very much WIP. Reviewing them on a
patch-by-patch basis is therefore nigh impossible; the specific
reviews below are thus on changes that could be traced back to a
specific patch. A round of cleanup would be appreciated.
0003: Direct SSL connections postmaster support [...] -extern bool pq_buffer_has_data(void); +extern size_t pq_buffer_has_data(void);
This should probably be renamed to pg_buffer_remaining_data or such,
if we change the signature like this.
+ /* Read from the "unread" buffered data first. c.f. libpq-be.h */ + if (port->raw_buf_remaining > 0) + { + /* consume up to len bytes from the raw_buf */ + if (len > port->raw_buf_remaining) + len = port->raw_buf_remaining;
Shouldn't we also try to read from the socket, instead of only
consuming bytes from the raw buffer if it contains bytes?
0008: Allow pipelining data after ssl request + /* + * At this point we should have no data already buffered. If we do, + * it was received before we performed the SSL handshake, so it wasn't + * encrypted and indeed may have been injected by a man-in-the-middle. + * We report this case to the client. + */ + if (port->raw_buf_remaining > 0) + ereport(FATAL, + (errcode(ERRCODE_PROTOCOL_VIOLATION), + errmsg("received unencrypted data after SSL request"), + errdetail("This could be either a client-software bug or evidence of an attempted man-in-the-middle attack.")));
We currently don't support 0-RTT SSL connections because (among other
reasons) we haven't yet imported many features from TLS1.3, but it
seems reasonable that clients may want to use 0RTT (or, session
resumption in 0 round trips), which would allow encrypted data after
the SSL startup packet.
It seems wise to add something to this note to these comments in
ProcessStartupPacket.
ALPN
Does the TLS ALPN spec allow protocol versions in the protocol tag? It
would be very useful to detect clients with new capabilities at the
first connection, rather than having to wait for one round trip, and
would allow one avenue for changing the protocol version.
Apart from this, I didn't really find any serious problems in the sum
of these patches. The intermediate states were not great though, with
various broken states in between.
Kind regards,
Matthias van de Meent
Attachments:
On 22/02/2024 01:43, Matthias van de Meent wrote:
On Wed, 10 Jan 2024 at 09:31, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
4. The number of combinations of sslmode, gssencmode and sslnegotiation
settings is scary. And we have very few tests for them.Yeah, it's not great. We could easily automate this better though. I
mean, can't we run the tests using a "cube" configuration, i.e. test
every combination of parameters? We would use a mapping function of
(psql connection parameter values -> expectations), which would be
along the lines of the attached pl testfile. I feel it's a bit more
approachable than the lists of manual option configurations, and makes
it a bit easier to program the logic of which connection security
option we should have used to connect.
The attached file would be a drop-in replacement; it's tested to work
with SSL only - without GSS - because I've been having issues getting
GSS working on my machine.
+1 testing all combinations. I don't think the 'mapper' function
approach in your version is much better than the original though. Maybe
it would be better with just one 'mapper' function that contains all the
rules, along the lines of: (This isn't valid perl, just pseudo-code)
sub expected_outcome
{
my ($user, $sslmode, $negotiation, $gssmode) = @_;
my @possible_outcomes = { 'plain', 'ssl', 'gss' }
delete $possible_outcomes{'plain'} if $sslmode eq 'require';
delete $possible_outcomes{'ssl'} if $sslmode eq 'disable';
delete $possible_outcomes{'plain'} if $user eq 'ssluser';
delete $possible_outcomes{'plain'} if $user eq 'ssluser';
if $sslmode eq 'allow' {
# move 'plain' before 'ssl' in the list
}
if $sslmode eq 'prefer' {
# move 'ssl' before 'plain' in the list
}
# more rules here
# If there are no outcomes left in $possible_outcomes, return 'fail'
# If there's exactly one outcome left, return that.
# If there's more, return the first one.
}
Or maybe a table that lists all the combinations and the expected
outcome. Something lieke this:
nossluser nogssuser ssluser gssuser
sslmode=require fail ...
sslmode=prefer plain
sslmode=disable plain
The problem is that there are more than two dimensions. So maybe an
exhaustive list like this:
user sslmode gssmode outcome
nossluser require disable fail
nossluser prefer disable plain
nossluser disable disable plain
ssluser require disable ssl
...
I'm just throwing around ideas here, can you experiment with different
approaches and see what looks best?
ALPN
Does the TLS ALPN spec allow protocol versions in the protocol tag? It
would be very useful to detect clients with new capabilities at the
first connection, rather than having to wait for one round trip, and
would allow one avenue for changing the protocol version.
Looking at the list of registered ALPN tags [0], I can see "http/0.9";
"http/1.0" and "http/1.1". I think we'd want to changing the major
protocol version in a way that would introduce a new roundtrip, though.
--
Heikki Linnakangas
Neon (https://neon.tech)
On Thu, 22 Feb 2024 at 18:02, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 22/02/2024 01:43, Matthias van de Meent wrote:
On Wed, 10 Jan 2024 at 09:31, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
4. The number of combinations of sslmode, gssencmode and sslnegotiation
settings is scary. And we have very few tests for them.Yeah, it's not great. We could easily automate this better though. I
mean, can't we run the tests using a "cube" configuration, i.e. test
every combination of parameters? We would use a mapping function of
(psql connection parameter values -> expectations), which would be
along the lines of the attached pl testfile. I feel it's a bit more
approachable than the lists of manual option configurations, and makes
it a bit easier to program the logic of which connection security
option we should have used to connect.
The attached file would be a drop-in replacement; it's tested to work
with SSL only - without GSS - because I've been having issues getting
GSS working on my machine.+1 testing all combinations. I don't think the 'mapper' function
approach in your version is much better than the original though. Maybe
it would be better with just one 'mapper' function that contains all the
rules, along the lines of: (This isn't valid perl, just pseudo-code)sub expected_outcome
{
[...]
}
Or maybe a table that lists all the combinations and the expected
outcome. Something lieke this:
[...]
The problem is that there are more than two dimensions. So maybe an
exhaustive list like this:user sslmode gssmode outcome
nossluser require disable fail
...
I'm just throwing around ideas here, can you experiment with different
approaches and see what looks best?
One issue with exhaustive tables is that they would require a product
of all options to be listed, and that'd require at least 216 rows to
manage: server_ssl 2 * server_gss 2 * users 3 * client_ssl 4 *
client_gss 3 * client_ssldirect 3 = 216 different states. I think the
expected_autcome version is easier in that regard.
Attached an updated version using a single unified connection type
validator using an approach similar to yours. Note that it does fail 8
tests, all of which are attributed to the current handling of
`sslmode=require gssencmode=prefer`: right now, we allow GSS in that
case, even though the user require-d sslmode.
An alternative check that does pass tests with the code of the patch
is commented out, at lines 209-216.
ALPN
Does the TLS ALPN spec allow protocol versions in the protocol tag? It
would be very useful to detect clients with new capabilities at the
first connection, rather than having to wait for one round trip, and
would allow one avenue for changing the protocol version.Looking at the list of registered ALPN tags [0], I can see "http/0.9";
"http/1.0" and "http/1.1".
Ah, nice.
I think we'd want to changing the major
protocol version in a way that would introduce a new roundtrip, though.
I don't think I understand what you meant here, could you correct the
sentence or expand why we want to do that?
Note that with ALPN you could negotiate postgres/3.0 or postgres/4.0
during the handshake, which could save round-trips.
Kind regards,
Matthias van de Meent
Neon (https://neon.tech)
Attachments:
On 28/02/2024 14:00, Matthias van de Meent wrote:
I don't think I understand what you meant here, could you correct the
sentence or expand why we want to do that?
Note that with ALPN you could negotiate postgres/3.0 or postgres/4.0
during the handshake, which could save round-trips.
Sorry, I missed "avoid" there. I meant:
I think we'd want to *avoid* changing the major protocol version in a
way that would introduce a new roundtrip, though.
--
Heikki Linnakangas
Neon (https://neon.tech)
On Wed, Feb 28, 2024 at 4:10 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
I think we'd want to *avoid* changing the major protocol version in a
way that would introduce a new roundtrip, though.
I'm starting to get up to speed with this patchset. So far I'm mostly
testing how it works; I have yet to take an in-depth look at the
implementation.
I'll squint more closely at the MITM-protection changes in 0008 later.
First impressions, though: it looks like that code has gotten much
less straightforward, which I think is dangerous given the attack it's
preventing. (Off-topic: I'm skeptical of future 0-RTT support. Our
protocol doesn't seem particularly replay-safe to me.)
If we're interested in ALPN negotiation in the future, we may also
want to look at GREASE [1]https://www.rfc-editor.org/rfc/rfc8701.html to keep those options open in the presence
of third-party implementations. Unfortunately OpenSSL doesn't do this
automatically yet.
If we don't have a reason not to, it'd be good to follow the strictest
recommendations from [2]https://alpaca-attack.com/libs.html to avoid cross-protocol attacks. (For anyone
currently running web servers and Postgres on the same host, they
really don't want browsers "talking" to their Postgres servers.) That
would mean checking the negotiated ALPN on both the server and client
side, and failing if it's not what we expect.
I'm not excited about the proliferation of connection options. I don't
have a lot of ideas on how to fix it, though, other than to note that
the current sslnegotiation option names are very unintuitive to me:
- "postgres": only legacy handshakes
- "direct": might be direct... or maybe legacy
- "requiredirect": only direct handshakes... unless other options are
enabled and then we fall back again to legacy? How many people willing
to break TLS compatibility with old servers via "requiredirect" are
going to be okay with lazy fallback to GSS or otherwise?
Heikki mentioned possibly hard-coding a TLS alert if direct SSL is
attempted without server TLS support. I think that's a cool idea, but
without an official "TLS not supported" alert code (which, honestly,
would be strange to standardize) I'm kinda -0.5 on it. If the client
tells me about a handshake_failure or similar, I'm going to start
investigating protocol versions and ciphersuites; I'm not going to
think to myself that maybe the server lacks TLS support altogether.
(Plus, we need to have a good error message when connecting to older
servers anyway. I think we should be able to key off of the EOF coming
back from OpenSSL; it'd be a good excuse to give that part of the code
some love.)
For the record, I'm adding some one-off tests for this feature to a
local copy of my OAuth pytest suite, which is designed to do the kinds
of testing you're running into trouble with. It's not in any way
viable for a PG17 commit, but if you're interested I can make the
patches available.
--Jacob
[1]: https://www.rfc-editor.org/rfc/rfc8701.html
[2]: https://alpaca-attack.com/libs.html
On 01/03/2024 23:49, Jacob Champion wrote:
On Wed, Feb 28, 2024 at 4:10 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
I think we'd want to *avoid* changing the major protocol version in a
way that would introduce a new roundtrip, though.I'm starting to get up to speed with this patchset. So far I'm mostly
testing how it works; I have yet to take an in-depth look at the
implementation.
Thank you!
I'll squint more closely at the MITM-protection changes in 0008 later.
First impressions, though: it looks like that code has gotten much
less straightforward, which I think is dangerous given the attack it's
preventing. (Off-topic: I'm skeptical of future 0-RTT support. Our
protocol doesn't seem particularly replay-safe to me.)
Let's drop that patch. AFAICS it's not needed by the rest of the patches.
If we're interested in ALPN negotiation in the future, we may also
want to look at GREASE [1] to keep those options open in the presence
of third-party implementations. Unfortunately OpenSSL doesn't do this
automatically yet.
Can you elaborate? Do we need to do something extra in the server to be
compatible with GREASE?
If we don't have a reason not to, it'd be good to follow the strictest
recommendations from [2] to avoid cross-protocol attacks. (For anyone
currently running web servers and Postgres on the same host, they
really don't want browsers "talking" to their Postgres servers.) That
would mean checking the negotiated ALPN on both the server and client
side, and failing if it's not what we expect.
Hmm, I thought that's what the patches does. But looking closer, libpq
is not checking that ALPN was used. We should add that. Am I right?
I'm not excited about the proliferation of connection options. I don't
have a lot of ideas on how to fix it, though, other than to note that
the current sslnegotiation option names are very unintuitive to me:
- "postgres": only legacy handshakes
- "direct": might be direct... or maybe legacy
- "requiredirect": only direct handshakes... unless other options are
enabled and then we fall back again to legacy? How many people willing
to break TLS compatibility with old servers via "requiredirect" are
going to be okay with lazy fallback to GSS or otherwise?
Yeah, this is my biggest complaint about all this. Not so much the names
of the options, but the number of combinations of different options, and
how we're going to test them all. I don't have any great solutions,
except adding a lot of tests to cover them, like Matthias did.
Heikki mentioned possibly hard-coding a TLS alert if direct SSL is
attempted without server TLS support. I think that's a cool idea, but
without an official "TLS not supported" alert code (which, honestly,
would be strange to standardize) I'm kinda -0.5 on it. If the client
tells me about a handshake_failure or similar, I'm going to start
investigating protocol versions and ciphersuites; I'm not going to
think to myself that maybe the server lacks TLS support altogether.
Agreed.
(Plus, we need to have a good error message when connecting to older
servers anyway.I think we should be able to key off of the EOF coming
back from OpenSSL; it'd be a good excuse to give that part of the code
some love.)
Hmm, if OpenSSL sends ClientHello and the server responds with a
Postgres error packet, OpenSSL will presumably consume the error packet
or at least part of it. But with our custom BIO, we can peek at the
server response before handing it to OpenSSL.
If it helps, we could backport a nicer error message to old server
versions, similar to what we did with SCRAM in commit 96d0f988b1.
For the record, I'm adding some one-off tests for this feature to a
local copy of my OAuth pytest suite, which is designed to do the kinds
of testing you're running into trouble with. It's not in any way
viable for a PG17 commit, but if you're interested I can make the
patches available.
Yes please, it would be nice to see what tests you've performed, and
have it archived.
--
Heikki Linnakangas
Neon (https://neon.tech)
I hope I didn't joggle your elbow reviewing this, Jacob, but I spent
some time rebase and fix various little things:
- Incorporated Matthias's test changes
- Squashed the client, server and documentation patches. Not much point
in keeping them separate, as one requires the other, and if you're only
interested e.g. in the server parts, just look at src/backend.
- Squashed some of my refactorings with the main patches, because I'm
certain enough that they're desirable. I kept the last libpq state
machine refactoring separate though. I'm pretty sure we need a
refactoring like that, but I'm not 100% sure about the details.
- Added some comments to the new state machine logic in fe-connect.c.
- Removed the XXX comments about TLS alerts.
- Removed the "Allow pipelining data after ssl request" patch
- Reordered the patches so that the first two patches add the tests
different combinations of sslmode, gssencmode and server support. That
could be committed separately, without the rest of the patches. A later
patch expands the tests for the new sslnegotiation option.
The tests are still not distinguishing whether a connection was
established in direct or negotiated mode. So if we e.g. had a bug that
accidentally disabled direct SSL connection completely and always used
negotiated mode, the tests would still pass. I'd like to see some tests
that would catch that.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v8-0001-Move-Kerberos-module.patchtext/x-patch; charset=UTF-8; name=v8-0001-Move-Kerberos-module.patchDownload
From c3b88ffb05a2a8b50e1af3220bf8f524e8bbae46 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 5 Mar 2024 15:40:30 +0200
Subject: [PATCH v8 1/6] Move Kerberos module
So that we can reuse it in new tests.
---
src/test/kerberos/t/001_auth.pl | 174 ++--------------
src/test/perl/PostgreSQL/Test/Kerberos.pm | 229 ++++++++++++++++++++++
2 files changed, 240 insertions(+), 163 deletions(-)
create mode 100644 src/test/perl/PostgreSQL/Test/Kerberos.pm
diff --git a/src/test/kerberos/t/001_auth.pl b/src/test/kerberos/t/001_auth.pl
index 2a81ce8834b..9d3fac83aaa 100644
--- a/src/test/kerberos/t/001_auth.pl
+++ b/src/test/kerberos/t/001_auth.pl
@@ -21,6 +21,7 @@ use strict;
use warnings FATAL => 'all';
use PostgreSQL::Test::Utils;
use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Kerberos;
use Test::More;
use Time::HiRes qw(usleep);
@@ -34,177 +35,27 @@ elsif (!$ENV{PG_TEST_EXTRA} || $ENV{PG_TEST_EXTRA} !~ /\bkerberos\b/)
'Potentially unsafe test GSSAPI/Kerberos not enabled in PG_TEST_EXTRA';
}
-my ($krb5_bin_dir, $krb5_sbin_dir);
-
-if ($^O eq 'darwin' && -d "/opt/homebrew")
-{
- # typical paths for Homebrew on ARM
- $krb5_bin_dir = '/opt/homebrew/opt/krb5/bin';
- $krb5_sbin_dir = '/opt/homebrew/opt/krb5/sbin';
-}
-elsif ($^O eq 'darwin')
-{
- # typical paths for Homebrew on Intel
- $krb5_bin_dir = '/usr/local/opt/krb5/bin';
- $krb5_sbin_dir = '/usr/local/opt/krb5/sbin';
-}
-elsif ($^O eq 'freebsd')
-{
- $krb5_bin_dir = '/usr/local/bin';
- $krb5_sbin_dir = '/usr/local/sbin';
-}
-elsif ($^O eq 'linux')
-{
- $krb5_sbin_dir = '/usr/sbin';
-}
-
-my $krb5_config = 'krb5-config';
-my $kinit = 'kinit';
-my $klist = 'klist';
-my $kdb5_util = 'kdb5_util';
-my $kadmin_local = 'kadmin.local';
-my $krb5kdc = 'krb5kdc';
-
-if ($krb5_bin_dir && -d $krb5_bin_dir)
-{
- $krb5_config = $krb5_bin_dir . '/' . $krb5_config;
- $kinit = $krb5_bin_dir . '/' . $kinit;
- $klist = $krb5_bin_dir . '/' . $klist;
-}
-if ($krb5_sbin_dir && -d $krb5_sbin_dir)
-{
- $kdb5_util = $krb5_sbin_dir . '/' . $kdb5_util;
- $kadmin_local = $krb5_sbin_dir . '/' . $kadmin_local;
- $krb5kdc = $krb5_sbin_dir . '/' . $krb5kdc;
-}
-
-my $host = 'auth-test-localhost.postgresql.example.com';
-my $hostaddr = '127.0.0.1';
-my $realm = 'EXAMPLE.COM';
-
-my $krb5_conf = "${PostgreSQL::Test::Utils::tmp_check}/krb5.conf";
-my $kdc_conf = "${PostgreSQL::Test::Utils::tmp_check}/kdc.conf";
-my $krb5_cache = "${PostgreSQL::Test::Utils::tmp_check}/krb5cc";
-my $krb5_log = "${PostgreSQL::Test::Utils::log_path}/krb5libs.log";
-my $kdc_log = "${PostgreSQL::Test::Utils::log_path}/krb5kdc.log";
-my $kdc_port = PostgreSQL::Test::Cluster::get_free_port();
-my $kdc_datadir = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc";
-my $kdc_pidfile = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc.pid";
-my $keytab = "${PostgreSQL::Test::Utils::tmp_check}/krb5.keytab";
-
my $pgpass = "${PostgreSQL::Test::Utils::tmp_check}/.pgpass";
my $dbname = 'postgres';
my $username = 'test1';
my $application = '001_auth.pl';
-note "setting up Kerberos";
-
-my ($stdout, $krb5_version);
-run_log [ $krb5_config, '--version' ], '>', \$stdout
- or BAIL_OUT("could not execute krb5-config");
-BAIL_OUT("Heimdal is not supported") if $stdout =~ m/heimdal/;
-$stdout =~ m/Kerberos 5 release ([0-9]+\.[0-9]+)/
- or BAIL_OUT("could not get Kerberos version");
-$krb5_version = $1;
-
# Construct a pgpass file to make sure we don't use it
append_to_file($pgpass, '*:*:*:*:abc123');
chmod 0600, $pgpass;
-# Build the krb5.conf to use.
-#
-# Explicitly specify the default (test) realm and the KDC for
-# that realm to avoid the Kerberos library trying to look up
-# that information in DNS, and also because we're using a
-# non-standard KDC port.
-#
-# Also explicitly disable DNS lookups since this isn't really
-# our domain and we shouldn't be causing random DNS requests
-# to be sent out (not to mention that broken DNS environments
-# can cause the tests to take an extra long time and timeout).
-#
-# Reverse DNS is explicitly disabled to avoid any issue with a
-# captive portal or other cases where the reverse DNS succeeds
-# and the Kerberos library uses that as the canonical name of
-# the host and then tries to acquire a cross-realm ticket.
-append_to_file(
- $krb5_conf,
- qq![logging]
-default = FILE:$krb5_log
-kdc = FILE:$kdc_log
-
-[libdefaults]
-dns_lookup_realm = false
-dns_lookup_kdc = false
-default_realm = $realm
-forwardable = false
-rdns = false
-
-[realms]
-$realm = {
- kdc = $hostaddr:$kdc_port
-}
-!);
-
-append_to_file(
- $kdc_conf,
- qq![kdcdefaults]
-!);
-
-# For new-enough versions of krb5, use the _listen settings rather
-# than the _ports settings so that we can bind to localhost only.
-if ($krb5_version >= 1.15)
-{
- append_to_file(
- $kdc_conf,
- qq!kdc_listen = $hostaddr:$kdc_port
-kdc_tcp_listen = $hostaddr:$kdc_port
-!);
-}
-else
-{
- append_to_file(
- $kdc_conf,
- qq!kdc_ports = $kdc_port
-kdc_tcp_ports = $kdc_port
-!);
-}
-append_to_file(
- $kdc_conf,
- qq!
-[realms]
-$realm = {
- database_name = $kdc_datadir/principal
- admin_keytab = FILE:$kdc_datadir/kadm5.keytab
- acl_file = $kdc_datadir/kadm5.acl
- key_stash_file = $kdc_datadir/_k5.$realm
-}!);
-
-mkdir $kdc_datadir or die;
-
-# Ensure that we use test's config and cache files, not global ones.
-$ENV{'KRB5_CONFIG'} = $krb5_conf;
-$ENV{'KRB5_KDC_PROFILE'} = $kdc_conf;
-$ENV{'KRB5CCNAME'} = $krb5_cache;
+note "setting up Kerberos";
-my $service_principal = "$ENV{with_krb_srvnam}/$host";
+my $host = 'auth-test-localhost.postgresql.example.com';
+my $hostaddr = '127.0.0.1';
+my $realm = 'EXAMPLE.COM';
-system_or_bail $kdb5_util, 'create', '-s', '-P', 'secret0';
+my $krb = PostgreSQL::Test::Kerberos->new($host, $hostaddr, $realm);
my $test1_password = 'secret1';
-system_or_bail $kadmin_local, '-q', "addprinc -pw $test1_password test1";
-
-system_or_bail $kadmin_local, '-q', "addprinc -randkey $service_principal";
-system_or_bail $kadmin_local, '-q', "ktadd -k $keytab $service_principal";
-
-system_or_bail $krb5kdc, '-P', $kdc_pidfile;
-
-END
-{
- kill 'INT', `cat $kdc_pidfile` if defined($kdc_pidfile) && -f $kdc_pidfile;
-}
+$krb->create_principal('test1', $test1_password);
note "setting up PostgreSQL instance";
@@ -213,7 +64,7 @@ $node->init;
$node->append_conf(
'postgresql.conf', qq{
listen_addresses = '$hostaddr'
-krb_server_keyfile = '$keytab'
+krb_server_keyfile = '$krb->{keytab}'
log_connections = on
lc_messages = 'C'
});
@@ -327,8 +178,7 @@ $node->restart;
test_access($node, 'test1', 'SELECT true', 2, '', 'fails without ticket');
-run_log [ $kinit, 'test1' ], \$test1_password or BAIL_OUT($?);
-run_log [ $klist, '-f' ] or BAIL_OUT($?);
+$krb->create_ticket('test1', $test1_password);
test_access(
$node,
@@ -470,10 +320,8 @@ $node->append_conf(
hostgssenc all all $hostaddr/32 gss map=mymap
});
-string_replace_file($krb5_conf, "forwardable = false", "forwardable = true");
-
-run_log [ $kinit, 'test1' ], \$test1_password or BAIL_OUT($?);
-run_log [ $klist, '-f' ] or BAIL_OUT($?);
+# Re-create the ticket, with the forwardable flag set
+$krb->create_ticket('test1', $test1_password, forwardable => 1);
test_access(
$node,
diff --git a/src/test/perl/PostgreSQL/Test/Kerberos.pm b/src/test/perl/PostgreSQL/Test/Kerberos.pm
new file mode 100644
index 00000000000..5bb00c76f14
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/Kerberos.pm
@@ -0,0 +1,229 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+# Sets up a stand-alone KDC for testing PostgreSQL GSSAPI / Kerberos
+# functionality.
+
+package PostgreSQL::Test::Kerberos;
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+
+our ($krb5_bin_dir, $krb5_sbin_dir, $krb5_config, $kinit, $klist,
+ $kdb5_util, $kadmin_local, $krb5kdc,
+ $krb5_conf, $kdc_conf, $krb5_cache, $krb5_log, $kdc_log,
+ $kdc_port, $kdc_datadir, $kdc_pidfile, $keytab);
+
+INIT
+{
+ if ($^O eq 'darwin' && -d "/opt/homebrew")
+ {
+ # typical paths for Homebrew on ARM
+ $krb5_bin_dir = '/opt/homebrew/opt/krb5/bin';
+ $krb5_sbin_dir = '/opt/homebrew/opt/krb5/sbin';
+ }
+ elsif ($^O eq 'darwin')
+ {
+ # typical paths for Homebrew on Intel
+ $krb5_bin_dir = '/usr/local/opt/krb5/bin';
+ $krb5_sbin_dir = '/usr/local/opt/krb5/sbin';
+ }
+ elsif ($^O eq 'freebsd')
+ {
+ $krb5_bin_dir = '/usr/local/bin';
+ $krb5_sbin_dir = '/usr/local/sbin';
+ }
+ elsif ($^O eq 'linux')
+ {
+ $krb5_sbin_dir = '/usr/sbin';
+ }
+
+ $krb5_config = 'krb5-config';
+ $kinit = 'kinit';
+ $klist = 'klist';
+ $kdb5_util = 'kdb5_util';
+ $kadmin_local = 'kadmin.local';
+ $krb5kdc = 'krb5kdc';
+
+ if ($krb5_bin_dir && -d $krb5_bin_dir)
+ {
+ $krb5_config = $krb5_bin_dir . '/' . $krb5_config;
+ $kinit = $krb5_bin_dir . '/' . $kinit;
+ $klist = $krb5_bin_dir . '/' . $klist;
+ }
+ if ($krb5_sbin_dir && -d $krb5_sbin_dir)
+ {
+ $kdb5_util = $krb5_sbin_dir . '/' . $kdb5_util;
+ $kadmin_local = $krb5_sbin_dir . '/' . $kadmin_local;
+ $krb5kdc = $krb5_sbin_dir . '/' . $krb5kdc;
+ }
+
+ $krb5_conf = "${PostgreSQL::Test::Utils::tmp_check}/krb5.conf";
+ $kdc_conf = "${PostgreSQL::Test::Utils::tmp_check}/kdc.conf";
+ $krb5_cache = "${PostgreSQL::Test::Utils::tmp_check}/krb5cc";
+ $krb5_log = "${PostgreSQL::Test::Utils::log_path}/krb5libs.log";
+ $kdc_log = "${PostgreSQL::Test::Utils::log_path}/krb5kdc.log";
+ $kdc_port = PostgreSQL::Test::Cluster::get_free_port();
+ $kdc_datadir = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc";
+ $kdc_pidfile = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc.pid";
+ $keytab = "${PostgreSQL::Test::Utils::tmp_check}/krb5.keytab";
+}
+
+=pod
+
+=item PostgreSQL::Test::Kerberos->new(host, hostaddr, realm, %params)
+
+Sets up a new Kerberos realm and KDC. This function assigns a free
+port for the KDC. The KDC will be shut down automatically when the
+test script exits.
+
+=over
+
+=item host => 'auth-test-localhost.postgresql.example.com'
+
+Hostname to use in the service principal.
+
+=item hostaddr => '127.0.0.1'
+
+Network interface the KDC will listen on.
+
+=item realm => 'EXAMPLE.COM'
+
+Name of the Kerberos realm.
+
+=back
+
+=cut
+
+sub new
+{
+ my $class = shift;
+ my ($host, $hostaddr, $realm) = @_;
+
+ my ($stdout, $krb5_version);
+ run_log [ $krb5_config, '--version' ], '>', \$stdout
+ or BAIL_OUT("could not execute krb5-config");
+ BAIL_OUT("Heimdal is not supported") if $stdout =~ m/heimdal/;
+ $stdout =~ m/Kerberos 5 release ([0-9]+\.[0-9]+)/
+ or BAIL_OUT("could not get Kerberos version");
+ $krb5_version = $1;
+
+ # Build the krb5.conf to use.
+ #
+ # Explicitly specify the default (test) realm and the KDC for
+ # that realm to avoid the Kerberos library trying to look up
+ # that information in DNS, and also because we're using a
+ # non-standard KDC port.
+ #
+ # Also explicitly disable DNS lookups since this isn't really
+ # our domain and we shouldn't be causing random DNS requests
+ # to be sent out (not to mention that broken DNS environments
+ # can cause the tests to take an extra long time and timeout).
+ #
+ # Reverse DNS is explicitly disabled to avoid any issue with a
+ # captive portal or other cases where the reverse DNS succeeds
+ # and the Kerberos library uses that as the canonical name of
+ # the host and then tries to acquire a cross-realm ticket.
+ append_to_file(
+ $krb5_conf,
+ qq![logging]
+default = FILE:$krb5_log
+kdc = FILE:$kdc_log
+
+[libdefaults]
+dns_lookup_realm = false
+dns_lookup_kdc = false
+default_realm = $realm
+forwardable = false
+rdns = false
+
+[realms]
+$realm = {
+ kdc = $hostaddr:$kdc_port
+}
+!);
+
+ append_to_file(
+ $kdc_conf,
+ qq![kdcdefaults]
+!);
+
+ # For new-enough versions of krb5, use the _listen settings rather
+ # than the _ports settings so that we can bind to localhost only.
+ if ($krb5_version >= 1.15)
+ {
+ append_to_file(
+ $kdc_conf,
+ qq!kdc_listen = $hostaddr:$kdc_port
+kdc_tcp_listen = $hostaddr:$kdc_port
+!);
+ }
+ else
+ {
+ append_to_file(
+ $kdc_conf,
+ qq!kdc_ports = $kdc_port
+kdc_tcp_ports = $kdc_port
+!);
+ }
+ append_to_file(
+ $kdc_conf,
+ qq!
+[realms]
+$realm = {
+ database_name = $kdc_datadir/principal
+ admin_keytab = FILE:$kdc_datadir/kadm5.keytab
+ acl_file = $kdc_datadir/kadm5.acl
+ key_stash_file = $kdc_datadir/_k5.$realm
+}!);
+
+ mkdir $kdc_datadir or die;
+
+ # Ensure that we use test's config and cache files, not global ones.
+ $ENV{'KRB5_CONFIG'} = $krb5_conf;
+ $ENV{'KRB5_KDC_PROFILE'} = $kdc_conf;
+ $ENV{'KRB5CCNAME'} = $krb5_cache;
+
+ my $service_principal = "$ENV{with_krb_srvnam}/$host";
+
+ system_or_bail $kdb5_util, 'create', '-s', '-P', 'secret0';
+
+ system_or_bail $kadmin_local, '-q', "addprinc -randkey $service_principal";
+ system_or_bail $kadmin_local, '-q', "ktadd -k $keytab $service_principal";
+
+ system_or_bail $krb5kdc, '-P', $kdc_pidfile;
+
+ my $self = {};
+ $self->{keytab} = $keytab;
+
+ bless $self, $class;
+
+ return $self;
+}
+
+sub create_principal
+{
+ my ($self, $principal, $password) = @_;
+
+ system_or_bail $kadmin_local, '-q', "addprinc -pw $password $principal";
+}
+
+sub create_ticket
+{
+ my ($self, $principal, $password, %params) = @_;
+
+ my @cmd = ($kinit, $principal);
+
+ push @cmd, '-f' if ($params{forwardable});
+
+ run_log [@cmd], \$password or BAIL_OUT($?);
+ run_log [ $klist, '-f' ] or BAIL_OUT($?);
+}
+
+END
+{
+ kill 'INT', `cat $kdc_pidfile` if defined($kdc_pidfile) && -f $kdc_pidfile;
+}
+
+1;
--
2.39.2
v8-0002-Add-tests-for-libpq-choosing-encryption-mode.patchtext/x-patch; charset=UTF-8; name=v8-0002-Add-tests-for-libpq-choosing-encryption-mode.patchDownload
From 90140f1093281691bdba185d96f3b07e27d4a50e Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 5 Mar 2024 15:41:09 +0200
Subject: [PATCH v8 2/6] Add tests for libpq choosing encryption mode
Author: Heikki Linnakangas, Matthias van de Meent
Discussion: XX
---
src/test/libpq_encryption/Makefile | 25 ++
src/test/libpq_encryption/README | 23 ++
src/test/libpq_encryption/meson.build | 19 +
.../t/001_negotiate_encryption.pl | 369 ++++++++++++++++++
src/test/perl/PostgreSQL/Test/Kerberos.pm | 6 +-
5 files changed, 439 insertions(+), 3 deletions(-)
create mode 100644 src/test/libpq_encryption/Makefile
create mode 100644 src/test/libpq_encryption/README
create mode 100644 src/test/libpq_encryption/meson.build
create mode 100644 src/test/libpq_encryption/t/001_negotiate_encryption.pl
diff --git a/src/test/libpq_encryption/Makefile b/src/test/libpq_encryption/Makefile
new file mode 100644
index 00000000000..710929c4cce
--- /dev/null
+++ b/src/test/libpq_encryption/Makefile
@@ -0,0 +1,25 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for src/test/libpq_encryption
+#
+# Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/test/ldap/libpq_encryption
+#
+#-------------------------------------------------------------------------
+
+subdir = src/test/libpq_encryption
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+export OPENSSL with_ssl with_gssapi with_krb_srvnam
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
+
+clean distclean:
+ rm -rf tmp_check
diff --git a/src/test/libpq_encryption/README b/src/test/libpq_encryption/README
new file mode 100644
index 00000000000..66430b54316
--- /dev/null
+++ b/src/test/libpq_encryption/README
@@ -0,0 +1,23 @@
+src/test/libpq_encryption/README
+
+Tests for negotiating network encryption method
+===============================================
+
+
+Running the tests
+=================
+
+NOTE: You must have given the --enable-tap-tests argument to configure.
+
+Run
+ make check PG_TEST_EXTRA=libpq_encryption
+
+XXX You can use "make installcheck" if you previously did "make install".
+In that case, the code in the installation tree is tested. With
+"make check", a temporary installation tree is built from the current
+sources and then tested.
+
+XXX Either way, this test initializes, starts, and stops a test Postgres
+cluster, as well as a test LDAP server.
+
+See src/test/perl/README for more info about running these tests.
diff --git a/src/test/libpq_encryption/meson.build b/src/test/libpq_encryption/meson.build
new file mode 100644
index 00000000000..04f479e9fe7
--- /dev/null
+++ b/src/test/libpq_encryption/meson.build
@@ -0,0 +1,19 @@
+# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+
+tests += {
+ 'name': 'ldap',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'tap': {
+ 'tests': [
+ 't/001_auth.pl',
+ 't/002_bindpasswd.pl',
+ ],
+ 'env': {
+ 'with_ssl': ssl_library,
+ 'OPENSSL': openssl.found() ? openssl.path() : '',
+ 'with_gssapi': gssapi.found() ? 'yes' : 'no',
+ 'with_krb_srvnam': 'postgres',
+ },
+ },
+}
diff --git a/src/test/libpq_encryption/t/001_negotiate_encryption.pl b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
new file mode 100644
index 00000000000..b8646c5bc97
--- /dev/null
+++ b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
@@ -0,0 +1,369 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+# Test negotiation of SSL and GSSAPI encryption
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Kerberos;
+use File::Basename;
+use File::Copy;
+use Test::More;
+
+if (!$ENV{PG_TEST_EXTRA} || $ENV{PG_TEST_EXTRA} !~ /\blibpq_encryption\b/)
+{
+ plan skip_all =>
+ 'Potentially unsafe test libpq_encryption not enabled in PG_TEST_EXTRA';
+}
+
+my $host = 'enc-test-localhost.postgresql.example.com';
+my $hostaddr = '127.0.0.1';
+my $servercidr = '127.0.0.1/32';
+
+note "setting up PostgreSQL instance";
+
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init;
+$node->append_conf(
+ 'postgresql.conf', qq{
+listen_addresses = '$hostaddr'
+log_connections = on
+lc_messages = 'C'
+});
+my $pgdata = $node->data_dir;
+
+my $dbname = 'postgres';
+my $username = 'enctest';
+my $application = '001_negotiate_encryption.pl';
+
+my $gssuser_password = 'secret1';
+
+my $krb;
+
+my $ssl_supported = $ENV{with_ssl} eq 'openssl';
+my $gss_supported = $ENV{with_gssapi} eq 'yes';
+
+if ($gss_supported != 0)
+{
+ note "setting up Kerberos";
+
+ my $realm = 'EXAMPLE.COM';
+ $krb = PostgreSQL::Test::Kerberos->new($host, $hostaddr, $realm);
+ $node->append_conf('postgresql.conf', "krb_server_keyfile = '$krb->{keytab}'\n");
+}
+
+if ($ssl_supported != 0)
+{
+ my $certdir = dirname(__FILE__) . "/../../ssl/ssl";
+
+ copy "$certdir/server-cn-only.crt", "$pgdata/server.crt"
+ || die "copying server.crt: $!";
+ copy "$certdir/server-cn-only.key", "$pgdata/server.key"
+ || die "copying server.key: $!";
+ chmod(0600, "$pgdata/server.key");
+
+ # Start with SSL disabled.
+ $node->append_conf('postgresql.conf', "ssl = off\n");
+}
+
+$node->start;
+
+$node->safe_psql('postgres', 'CREATE USER localuser;');
+$node->safe_psql('postgres', 'CREATE USER testuser;');
+$node->safe_psql('postgres', 'CREATE USER ssluser;');
+$node->safe_psql('postgres', 'CREATE USER nossluser;');
+$node->safe_psql('postgres', 'CREATE USER gssuser;');
+$node->safe_psql('postgres', 'CREATE USER nogssuser;');
+
+my $unixdir = $node->safe_psql('postgres', 'SHOW unix_socket_directories;');
+chomp($unixdir);
+
+$node->safe_psql('postgres', q{
+CREATE FUNCTION current_enc() RETURNS text LANGUAGE plpgsql AS $$
+DECLARE
+ ssl_in_use bool;
+ gss_in_use bool;
+BEGIN
+ ssl_in_use = (SELECT ssl FROM pg_stat_ssl WHERE pid = pg_backend_pid());
+ gss_in_use = (SELECT encrypted FROM pg_stat_gssapi WHERE pid = pg_backend_pid());
+
+ raise log 'ssl % gss %', ssl_in_use, gss_in_use;
+
+ IF ssl_in_use AND gss_in_use THEN
+ RETURN 'ssl+gss'; -- shouldn't happen
+ ELSIF ssl_in_use THEN
+ RETURN 'ssl';
+ ELSIF gss_in_use THEN
+ RETURN 'gss';
+ ELSE
+ RETURN 'plain';
+ END IF;
+END;
+$$;
+});
+
+# Only accept SSL connections from $servercidr. Our tests don't depend on this
+# but seems best to keep it as narrow as possible for security reasons.
+#
+# When connecting to certdb, also check the client certificate.
+open my $hba, '>', "$pgdata/pg_hba.conf";
+print $hba qq{
+# TYPE DATABASE USER ADDRESS METHOD OPTIONS
+local postgres localuser trust
+host postgres testuser $servercidr trust
+hostnossl postgres nossluser $servercidr trust
+hostnogssenc postgres nogssuser $servercidr trust
+};
+
+print $hba qq{
+hostssl postgres ssluser $servercidr trust
+} if ($ssl_supported != 0);
+
+print $hba qq{
+hostgssenc postgres gssuser $servercidr trust
+} if ($gss_supported != 0);
+close $hba;
+$node->reload;
+
+sub connect_test
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $connstr, $expected_enc, @expect_log_msgs)
+ = @_;
+
+ my $test_name = " '$connstr' -> $expected_enc";
+
+ my $connstr_full = "";
+ $connstr_full .= "dbname=postgres " unless $connstr =~ m/dbname=/;
+ $connstr_full .= "host=$host hostaddr=$hostaddr " unless $connstr =~ m/host=/;
+ $connstr_full .= $connstr;
+
+ my $log_location = -s $node->logfile;
+
+ my ($ret, $stdout, $stderr) = $node->psql(
+ 'postgres',
+ 'SELECT current_enc()',
+ extra_params => ['-w'],
+ connstr => "$connstr_full",
+ on_error_stop => 0);
+
+ my $result = $ret == 0 ? $stdout : 'fail';
+
+ is($result, $expected_enc, $test_name);
+
+ if (@expect_log_msgs)
+ {
+ # Match every message literally.
+ my @regexes = map { qr/\Q$_\E/ } @expect_log_msgs;
+ my %params = ();
+ $params{log_like} = \@regexes;
+ $node->log_check($test_name, $log_location, %params);
+ }
+}
+
+# Return the encryption mode that we expect to be chosen by libpq,
+# when connecting with given the user, gssmode, sslmode settings.
+sub resolve_connection_type
+{
+ my ($config) = @_;
+ my $user = $config->{user};
+ my $gssmode = $config->{gssmode};
+ my $sslmode = $config->{sslmode};
+
+ my @conntypes = qw(plain);
+
+ # Add connection types supported by the server to the pool
+ push(@conntypes, "ssl") if $config->{server_ssl} == 1;
+ push(@conntypes, "gss") if $config->{server_gss} == 1;
+
+ # User configurations:
+ # gssuser/ssluser require the relevant connection type,
+ @conntypes = grep {/gss/} @conntypes if $user eq 'gssuser';
+ @conntypes = grep {/ssl/} @conntypes if $user eq 'ssluser';
+
+ # nogssuser/nossluser require anything but the relevant connection type.
+ @conntypes = grep {!/gss/} @conntypes if $user eq 'nogssuser';
+ @conntypes = grep {!/ssl/} @conntypes if $user eq 'nossluser';
+
+ print STDOUT "After user filter: @conntypes\n";
+
+ # remove disabled connection modes
+ @conntypes = grep {!/gss/} @conntypes if $gssmode eq 'disable';
+ @conntypes = grep {!/ssl/} @conntypes if $sslmode eq 'disable';
+
+ # If gssmode=require, drop all non-GSS modes.
+ if ($gssmode eq 'require')
+ {
+ @conntypes = grep {/gss/} @conntypes;
+ }
+
+ # If sslmode=require, drop plain mode.
+ #
+ # NOTE: GSS is also allowed with sslmode=require.
+ if ($sslmode eq 'require')
+ {
+ @conntypes = grep {!/plain/} @conntypes;
+ }
+
+ print STDOUT "After mode require filter: @conntypes\n";
+
+ # Handle priorities of the various types.
+ # Note that this doesn't need to care about require/disable/etc, those
+ # filters were applied before we get here.
+ # Also note that preference is 1 > 2 > 3 > 4 > 5, so first preference
+ # without ssl or gss 'prefer/require' is plain connections.
+ my %order = (plain=>3, gss=>4, ssl=>5);
+
+ $order{ssl} = 2 if $sslmode eq "prefer";
+ $order{gss} = 1 if $gssmode eq "prefer";
+ @conntypes = sort { $order{$a} cmp $order{$b} } @conntypes;
+
+ # If there are no connection types available after filtering requirements,
+ # the connection fails.
+ return "fail" if @conntypes == 0;
+ # Else, we get to connect using the connection type with the highest
+ # priority.
+ return $conntypes[0];
+}
+
+# First test with SSL disabled in the server
+
+# Test the cube of parameters: user, sslmode, and gssencmode
+sub test_modes
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($pg_node, $node_conf,
+ $test_users, $ssl_modes, $gss_modes) = @_;
+
+ foreach my $test_user (@{$test_users})
+ {
+ foreach my $client_mode (@{$ssl_modes})
+ {
+ foreach my $gssencmode (@{$gss_modes})
+ {
+ my %params = (
+ server_ssl=>$node_conf->{server_ssl},
+ server_gss=>$node_conf->{server_gss},
+ user=>$test_user,
+ sslmode=>$client_mode,
+ gssmode=>$gssencmode,
+ );
+ my $res = resolve_connection_type(\%params);
+ connect_test($pg_node, "user=$test_user sslmode=$client_mode gssencmode=$gssencmode", $res);
+ }
+ }
+ }
+}
+
+my $sslmodes = ['disable', 'allow', 'prefer', 'require'];
+my $gssencmodes = ['disable', 'prefer', 'require'];
+
+my $server_config = {
+ server_ssl => 0,
+ server_gss => 0,
+};
+
+note("Running tests with SSL and GSS disabled in server");
+test_modes($node, $server_config,
+ ['testuser'],
+ $sslmodes, $gssencmodes);
+
+# Enable SSL in the server
+SKIP:
+{
+ skip "SSL not supported by this build" if $ssl_supported == 0;
+
+ $node->adjust_conf('postgresql.conf', 'ssl', 'on');
+ $node->restart;
+ $server_config->{server_ssl} = 1;
+
+ note("Running tests with SSL enabled in server");
+ test_modes($node, $server_config,
+ ['testuser', 'ssluser', 'nossluser'],
+ $sslmodes, ['disable']);
+
+ $node->adjust_conf('postgresql.conf', 'ssl', 'off');
+ $node->reload;
+ $server_config->{server_ssl} = 0;
+}
+
+# Test GSSAPI
+SKIP:
+{
+ skip "GSSAPI/Kerberos not supported by this build" if $gss_supported == 0;
+
+ # No ticket
+ connect_test($node, 'user=testuser sslmode=disable gssencmode=require', 'fail');
+
+ $krb->create_principal('gssuser', $gssuser_password);
+ $krb->create_ticket('gssuser', $gssuser_password);
+ $server_config->{server_gss} = 1;
+
+ note("Running tests with GSS enabled in server");
+ test_modes($node, $server_config,
+ ['testuser', 'gssuser', 'nogssuser'],
+ $sslmodes, $gssencmodes);
+
+ # Check that logs match the expected 'no pg_hba.conf entry' line, too, as
+ # that is not tested by test_modes.
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=require', 'fail',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+
+ # With 'gssencmode=prefer', libpq will first negotiate GSSAPI
+ # encryption, but the connection will fail because pg_hba.conf
+ # forbids GSSAPI encryption for this user. It will then reconnect
+ # with SSL, but the server doesn't support it, so it will continue
+ # with no encryption.
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer', 'plain',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+}
+
+# Server supports both SSL and GSSAPI
+SKIP:
+{
+ skip "GSSAPI/Kerberos or SSL not supported by this build" unless ($ssl_supported && $gss_supported);
+
+ # SSL is still disabled
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer', 'gss');
+
+ # Enable SSL
+ $node->adjust_conf('postgresql.conf', 'ssl', 'on');
+ $node->reload;
+ $server_config->{server_ssl} = 1;
+
+ note("Running tests with both GSS and SSL enabled in server");
+ test_modes($node, $server_config,
+ ['testuser', 'gssuser', 'ssluser', 'nogssuser', 'nossluser'],
+ $sslmodes, $gssencmodes);
+
+ # Test case that server supports GSSAPI, but it's not allowed for
+ # this user. Special cased because we check output
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=require', 'fail',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+
+ # with 'gssencmode=prefer', libpq will first negotiate GSSAPI
+ # encryption, but the connection will fail because pg_hba.conf
+ # forbids GSSAPI encryption for this user. It will then reconnect
+ # with SSL.
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer', 'ssl',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+
+ # Setting both sslmode=require and gssencmode=require fails if GSSAPI is not
+ # available.
+ connect_test($node, 'user=nogssuser sslmode=require gssencmode=require', 'fail');
+}
+
+# Test negotiation over unix domain sockets.
+SKIP:
+{
+ skip "Unix domain sockets not supported" unless ($unixdir ne "");
+
+ connect_test($node, "user=localuser sslmode=require gssencmode=prefer host=$unixdir", 'plain');
+ connect_test($node, "user=localuser sslmode=prefer gssencmode=require host=$unixdir", 'fail');
+}
+
+done_testing();
diff --git a/src/test/perl/PostgreSQL/Test/Kerberos.pm b/src/test/perl/PostgreSQL/Test/Kerberos.pm
index 5bb00c76f14..e6063703c3a 100644
--- a/src/test/perl/PostgreSQL/Test/Kerberos.pm
+++ b/src/test/perl/PostgreSQL/Test/Kerberos.pm
@@ -103,10 +103,10 @@ sub new
my ($stdout, $krb5_version);
run_log [ $krb5_config, '--version' ], '>', \$stdout
- or BAIL_OUT("could not execute krb5-config");
- BAIL_OUT("Heimdal is not supported") if $stdout =~ m/heimdal/;
+ or die("could not execute krb5-config");
+ die("Heimdal is not supported") if $stdout =~ m/heimdal/;
$stdout =~ m/Kerberos 5 release ([0-9]+\.[0-9]+)/
- or BAIL_OUT("could not get Kerberos version");
+ or die("could not get Kerberos version");
$krb5_version = $1;
# Build the krb5.conf to use.
--
2.39.2
v8-0003-Direct-SSL-connections-client-and-server-support.patchtext/x-patch; charset=UTF-8; name=v8-0003-Direct-SSL-connections-client-and-server-support.patchDownload
From 060188083d483d725939d77efb51ccd7d16e7ba2 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Wed, 15 Mar 2023 15:51:02 -0400
Subject: [PATCH v8 3/6] Direct SSL connections, client and server support
Author: Greg Stark, Heikki Linnakangas
---
doc/src/sgml/libpq.sgml | 102 +++++++++++++++++++++++++---
doc/src/sgml/protocol.sgml | 37 ++++++++++
src/backend/libpq/be-secure.c | 52 +++++++++++++-
src/backend/libpq/pqcomm.c | 9 +--
src/backend/postmaster/postmaster.c | 84 +++++++++++++++++++++--
src/include/libpq/libpq-be.h | 13 ++++
src/include/libpq/libpq.h | 2 +-
src/interfaces/libpq/fe-connect.c | 99 +++++++++++++++++++++++++--
src/interfaces/libpq/libpq-fe.h | 3 +-
src/interfaces/libpq/libpq-int.h | 4 ++
10 files changed, 375 insertions(+), 30 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 1d8998efb2a..88bcfe44d8e 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1691,10 +1691,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
Note that if <acronym>GSSAPI</acronym> encryption is possible,
that will be used in preference to <acronym>SSL</acronym>
encryption, regardless of the value of <literal>sslmode</literal>.
- To force use of <acronym>SSL</acronym> encryption in an
- environment that has working <acronym>GSSAPI</acronym>
- infrastructure (such as a Kerberos server), also
- set <literal>gssencmode</literal> to <literal>disable</literal>.
+ To negotiate <acronym>SSL</acronym> encryption in an environment that
+ has working <acronym>GSSAPI</acronym> infrastructure (such as a
+ Kerberos server), also set <literal>gssencmode</literal>
+ to <literal>disable</literal>.
+ Use of non-default values of <literal>sslnegotiation</literal> can
+ also cause <acronym>SSL</acronym> to be used instead of
+ negotiating <acronym>GSSAPI</acronym> encryption.
</para>
</listitem>
</varlistentry>
@@ -1721,6 +1724,75 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
</listitem>
</varlistentry>
+ <varlistentry id="libpq-connect-sslnegotiation" xreflabel="sslnegotiation">
+ <term><literal>sslnegotiation</literal></term>
+ <listitem>
+ <para>
+ This option controls whether <productname>PostgreSQL</productname>
+ will perform its protocol negotiation to request encryption from the
+ server or will just directly make a standard <acronym>SSL</acronym>
+ connection. Traditional <productname>PostgreSQL</productname>
+ protocol negotiation is the default and the most flexible with
+ different server configurations. If the server is known to support
+ direct <acronym>SSL</acronym> connections then the latter requires one
+ fewer round trip reducing connection latency and also allows the use
+ of protocol agnostic SSL network tools.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>postgres</literal></term>
+ <listitem>
+ <para>
+ perform <productname>PostgreSQL</productname> protocol
+ negotiation. This is the default if the option is not provided.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>direct</literal></term>
+ <listitem>
+ <para>
+ first attempt to establish a standard SSL connection and if that
+ fails reconnect and perform the negotiation. This fallback
+ process adds significant latency if the initial SSL connection
+ fails.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>requiredirect</literal></term>
+ <listitem>
+ <para>
+ attempt to establish a standard SSL connection and if that fails
+ return a connection failure immediately.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ If <literal>sslmode</literal> set to <literal>disable</literal>
+ or <literal>allow</literal> then <literal>sslnegotiation</literal> is
+ ignored. If <literal>gssencmode</literal> is set
+ to <literal>require</literal> then <literal>sslnegotiation</literal>
+ must be the default <literal>postgres</literal> value.
+ </para>
+
+ <para>
+ Moreover, note if <literal>gssencmode</literal> is set
+ to <literal>prefer</literal> and <literal>sslnegotiation</literal>
+ to <literal>direct</literal> then the effective preference will be
+ direct <acronym>SSL</acronym> connections, followed by
+ negotiated <acronym>GSS</acronym> connections, followed by
+ negotiated <acronym>SSL</acronym> connections, possibly followed
+ lastly by unencrypted connections.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-connect-sslcompression" xreflabel="sslcompression">
<term><literal>sslcompression</literal></term>
<listitem>
@@ -1954,11 +2026,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
<para>
The Server Name Indication can be used by SSL-aware proxies to route
- connections without having to decrypt the SSL stream. (Note that this
- requires a proxy that is aware of the PostgreSQL protocol handshake,
- not just any SSL proxy.) However, <acronym>SNI</acronym> makes the
- destination host name appear in cleartext in the network traffic, so
- it might be undesirable in some cases.
+ connections without having to decrypt the SSL stream. (Note that
+ unless the proxy is aware of the PostgreSQL protocol handshake this
+ would require setting <literal>sslnegotiation</literal>
+ to <literal>direct</literal> or <literal>requiredirect</literal>.)
+ However, <acronym>SNI</acronym> makes the destination host name appear
+ in cleartext in the network traffic, so it might be undesirable in
+ some cases.
</para>
</listitem>
</varlistentry>
@@ -8149,6 +8223,16 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
</para>
</listitem>
+ <listitem>
+ <para>
+ <indexterm>
+ <primary><envar>PGSSLNEGOTIATION</envar></primary>
+ </indexterm>
+ <envar>PGSSLNEGOTIATION</envar> behaves the same as the <xref
+ linkend="libpq-connect-sslnegotiation"/> connection parameter.
+ </para>
+ </listitem>
+
<listitem>
<para>
<indexterm>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index a5cb19357f5..b215602c5ed 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -1529,17 +1529,54 @@ SELCT 1/0;<!-- this typo is intentional -->
bytes.
</para>
+ <para>
+ Likewise the server expects the client to not begin
+ the <acronym>SSL</acronym> negotiation until it receives the server's
+ single byte response to the <acronym>SSL</acronym> request. If the
+ client begins the <acronym>SSL</acronym> negotiation immediately without
+ waiting for the server response to be received it can reduce connection
+ latency by one round-trip. However this comes at the cost of not being
+ able to handle the case where the server sends a negative response to the
+ <acronym>SSL</acronym> request. In that case instead of continuing with either GSSAPI or an
+ unencrypted connection or a protocol error the server will simply
+ disconnect.
+ </para>
+
<para>
An initial SSLRequest can also be used in a connection that is being
opened to send a CancelRequest message.
</para>
+ <para>
+ A second alternate way to initiate <acronym>SSL</acronym> encryption is
+ available. The server will recognize connections which immediately
+ begin <acronym>SSL</acronym> negotiation without any previous SSLRequest
+ packets. Once the <acronym>SSL</acronym> connection is established the
+ server will expect a normal startup-request packet and continue
+ negotiation over the encrypted channel. In this case any other requests
+ for encryption will be refused. This method is not preferred for general
+ purpose tools as it cannot negotiate the best connection encryption
+ available or handle unencrypted connections. However it is useful for
+ environments where both the server and client are controlled together.
+ In that case it avoids one round trip of latency and allows the use of
+ network tools that depend on standard <acronym>SSL</acronym> connections.
+ When using <acronym>SSL</acronym> connections in this style the client is
+ required to use the ALPN extension defined
+ by <ulink url="https://tools.ietf.org/html/rfc7301">RFC 7301</ulink> to
+ protect against protocol confusion attacks.
+ The <productname>PostgreSQL</productname> protocol is "TBD-pgsql" as
+ registered
+ at <ulink url="https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids">IANA
+ TLS ALPN Protocol IDs</ulink> registry.
+ </para>
+
<para>
While the protocol itself does not provide a way for the server to
force <acronym>SSL</acronym> encryption, the administrator can
configure the server to reject unencrypted sessions as a byproduct
of authentication checking.
</para>
+
</sect2>
<sect2 id="protocol-flow-gssapi">
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 5612c29f8b2..1d1329d1d95 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -109,18 +109,51 @@ secure_loaded_verify_locations(void)
int
secure_open_server(Port *port)
{
+#ifdef USE_SSL
int r = 0;
+ ssize_t len;
+
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ char *buf = palloc(len);
+
+ pq_startmsgread();
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ pq_endmsgread();
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+ Assert(pq_buffer_has_data() == 0);
-#ifdef USE_SSL
r = be_tls_open_server(port);
+ if (port->raw_buf_remaining > 0)
+ {
+ /*
+ * This shouldn't be possible -- it would mean the client sent
+ * encrypted data before we established a session key...
+ */
+ elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ return STATUS_ERROR;
+ }
+ if (port->raw_buf != NULL)
+ {
+ pfree(port->raw_buf);
+ port->raw_buf = NULL;
+ }
+
ereport(DEBUG2,
(errmsg_internal("SSL connection from DN:\"%s\" CN:\"%s\"",
port->peer_dn ? port->peer_dn : "(anonymous)",
port->peer_cn ? port->peer_cn : "(anonymous)")));
-#endif
-
return r;
+#else
+ return 0;
+#endif
}
/*
@@ -232,6 +265,19 @@ secure_raw_read(Port *port, void *ptr, size_t len)
{
ssize_t n;
+ /* Read from the "unread" buffered data first. c.f. libpq-be.h */
+ if (port->raw_buf_remaining > 0)
+ {
+ /* consume up to len bytes from the raw_buf */
+ if (len > port->raw_buf_remaining)
+ len = port->raw_buf_remaining;
+ Assert(port->raw_buf);
+ memcpy(ptr, port->raw_buf + port->raw_buf_consumed, len);
+ port->raw_buf_consumed += len;
+ port->raw_buf_remaining -= len;
+ return len;
+ }
+
/*
* Try to read from the socket without blocking. If it succeeds we're
* done, otherwise we'll wait for the socket using the latch mechanism.
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index c606bf34473..caa502b6373 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -1134,15 +1134,16 @@ pq_discardbytes(size_t len)
}
/* --------------------------------
- * pq_buffer_has_data - is any buffered data available to read?
+ * pq_buffer_has_data - return number of bytes in receive buffer
*
- * This will *not* attempt to read more data.
+ * This will *not* attempt to read more data. And reading up to that number of
+ * bytes should not cause reading any more data either.
* --------------------------------
*/
-bool
+size_t
pq_buffer_has_data(void)
{
- return (PqRecvPointer < PqRecvLength);
+ return (PqRecvLength - PqRecvPointer);
}
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index f3c09e8dc0d..ab82faf6aa0 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -423,6 +423,7 @@ static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
static int BackendStartup(Port *port);
+static int ProcessSSLStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -1922,6 +1923,69 @@ ServerLoop(void)
}
}
+/*
+ * Check for a native direct SSL connection.
+ *
+ * This happens before startup packets so we are careful not to actual read
+ * any bytes from the stream if it's not a direct SSL connection.
+ */
+static int
+ProcessSSLStartup(Port *port)
+{
+ int firstbyte;
+
+ Assert(!port->ssl_in_use);
+
+ pq_startmsgread();
+ firstbyte = pq_peekbyte();
+ pq_endmsgread();
+ if (firstbyte == EOF)
+ {
+ /*
+ * Like in ProcessStartupPacket, if we get no data at all, don't
+ * clutter the log with a complaint.
+ */
+ return STATUS_ERROR;
+ }
+
+ /*
+ * First byte indicates standard SSL handshake message
+ *
+ * (It can't be a Postgres startup length because in network byte order
+ * that would be a startup packet hundreds of megabytes long)
+ */
+ if (firstbyte == 0x16)
+ {
+ elog(LOG, "Detected direct SSL handshake");
+
+#ifdef USE_SSL
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ {
+ /* SSL not supported */
+ return STATUS_ERROR;
+ }
+ else if (secure_open_server(port) == -1)
+ {
+ /*
+ * we assume secure_open_server() sent an appropriate TLS alert
+ * already
+ */
+ return STATUS_ERROR;
+ }
+#else
+ /* SSL not supported by this build */
+ return STATUS_ERROR;
+#endif
+ }
+
+ if (port->ssl_in_use)
+ ereport(DEBUG2,
+ (errmsg_internal("Direct SSL connection established")));
+
+ return STATUS_OK;
+}
+
+
/*
* Read a client's startup packet and do something according to it.
*
@@ -2035,8 +2099,13 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
char SSLok;
#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+
+ /*
+ * No SSL when disabled or on Unix sockets.
+ *
+ * Also no SSL negotiation if we already have a direct SSL connection
+ */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX || port->ssl_in_use)
SSLok = 'N';
else
SSLok = 'S'; /* Support for SSL */
@@ -2044,11 +2113,10 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
SSLok = 'N'; /* No support for SSL */
#endif
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
+ while (secure_write(port, &SSLok, 1) != 1)
{
if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
+ continue; /* if interrupted, just retry */
ereport(COMMERROR,
(errcode_for_socket_access(),
errmsg("failed to send SSL negotiation response: %m")));
@@ -2089,7 +2157,7 @@ retry1:
GSSok = 'G';
#endif
- while (send(port->sock, &GSSok, 1, 0) != 1)
+ while (secure_write(port, &GSSok, 1) != 1)
{
if (errno == EINTR)
continue;
@@ -4358,7 +4426,9 @@ BackendInitialize(Port *port)
* Receive the startup packet (which might turn out to be a cancel request
* packet).
*/
- status = ProcessStartupPacket(port, false, false);
+ status = ProcessSSLStartup(port);
+ if (status == STATUS_OK)
+ status = ProcessStartupPacket(port, false, false);
/*
* If we're going to reject the connection due to database state, say so
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 47d66d55241..1f7ce041359 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -227,6 +227,19 @@ typedef struct Port
SSL *ssl;
X509 *peer;
#endif
+
+ /*
+ * This is a bit of a hack. The raw_buf is data that was previously read
+ * and buffered in a higher layer but then "unread" and needs to be read
+ * again while establishing an SSL connection via the SSL library layer.
+ *
+ * There's no API to "unread", the upper layer just places the data in the
+ * Port structure in raw_buf and sets raw_buf_remaining to the amount of
+ * bytes unread and raw_buf_consumed to 0.
+ */
+ char *raw_buf;
+ ssize_t raw_buf_consumed,
+ raw_buf_remaining;
} Port;
#ifdef USE_SSL
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 6171a0d17a5..79c5f6b754b 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -80,7 +80,7 @@ extern int pq_getmessage(StringInfo s, int maxlen);
extern int pq_getbyte(void);
extern int pq_peekbyte(void);
extern int pq_getbyte_if_available(unsigned char *c);
-extern bool pq_buffer_has_data(void);
+extern size_t pq_buffer_has_data(void);
extern int pq_putmessage_v2(char msgtype, const char *s, size_t len);
extern bool pq_check_connection(void);
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index d4e10a0c4f3..7bd371fafb5 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -129,6 +129,7 @@ static int ldapServiceLookup(const char *purl, PQconninfoOption *options,
#define DefaultSSLMode "disable"
#define DefaultSSLCertMode "disable"
#endif
+#define DefaultSSLNegotiation "postgres"
#ifdef ENABLE_GSS
#include "fe-gssapi-common.h"
#define DefaultGSSMode "prefer"
@@ -272,6 +273,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-Mode", "", 12, /* sizeof("verify-full") == 12 */
offsetof(struct pg_conn, sslmode)},
+ {"sslnegotiation", "PGSSLNEGOTIATION", DefaultSSLNegotiation, NULL,
+ "SSL-Negotiation", "", 14, /* strlen("requiredirect") == 14 */
+ offsetof(struct pg_conn, sslnegotiation)},
+
{"sslcompression", "PGSSLCOMPRESSION", "0", NULL,
"SSL-Compression", "", 1,
offsetof(struct pg_conn, sslcompression)},
@@ -1517,10 +1522,37 @@ pqConnectOptions2(PGconn *conn)
}
#endif
}
+
+ /*
+ * validate sslnegotiation option, default is "postgres" for the postgres
+ * style negotiated connection with an extra round trip but more options.
+ */
+ if (conn->sslnegotiation)
+ {
+ if (strcmp(conn->sslnegotiation, "postgres") != 0
+ && strcmp(conn->sslnegotiation, "direct") != 0
+ && strcmp(conn->sslnegotiation, "requiredirect") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
+ "sslnegotiation", conn->sslnegotiation);
+ return false;
+ }
+
+#ifndef USE_SSL
+ if (conn->sslnegotiation[0] != 'p')
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "sslnegotiation value \"%s\" invalid when SSL support is not compiled in",
+ conn->sslnegotiation);
+ return false;
+ }
+#endif
+ }
else
{
- conn->sslmode = strdup(DefaultSSLMode);
- if (!conn->sslmode)
+ conn->sslnegotiation = strdup(DefaultSSLNegotiation);
+ if (!conn->sslnegotiation)
goto oom_error;
}
@@ -1643,6 +1675,21 @@ pqConnectOptions2(PGconn *conn)
conn->gssencmode);
return false;
}
+#endif
+#ifdef USE_SSL
+
+ /*
+ * GSS is incompatible with direct SSL connections so it requires the
+ * default postgres style connection ssl negotiation
+ */
+ if (strcmp(conn->gssencmode, "require") == 0 &&
+ strcmp(conn->sslnegotiation, "postgres") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when Direct SSL negotiation is enabled",
+ conn->gssencmode);
+ return false;
+ }
#endif
}
else
@@ -2730,11 +2777,12 @@ keep_going: /* We will come back to here until there is
/* initialize these values based on SSL mode */
conn->allow_ssl_try = (conn->sslmode[0] != 'd'); /* "disable" */
conn->wait_ssl_try = (conn->sslmode[0] == 'a'); /* "allow" */
+ /* direct ssl is incompatible with "allow" or "disabled" ssl */
+ conn->allow_direct_ssl_try = conn->allow_ssl_try && !conn->wait_ssl_try && (conn->sslnegotiation[0] != 'p');
#endif
#ifdef ENABLE_GSS
conn->try_gss = (conn->gssencmode[0] != 'd'); /* "disable" */
#endif
-
reset_connection_state_machine = false;
need_new_connection = true;
}
@@ -3192,6 +3240,29 @@ keep_going: /* We will come back to here until there is
if (pqsecure_initialize(conn, false, true) < 0)
goto error_return;
+ /*
+ * If SSL is enabled and direct SSL connections are enabled
+ * and we haven't already established an SSL connection (or
+ * already tried a direct connection and failed or succeeded)
+ * then try just enabling SSL directly.
+ *
+ * If we fail then we'll either fail the connection (if
+ * sslnegotiation is set to requiredirect or turn
+ * allow_direct_ssl_try to false
+ */
+ if (conn->allow_ssl_try
+ && !conn->wait_ssl_try
+ && conn->allow_direct_ssl_try
+ && !conn->ssl_in_use
+#ifdef ENABLE_GSS
+ && !conn->gssenc
+#endif
+ )
+ {
+ conn->status = CONNECTION_SSL_STARTUP;
+ return PGRES_POLLING_WRITING;
+ }
+
/*
* If SSL is enabled and we haven't already got encryption of
* some sort running, request SSL instead of sending the
@@ -3268,9 +3339,11 @@ keep_going: /* We will come back to here until there is
/*
* On first time through, get the postmaster's response to our
- * SSL negotiation packet.
+ * SSL negotiation packet. If we are trying a direct ssl
+ * connection skip reading the negotiation packet and go
+ * straight to initiating an ssl connection.
*/
- if (!conn->ssl_in_use)
+ if (!conn->ssl_in_use && !conn->allow_direct_ssl_try)
{
/*
* We use pqReadData here since it has the logic to
@@ -3376,6 +3449,21 @@ keep_going: /* We will come back to here until there is
}
if (pollres == PGRES_POLLING_FAILED)
{
+ /*
+ * Failed direct ssl connection, possibly try a new
+ * connection with postgres negotiation
+ */
+ if (conn->allow_direct_ssl_try)
+ {
+ /* if it's requiredirect then it's a hard failure */
+ if (conn->sslnegotiation[0] == 'r')
+ goto error_return;
+ /* otherwise only retry using postgres connection */
+ conn->allow_direct_ssl_try = false;
+ need_new_connection = true;
+ goto keep_going;
+ }
+
/*
* Failed ... if sslmode is "prefer" then do a non-SSL
* retry
@@ -4371,6 +4459,7 @@ freePGconn(PGconn *conn)
free(conn->keepalives_interval);
free(conn->keepalives_count);
free(conn->sslmode);
+ free(conn->sslnegotiation);
free(conn->sslcert);
free(conn->sslkey);
if (conn->sslpassword)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3f..8dd7a9af457 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -78,7 +78,8 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_DIRECT_SSL_STARTUP /* Starting SSL without PG Negotiation. */
} ConnStatusType;
typedef enum
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 82c18f870d2..640a3ae0f5a 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -388,6 +388,8 @@ struct pg_conn
char *keepalives_count; /* maximum number of TCP keepalive
* retransmits */
char *sslmode; /* SSL mode (require,prefer,allow,disable) */
+ char *sslnegotiation; /* SSL initiation style
+ * (postgres,direct,requiredirect) */
char *sslcompression; /* SSL compression (0 or 1) */
char *sslkey; /* client key filename */
char *sslcert; /* client certificate filename */
@@ -549,6 +551,8 @@ struct pg_conn
bool ssl_cert_sent; /* Did we send one in reply? */
#ifdef USE_SSL
+ bool allow_direct_ssl_try; /* Try to make a direct SSL connection
+ * without an "SSL negotiation packet" */
bool allow_ssl_try; /* Allowed to try SSL negotiation */
bool wait_ssl_try; /* Delay SSL negotiation until after
* attempting normal connection */
--
2.39.2
v8-0004-Direct-SSL-connections-ALPN-support.patchtext/x-patch; charset=UTF-8; name=v8-0004-Direct-SSL-connections-ALPN-support.patchDownload
From 3a6f22bdd0b231cc645b67cccd7511d6a38095c4 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Mon, 20 Mar 2023 14:09:30 -0400
Subject: [PATCH v8 4/6] Direct SSL connections ALPN support
Move code to check for alpn
- if non-direct SSL is used, and client sends an unexpected ALPN
protocol, that's now an error ?
Author: Greg Stark, Heikki Linnakangas
---
src/backend/libpq/be-secure-openssl.c | 84 +++++++++++++++++++
src/backend/libpq/be-secure.c | 3 +
src/backend/postmaster/postmaster.c | 13 +++
src/backend/utils/misc/guc_tables.c | 9 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/bin/psql/command.c | 7 +-
src/include/libpq/libpq-be.h | 1 +
src/include/libpq/libpq.h | 1 +
src/include/libpq/pqcomm.h | 19 +++++
src/interfaces/libpq/fe-connect.c | 5 ++
src/interfaces/libpq/fe-secure-openssl.c | 35 ++++++++
src/interfaces/libpq/libpq-int.h | 1 +
12 files changed, 177 insertions(+), 2 deletions(-)
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
index e12b1cc9e3b..afde13c2fad 100644
--- a/src/backend/libpq/be-secure-openssl.c
+++ b/src/backend/libpq/be-secure-openssl.c
@@ -67,6 +67,12 @@ static int ssl_external_passwd_cb(char *buf, int size, int rwflag, void *userdat
static int dummy_ssl_passwd_cb(char *buf, int size, int rwflag, void *userdata);
static int verify_cb(int ok, X509_STORE_CTX *ctx);
static void info_cb(const SSL *ssl, int type, int args);
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata);
static bool initialize_dh(SSL_CTX *context, bool isServerStart);
static bool initialize_ecdh(SSL_CTX *context, bool isServerStart);
static const char *SSLerrmessage(unsigned long ecode);
@@ -432,6 +438,16 @@ be_tls_open_server(Port *port)
/* set up debugging/info callback */
SSL_CTX_set_info_callback(SSL_context, info_cb);
+ if (ssl_enable_alpn)
+ {
+ elog(DEBUG2, "Enabling OpenSSL ALPN callback");
+ SSL_CTX_set_alpn_select_cb(SSL_context, alpn_cb, port);
+ }
+ else
+ {
+ elog(DEBUG2, "OpenSSL ALPN is disabled, not setting callback");
+ }
+
if (!(port->ssl = SSL_new(SSL_context)))
{
ereport(COMMERROR,
@@ -571,6 +587,32 @@ aloop:
return -1;
}
+ /* Get the protocol selected by ALPN */
+ port->alpn_used = false;
+ {
+ const unsigned char *selected;
+ unsigned int len;
+
+ SSL_get0_alpn_selected(port->ssl, &selected, &len);
+
+ /* If ALPN is used, check that we negotiated the expected protocol */
+ if (selected != NULL)
+ {
+ if (len == strlen(PG_ALPN_PROTOCOL) &&
+ memcmp(selected, PG_ALPN_PROTOCOL, strlen(PG_ALPN_PROTOCOL)) == 0)
+ {
+ port->alpn_used = true;
+ }
+ else
+ {
+ /* shouldn't happen */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received SSL connection request with unexpected ALPN protocol")));
+ }
+ }
+ }
+
/* Get client certificate, if available. */
port->peer = SSL_get_peer_certificate(port->ssl);
@@ -1259,6 +1301,48 @@ info_cb(const SSL *ssl, int type, int args)
}
}
+/* See pqcomm.h comments on OpenSSL implementation of ALPN (RFC 7301) */
+static const unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
+
+/*
+ * Server callback for ALPN negotiation. We use the standard "helper" function
+ * even though currently we only accept one value.
+ */
+static int
+alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata)
+{
+ /*
+ * Why does OpenSSL provide a helper function that requires a nonconst
+ * vector when the callback is declared to take a const vector? What are
+ * we to do with that?
+ */
+ int retval;
+
+ Assert(userdata != NULL);
+ Assert(out != NULL);
+ Assert(outlen != NULL);
+ Assert(in != NULL);
+
+ retval = SSL_select_next_proto((unsigned char **) out, outlen,
+ alpn_protos, sizeof(alpn_protos),
+ in, inlen);
+ if (*out == NULL || *outlen > sizeof(alpn_protos) || outlen <= 0)
+ return SSL_TLSEXT_ERR_NOACK; /* can't happen */
+
+ if (retval == OPENSSL_NPN_NEGOTIATED)
+ return SSL_TLSEXT_ERR_OK;
+ else if (retval == OPENSSL_NPN_NO_OVERLAP)
+ return SSL_TLSEXT_ERR_NOACK;
+ else
+ return SSL_TLSEXT_ERR_NOACK;
+}
+
+
/*
* Set DH parameters for generating ephemeral DH keys. The
* DH parameters can take a long time to compute, so they must be
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 1d1329d1d95..20a1a4ad551 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -58,6 +58,9 @@ bool SSLPreferServerCiphers;
int ssl_min_protocol_version = PG_TLS1_2_VERSION;
int ssl_max_protocol_version = PG_TLS_ANY;
+/* GUC variable: if false ignore ALPN negotiation */
+bool ssl_enable_alpn = true;
+
/* ------------------------------------------------------------ */
/* Procedures common to all secure sessions */
/* ------------------------------------------------------------ */
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index ab82faf6aa0..cc6d7b477e2 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1958,6 +1958,11 @@ ProcessSSLStartup(Port *port)
{
elog(LOG, "Detected direct SSL handshake");
+ if (!ssl_enable_alpn)
+ {
+ elog(WARNING, "Received direct SSL connection without ssl_enable_alpn enabled");
+ }
+
#ifdef USE_SSL
if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
{
@@ -1972,6 +1977,14 @@ ProcessSSLStartup(Port *port)
*/
return STATUS_ERROR;
}
+
+ if (!port->alpn_used)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request without required ALPN protocol negotiation extension")));
+ return STATUS_ERROR;
+ }
#else
/* SSL not supported by this build */
return STATUS_ERROR;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 45013582a74..b7f7b2837aa 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1085,6 +1085,15 @@ struct config_bool ConfigureNamesBool[] =
true,
NULL, NULL, NULL
},
+ {
+ {"ssl_enable_alpn", PGC_SIGHUP, CONN_AUTH_SSL,
+ gettext_noop("Respond to TLS ALPN Extension Requests."),
+ NULL,
+ },
+ &ssl_enable_alpn,
+ true,
+ NULL, NULL, NULL
+ },
{
{"fsync", PGC_SIGHUP, WAL_SETTINGS,
gettext_noop("Forces synchronization of updates to disk."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index edcc0282b2d..880cd9438cb 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -127,6 +127,7 @@
#ssl_dh_params_file = ''
#ssl_passphrase_command = ''
#ssl_passphrase_command_supports_reload = off
+#ssl_enable_alpn = on
#------------------------------------------------------------------------------
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 5c906e48068..07f13f9c04e 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -3814,6 +3814,7 @@ printSSLInfo(void)
const char *protocol;
const char *cipher;
const char *compression;
+ const char *alpn;
if (!PQsslInUse(pset.db))
return; /* no SSL */
@@ -3821,11 +3822,13 @@ printSSLInfo(void)
protocol = PQsslAttribute(pset.db, "protocol");
cipher = PQsslAttribute(pset.db, "cipher");
compression = PQsslAttribute(pset.db, "compression");
+ alpn = PQsslAttribute(pset.db, "alpn");
- printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s)\n"),
+ printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s, ALPN: %s)\n"),
protocol ? protocol : _("unknown"),
cipher ? cipher : _("unknown"),
- (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"));
+ (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"),
+ alpn ? alpn : _("none"));
}
/*
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 1f7ce041359..53c08370a4c 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -218,6 +218,7 @@ typedef struct Port
char *peer_cn;
char *peer_dn;
bool peer_cert_valid;
+ bool alpn_used;
/*
* OpenSSL structures. (Keep these last so that the locations of other
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index 79c5f6b754b..b65b32f2093 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -123,6 +123,7 @@ extern PGDLLIMPORT char *SSLECDHCurve;
extern PGDLLIMPORT bool SSLPreferServerCiphers;
extern PGDLLIMPORT int ssl_min_protocol_version;
extern PGDLLIMPORT int ssl_max_protocol_version;
+extern PGDLLIMPORT bool ssl_enable_alpn;
enum ssl_protocol_versions
{
diff --git a/src/include/libpq/pqcomm.h b/src/include/libpq/pqcomm.h
index 9ae469c86c4..fb93c820530 100644
--- a/src/include/libpq/pqcomm.h
+++ b/src/include/libpq/pqcomm.h
@@ -139,6 +139,25 @@ typedef struct CancelRequestPacket
uint32 cancelAuthCode; /* secret key to authorize cancel */
} CancelRequestPacket;
+/* Application-Layer Protocol Negotiation is required for direct connections
+ * to avoid protocol confusion attacks (e.g https://alpaca-attack.com/).
+ *
+ * ALPN is specified in RFC 7301
+ *
+ * This string should be registered at:
+ * https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids
+ *
+ * OpenSSL uses this wire-format for the list of alpn protocols even in the
+ * API. Both server and client take the same format parameter but the client
+ * actually sends it to the server as-is and the server it specifies the
+ * preference order to use to choose the one selected to send back.
+ *
+ * c.f. https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_alpn_select_cb.html
+ *
+ * The #define can be used to initialize a char[] vector to use directly in the API
+ */
+#define PG_ALPN_PROTOCOL "TBD-pgsql"
+#define PG_ALPN_PROTOCOL_VECTOR { 9, 'T','B','D','-','p','g','s','q','l' }
/*
* A client can also start by sending a SSL or GSSAPI negotiation request to
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 7bd371fafb5..4bbed0451ed 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -313,6 +313,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-SNI", "", 1,
offsetof(struct pg_conn, sslsni)},
+ {"sslalpn", "PGSSLALPN", "1", NULL,
+ "SSL-ALPN", "", 1,
+ offsetof(struct pg_conn, sslalpn)},
+
{"requirepeer", "PGREQUIREPEER", NULL, NULL,
"Require-Peer", "", 10,
offsetof(struct pg_conn, requirepeer)},
@@ -4473,6 +4477,7 @@ freePGconn(PGconn *conn)
free(conn->sslcrldir);
free(conn->sslcompression);
free(conn->sslsni);
+ free(conn->sslalpn);
free(conn->requirepeer);
free(conn->require_auth);
free(conn->ssl_min_protocol_version);
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 81108822620..a8fd4c19efb 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -885,6 +885,9 @@ destroy_ssl_system(void)
#endif
}
+/* See pqcomm.h comments on OpenSSL implementation of ALPN (RFC 7301) */
+static unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
+
/*
* Create per-connection SSL object, and load the client certificate,
* private key, and trusted CA certs.
@@ -1234,6 +1237,22 @@ initialize_SSL(PGconn *conn)
}
}
+ if (conn->sslalpn && conn->sslalpn[0] == '1')
+ {
+ int retval;
+
+ retval = SSL_set_alpn_protos(conn->ssl, alpn_protos, sizeof(alpn_protos));
+
+ if (retval != 0)
+ {
+ char *err = SSLerrmessage(ERR_get_error());
+
+ libpq_append_conn_error(conn, "could not set ssl alpn extension: %s", err);
+ SSLerrfree(err);
+ return -1;
+ }
+ }
+
/*
* Read the SSL key. If a key is specified, treat it as an engine:key
* combination if there is colon present - we don't support files with
@@ -1736,6 +1755,7 @@ PQsslAttributeNames(PGconn *conn)
"cipher",
"compression",
"protocol",
+ "alpn",
NULL
};
static const char *const empty_attrs[] = {NULL};
@@ -1790,6 +1810,21 @@ PQsslAttribute(PGconn *conn, const char *attribute_name)
if (strcmp(attribute_name, "protocol") == 0)
return SSL_get_version(conn->ssl);
+ if (strcmp(attribute_name, "alpn") == 0)
+ {
+ const unsigned char *data;
+ unsigned int len;
+ static char alpn_str[256]; /* alpn doesn't support longer than 255
+ * bytes */
+
+ SSL_get0_alpn_selected(conn->ssl, &data, &len);
+ if (data == NULL || len == 0 || len > sizeof(alpn_str) - 1)
+ return NULL;
+ memcpy(alpn_str, data, len);
+ alpn_str[len] = 0;
+ return alpn_str;
+ }
+
return NULL; /* unknown attribute */
}
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 640a3ae0f5a..97d78f02b92 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -399,6 +399,7 @@ struct pg_conn
char *sslcrl; /* certificate revocation list filename */
char *sslcrldir; /* certificate revocation list directory name */
char *sslsni; /* use SSL SNI extension (0 or 1) */
+ char *sslalpn; /* use SSL ALPN extension (0 or 1) */
char *requirepeer; /* required peer credentials for local sockets */
char *gssencmode; /* GSS mode (require,prefer,disable) */
char *krbsrvname; /* Kerberos service name */
--
2.39.2
v8-0005-Add-tests-for-sslnegotiation.patchtext/x-patch; charset=UTF-8; name=v8-0005-Add-tests-for-sslnegotiation.patchDownload
From c3e21363f0d15adbbd8f6a90fba0e38c36c60188 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 10:25:08 +0200
Subject: [PATCH v8 5/6] Add tests for sslnegotiation
Author: Heikki Linnakangas, Matthias van de Meent
---
.../t/001_negotiate_encryption.pl | 25 +++++++++++++------
1 file changed, 18 insertions(+), 7 deletions(-)
diff --git a/src/test/libpq_encryption/t/001_negotiate_encryption.pl b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
index b8646c5bc97..b8b4cd2726d 100644
--- a/src/test/libpq_encryption/t/001_negotiate_encryption.pl
+++ b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
@@ -231,13 +231,13 @@ sub resolve_connection_type
# First test with SSL disabled in the server
-# Test the cube of parameters: user, sslmode, and gssencmode
+# Test the cube of parameters: user, sslmode, sslnegotiation and gssencmode
sub test_modes
{
local $Test::Builder::Level = $Test::Builder::Level + 1;
my ($pg_node, $node_conf,
- $test_users, $ssl_modes, $gss_modes) = @_;
+ $test_users, $ssl_modes, $ssl_negotiations, $gss_modes) = @_;
foreach my $test_user (@{$test_users})
{
@@ -253,13 +253,18 @@ sub test_modes
gssmode=>$gssencmode,
);
my $res = resolve_connection_type(\%params);
- connect_test($pg_node, "user=$test_user sslmode=$client_mode gssencmode=$gssencmode", $res);
+ # Negotiation type doesn't matter for supported connection types
+ foreach my $negotiation (@{$ssl_negotiations})
+ {
+ connect_test($pg_node, "user=$test_user sslmode=$client_mode sslnegotiation=$negotiation gssencmode=$gssencmode", $res);
+ }
}
}
}
}
my $sslmodes = ['disable', 'allow', 'prefer', 'require'];
+my $sslnegotiations = ['postgres', 'direct', 'requiredirect'];
my $gssencmodes = ['disable', 'prefer', 'require'];
my $server_config = {
@@ -270,7 +275,7 @@ my $server_config = {
note("Running tests with SSL and GSS disabled in server");
test_modes($node, $server_config,
['testuser'],
- $sslmodes, $gssencmodes);
+ $sslmodes, $sslnegotiations, $gssencmodes);
# Enable SSL in the server
SKIP:
@@ -284,7 +289,7 @@ SKIP:
note("Running tests with SSL enabled in server");
test_modes($node, $server_config,
['testuser', 'ssluser', 'nossluser'],
- $sslmodes, ['disable']);
+ $sslmodes, $sslnegotiations, ['disable']);
$node->adjust_conf('postgresql.conf', 'ssl', 'off');
$node->reload;
@@ -306,7 +311,7 @@ SKIP:
note("Running tests with GSS enabled in server");
test_modes($node, $server_config,
['testuser', 'gssuser', 'nogssuser'],
- $sslmodes, $gssencmodes);
+ $sslmodes, $sslnegotiations, $gssencmodes);
# Check that logs match the expected 'no pg_hba.conf entry' line, too, as
# that is not tested by test_modes.
@@ -320,6 +325,10 @@ SKIP:
# with no encryption.
connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer', 'plain',
'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer sslnegotiation=direct', 'plain',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer sslnegotiation=requiredirect', 'plain',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
}
# Server supports both SSL and GSSAPI
@@ -329,6 +338,8 @@ SKIP:
# SSL is still disabled
connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer sslnegotiation=direct', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer sslnegotiation=requiredirect', 'gss');
# Enable SSL
$node->adjust_conf('postgresql.conf', 'ssl', 'on');
@@ -338,7 +349,7 @@ SKIP:
note("Running tests with both GSS and SSL enabled in server");
test_modes($node, $server_config,
['testuser', 'gssuser', 'ssluser', 'nogssuser', 'nossluser'],
- $sslmodes, $gssencmodes);
+ $sslmodes, $sslnegotiations, $gssencmodes);
# Test case that server supports GSSAPI, but it's not allowed for
# this user. Special cased because we check output
--
2.39.2
v8-0006-WIP-refactor-state-machine-in-libpq.patchtext/x-patch; charset=UTF-8; name=v8-0006-WIP-refactor-state-machine-in-libpq.patchDownload
From 0718d379ca12459469d503340bf35b1acf0cdbc2 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 5 Mar 2024 16:03:52 +0200
Subject: [PATCH v8 6/6] WIP: refactor state machine in libpq
---
src/interfaces/libpq/fe-connect.c | 506 +++++++++++++----------
src/interfaces/libpq/fe-secure-openssl.c | 12 +-
src/interfaces/libpq/libpq-fe.h | 3 +-
src/interfaces/libpq/libpq-int.h | 18 +-
4 files changed, 315 insertions(+), 224 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 4bbed0451ed..5b42384d897 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -396,6 +396,12 @@ static const char uri_designator[] = "postgresql://";
static const char short_uri_designator[] = "postgres://";
static bool connectOptions1(PGconn *conn, const char *conninfo);
+static bool init_allowed_encryption_methods(PGconn *conn);
+#if defined(USE_SSL) || defined(USE_GSS)
+static int encryption_negotiation_failed(PGconn *conn);
+#endif
+static bool connection_failed(PGconn *conn);
+static bool select_next_encryption_method(PGconn *conn, bool negotiation_failure);
static PGPing internal_ping(PGconn *conn);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
@@ -1679,21 +1685,6 @@ pqConnectOptions2(PGconn *conn)
conn->gssencmode);
return false;
}
-#endif
-#ifdef USE_SSL
-
- /*
- * GSS is incompatible with direct SSL connections so it requires the
- * default postgres style connection ssl negotiation
- */
- if (strcmp(conn->gssencmode, "require") == 0 &&
- strcmp(conn->sslnegotiation, "postgres") != 0)
- {
- conn->status = CONNECTION_BAD;
- libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when Direct SSL negotiation is enabled",
- conn->gssencmode);
- return false;
- }
#endif
}
else
@@ -2777,16 +2768,9 @@ keep_going: /* We will come back to here until there is
*/
conn->pversion = PG_PROTOCOL(3, 0);
conn->send_appname = true;
-#ifdef USE_SSL
- /* initialize these values based on SSL mode */
- conn->allow_ssl_try = (conn->sslmode[0] != 'd'); /* "disable" */
- conn->wait_ssl_try = (conn->sslmode[0] == 'a'); /* "allow" */
- /* direct ssl is incompatible with "allow" or "disabled" ssl */
- conn->allow_direct_ssl_try = conn->allow_ssl_try && !conn->wait_ssl_try && (conn->sslnegotiation[0] != 'p');
-#endif
-#ifdef ENABLE_GSS
- conn->try_gss = (conn->gssencmode[0] != 'd'); /* "disable" */
-#endif
+ conn->failed_enc_methods = 0;
+ conn->current_enc_method = 0;
+ conn->allowed_enc_methods = 0;
reset_connection_state_machine = false;
need_new_connection = true;
}
@@ -2812,6 +2796,34 @@ keep_going: /* We will come back to here until there is
need_new_connection = false;
}
+ /* Decide what to do next, if SSL or GSS negotiation fails */
+#define ENCRYPTION_NEGOTIATION_FAILED() \
+ do { \
+ switch (encryption_negotiation_failed(conn)) \
+ { \
+ case 0: \
+ goto error_return; \
+ case 1: \
+ conn->status = CONNECTION_MADE; \
+ return PGRES_POLLING_WRITING; \
+ case 2: \
+ need_new_connection = true; \
+ goto keep_going; \
+ } \
+ } while(0);
+
+ /* Decide what to do next, if connection fails */
+#define CONNECTION_FAILED() \
+ do { \
+ if (connection_failed(conn)) \
+ { \
+ need_new_connection = true; \
+ goto keep_going; \
+ } \
+ else \
+ goto error_return; \
+ } while(0);
+
/* Now try to advance the state machine for this connection */
switch (conn->status)
{
@@ -3126,18 +3138,6 @@ keep_going: /* We will come back to here until there is
goto error_return;
}
- /*
- * Make sure we can write before advancing to next step.
- */
- conn->status = CONNECTION_MADE;
- return PGRES_POLLING_WRITING;
- }
-
- case CONNECTION_MADE:
- {
- char *startpacket;
- int packetlen;
-
/*
* Implement requirepeer check, if requested and it's a
* Unix-domain socket.
@@ -3186,30 +3186,31 @@ keep_going: /* We will come back to here until there is
#endif /* WIN32 */
}
- if (conn->raddr.addr.ss_family == AF_UNIX)
- {
- /* Don't request SSL or GSSAPI over Unix sockets */
-#ifdef USE_SSL
- conn->allow_ssl_try = false;
-#endif
-#ifdef ENABLE_GSS
- conn->try_gss = false;
-#endif
- }
+ /* Choose encryption method to try first */
+ if (!init_allowed_encryption_methods(conn))
+ goto error_return;
+
+ /*
+ * Make sure we can write before advancing to next step.
+ */
+ conn->status = CONNECTION_MADE;
+ return PGRES_POLLING_WRITING;
+ }
+
+ case CONNECTION_MADE:
+ {
+ char *startpacket;
+ int packetlen;
#ifdef ENABLE_GSS
/*
- * If GSSAPI encryption is enabled, then call
- * pg_GSS_have_cred_cache() which will return true if we can
- * acquire credentials (and give us a handle to use in
- * conn->gcred), and then send a packet to the server asking
- * for GSSAPI Encryption (and skip past SSL negotiation and
- * regular startup below).
+ * If GSSAPI encryption is enabled, send a packet to the
+ * server asking for GSSAPI Encryption and proceed with GSSAPI
+ * handshake. We will come back here after GSSAPI encryption
+ * has been established, with conn->gctx set.
*/
- if (conn->try_gss && !conn->gctx)
- conn->try_gss = pg_GSS_have_cred_cache(&conn->gcred);
- if (conn->try_gss && !conn->gctx)
+ if (conn->current_enc_method == ENC_GSSAPI && !conn->gctx)
{
ProtocolVersion pv = pg_hton32(NEGOTIATE_GSS_CODE);
@@ -3224,12 +3225,6 @@ keep_going: /* We will come back to here until there is
conn->status = CONNECTION_GSS_STARTUP;
return PGRES_POLLING_READING;
}
- else if (!conn->gctx && conn->gssencmode[0] == 'r')
- {
- libpq_append_conn_error(conn,
- "GSSAPI encryption required but was impossible (possibly no credential cache, no server support, or using a local socket)");
- goto error_return;
- }
#endif
#ifdef USE_SSL
@@ -3245,39 +3240,22 @@ keep_going: /* We will come back to here until there is
goto error_return;
/*
- * If SSL is enabled and direct SSL connections are enabled
- * and we haven't already established an SSL connection (or
- * already tried a direct connection and failed or succeeded)
- * then try just enabling SSL directly.
- *
- * If we fail then we'll either fail the connection (if
- * sslnegotiation is set to requiredirect or turn
- * allow_direct_ssl_try to false
+ * If direct SSL is enabled, jump right into SSL handshake. We
+ * will come back here after SSL encryption has been
+ * established, with ssl_in_use set.
*/
- if (conn->allow_ssl_try
- && !conn->wait_ssl_try
- && conn->allow_direct_ssl_try
- && !conn->ssl_in_use
-#ifdef ENABLE_GSS
- && !conn->gssenc
-#endif
- )
+ if (conn->current_enc_method == ENC_DIRECT_SSL && !conn->ssl_in_use)
{
conn->status = CONNECTION_SSL_STARTUP;
return PGRES_POLLING_WRITING;
}
/*
- * If SSL is enabled and we haven't already got encryption of
- * some sort running, request SSL instead of sending the
- * startup message.
+ * If negotiated SSL is enabled, request SSL and proceed with
+ * SSL handshake. We will come back here after SSL encryption
+ * has been established, with ssl_in_use set.
*/
- if (conn->allow_ssl_try && !conn->wait_ssl_try &&
- !conn->ssl_in_use
-#ifdef ENABLE_GSS
- && !conn->gssenc
-#endif
- )
+ if (conn->current_enc_method == ENC_NEGOTIATED_SSL && !conn->ssl_in_use)
{
ProtocolVersion pv;
@@ -3302,8 +3280,11 @@ keep_going: /* We will come back to here until there is
#endif /* USE_SSL */
/*
- * Build the startup packet.
+ * We have now established encryption, or we are happy to
+ * proceed without.
*/
+
+ /* Build the startup packet. */
startpacket = pqBuildStartupPacket3(conn, &packetlen,
EnvironmentOptions);
if (!startpacket)
@@ -3344,10 +3325,9 @@ keep_going: /* We will come back to here until there is
/*
* On first time through, get the postmaster's response to our
* SSL negotiation packet. If we are trying a direct ssl
- * connection skip reading the negotiation packet and go
- * straight to initiating an ssl connection.
+ * connection, go straight to initiating ssl.
*/
- if (!conn->ssl_in_use && !conn->allow_direct_ssl_try)
+ if (!conn->ssl_in_use && conn->current_enc_method == ENC_NEGOTIATED_SSL)
{
/*
* We use pqReadData here since it has the logic to
@@ -3377,34 +3357,14 @@ keep_going: /* We will come back to here until there is
{
/* mark byte consumed */
conn->inStart = conn->inCursor;
-
- /*
- * Set up global SSL state if required. The crypto
- * state has already been set if libpq took care of
- * doing that, so there is no need to make that happen
- * again.
- */
- if (pqsecure_initialize(conn, true, false) != 0)
- goto error_return;
}
else if (SSLok == 'N')
{
/* mark byte consumed */
conn->inStart = conn->inCursor;
/* OK to do without SSL? */
- if (conn->sslmode[0] == 'r' || /* "require" */
- conn->sslmode[0] == 'v') /* "verify-ca" or
- * "verify-full" */
- {
- /* Require SSL, but server does not want it */
- libpq_append_conn_error(conn, "server does not support SSL, but SSL was required");
- goto error_return;
- }
- /* Otherwise, proceed with normal startup */
- conn->allow_ssl_try = false;
/* We can proceed using this connection */
- conn->status = CONNECTION_MADE;
- return PGRES_POLLING_WRITING;
+ ENCRYPTION_NEGOTIATION_FAILED();
}
else if (SSLok == 'E')
{
@@ -3429,6 +3389,14 @@ keep_going: /* We will come back to here until there is
}
}
+ /*
+ * Set up global SSL state if required. The crypto state has
+ * already been set if libpq took care of doing that, so there
+ * is no need to make that happen again.
+ */
+ if (pqsecure_initialize(conn, true, false) != 0)
+ goto error_return;
+
/*
* Begin or continue the SSL negotiation process.
*/
@@ -3457,32 +3425,7 @@ keep_going: /* We will come back to here until there is
* Failed direct ssl connection, possibly try a new
* connection with postgres negotiation
*/
- if (conn->allow_direct_ssl_try)
- {
- /* if it's requiredirect then it's a hard failure */
- if (conn->sslnegotiation[0] == 'r')
- goto error_return;
- /* otherwise only retry using postgres connection */
- conn->allow_direct_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
-
- /*
- * Failed ... if sslmode is "prefer" then do a non-SSL
- * retry
- */
- if (conn->sslmode[0] == 'p' /* "prefer" */
- && conn->allow_ssl_try /* redundant? */
- && !conn->wait_ssl_try) /* redundant? */
- {
- /* only retry once */
- conn->allow_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
- /* Else it's a hard failure */
- goto error_return;
+ CONNECTION_FAILED();
}
/* Else, return POLLING_READING or POLLING_WRITING status */
return pollres;
@@ -3501,7 +3444,7 @@ keep_going: /* We will come back to here until there is
* If we haven't yet, get the postmaster's response to our
* negotiation packet
*/
- if (conn->try_gss && !conn->gctx)
+ if (!conn->gctx)
{
char gss_ok;
int rdresult = pqReadData(conn);
@@ -3525,9 +3468,7 @@ keep_going: /* We will come back to here until there is
* error message on retry). Server gets fussy if we
* don't hang up the socket, though.
*/
- conn->try_gss = false;
- need_new_connection = true;
- goto keep_going;
+ CONNECTION_FAILED();
}
/* mark byte consumed */
@@ -3535,17 +3476,8 @@ keep_going: /* We will come back to here until there is
if (gss_ok == 'N')
{
- /* Server doesn't want GSSAPI; fall back if we can */
- if (conn->gssencmode[0] == 'r')
- {
- libpq_append_conn_error(conn, "server doesn't support GSSAPI encryption, but it was required");
- goto error_return;
- }
-
- conn->try_gss = false;
/* We can proceed using this connection */
- conn->status = CONNECTION_MADE;
- return PGRES_POLLING_WRITING;
+ ENCRYPTION_NEGOTIATION_FAILED();
}
else if (gss_ok != 'G')
{
@@ -3577,18 +3509,7 @@ keep_going: /* We will come back to here until there is
}
else if (pollres == PGRES_POLLING_FAILED)
{
- if (conn->gssencmode[0] == 'p')
- {
- /*
- * We failed, but we can retry on "prefer". Have to
- * drop the current connection to do so, though.
- */
- conn->try_gss = false;
- need_new_connection = true;
- goto keep_going;
- }
- /* Else it's a hard failure */
- goto error_return;
+ CONNECTION_FAILED();
}
/* Else, return POLLING_READING or POLLING_WRITING status */
return pollres;
@@ -3764,55 +3685,7 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
-#ifdef ENABLE_GSS
-
- /*
- * If gssencmode is "prefer" and we're using GSSAPI, retry
- * without it.
- */
- if (conn->gssenc && conn->gssencmode[0] == 'p')
- {
- /* only retry once */
- conn->try_gss = false;
- need_new_connection = true;
- goto keep_going;
- }
-#endif
-
-#ifdef USE_SSL
-
- /*
- * if sslmode is "allow" and we haven't tried an SSL
- * connection already, then retry with an SSL connection
- */
- if (conn->sslmode[0] == 'a' /* "allow" */
- && !conn->ssl_in_use
- && conn->allow_ssl_try
- && conn->wait_ssl_try)
- {
- /* only retry once */
- conn->wait_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
-
- /*
- * if sslmode is "prefer" and we're in an SSL connection,
- * then do a non-SSL retry
- */
- if (conn->sslmode[0] == 'p' /* "prefer" */
- && conn->ssl_in_use
- && conn->allow_ssl_try /* redundant? */
- && !conn->wait_ssl_try) /* redundant? */
- {
- /* only retry once */
- conn->allow_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
-#endif
-
- goto error_return;
+ CONNECTION_FAILED();
}
else if (beresp == PqMsg_NegotiateProtocolVersion)
{
@@ -4252,6 +4125,213 @@ error_return:
return PGRES_POLLING_FAILED;
}
+static bool
+init_allowed_encryption_methods(PGconn *conn)
+{
+ if (conn->raddr.addr.ss_family == AF_UNIX)
+ {
+ /* Don't request SSL or GSSAPI over Unix sockets */
+ conn->allowed_enc_methods &= ~(ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL | ENC_GSSAPI);
+
+ /* to give a better error message */
+
+ /*
+ * XXX: we probably should not do this. sslmode=require works
+ * differently
+ */
+ if (conn->gssencmode[0] == 'r')
+ {
+ libpq_append_conn_error(conn,
+ "GSSAPI encryption required but it is not supported over a local socket)");
+ conn->allowed_enc_methods = 0;
+ conn->current_enc_method = ENC_ERROR;
+ return false;
+ }
+
+ conn->allowed_enc_methods = ENC_PLAINTEXT;
+ conn->current_enc_method = ENC_PLAINTEXT;
+ return true;
+ }
+
+ /* initialize these values based on sslmode and gssencmode */
+ conn->allowed_enc_methods = 0;
+
+#ifdef USE_SSL
+ /* sslmode anything but 'disable', and GSSAPI not required */
+ if (conn->sslmode[0] != 'd' && conn->gssencmode[0] != 'r')
+ {
+ if (conn->sslnegotiation[0] == 'p')
+ conn->allowed_enc_methods |= ENC_NEGOTIATED_SSL;
+ else if (conn->sslnegotiation[0] == 'd')
+ conn->allowed_enc_methods |= ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL;
+ else if (conn->sslnegotiation[0] == 'r')
+ conn->allowed_enc_methods |= ENC_DIRECT_SSL;
+ }
+#endif
+
+#ifdef ENABLE_GSS
+ if (conn->gssencmode[0] != 'd')
+ conn->allowed_enc_methods |= ENC_GSSAPI;
+#endif
+
+ if ((conn->sslmode[0] == 'd' || conn->sslmode[0] == 'p' || conn->sslmode[0] == 'a') &&
+ (conn->gssencmode[0] == 'd' || conn->gssencmode[0] == 'p'))
+ {
+ conn->allowed_enc_methods |= ENC_PLAINTEXT;
+ }
+
+ return select_next_encryption_method(conn, false);
+}
+
+/*
+ * Out-of-line portion of the ENCRYPTION_NEGOTIATION_FAILED() macro in the
+ * PQconnectPoll state machine.
+ *
+ * Return value:
+ * 0: connection failed and we are out of encryption methods to try. return an error
+ * 1: Retry with next connection method. The TCP connection is still valid and in
+ * known state, so we can proceed with the negotiating next method without
+ * reconnecting.
+ * 2: Disconnect, and retry with next connection method.
+ *
+ * conn->current_enc_method is updated to the next method to try.
+ */
+#if defined(USE_SSL) || defined(USE_GSS)
+static int
+encryption_negotiation_failed(PGconn *conn)
+{
+ Assert((conn->failed_enc_methods & conn->current_enc_method) == 0);
+ conn->failed_enc_methods |= conn->current_enc_method;
+
+ if (select_next_encryption_method(conn, true))
+ {
+ if (conn->current_enc_method == ENC_DIRECT_SSL)
+ return 2;
+ else
+ return 1;
+ }
+ else
+ return 0;
+}
+#endif
+
+/*
+ * Out-of-line portion of the CONNECTION_FAILED() macro
+ *
+ * Returns true, if we should retry the connection with different encryption method.
+ * conn->current_enc_method is updated to the next method to try.
+ */
+static bool
+connection_failed(PGconn *conn)
+{
+ Assert((conn->failed_enc_methods & conn->current_enc_method) == 0);
+ conn->failed_enc_methods |= conn->current_enc_method;
+
+ /*
+ * If the server reported an error after the SSL handshake, no point in
+ * retrying with negotiated vs direct SSL.
+ */
+ if ((conn->current_enc_method & (ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL)) != 0 &&
+ conn->ssl_handshake_started)
+ {
+ conn->failed_enc_methods |= (ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL) & conn->allowed_enc_methods;
+ }
+ else
+ conn->failed_enc_methods |= conn->current_enc_method;
+
+ return select_next_encryption_method(conn, false);
+}
+
+/*
+ * Choose the next encryption method to try. If this is a retry,
+ * conn->failed_enc_methods has already been updated. conn->current_enc_method
+ * is updated to the next method to try.
+ */
+static bool
+select_next_encryption_method(PGconn *conn, bool have_valid_connection)
+{
+ int remaining_methods;
+
+ remaining_methods = conn->allowed_enc_methods & ~conn->failed_enc_methods;
+
+ /*
+ * Try GSSAPI before SSL
+ */
+#ifdef ENABLE_GSS
+ if ((remaining_methods & ENC_GSSAPI) != 0)
+ {
+ /*
+ * If GSSAPI encryption is enabled, then call pg_GSS_have_cred_cache()
+ * which will return true if we can acquire credentials (and give us a
+ * handle to use in conn->gcred), and then send a packet to the server
+ * asking for GSSAPI Encryption (and skip past SSL negotiation and
+ * regular startup below).
+ */
+ if (!conn->gctx)
+ {
+ if (!pg_GSS_have_cred_cache(&conn->gcred))
+ {
+ conn->allowed_enc_methods &= ~ENC_GSSAPI;
+ remaining_methods &= ~ENC_GSSAPI;
+
+ if (conn->gssencmode[0] == 'r')
+ {
+ libpq_append_conn_error(conn,
+ "GSSAPI encryption required but no credential cache");
+ }
+ }
+ }
+ if ((remaining_methods & ENC_GSSAPI) != 0)
+ {
+ conn->current_enc_method = ENC_GSSAPI;
+ return true;
+ }
+ }
+#endif
+
+ /* With sslmode=allow, try plaintext connection before SSL. */
+ if (conn->sslmode[0] == 'a' && (remaining_methods & ENC_PLAINTEXT) != 0)
+ {
+ conn->current_enc_method = ENC_PLAINTEXT;
+ return true;
+ }
+
+ /*
+ * Try SSL. If enabled, try direct SSL. Unless we have a valid TCP
+ * connection that failed negotiating GSSAPI encryption; in that case we
+ * prefer to reuse the connection with negotiated SSL, instead of
+ * reconnecting to do direct SSL. The point of direct SSL is to avoid the
+ * roundtrip from the negotiation, but reconnecting would also incur a
+ * roundtrip.
+ */
+ if (have_valid_connection && (remaining_methods & ENC_NEGOTIATED_SSL) != 0)
+ {
+ conn->current_enc_method = ENC_NEGOTIATED_SSL;
+ return true;
+ }
+
+ if ((remaining_methods & ENC_DIRECT_SSL) != 0)
+ {
+ conn->current_enc_method = ENC_DIRECT_SSL;
+ return true;
+ }
+
+ if ((remaining_methods & ENC_NEGOTIATED_SSL) != 0)
+ {
+ conn->current_enc_method = ENC_NEGOTIATED_SSL;
+ return true;
+ }
+
+ if (conn->sslmode[0] != 'a' && (remaining_methods & ENC_PLAINTEXT) != 0)
+ {
+ conn->current_enc_method = ENC_PLAINTEXT;
+ return true;
+ }
+
+ /* No more options */
+ conn->current_enc_method = ENC_ERROR;
+ return false;
+}
/*
* internal_ping
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index a8fd4c19efb..9bfab813bea 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -1484,6 +1484,7 @@ open_client_SSL(PGconn *conn)
SOCK_ERRNO_SET(0);
ERR_clear_error();
r = SSL_connect(conn->ssl);
+
if (r <= 0)
{
int save_errno = SOCK_ERRNO;
@@ -1587,7 +1588,7 @@ open_client_SSL(PGconn *conn)
/*
* We already checked the server certificate in initialize_SSL() using
- * SSL_CTX_set_verify(), if root.crt exists.
+ * SSL_set_verify(), if root.crt exists.
*/
/* get server certificate */
@@ -1631,6 +1632,7 @@ pgtls_close(PGconn *conn)
SSL_free(conn->ssl);
conn->ssl = NULL;
conn->ssl_in_use = false;
+ conn->ssl_handshake_started = false;
destroy_needed = true;
}
@@ -1654,7 +1656,7 @@ pgtls_close(PGconn *conn)
{
/*
* In the non-SSL case, just remove the crypto callbacks if the
- * connection has then loaded. This code path has no dependency on
+ * connection has loaded them. This code path has no dependency on
* any pending SSL calls.
*/
if (conn->crypto_loaded)
@@ -1843,9 +1845,10 @@ static BIO_METHOD *my_bio_methods;
static int
my_sock_read(BIO *h, char *buf, int size)
{
+ PGconn *conn = (PGconn *) BIO_get_app_data(h);
int res;
- res = pqsecure_raw_read((PGconn *) BIO_get_app_data(h), buf, size);
+ res = pqsecure_raw_read(conn, buf, size);
BIO_clear_retry_flags(h);
if (res < 0)
{
@@ -1867,6 +1870,9 @@ my_sock_read(BIO *h, char *buf, int size)
}
}
+ if (res > 0)
+ conn->ssl_handshake_started = true;
+
return res;
}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 8dd7a9af457..595973fd8de 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -72,14 +72,13 @@ typedef enum
CONNECTION_AUTH_OK, /* Received authentication; waiting for
* backend startup. */
CONNECTION_SETENV, /* This state is no longer used. */
- CONNECTION_SSL_STARTUP, /* Negotiating SSL. */
+ CONNECTION_SSL_STARTUP, /* Performing SSL handshake. */
CONNECTION_NEEDED, /* Internal state: connect() needed */
CONNECTION_CHECK_WRITABLE, /* Checking if session is read-write. */
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
- CONNECTION_DIRECT_SSL_STARTUP /* Starting SSL without PG Negotiation. */
} ConnStatusType;
typedef enum
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 97d78f02b92..d3105e6c8d0 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -76,6 +76,7 @@ typedef struct
#include <openssl/ssl.h>
#include <openssl/err.h>
+
#ifndef OPENSSL_NO_ENGINE
#define USE_SSL_ENGINE
#endif
@@ -231,6 +232,12 @@ typedef enum
PGASYNC_PIPELINE_IDLE, /* "Idle" between commands in pipeline mode */
} PGAsyncStatusType;
+#define ENC_ERROR 0
+#define ENC_DIRECT_SSL 0x01
+#define ENC_GSSAPI 0x02
+#define ENC_NEGOTIATED_SSL 0x04
+#define ENC_PLAINTEXT 0x08
+
/* Target server type (decoded value of target_session_attrs) */
typedef enum
{
@@ -546,17 +553,17 @@ struct pg_conn
void *sasl_state;
int scram_sha_256_iterations;
+ uint8 allowed_enc_methods;
+ uint8 failed_enc_methods;
+ uint8 current_enc_method;
+
/* SSL structures */
bool ssl_in_use;
+ bool ssl_handshake_started;
bool ssl_cert_requested; /* Did the server ask us for a cert? */
bool ssl_cert_sent; /* Did we send one in reply? */
#ifdef USE_SSL
- bool allow_direct_ssl_try; /* Try to make a direct SSL connection
- * without an "SSL negotiation packet" */
- bool allow_ssl_try; /* Allowed to try SSL negotiation */
- bool wait_ssl_try; /* Delay SSL negotiation until after
- * attempting normal connection */
#ifdef USE_OPENSSL
SSL *ssl; /* SSL status, if have SSL connection */
X509 *peer; /* X509 cert of server */
@@ -579,7 +586,6 @@ struct pg_conn
gss_name_t gtarg_nam; /* GSS target name */
/* The following are encryption-only */
- bool try_gss; /* GSS attempting permitted */
bool gssenc; /* GSS encryption is usable */
gss_cred_id_t gcred; /* GSS credential temp storage. */
--
2.39.2
On Tue, Mar 5, 2024 at 6:09 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
I hope I didn't joggle your elbow reviewing this
Nope, not at all!
The tests are still not distinguishing whether a connection was
established in direct or negotiated mode. So if we e.g. had a bug that
accidentally disabled direct SSL connection completely and always used
negotiated mode, the tests would still pass. I'd like to see some tests
that would catch that.
+1
On Mon, Mar 4, 2024 at 7:29 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
On 01/03/2024 23:49, Jacob Champion wrote:
I'll squint more closely at the MITM-protection changes in 0008 later.
First impressions, though: it looks like that code has gotten much
less straightforward, which I think is dangerous given the attack it's
preventing. (Off-topic: I'm skeptical of future 0-RTT support. Our
protocol doesn't seem particularly replay-safe to me.)Let's drop that patch. AFAICS it's not needed by the rest of the patches.
Okay, sounds good.
If we're interested in ALPN negotiation in the future, we may also
want to look at GREASE [1] to keep those options open in the presence
of third-party implementations. Unfortunately OpenSSL doesn't do this
automatically yet.Can you elaborate?
Sure: now that we're letting middleboxes and proxies inspect and react
to connections based on ALPN, it's possible that some intermediary
might incorrectly fixate on the "postgres" ID (or whatever we choose
in the end), and shut down connections that carry additional protocols
rather than ignoring them. That would prevent future graceful upgrades
where the client sends both "postgres/X" and "postgres/X+1". While
that wouldn't be our fault, it'd be cold comfort to whoever has that
middlebox.
GREASE is a set of reserved protocol IDs that you can add randomly to
your ALPN list, so any middleboxes that fail to follow the rules will
just break outright rather than silently proliferating. (Hence the
pun: GREASE keeps the joints in the pipe from rusting into place.) The
RFC goes into more detail about how to do it. And I don't know if it's
necessary for a v1, but it'd be something to keep in mind.
Do we need to do something extra in the server to be
compatible with GREASE?
No, I think that as long as we use OpenSSL's APIs correctly on the
server side, we'll be compatible by default. This would be a
client-side implementation, to push random GREASE strings into the
ALPN list. (There is a risk that if/when OpenSSL finally starts
supporting this transparently, we'd need to remove it from our code.)
If we don't have a reason not to, it'd be good to follow the strictest
recommendations from [2] to avoid cross-protocol attacks. (For anyone
currently running web servers and Postgres on the same host, they
really don't want browsers "talking" to their Postgres servers.) That
would mean checking the negotiated ALPN on both the server and client
side, and failing if it's not what we expect.Hmm, I thought that's what the patches does. But looking closer, libpq
is not checking that ALPN was used. We should add that. Am I right?
Right. Also, it looks like the server isn't failing the TLS handshake
itself, but instead just dropping the connection after the handshake.
In a cross-protocol attack, there's a danger that the client (which is
not speaking our protocol) could still treat the server as
authoritative in that situation.
I'm not excited about the proliferation of connection options. I don't
have a lot of ideas on how to fix it, though, other than to note that
the current sslnegotiation option names are very unintuitive to me:
- "postgres": only legacy handshakes
- "direct": might be direct... or maybe legacy
- "requiredirect": only direct handshakes... unless other options are
enabled and then we fall back again to legacy? How many people willing
to break TLS compatibility with old servers via "requiredirect" are
going to be okay with lazy fallback to GSS or otherwise?Yeah, this is my biggest complaint about all this. Not so much the names
of the options, but the number of combinations of different options, and
how we're going to test them all. I don't have any great solutions,
except adding a lot of tests to cover them, like Matthias did.
The default gssencmode=prefer is especially problematic if I'm trying
to use sslnegotiation=requiredirect for security. It'll appear to work
at first, but if somehow I get a credential cache into my environment,
libpq will suddenly fall back to plaintext negotiation :(
(Plus, we need to have a good error message when connecting to older
servers anyway.I think we should be able to key off of the EOF coming
back from OpenSSL; it'd be a good excuse to give that part of the code
some love.)Hmm, if OpenSSL sends ClientHello and the server responds with a
Postgres error packet, OpenSSL will presumably consume the error packet
or at least part of it. But with our custom BIO, we can peek at the
server response before handing it to OpenSSL.
I don't think an error packet is going to come back with the
currently-shipped implementations. IIUC, COMMERROR packets are
swallowed instead of emitted before authentication completes. So I see
EOFs when trying to connect to older servers. Do you know of any
situations where we'd see an actual error message on the wire?
If it helps, we could backport a nicer error message to old server
versions, similar to what we did with SCRAM in commit 96d0f988b1.
That might be nice regardless, instead of pushing "invalid length of
startup packet" into the logs.
For the record, I'm adding some one-off tests for this feature to a
local copy of my OAuth pytest suite, which is designed to do the kinds
of testing you're running into trouble with. It's not in any way
viable for a PG17 commit, but if you're interested I can make the
patches available.Yes please, it would be nice to see what tests you've performed, and
have it archived.
I've cleaned it up a bit and put it up at [1]https://github.com/jchampio/pg-pytest-suite. (If you want, I can
attach the GitHub-generated ZIP, so the mailing list has a snapshot.)
These include happy-path tests for direct SSL, some failure modes, and
an example test that combines the GSS and SSL negotiation paths. So
there might be test bugs, but with the v8 patchset, I see the
following failures:
FAILED client/test_tls.py::test_direct_ssl_without_alpn - AssertionError: client sent unexpected data
I.e. the client doesn't disconnect if the server doesn't select our protocol.
FAILED client/test_tls.py::test_direct_ssl_failed_negotiation[direct-True] - AssertionError: Regex pattern did not match.
FAILED client/test_tls.py::test_direct_ssl_failed_negotiation[requiredirect-False] - AssertionError: Regex pattern did not match.
FAILED client/test_tls.py::test_gssapi_negotiation - AssertionError: Regex pattern did not match.
These are complaining about the "SSL SYSCALL error: EOF detected"
error messages that the client returns.
FAILED server/test_tls.py::test_direct_ssl_without_alpn[no application protocols] - Failed: DID NOT RAISE <class 'ssl.SSLError'>
FAILED server/test_tls.py::test_direct_ssl_without_alpn[incorrect application protocol] - Failed: DID NOT RAISE <class 'ssl.SSLError'>
I.e. the server allows the handshake to complete without a proper ALPN
selection.
Thanks,
--Jacob
I keep forgetting -- attached is the diff I'm carrying to plug
libpq_encryption into Meson. (The current patchset has a meson.build
for it, but it's not connected.)
--Jacob
Attachments:
meson.diff.txttext/plain; charset=US-ASCII; name=meson.diff.txtDownload
commit 64215f1e6b68208378b34cc0736d2f0eb1d45919
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Wed Feb 28 11:28:17 2024 -0800
WIP: mesonify
diff --git a/src/test/libpq_encryption/meson.build b/src/test/libpq_encryption/meson.build
index 04f479e9fe..ac1db10d74 100644
--- a/src/test/libpq_encryption/meson.build
+++ b/src/test/libpq_encryption/meson.build
@@ -1,13 +1,12 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
tests += {
- 'name': 'ldap',
+ 'name': 'libpq_encryption',
'sd': meson.current_source_dir(),
'bd': meson.current_build_dir(),
'tap': {
'tests': [
- 't/001_auth.pl',
- 't/002_bindpasswd.pl',
+ 't/001_negotiate_encryption.pl',
],
'env': {
'with_ssl': ssl_library,
diff --git a/src/test/meson.build b/src/test/meson.build
index c3d0dfedf1..702213bc6f 100644
--- a/src/test/meson.build
+++ b/src/test/meson.build
@@ -4,6 +4,7 @@ subdir('regress')
subdir('isolation')
subdir('authentication')
+subdir('libpq_encryption')
subdir('recovery')
subdir('subscription')
subdir('modules')
On Tue, 5 Mar 2024 at 15:08, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
I hope I didn't joggle your elbow reviewing this, Jacob, but I spent
some time rebase and fix various little things:
With the recent changes to backend startup committed by you, this
patchset has gotten major apply failures.
Could you provide a new version of the patchset so that it can be
reviewed in the context of current HEAD?
Kind regards,
Matthias van de Meent
Neon (https://neon.tech)
On 28/03/2024 13:15, Matthias van de Meent wrote:
On Tue, 5 Mar 2024 at 15:08, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
I hope I didn't joggle your elbow reviewing this, Jacob, but I spent
some time rebase and fix various little things:With the recent changes to backend startup committed by you, this
patchset has gotten major apply failures.Could you provide a new version of the patchset so that it can be
reviewed in the context of current HEAD?
Here you are.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
v9-0001-Move-Kerberos-module.patchtext/x-patch; charset=UTF-8; name=v9-0001-Move-Kerberos-module.patchDownload
From 83484696e470ab130bcd3038f0e28d494065071a Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 5 Mar 2024 15:40:30 +0200
Subject: [PATCH v9 1/6] Move Kerberos module
So that we can reuse it in new tests.
---
src/test/kerberos/t/001_auth.pl | 174 ++--------------
src/test/perl/PostgreSQL/Test/Kerberos.pm | 229 ++++++++++++++++++++++
2 files changed, 240 insertions(+), 163 deletions(-)
create mode 100644 src/test/perl/PostgreSQL/Test/Kerberos.pm
diff --git a/src/test/kerberos/t/001_auth.pl b/src/test/kerberos/t/001_auth.pl
index e51e87d0a2e..d4f1ec58092 100644
--- a/src/test/kerberos/t/001_auth.pl
+++ b/src/test/kerberos/t/001_auth.pl
@@ -21,6 +21,7 @@ use strict;
use warnings FATAL => 'all';
use PostgreSQL::Test::Utils;
use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Kerberos;
use Test::More;
use Time::HiRes qw(usleep);
@@ -34,177 +35,27 @@ elsif (!$ENV{PG_TEST_EXTRA} || $ENV{PG_TEST_EXTRA} !~ /\bkerberos\b/)
'Potentially unsafe test GSSAPI/Kerberos not enabled in PG_TEST_EXTRA';
}
-my ($krb5_bin_dir, $krb5_sbin_dir);
-
-if ($^O eq 'darwin' && -d "/opt/homebrew")
-{
- # typical paths for Homebrew on ARM
- $krb5_bin_dir = '/opt/homebrew/opt/krb5/bin';
- $krb5_sbin_dir = '/opt/homebrew/opt/krb5/sbin';
-}
-elsif ($^O eq 'darwin')
-{
- # typical paths for Homebrew on Intel
- $krb5_bin_dir = '/usr/local/opt/krb5/bin';
- $krb5_sbin_dir = '/usr/local/opt/krb5/sbin';
-}
-elsif ($^O eq 'freebsd')
-{
- $krb5_bin_dir = '/usr/local/bin';
- $krb5_sbin_dir = '/usr/local/sbin';
-}
-elsif ($^O eq 'linux')
-{
- $krb5_sbin_dir = '/usr/sbin';
-}
-
-my $krb5_config = 'krb5-config';
-my $kinit = 'kinit';
-my $klist = 'klist';
-my $kdb5_util = 'kdb5_util';
-my $kadmin_local = 'kadmin.local';
-my $krb5kdc = 'krb5kdc';
-
-if ($krb5_bin_dir && -d $krb5_bin_dir)
-{
- $krb5_config = $krb5_bin_dir . '/' . $krb5_config;
- $kinit = $krb5_bin_dir . '/' . $kinit;
- $klist = $krb5_bin_dir . '/' . $klist;
-}
-if ($krb5_sbin_dir && -d $krb5_sbin_dir)
-{
- $kdb5_util = $krb5_sbin_dir . '/' . $kdb5_util;
- $kadmin_local = $krb5_sbin_dir . '/' . $kadmin_local;
- $krb5kdc = $krb5_sbin_dir . '/' . $krb5kdc;
-}
-
-my $host = 'auth-test-localhost.postgresql.example.com';
-my $hostaddr = '127.0.0.1';
-my $realm = 'EXAMPLE.COM';
-
-my $krb5_conf = "${PostgreSQL::Test::Utils::tmp_check}/krb5.conf";
-my $kdc_conf = "${PostgreSQL::Test::Utils::tmp_check}/kdc.conf";
-my $krb5_cache = "${PostgreSQL::Test::Utils::tmp_check}/krb5cc";
-my $krb5_log = "${PostgreSQL::Test::Utils::log_path}/krb5libs.log";
-my $kdc_log = "${PostgreSQL::Test::Utils::log_path}/krb5kdc.log";
-my $kdc_port = PostgreSQL::Test::Cluster::get_free_port();
-my $kdc_datadir = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc";
-my $kdc_pidfile = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc.pid";
-my $keytab = "${PostgreSQL::Test::Utils::tmp_check}/krb5.keytab";
-
my $pgpass = "${PostgreSQL::Test::Utils::tmp_check}/.pgpass";
my $dbname = 'postgres';
my $username = 'test1';
my $application = '001_auth.pl';
-note "setting up Kerberos";
-
-my ($stdout, $krb5_version);
-run_log [ $krb5_config, '--version' ], '>', \$stdout
- or BAIL_OUT("could not execute krb5-config");
-BAIL_OUT("Heimdal is not supported") if $stdout =~ m/heimdal/;
-$stdout =~ m/Kerberos 5 release ([0-9]+\.[0-9]+)/
- or BAIL_OUT("could not get Kerberos version");
-$krb5_version = $1;
-
# Construct a pgpass file to make sure we don't use it
append_to_file($pgpass, '*:*:*:*:abc123');
chmod 0600, $pgpass or die $!;
-# Build the krb5.conf to use.
-#
-# Explicitly specify the default (test) realm and the KDC for
-# that realm to avoid the Kerberos library trying to look up
-# that information in DNS, and also because we're using a
-# non-standard KDC port.
-#
-# Also explicitly disable DNS lookups since this isn't really
-# our domain and we shouldn't be causing random DNS requests
-# to be sent out (not to mention that broken DNS environments
-# can cause the tests to take an extra long time and timeout).
-#
-# Reverse DNS is explicitly disabled to avoid any issue with a
-# captive portal or other cases where the reverse DNS succeeds
-# and the Kerberos library uses that as the canonical name of
-# the host and then tries to acquire a cross-realm ticket.
-append_to_file(
- $krb5_conf,
- qq![logging]
-default = FILE:$krb5_log
-kdc = FILE:$kdc_log
-
-[libdefaults]
-dns_lookup_realm = false
-dns_lookup_kdc = false
-default_realm = $realm
-forwardable = false
-rdns = false
-
-[realms]
-$realm = {
- kdc = $hostaddr:$kdc_port
-}
-!);
-
-append_to_file(
- $kdc_conf,
- qq![kdcdefaults]
-!);
-
-# For new-enough versions of krb5, use the _listen settings rather
-# than the _ports settings so that we can bind to localhost only.
-if ($krb5_version >= 1.15)
-{
- append_to_file(
- $kdc_conf,
- qq!kdc_listen = $hostaddr:$kdc_port
-kdc_tcp_listen = $hostaddr:$kdc_port
-!);
-}
-else
-{
- append_to_file(
- $kdc_conf,
- qq!kdc_ports = $kdc_port
-kdc_tcp_ports = $kdc_port
-!);
-}
-append_to_file(
- $kdc_conf,
- qq!
-[realms]
-$realm = {
- database_name = $kdc_datadir/principal
- admin_keytab = FILE:$kdc_datadir/kadm5.keytab
- acl_file = $kdc_datadir/kadm5.acl
- key_stash_file = $kdc_datadir/_k5.$realm
-}!);
-
-mkdir $kdc_datadir or die;
-
-# Ensure that we use test's config and cache files, not global ones.
-$ENV{'KRB5_CONFIG'} = $krb5_conf;
-$ENV{'KRB5_KDC_PROFILE'} = $kdc_conf;
-$ENV{'KRB5CCNAME'} = $krb5_cache;
+note "setting up Kerberos";
-my $service_principal = "$ENV{with_krb_srvnam}/$host";
+my $host = 'auth-test-localhost.postgresql.example.com';
+my $hostaddr = '127.0.0.1';
+my $realm = 'EXAMPLE.COM';
-system_or_bail $kdb5_util, 'create', '-s', '-P', 'secret0';
+my $krb = PostgreSQL::Test::Kerberos->new($host, $hostaddr, $realm);
my $test1_password = 'secret1';
-system_or_bail $kadmin_local, '-q', "addprinc -pw $test1_password test1";
-
-system_or_bail $kadmin_local, '-q', "addprinc -randkey $service_principal";
-system_or_bail $kadmin_local, '-q', "ktadd -k $keytab $service_principal";
-
-system_or_bail $krb5kdc, '-P', $kdc_pidfile;
-
-END
-{
- kill 'INT', `cat $kdc_pidfile` if defined($kdc_pidfile) && -f $kdc_pidfile;
-}
+$krb->create_principal('test1', $test1_password);
note "setting up PostgreSQL instance";
@@ -213,7 +64,7 @@ $node->init;
$node->append_conf(
'postgresql.conf', qq{
listen_addresses = '$hostaddr'
-krb_server_keyfile = '$keytab'
+krb_server_keyfile = '$krb->{keytab}'
log_connections = on
lc_messages = 'C'
});
@@ -327,8 +178,7 @@ $node->restart;
test_access($node, 'test1', 'SELECT true', 2, '', 'fails without ticket');
-run_log [ $kinit, 'test1' ], \$test1_password or BAIL_OUT($?);
-run_log [ $klist, '-f' ] or BAIL_OUT($?);
+$krb->create_ticket('test1', $test1_password);
test_access(
$node,
@@ -470,10 +320,8 @@ $node->append_conf(
hostgssenc all all $hostaddr/32 gss map=mymap
});
-string_replace_file($krb5_conf, "forwardable = false", "forwardable = true");
-
-run_log [ $kinit, 'test1' ], \$test1_password or BAIL_OUT($?);
-run_log [ $klist, '-f' ] or BAIL_OUT($?);
+# Re-create the ticket, with the forwardable flag set
+$krb->create_ticket('test1', $test1_password, forwardable => 1);
test_access(
$node,
diff --git a/src/test/perl/PostgreSQL/Test/Kerberos.pm b/src/test/perl/PostgreSQL/Test/Kerberos.pm
new file mode 100644
index 00000000000..5bb00c76f14
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/Kerberos.pm
@@ -0,0 +1,229 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+# Sets up a stand-alone KDC for testing PostgreSQL GSSAPI / Kerberos
+# functionality.
+
+package PostgreSQL::Test::Kerberos;
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+
+our ($krb5_bin_dir, $krb5_sbin_dir, $krb5_config, $kinit, $klist,
+ $kdb5_util, $kadmin_local, $krb5kdc,
+ $krb5_conf, $kdc_conf, $krb5_cache, $krb5_log, $kdc_log,
+ $kdc_port, $kdc_datadir, $kdc_pidfile, $keytab);
+
+INIT
+{
+ if ($^O eq 'darwin' && -d "/opt/homebrew")
+ {
+ # typical paths for Homebrew on ARM
+ $krb5_bin_dir = '/opt/homebrew/opt/krb5/bin';
+ $krb5_sbin_dir = '/opt/homebrew/opt/krb5/sbin';
+ }
+ elsif ($^O eq 'darwin')
+ {
+ # typical paths for Homebrew on Intel
+ $krb5_bin_dir = '/usr/local/opt/krb5/bin';
+ $krb5_sbin_dir = '/usr/local/opt/krb5/sbin';
+ }
+ elsif ($^O eq 'freebsd')
+ {
+ $krb5_bin_dir = '/usr/local/bin';
+ $krb5_sbin_dir = '/usr/local/sbin';
+ }
+ elsif ($^O eq 'linux')
+ {
+ $krb5_sbin_dir = '/usr/sbin';
+ }
+
+ $krb5_config = 'krb5-config';
+ $kinit = 'kinit';
+ $klist = 'klist';
+ $kdb5_util = 'kdb5_util';
+ $kadmin_local = 'kadmin.local';
+ $krb5kdc = 'krb5kdc';
+
+ if ($krb5_bin_dir && -d $krb5_bin_dir)
+ {
+ $krb5_config = $krb5_bin_dir . '/' . $krb5_config;
+ $kinit = $krb5_bin_dir . '/' . $kinit;
+ $klist = $krb5_bin_dir . '/' . $klist;
+ }
+ if ($krb5_sbin_dir && -d $krb5_sbin_dir)
+ {
+ $kdb5_util = $krb5_sbin_dir . '/' . $kdb5_util;
+ $kadmin_local = $krb5_sbin_dir . '/' . $kadmin_local;
+ $krb5kdc = $krb5_sbin_dir . '/' . $krb5kdc;
+ }
+
+ $krb5_conf = "${PostgreSQL::Test::Utils::tmp_check}/krb5.conf";
+ $kdc_conf = "${PostgreSQL::Test::Utils::tmp_check}/kdc.conf";
+ $krb5_cache = "${PostgreSQL::Test::Utils::tmp_check}/krb5cc";
+ $krb5_log = "${PostgreSQL::Test::Utils::log_path}/krb5libs.log";
+ $kdc_log = "${PostgreSQL::Test::Utils::log_path}/krb5kdc.log";
+ $kdc_port = PostgreSQL::Test::Cluster::get_free_port();
+ $kdc_datadir = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc";
+ $kdc_pidfile = "${PostgreSQL::Test::Utils::tmp_check}/krb5kdc.pid";
+ $keytab = "${PostgreSQL::Test::Utils::tmp_check}/krb5.keytab";
+}
+
+=pod
+
+=item PostgreSQL::Test::Kerberos->new(host, hostaddr, realm, %params)
+
+Sets up a new Kerberos realm and KDC. This function assigns a free
+port for the KDC. The KDC will be shut down automatically when the
+test script exits.
+
+=over
+
+=item host => 'auth-test-localhost.postgresql.example.com'
+
+Hostname to use in the service principal.
+
+=item hostaddr => '127.0.0.1'
+
+Network interface the KDC will listen on.
+
+=item realm => 'EXAMPLE.COM'
+
+Name of the Kerberos realm.
+
+=back
+
+=cut
+
+sub new
+{
+ my $class = shift;
+ my ($host, $hostaddr, $realm) = @_;
+
+ my ($stdout, $krb5_version);
+ run_log [ $krb5_config, '--version' ], '>', \$stdout
+ or BAIL_OUT("could not execute krb5-config");
+ BAIL_OUT("Heimdal is not supported") if $stdout =~ m/heimdal/;
+ $stdout =~ m/Kerberos 5 release ([0-9]+\.[0-9]+)/
+ or BAIL_OUT("could not get Kerberos version");
+ $krb5_version = $1;
+
+ # Build the krb5.conf to use.
+ #
+ # Explicitly specify the default (test) realm and the KDC for
+ # that realm to avoid the Kerberos library trying to look up
+ # that information in DNS, and also because we're using a
+ # non-standard KDC port.
+ #
+ # Also explicitly disable DNS lookups since this isn't really
+ # our domain and we shouldn't be causing random DNS requests
+ # to be sent out (not to mention that broken DNS environments
+ # can cause the tests to take an extra long time and timeout).
+ #
+ # Reverse DNS is explicitly disabled to avoid any issue with a
+ # captive portal or other cases where the reverse DNS succeeds
+ # and the Kerberos library uses that as the canonical name of
+ # the host and then tries to acquire a cross-realm ticket.
+ append_to_file(
+ $krb5_conf,
+ qq![logging]
+default = FILE:$krb5_log
+kdc = FILE:$kdc_log
+
+[libdefaults]
+dns_lookup_realm = false
+dns_lookup_kdc = false
+default_realm = $realm
+forwardable = false
+rdns = false
+
+[realms]
+$realm = {
+ kdc = $hostaddr:$kdc_port
+}
+!);
+
+ append_to_file(
+ $kdc_conf,
+ qq![kdcdefaults]
+!);
+
+ # For new-enough versions of krb5, use the _listen settings rather
+ # than the _ports settings so that we can bind to localhost only.
+ if ($krb5_version >= 1.15)
+ {
+ append_to_file(
+ $kdc_conf,
+ qq!kdc_listen = $hostaddr:$kdc_port
+kdc_tcp_listen = $hostaddr:$kdc_port
+!);
+ }
+ else
+ {
+ append_to_file(
+ $kdc_conf,
+ qq!kdc_ports = $kdc_port
+kdc_tcp_ports = $kdc_port
+!);
+ }
+ append_to_file(
+ $kdc_conf,
+ qq!
+[realms]
+$realm = {
+ database_name = $kdc_datadir/principal
+ admin_keytab = FILE:$kdc_datadir/kadm5.keytab
+ acl_file = $kdc_datadir/kadm5.acl
+ key_stash_file = $kdc_datadir/_k5.$realm
+}!);
+
+ mkdir $kdc_datadir or die;
+
+ # Ensure that we use test's config and cache files, not global ones.
+ $ENV{'KRB5_CONFIG'} = $krb5_conf;
+ $ENV{'KRB5_KDC_PROFILE'} = $kdc_conf;
+ $ENV{'KRB5CCNAME'} = $krb5_cache;
+
+ my $service_principal = "$ENV{with_krb_srvnam}/$host";
+
+ system_or_bail $kdb5_util, 'create', '-s', '-P', 'secret0';
+
+ system_or_bail $kadmin_local, '-q', "addprinc -randkey $service_principal";
+ system_or_bail $kadmin_local, '-q', "ktadd -k $keytab $service_principal";
+
+ system_or_bail $krb5kdc, '-P', $kdc_pidfile;
+
+ my $self = {};
+ $self->{keytab} = $keytab;
+
+ bless $self, $class;
+
+ return $self;
+}
+
+sub create_principal
+{
+ my ($self, $principal, $password) = @_;
+
+ system_or_bail $kadmin_local, '-q', "addprinc -pw $password $principal";
+}
+
+sub create_ticket
+{
+ my ($self, $principal, $password, %params) = @_;
+
+ my @cmd = ($kinit, $principal);
+
+ push @cmd, '-f' if ($params{forwardable});
+
+ run_log [@cmd], \$password or BAIL_OUT($?);
+ run_log [ $klist, '-f' ] or BAIL_OUT($?);
+}
+
+END
+{
+ kill 'INT', `cat $kdc_pidfile` if defined($kdc_pidfile) && -f $kdc_pidfile;
+}
+
+1;
--
2.39.2
v9-0002-Add-tests-for-libpq-choosing-encryption-mode.patchtext/x-patch; charset=UTF-8; name=v9-0002-Add-tests-for-libpq-choosing-encryption-mode.patchDownload
From e52c701931f5c4b67d2014d6d27c925501f15464 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 5 Mar 2024 15:41:09 +0200
Subject: [PATCH v9 2/6] Add tests for libpq choosing encryption mode
Author: Heikki Linnakangas, Matthias van de Meent
Discussion: XX
---
src/test/libpq_encryption/Makefile | 25 ++
src/test/libpq_encryption/README | 23 ++
src/test/libpq_encryption/meson.build | 19 +
.../t/001_negotiate_encryption.pl | 369 ++++++++++++++++++
src/test/perl/PostgreSQL/Test/Kerberos.pm | 6 +-
5 files changed, 439 insertions(+), 3 deletions(-)
create mode 100644 src/test/libpq_encryption/Makefile
create mode 100644 src/test/libpq_encryption/README
create mode 100644 src/test/libpq_encryption/meson.build
create mode 100644 src/test/libpq_encryption/t/001_negotiate_encryption.pl
diff --git a/src/test/libpq_encryption/Makefile b/src/test/libpq_encryption/Makefile
new file mode 100644
index 00000000000..710929c4cce
--- /dev/null
+++ b/src/test/libpq_encryption/Makefile
@@ -0,0 +1,25 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for src/test/libpq_encryption
+#
+# Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/test/ldap/libpq_encryption
+#
+#-------------------------------------------------------------------------
+
+subdir = src/test/libpq_encryption
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+export OPENSSL with_ssl with_gssapi with_krb_srvnam
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
+
+clean distclean:
+ rm -rf tmp_check
diff --git a/src/test/libpq_encryption/README b/src/test/libpq_encryption/README
new file mode 100644
index 00000000000..66430b54316
--- /dev/null
+++ b/src/test/libpq_encryption/README
@@ -0,0 +1,23 @@
+src/test/libpq_encryption/README
+
+Tests for negotiating network encryption method
+===============================================
+
+
+Running the tests
+=================
+
+NOTE: You must have given the --enable-tap-tests argument to configure.
+
+Run
+ make check PG_TEST_EXTRA=libpq_encryption
+
+XXX You can use "make installcheck" if you previously did "make install".
+In that case, the code in the installation tree is tested. With
+"make check", a temporary installation tree is built from the current
+sources and then tested.
+
+XXX Either way, this test initializes, starts, and stops a test Postgres
+cluster, as well as a test LDAP server.
+
+See src/test/perl/README for more info about running these tests.
diff --git a/src/test/libpq_encryption/meson.build b/src/test/libpq_encryption/meson.build
new file mode 100644
index 00000000000..04f479e9fe7
--- /dev/null
+++ b/src/test/libpq_encryption/meson.build
@@ -0,0 +1,19 @@
+# Copyright (c) 2022-2024, PostgreSQL Global Development Group
+
+tests += {
+ 'name': 'ldap',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'tap': {
+ 'tests': [
+ 't/001_auth.pl',
+ 't/002_bindpasswd.pl',
+ ],
+ 'env': {
+ 'with_ssl': ssl_library,
+ 'OPENSSL': openssl.found() ? openssl.path() : '',
+ 'with_gssapi': gssapi.found() ? 'yes' : 'no',
+ 'with_krb_srvnam': 'postgres',
+ },
+ },
+}
diff --git a/src/test/libpq_encryption/t/001_negotiate_encryption.pl b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
new file mode 100644
index 00000000000..b8646c5bc97
--- /dev/null
+++ b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
@@ -0,0 +1,369 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+# Test negotiation of SSL and GSSAPI encryption
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Kerberos;
+use File::Basename;
+use File::Copy;
+use Test::More;
+
+if (!$ENV{PG_TEST_EXTRA} || $ENV{PG_TEST_EXTRA} !~ /\blibpq_encryption\b/)
+{
+ plan skip_all =>
+ 'Potentially unsafe test libpq_encryption not enabled in PG_TEST_EXTRA';
+}
+
+my $host = 'enc-test-localhost.postgresql.example.com';
+my $hostaddr = '127.0.0.1';
+my $servercidr = '127.0.0.1/32';
+
+note "setting up PostgreSQL instance";
+
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init;
+$node->append_conf(
+ 'postgresql.conf', qq{
+listen_addresses = '$hostaddr'
+log_connections = on
+lc_messages = 'C'
+});
+my $pgdata = $node->data_dir;
+
+my $dbname = 'postgres';
+my $username = 'enctest';
+my $application = '001_negotiate_encryption.pl';
+
+my $gssuser_password = 'secret1';
+
+my $krb;
+
+my $ssl_supported = $ENV{with_ssl} eq 'openssl';
+my $gss_supported = $ENV{with_gssapi} eq 'yes';
+
+if ($gss_supported != 0)
+{
+ note "setting up Kerberos";
+
+ my $realm = 'EXAMPLE.COM';
+ $krb = PostgreSQL::Test::Kerberos->new($host, $hostaddr, $realm);
+ $node->append_conf('postgresql.conf', "krb_server_keyfile = '$krb->{keytab}'\n");
+}
+
+if ($ssl_supported != 0)
+{
+ my $certdir = dirname(__FILE__) . "/../../ssl/ssl";
+
+ copy "$certdir/server-cn-only.crt", "$pgdata/server.crt"
+ || die "copying server.crt: $!";
+ copy "$certdir/server-cn-only.key", "$pgdata/server.key"
+ || die "copying server.key: $!";
+ chmod(0600, "$pgdata/server.key");
+
+ # Start with SSL disabled.
+ $node->append_conf('postgresql.conf', "ssl = off\n");
+}
+
+$node->start;
+
+$node->safe_psql('postgres', 'CREATE USER localuser;');
+$node->safe_psql('postgres', 'CREATE USER testuser;');
+$node->safe_psql('postgres', 'CREATE USER ssluser;');
+$node->safe_psql('postgres', 'CREATE USER nossluser;');
+$node->safe_psql('postgres', 'CREATE USER gssuser;');
+$node->safe_psql('postgres', 'CREATE USER nogssuser;');
+
+my $unixdir = $node->safe_psql('postgres', 'SHOW unix_socket_directories;');
+chomp($unixdir);
+
+$node->safe_psql('postgres', q{
+CREATE FUNCTION current_enc() RETURNS text LANGUAGE plpgsql AS $$
+DECLARE
+ ssl_in_use bool;
+ gss_in_use bool;
+BEGIN
+ ssl_in_use = (SELECT ssl FROM pg_stat_ssl WHERE pid = pg_backend_pid());
+ gss_in_use = (SELECT encrypted FROM pg_stat_gssapi WHERE pid = pg_backend_pid());
+
+ raise log 'ssl % gss %', ssl_in_use, gss_in_use;
+
+ IF ssl_in_use AND gss_in_use THEN
+ RETURN 'ssl+gss'; -- shouldn't happen
+ ELSIF ssl_in_use THEN
+ RETURN 'ssl';
+ ELSIF gss_in_use THEN
+ RETURN 'gss';
+ ELSE
+ RETURN 'plain';
+ END IF;
+END;
+$$;
+});
+
+# Only accept SSL connections from $servercidr. Our tests don't depend on this
+# but seems best to keep it as narrow as possible for security reasons.
+#
+# When connecting to certdb, also check the client certificate.
+open my $hba, '>', "$pgdata/pg_hba.conf";
+print $hba qq{
+# TYPE DATABASE USER ADDRESS METHOD OPTIONS
+local postgres localuser trust
+host postgres testuser $servercidr trust
+hostnossl postgres nossluser $servercidr trust
+hostnogssenc postgres nogssuser $servercidr trust
+};
+
+print $hba qq{
+hostssl postgres ssluser $servercidr trust
+} if ($ssl_supported != 0);
+
+print $hba qq{
+hostgssenc postgres gssuser $servercidr trust
+} if ($gss_supported != 0);
+close $hba;
+$node->reload;
+
+sub connect_test
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $connstr, $expected_enc, @expect_log_msgs)
+ = @_;
+
+ my $test_name = " '$connstr' -> $expected_enc";
+
+ my $connstr_full = "";
+ $connstr_full .= "dbname=postgres " unless $connstr =~ m/dbname=/;
+ $connstr_full .= "host=$host hostaddr=$hostaddr " unless $connstr =~ m/host=/;
+ $connstr_full .= $connstr;
+
+ my $log_location = -s $node->logfile;
+
+ my ($ret, $stdout, $stderr) = $node->psql(
+ 'postgres',
+ 'SELECT current_enc()',
+ extra_params => ['-w'],
+ connstr => "$connstr_full",
+ on_error_stop => 0);
+
+ my $result = $ret == 0 ? $stdout : 'fail';
+
+ is($result, $expected_enc, $test_name);
+
+ if (@expect_log_msgs)
+ {
+ # Match every message literally.
+ my @regexes = map { qr/\Q$_\E/ } @expect_log_msgs;
+ my %params = ();
+ $params{log_like} = \@regexes;
+ $node->log_check($test_name, $log_location, %params);
+ }
+}
+
+# Return the encryption mode that we expect to be chosen by libpq,
+# when connecting with given the user, gssmode, sslmode settings.
+sub resolve_connection_type
+{
+ my ($config) = @_;
+ my $user = $config->{user};
+ my $gssmode = $config->{gssmode};
+ my $sslmode = $config->{sslmode};
+
+ my @conntypes = qw(plain);
+
+ # Add connection types supported by the server to the pool
+ push(@conntypes, "ssl") if $config->{server_ssl} == 1;
+ push(@conntypes, "gss") if $config->{server_gss} == 1;
+
+ # User configurations:
+ # gssuser/ssluser require the relevant connection type,
+ @conntypes = grep {/gss/} @conntypes if $user eq 'gssuser';
+ @conntypes = grep {/ssl/} @conntypes if $user eq 'ssluser';
+
+ # nogssuser/nossluser require anything but the relevant connection type.
+ @conntypes = grep {!/gss/} @conntypes if $user eq 'nogssuser';
+ @conntypes = grep {!/ssl/} @conntypes if $user eq 'nossluser';
+
+ print STDOUT "After user filter: @conntypes\n";
+
+ # remove disabled connection modes
+ @conntypes = grep {!/gss/} @conntypes if $gssmode eq 'disable';
+ @conntypes = grep {!/ssl/} @conntypes if $sslmode eq 'disable';
+
+ # If gssmode=require, drop all non-GSS modes.
+ if ($gssmode eq 'require')
+ {
+ @conntypes = grep {/gss/} @conntypes;
+ }
+
+ # If sslmode=require, drop plain mode.
+ #
+ # NOTE: GSS is also allowed with sslmode=require.
+ if ($sslmode eq 'require')
+ {
+ @conntypes = grep {!/plain/} @conntypes;
+ }
+
+ print STDOUT "After mode require filter: @conntypes\n";
+
+ # Handle priorities of the various types.
+ # Note that this doesn't need to care about require/disable/etc, those
+ # filters were applied before we get here.
+ # Also note that preference is 1 > 2 > 3 > 4 > 5, so first preference
+ # without ssl or gss 'prefer/require' is plain connections.
+ my %order = (plain=>3, gss=>4, ssl=>5);
+
+ $order{ssl} = 2 if $sslmode eq "prefer";
+ $order{gss} = 1 if $gssmode eq "prefer";
+ @conntypes = sort { $order{$a} cmp $order{$b} } @conntypes;
+
+ # If there are no connection types available after filtering requirements,
+ # the connection fails.
+ return "fail" if @conntypes == 0;
+ # Else, we get to connect using the connection type with the highest
+ # priority.
+ return $conntypes[0];
+}
+
+# First test with SSL disabled in the server
+
+# Test the cube of parameters: user, sslmode, and gssencmode
+sub test_modes
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($pg_node, $node_conf,
+ $test_users, $ssl_modes, $gss_modes) = @_;
+
+ foreach my $test_user (@{$test_users})
+ {
+ foreach my $client_mode (@{$ssl_modes})
+ {
+ foreach my $gssencmode (@{$gss_modes})
+ {
+ my %params = (
+ server_ssl=>$node_conf->{server_ssl},
+ server_gss=>$node_conf->{server_gss},
+ user=>$test_user,
+ sslmode=>$client_mode,
+ gssmode=>$gssencmode,
+ );
+ my $res = resolve_connection_type(\%params);
+ connect_test($pg_node, "user=$test_user sslmode=$client_mode gssencmode=$gssencmode", $res);
+ }
+ }
+ }
+}
+
+my $sslmodes = ['disable', 'allow', 'prefer', 'require'];
+my $gssencmodes = ['disable', 'prefer', 'require'];
+
+my $server_config = {
+ server_ssl => 0,
+ server_gss => 0,
+};
+
+note("Running tests with SSL and GSS disabled in server");
+test_modes($node, $server_config,
+ ['testuser'],
+ $sslmodes, $gssencmodes);
+
+# Enable SSL in the server
+SKIP:
+{
+ skip "SSL not supported by this build" if $ssl_supported == 0;
+
+ $node->adjust_conf('postgresql.conf', 'ssl', 'on');
+ $node->restart;
+ $server_config->{server_ssl} = 1;
+
+ note("Running tests with SSL enabled in server");
+ test_modes($node, $server_config,
+ ['testuser', 'ssluser', 'nossluser'],
+ $sslmodes, ['disable']);
+
+ $node->adjust_conf('postgresql.conf', 'ssl', 'off');
+ $node->reload;
+ $server_config->{server_ssl} = 0;
+}
+
+# Test GSSAPI
+SKIP:
+{
+ skip "GSSAPI/Kerberos not supported by this build" if $gss_supported == 0;
+
+ # No ticket
+ connect_test($node, 'user=testuser sslmode=disable gssencmode=require', 'fail');
+
+ $krb->create_principal('gssuser', $gssuser_password);
+ $krb->create_ticket('gssuser', $gssuser_password);
+ $server_config->{server_gss} = 1;
+
+ note("Running tests with GSS enabled in server");
+ test_modes($node, $server_config,
+ ['testuser', 'gssuser', 'nogssuser'],
+ $sslmodes, $gssencmodes);
+
+ # Check that logs match the expected 'no pg_hba.conf entry' line, too, as
+ # that is not tested by test_modes.
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=require', 'fail',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+
+ # With 'gssencmode=prefer', libpq will first negotiate GSSAPI
+ # encryption, but the connection will fail because pg_hba.conf
+ # forbids GSSAPI encryption for this user. It will then reconnect
+ # with SSL, but the server doesn't support it, so it will continue
+ # with no encryption.
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer', 'plain',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+}
+
+# Server supports both SSL and GSSAPI
+SKIP:
+{
+ skip "GSSAPI/Kerberos or SSL not supported by this build" unless ($ssl_supported && $gss_supported);
+
+ # SSL is still disabled
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer', 'gss');
+
+ # Enable SSL
+ $node->adjust_conf('postgresql.conf', 'ssl', 'on');
+ $node->reload;
+ $server_config->{server_ssl} = 1;
+
+ note("Running tests with both GSS and SSL enabled in server");
+ test_modes($node, $server_config,
+ ['testuser', 'gssuser', 'ssluser', 'nogssuser', 'nossluser'],
+ $sslmodes, $gssencmodes);
+
+ # Test case that server supports GSSAPI, but it's not allowed for
+ # this user. Special cased because we check output
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=require', 'fail',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+
+ # with 'gssencmode=prefer', libpq will first negotiate GSSAPI
+ # encryption, but the connection will fail because pg_hba.conf
+ # forbids GSSAPI encryption for this user. It will then reconnect
+ # with SSL.
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer', 'ssl',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+
+ # Setting both sslmode=require and gssencmode=require fails if GSSAPI is not
+ # available.
+ connect_test($node, 'user=nogssuser sslmode=require gssencmode=require', 'fail');
+}
+
+# Test negotiation over unix domain sockets.
+SKIP:
+{
+ skip "Unix domain sockets not supported" unless ($unixdir ne "");
+
+ connect_test($node, "user=localuser sslmode=require gssencmode=prefer host=$unixdir", 'plain');
+ connect_test($node, "user=localuser sslmode=prefer gssencmode=require host=$unixdir", 'fail');
+}
+
+done_testing();
diff --git a/src/test/perl/PostgreSQL/Test/Kerberos.pm b/src/test/perl/PostgreSQL/Test/Kerberos.pm
index 5bb00c76f14..e6063703c3a 100644
--- a/src/test/perl/PostgreSQL/Test/Kerberos.pm
+++ b/src/test/perl/PostgreSQL/Test/Kerberos.pm
@@ -103,10 +103,10 @@ sub new
my ($stdout, $krb5_version);
run_log [ $krb5_config, '--version' ], '>', \$stdout
- or BAIL_OUT("could not execute krb5-config");
- BAIL_OUT("Heimdal is not supported") if $stdout =~ m/heimdal/;
+ or die("could not execute krb5-config");
+ die("Heimdal is not supported") if $stdout =~ m/heimdal/;
$stdout =~ m/Kerberos 5 release ([0-9]+\.[0-9]+)/
- or BAIL_OUT("could not get Kerberos version");
+ or die("could not get Kerberos version");
$krb5_version = $1;
# Build the krb5.conf to use.
--
2.39.2
v9-0003-Direct-SSL-connections-client-and-server-support.patchtext/x-patch; charset=UTF-8; name=v9-0003-Direct-SSL-connections-client-and-server-support.patchDownload
From c050ee7cd559dc1a3575cb11b4013bf3e10839b9 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Wed, 15 Mar 2023 15:51:02 -0400
Subject: [PATCH v9 3/6] Direct SSL connections, client and server support
Author: Greg Stark, Heikki Linnakangas
---
doc/src/sgml/libpq.sgml | 102 ++++++++++++++++++++++++++---
doc/src/sgml/protocol.sgml | 37 +++++++++++
src/backend/libpq/be-secure.c | 52 ++++++++++++++-
src/backend/libpq/pqcomm.c | 9 +--
src/backend/tcop/backend_startup.c | 83 +++++++++++++++++++++--
src/include/libpq/libpq-be.h | 13 ++++
src/include/libpq/libpq.h | 2 +-
src/interfaces/libpq/fe-connect.c | 99 ++++++++++++++++++++++++++--
src/interfaces/libpq/libpq-fe.h | 3 +-
src/interfaces/libpq/libpq-int.h | 4 ++
10 files changed, 374 insertions(+), 30 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index d3e87056f2c..0166355d834 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1700,10 +1700,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
Note that if <acronym>GSSAPI</acronym> encryption is possible,
that will be used in preference to <acronym>SSL</acronym>
encryption, regardless of the value of <literal>sslmode</literal>.
- To force use of <acronym>SSL</acronym> encryption in an
- environment that has working <acronym>GSSAPI</acronym>
- infrastructure (such as a Kerberos server), also
- set <literal>gssencmode</literal> to <literal>disable</literal>.
+ To negotiate <acronym>SSL</acronym> encryption in an environment that
+ has working <acronym>GSSAPI</acronym> infrastructure (such as a
+ Kerberos server), also set <literal>gssencmode</literal>
+ to <literal>disable</literal>.
+ Use of non-default values of <literal>sslnegotiation</literal> can
+ also cause <acronym>SSL</acronym> to be used instead of
+ negotiating <acronym>GSSAPI</acronym> encryption.
</para>
</listitem>
</varlistentry>
@@ -1730,6 +1733,75 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
</listitem>
</varlistentry>
+ <varlistentry id="libpq-connect-sslnegotiation" xreflabel="sslnegotiation">
+ <term><literal>sslnegotiation</literal></term>
+ <listitem>
+ <para>
+ This option controls whether <productname>PostgreSQL</productname>
+ will perform its protocol negotiation to request encryption from the
+ server or will just directly make a standard <acronym>SSL</acronym>
+ connection. Traditional <productname>PostgreSQL</productname>
+ protocol negotiation is the default and the most flexible with
+ different server configurations. If the server is known to support
+ direct <acronym>SSL</acronym> connections then the latter requires one
+ fewer round trip reducing connection latency and also allows the use
+ of protocol agnostic SSL network tools.
+ </para>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>postgres</literal></term>
+ <listitem>
+ <para>
+ perform <productname>PostgreSQL</productname> protocol
+ negotiation. This is the default if the option is not provided.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>direct</literal></term>
+ <listitem>
+ <para>
+ first attempt to establish a standard SSL connection and if that
+ fails reconnect and perform the negotiation. This fallback
+ process adds significant latency if the initial SSL connection
+ fails.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>requiredirect</literal></term>
+ <listitem>
+ <para>
+ attempt to establish a standard SSL connection and if that fails
+ return a connection failure immediately.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ <para>
+ If <literal>sslmode</literal> set to <literal>disable</literal>
+ or <literal>allow</literal> then <literal>sslnegotiation</literal> is
+ ignored. If <literal>gssencmode</literal> is set
+ to <literal>require</literal> then <literal>sslnegotiation</literal>
+ must be the default <literal>postgres</literal> value.
+ </para>
+
+ <para>
+ Moreover, note if <literal>gssencmode</literal> is set
+ to <literal>prefer</literal> and <literal>sslnegotiation</literal>
+ to <literal>direct</literal> then the effective preference will be
+ direct <acronym>SSL</acronym> connections, followed by
+ negotiated <acronym>GSS</acronym> connections, followed by
+ negotiated <acronym>SSL</acronym> connections, possibly followed
+ lastly by unencrypted connections.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-connect-sslcompression" xreflabel="sslcompression">
<term><literal>sslcompression</literal></term>
<listitem>
@@ -1963,11 +2035,13 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
<para>
The Server Name Indication can be used by SSL-aware proxies to route
- connections without having to decrypt the SSL stream. (Note that this
- requires a proxy that is aware of the PostgreSQL protocol handshake,
- not just any SSL proxy.) However, <acronym>SNI</acronym> makes the
- destination host name appear in cleartext in the network traffic, so
- it might be undesirable in some cases.
+ connections without having to decrypt the SSL stream. (Note that
+ unless the proxy is aware of the PostgreSQL protocol handshake this
+ would require setting <literal>sslnegotiation</literal>
+ to <literal>direct</literal> or <literal>requiredirect</literal>.)
+ However, <acronym>SNI</acronym> makes the destination host name appear
+ in cleartext in the network traffic, so it might be undesirable in
+ some cases.
</para>
</listitem>
</varlistentry>
@@ -8585,6 +8659,16 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough)
</para>
</listitem>
+ <listitem>
+ <para>
+ <indexterm>
+ <primary><envar>PGSSLNEGOTIATION</envar></primary>
+ </indexterm>
+ <envar>PGSSLNEGOTIATION</envar> behaves the same as the <xref
+ linkend="libpq-connect-sslnegotiation"/> connection parameter.
+ </para>
+ </listitem>
+
<listitem>
<para>
<indexterm>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index a5cb19357f5..b215602c5ed 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -1529,17 +1529,54 @@ SELCT 1/0;<!-- this typo is intentional -->
bytes.
</para>
+ <para>
+ Likewise the server expects the client to not begin
+ the <acronym>SSL</acronym> negotiation until it receives the server's
+ single byte response to the <acronym>SSL</acronym> request. If the
+ client begins the <acronym>SSL</acronym> negotiation immediately without
+ waiting for the server response to be received it can reduce connection
+ latency by one round-trip. However this comes at the cost of not being
+ able to handle the case where the server sends a negative response to the
+ <acronym>SSL</acronym> request. In that case instead of continuing with either GSSAPI or an
+ unencrypted connection or a protocol error the server will simply
+ disconnect.
+ </para>
+
<para>
An initial SSLRequest can also be used in a connection that is being
opened to send a CancelRequest message.
</para>
+ <para>
+ A second alternate way to initiate <acronym>SSL</acronym> encryption is
+ available. The server will recognize connections which immediately
+ begin <acronym>SSL</acronym> negotiation without any previous SSLRequest
+ packets. Once the <acronym>SSL</acronym> connection is established the
+ server will expect a normal startup-request packet and continue
+ negotiation over the encrypted channel. In this case any other requests
+ for encryption will be refused. This method is not preferred for general
+ purpose tools as it cannot negotiate the best connection encryption
+ available or handle unencrypted connections. However it is useful for
+ environments where both the server and client are controlled together.
+ In that case it avoids one round trip of latency and allows the use of
+ network tools that depend on standard <acronym>SSL</acronym> connections.
+ When using <acronym>SSL</acronym> connections in this style the client is
+ required to use the ALPN extension defined
+ by <ulink url="https://tools.ietf.org/html/rfc7301">RFC 7301</ulink> to
+ protect against protocol confusion attacks.
+ The <productname>PostgreSQL</productname> protocol is "TBD-pgsql" as
+ registered
+ at <ulink url="https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids">IANA
+ TLS ALPN Protocol IDs</ulink> registry.
+ </para>
+
<para>
While the protocol itself does not provide a way for the server to
force <acronym>SSL</acronym> encryption, the administrator can
configure the server to reject unencrypted sessions as a byproduct
of authentication checking.
</para>
+
</sect2>
<sect2 id="protocol-flow-gssapi">
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 5612c29f8b2..1d1329d1d95 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -109,18 +109,51 @@ secure_loaded_verify_locations(void)
int
secure_open_server(Port *port)
{
+#ifdef USE_SSL
int r = 0;
+ ssize_t len;
+
+ /* push unencrypted buffered data back through SSL setup */
+ len = pq_buffer_has_data();
+ if (len > 0)
+ {
+ char *buf = palloc(len);
+
+ pq_startmsgread();
+ if (pq_getbytes(buf, len) == EOF)
+ return STATUS_ERROR; /* shouldn't be possible */
+ pq_endmsgread();
+ port->raw_buf = buf;
+ port->raw_buf_remaining = len;
+ port->raw_buf_consumed = 0;
+ }
+ Assert(pq_buffer_has_data() == 0);
-#ifdef USE_SSL
r = be_tls_open_server(port);
+ if (port->raw_buf_remaining > 0)
+ {
+ /*
+ * This shouldn't be possible -- it would mean the client sent
+ * encrypted data before we established a session key...
+ */
+ elog(LOG, "Buffered unencrypted data remains after negotiating native SSL connection");
+ return STATUS_ERROR;
+ }
+ if (port->raw_buf != NULL)
+ {
+ pfree(port->raw_buf);
+ port->raw_buf = NULL;
+ }
+
ereport(DEBUG2,
(errmsg_internal("SSL connection from DN:\"%s\" CN:\"%s\"",
port->peer_dn ? port->peer_dn : "(anonymous)",
port->peer_cn ? port->peer_cn : "(anonymous)")));
-#endif
-
return r;
+#else
+ return 0;
+#endif
}
/*
@@ -232,6 +265,19 @@ secure_raw_read(Port *port, void *ptr, size_t len)
{
ssize_t n;
+ /* Read from the "unread" buffered data first. c.f. libpq-be.h */
+ if (port->raw_buf_remaining > 0)
+ {
+ /* consume up to len bytes from the raw_buf */
+ if (len > port->raw_buf_remaining)
+ len = port->raw_buf_remaining;
+ Assert(port->raw_buf);
+ memcpy(ptr, port->raw_buf + port->raw_buf_consumed, len);
+ port->raw_buf_consumed += len;
+ port->raw_buf_remaining -= len;
+ return len;
+ }
+
/*
* Try to read from the socket without blocking. If it succeeds we're
* done, otherwise we'll wait for the socket using the latch mechanism.
diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c
index 6497100a1a4..58f6e3da4f8 100644
--- a/src/backend/libpq/pqcomm.c
+++ b/src/backend/libpq/pqcomm.c
@@ -1114,15 +1114,16 @@ pq_discardbytes(size_t len)
}
/* --------------------------------
- * pq_buffer_has_data - is any buffered data available to read?
+ * pq_buffer_has_data - return number of bytes in receive buffer
*
- * This will *not* attempt to read more data.
+ * This will *not* attempt to read more data. And reading up to that number of
+ * bytes should not cause reading any more data either.
* --------------------------------
*/
-bool
+size_t
pq_buffer_has_data(void)
{
- return (PqRecvPointer < PqRecvLength);
+ return (PqRecvLength - PqRecvPointer);
}
diff --git a/src/backend/tcop/backend_startup.c b/src/backend/tcop/backend_startup.c
index 0b9f899cd8b..d2ce8e47a8e 100644
--- a/src/backend/tcop/backend_startup.c
+++ b/src/backend/tcop/backend_startup.c
@@ -38,6 +38,7 @@
#include "utils/timeout.h"
static void BackendInitialize(ClientSocket *client_sock, CAC_state cac);
+static int ProcessSSLStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void process_startup_packet_die(SIGNAL_ARGS);
@@ -252,7 +253,9 @@ BackendInitialize(ClientSocket *client_sock, CAC_state cac)
* Receive the startup packet (which might turn out to be a cancel request
* packet).
*/
- status = ProcessStartupPacket(port, false, false);
+ status = ProcessSSLStartup(port);
+ if (status == STATUS_OK)
+ status = ProcessStartupPacket(port, false, false);
/*
* If we're going to reject the connection due to database state, say so
@@ -344,6 +347,68 @@ BackendInitialize(ClientSocket *client_sock, CAC_state cac)
set_ps_display("initializing");
}
+/*
+ * Check for a native direct SSL connection.
+ *
+ * This happens before startup packets so we are careful not to actual read
+ * any bytes from the stream if it's not a direct SSL connection.
+ */
+static int
+ProcessSSLStartup(Port *port)
+{
+ int firstbyte;
+
+ Assert(!port->ssl_in_use);
+
+ pq_startmsgread();
+ firstbyte = pq_peekbyte();
+ pq_endmsgread();
+ if (firstbyte == EOF)
+ {
+ /*
+ * Like in ProcessStartupPacket, if we get no data at all, don't
+ * clutter the log with a complaint.
+ */
+ return STATUS_ERROR;
+ }
+
+ /*
+ * First byte indicates standard SSL handshake message
+ *
+ * (It can't be a Postgres startup length because in network byte order
+ * that would be a startup packet hundreds of megabytes long)
+ */
+ if (firstbyte == 0x16)
+ {
+ elog(LOG, "Detected direct SSL handshake");
+
+#ifdef USE_SSL
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+ {
+ /* SSL not supported */
+ return STATUS_ERROR;
+ }
+ else if (secure_open_server(port) == -1)
+ {
+ /*
+ * we assume secure_open_server() sent an appropriate TLS alert
+ * already
+ */
+ return STATUS_ERROR;
+ }
+#else
+ /* SSL not supported by this build */
+ return STATUS_ERROR;
+#endif
+ }
+
+ if (port->ssl_in_use)
+ ereport(DEBUG2,
+ (errmsg_internal("Direct SSL connection established")));
+
+ return STATUS_OK;
+}
+
/*
* Read a client's startup packet and do something according to it.
*
@@ -465,8 +530,13 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
char SSLok;
#ifdef USE_SSL
- /* No SSL when disabled or on Unix sockets */
- if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
+
+ /*
+ * No SSL when disabled or on Unix sockets.
+ *
+ * Also no SSL negotiation if we already have a direct SSL connection
+ */
+ if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX || port->ssl_in_use)
SSLok = 'N';
else
SSLok = 'S'; /* Support for SSL */
@@ -474,11 +544,10 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done)
SSLok = 'N'; /* No support for SSL */
#endif
-retry1:
- if (send(port->sock, &SSLok, 1, 0) != 1)
+ while (secure_write(port, &SSLok, 1) != 1)
{
if (errno == EINTR)
- goto retry1; /* if interrupted, just retry */
+ continue; /* if interrupted, just retry */
ereport(COMMERROR,
(errcode_for_socket_access(),
errmsg("failed to send SSL negotiation response: %m")));
@@ -519,7 +588,7 @@ retry1:
GSSok = 'G';
#endif
- while (send(port->sock, &GSSok, 1, 0) != 1)
+ while (secure_write(port, &GSSok, 1) != 1)
{
if (errno == EINTR)
continue;
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 4dce7677510..86f773999ed 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -212,6 +212,19 @@ typedef struct Port
SSL *ssl;
X509 *peer;
#endif
+
+ /*
+ * This is a bit of a hack. The raw_buf is data that was previously read
+ * and buffered in a higher layer but then "unread" and needs to be read
+ * again while establishing an SSL connection via the SSL library layer.
+ *
+ * There's no API to "unread", the upper layer just places the data in the
+ * Port structure in raw_buf and sets raw_buf_remaining to the amount of
+ * bytes unread and raw_buf_consumed to 0.
+ */
+ char *raw_buf;
+ ssize_t raw_buf_consumed,
+ raw_buf_remaining;
} Port;
/*
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index be054b59dd1..a9c89e8179b 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -79,7 +79,7 @@ extern int pq_getmessage(StringInfo s, int maxlen);
extern int pq_getbyte(void);
extern int pq_peekbyte(void);
extern int pq_getbyte_if_available(unsigned char *c);
-extern bool pq_buffer_has_data(void);
+extern size_t pq_buffer_has_data(void);
extern int pq_putmessage_v2(char msgtype, const char *s, size_t len);
extern bool pq_check_connection(void);
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 01e49c6975e..a559c80e258 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -129,6 +129,7 @@ static int ldapServiceLookup(const char *purl, PQconninfoOption *options,
#define DefaultSSLMode "disable"
#define DefaultSSLCertMode "disable"
#endif
+#define DefaultSSLNegotiation "postgres"
#ifdef ENABLE_GSS
#include "fe-gssapi-common.h"
#define DefaultGSSMode "prefer"
@@ -272,6 +273,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-Mode", "", 12, /* sizeof("verify-full") == 12 */
offsetof(struct pg_conn, sslmode)},
+ {"sslnegotiation", "PGSSLNEGOTIATION", DefaultSSLNegotiation, NULL,
+ "SSL-Negotiation", "", 14, /* strlen("requiredirect") == 14 */
+ offsetof(struct pg_conn, sslnegotiation)},
+
{"sslcompression", "PGSSLCOMPRESSION", "0", NULL,
"SSL-Compression", "", 1,
offsetof(struct pg_conn, sslcompression)},
@@ -1565,10 +1570,37 @@ pqConnectOptions2(PGconn *conn)
}
#endif
}
+
+ /*
+ * validate sslnegotiation option, default is "postgres" for the postgres
+ * style negotiated connection with an extra round trip but more options.
+ */
+ if (conn->sslnegotiation)
+ {
+ if (strcmp(conn->sslnegotiation, "postgres") != 0
+ && strcmp(conn->sslnegotiation, "direct") != 0
+ && strcmp(conn->sslnegotiation, "requiredirect") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "invalid %s value: \"%s\"",
+ "sslnegotiation", conn->sslnegotiation);
+ return false;
+ }
+
+#ifndef USE_SSL
+ if (conn->sslnegotiation[0] != 'p')
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "sslnegotiation value \"%s\" invalid when SSL support is not compiled in",
+ conn->sslnegotiation);
+ return false;
+ }
+#endif
+ }
else
{
- conn->sslmode = strdup(DefaultSSLMode);
- if (!conn->sslmode)
+ conn->sslnegotiation = strdup(DefaultSSLNegotiation);
+ if (!conn->sslnegotiation)
goto oom_error;
}
@@ -1691,6 +1723,21 @@ pqConnectOptions2(PGconn *conn)
conn->gssencmode);
return false;
}
+#endif
+#ifdef USE_SSL
+
+ /*
+ * GSS is incompatible with direct SSL connections so it requires the
+ * default postgres style connection ssl negotiation
+ */
+ if (strcmp(conn->gssencmode, "require") == 0 &&
+ strcmp(conn->sslnegotiation, "postgres") != 0)
+ {
+ conn->status = CONNECTION_BAD;
+ libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when Direct SSL negotiation is enabled",
+ conn->gssencmode);
+ return false;
+ }
#endif
}
else
@@ -2793,11 +2840,12 @@ keep_going: /* We will come back to here until there is
/* initialize these values based on SSL mode */
conn->allow_ssl_try = (conn->sslmode[0] != 'd'); /* "disable" */
conn->wait_ssl_try = (conn->sslmode[0] == 'a'); /* "allow" */
+ /* direct ssl is incompatible with "allow" or "disabled" ssl */
+ conn->allow_direct_ssl_try = conn->allow_ssl_try && !conn->wait_ssl_try && (conn->sslnegotiation[0] != 'p');
#endif
#ifdef ENABLE_GSS
conn->try_gss = (conn->gssencmode[0] != 'd'); /* "disable" */
#endif
-
reset_connection_state_machine = false;
need_new_connection = true;
}
@@ -3255,6 +3303,29 @@ keep_going: /* We will come back to here until there is
if (pqsecure_initialize(conn, false, true) < 0)
goto error_return;
+ /*
+ * If SSL is enabled and direct SSL connections are enabled
+ * and we haven't already established an SSL connection (or
+ * already tried a direct connection and failed or succeeded)
+ * then try just enabling SSL directly.
+ *
+ * If we fail then we'll either fail the connection (if
+ * sslnegotiation is set to requiredirect or turn
+ * allow_direct_ssl_try to false
+ */
+ if (conn->allow_ssl_try
+ && !conn->wait_ssl_try
+ && conn->allow_direct_ssl_try
+ && !conn->ssl_in_use
+#ifdef ENABLE_GSS
+ && !conn->gssenc
+#endif
+ )
+ {
+ conn->status = CONNECTION_SSL_STARTUP;
+ return PGRES_POLLING_WRITING;
+ }
+
/*
* If SSL is enabled and we haven't already got encryption of
* some sort running, request SSL instead of sending the
@@ -3354,9 +3425,11 @@ keep_going: /* We will come back to here until there is
/*
* On first time through, get the postmaster's response to our
- * SSL negotiation packet.
+ * SSL negotiation packet. If we are trying a direct ssl
+ * connection skip reading the negotiation packet and go
+ * straight to initiating an ssl connection.
*/
- if (!conn->ssl_in_use)
+ if (!conn->ssl_in_use && !conn->allow_direct_ssl_try)
{
/*
* We use pqReadData here since it has the logic to
@@ -3462,6 +3535,21 @@ keep_going: /* We will come back to here until there is
}
if (pollres == PGRES_POLLING_FAILED)
{
+ /*
+ * Failed direct ssl connection, possibly try a new
+ * connection with postgres negotiation
+ */
+ if (conn->allow_direct_ssl_try)
+ {
+ /* if it's requiredirect then it's a hard failure */
+ if (conn->sslnegotiation[0] == 'r')
+ goto error_return;
+ /* otherwise only retry using postgres connection */
+ conn->allow_direct_ssl_try = false;
+ need_new_connection = true;
+ goto keep_going;
+ }
+
/*
* Failed ... if sslmode is "prefer" then do a non-SSL
* retry
@@ -4464,6 +4552,7 @@ freePGconn(PGconn *conn)
free(conn->keepalives_interval);
free(conn->keepalives_count);
free(conn->sslmode);
+ free(conn->sslnegotiation);
free(conn->sslcert);
free(conn->sslkey);
if (conn->sslpassword)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 09b485bd2bc..ef6ce9d0c0b 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -80,8 +80,9 @@ typedef enum
CONNECTION_CHECK_TARGET, /* Internal state: checking target server
* properties. */
CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
- CONNECTION_ALLOCATED /* Waiting for connection attempt to be
+ CONNECTION_ALLOCATED, /* Waiting for connection attempt to be
* started. */
+ CONNECTION_DIRECT_SSL_STARTUP /* Starting SSL without PG Negotiation. */
} ConnStatusType;
typedef enum
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 9c05f11a6e9..78a39dccfce 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -388,6 +388,8 @@ struct pg_conn
char *keepalives_count; /* maximum number of TCP keepalive
* retransmits */
char *sslmode; /* SSL mode (require,prefer,allow,disable) */
+ char *sslnegotiation; /* SSL initiation style
+ * (postgres,direct,requiredirect) */
char *sslcompression; /* SSL compression (0 or 1) */
char *sslkey; /* client key filename */
char *sslcert; /* client certificate filename */
@@ -553,6 +555,8 @@ struct pg_conn
bool ssl_cert_sent; /* Did we send one in reply? */
#ifdef USE_SSL
+ bool allow_direct_ssl_try; /* Try to make a direct SSL connection
+ * without an "SSL negotiation packet" */
bool allow_ssl_try; /* Allowed to try SSL negotiation */
bool wait_ssl_try; /* Delay SSL negotiation until after
* attempting normal connection */
--
2.39.2
v9-0004-Direct-SSL-connections-ALPN-support.patchtext/x-patch; charset=UTF-8; name=v9-0004-Direct-SSL-connections-ALPN-support.patchDownload
From f0bac7c748804693818746fd99b4c09e3e9e3317 Mon Sep 17 00:00:00 2001
From: Greg Stark <stark@mit.edu>
Date: Mon, 20 Mar 2023 14:09:30 -0400
Subject: [PATCH v9 4/6] Direct SSL connections ALPN support
Move code to check for alpn
- if non-direct SSL is used, and client sends an unexpected ALPN
protocol, that's now an error ?
Author: Greg Stark, Heikki Linnakangas
---
src/backend/libpq/be-secure-openssl.c | 84 +++++++++++++++++++
src/backend/libpq/be-secure.c | 3 +
src/backend/tcop/backend_startup.c | 13 +++
src/backend/utils/misc/guc_tables.c | 9 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/bin/psql/command.c | 7 +-
src/include/libpq/libpq-be.h | 1 +
src/include/libpq/libpq.h | 1 +
src/include/libpq/pqcomm.h | 19 +++++
src/interfaces/libpq/fe-connect.c | 5 ++
src/interfaces/libpq/fe-secure-openssl.c | 35 ++++++++
src/interfaces/libpq/libpq-int.h | 1 +
12 files changed, 177 insertions(+), 2 deletions(-)
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
index 72e43af3537..6f950f9c03c 100644
--- a/src/backend/libpq/be-secure-openssl.c
+++ b/src/backend/libpq/be-secure-openssl.c
@@ -67,6 +67,12 @@ static int ssl_external_passwd_cb(char *buf, int size, int rwflag, void *userdat
static int dummy_ssl_passwd_cb(char *buf, int size, int rwflag, void *userdata);
static int verify_cb(int ok, X509_STORE_CTX *ctx);
static void info_cb(const SSL *ssl, int type, int args);
+static int alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata);
static bool initialize_dh(SSL_CTX *context, bool isServerStart);
static bool initialize_ecdh(SSL_CTX *context, bool isServerStart);
static const char *SSLerrmessage(unsigned long ecode);
@@ -432,6 +438,16 @@ be_tls_open_server(Port *port)
/* set up debugging/info callback */
SSL_CTX_set_info_callback(SSL_context, info_cb);
+ if (ssl_enable_alpn)
+ {
+ elog(DEBUG2, "Enabling OpenSSL ALPN callback");
+ SSL_CTX_set_alpn_select_cb(SSL_context, alpn_cb, port);
+ }
+ else
+ {
+ elog(DEBUG2, "OpenSSL ALPN is disabled, not setting callback");
+ }
+
if (!(port->ssl = SSL_new(SSL_context)))
{
ereport(COMMERROR,
@@ -571,6 +587,32 @@ aloop:
return -1;
}
+ /* Get the protocol selected by ALPN */
+ port->alpn_used = false;
+ {
+ const unsigned char *selected;
+ unsigned int len;
+
+ SSL_get0_alpn_selected(port->ssl, &selected, &len);
+
+ /* If ALPN is used, check that we negotiated the expected protocol */
+ if (selected != NULL)
+ {
+ if (len == strlen(PG_ALPN_PROTOCOL) &&
+ memcmp(selected, PG_ALPN_PROTOCOL, strlen(PG_ALPN_PROTOCOL)) == 0)
+ {
+ port->alpn_used = true;
+ }
+ else
+ {
+ /* shouldn't happen */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received SSL connection request with unexpected ALPN protocol")));
+ }
+ }
+ }
+
/* Get client certificate, if available. */
port->peer = SSL_get_peer_certificate(port->ssl);
@@ -1259,6 +1301,48 @@ info_cb(const SSL *ssl, int type, int args)
}
}
+/* See pqcomm.h comments on OpenSSL implementation of ALPN (RFC 7301) */
+static const unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
+
+/*
+ * Server callback for ALPN negotiation. We use the standard "helper" function
+ * even though currently we only accept one value.
+ */
+static int
+alpn_cb(SSL *ssl,
+ const unsigned char **out,
+ unsigned char *outlen,
+ const unsigned char *in,
+ unsigned int inlen,
+ void *userdata)
+{
+ /*
+ * Why does OpenSSL provide a helper function that requires a nonconst
+ * vector when the callback is declared to take a const vector? What are
+ * we to do with that?
+ */
+ int retval;
+
+ Assert(userdata != NULL);
+ Assert(out != NULL);
+ Assert(outlen != NULL);
+ Assert(in != NULL);
+
+ retval = SSL_select_next_proto((unsigned char **) out, outlen,
+ alpn_protos, sizeof(alpn_protos),
+ in, inlen);
+ if (*out == NULL || *outlen > sizeof(alpn_protos) || outlen <= 0)
+ return SSL_TLSEXT_ERR_NOACK; /* can't happen */
+
+ if (retval == OPENSSL_NPN_NEGOTIATED)
+ return SSL_TLSEXT_ERR_OK;
+ else if (retval == OPENSSL_NPN_NO_OVERLAP)
+ return SSL_TLSEXT_ERR_NOACK;
+ else
+ return SSL_TLSEXT_ERR_NOACK;
+}
+
+
/*
* Set DH parameters for generating ephemeral DH keys. The
* DH parameters can take a long time to compute, so they must be
diff --git a/src/backend/libpq/be-secure.c b/src/backend/libpq/be-secure.c
index 1d1329d1d95..20a1a4ad551 100644
--- a/src/backend/libpq/be-secure.c
+++ b/src/backend/libpq/be-secure.c
@@ -58,6 +58,9 @@ bool SSLPreferServerCiphers;
int ssl_min_protocol_version = PG_TLS1_2_VERSION;
int ssl_max_protocol_version = PG_TLS_ANY;
+/* GUC variable: if false ignore ALPN negotiation */
+bool ssl_enable_alpn = true;
+
/* ------------------------------------------------------------ */
/* Procedures common to all secure sessions */
/* ------------------------------------------------------------ */
diff --git a/src/backend/tcop/backend_startup.c b/src/backend/tcop/backend_startup.c
index d2ce8e47a8e..5801d7576ba 100644
--- a/src/backend/tcop/backend_startup.c
+++ b/src/backend/tcop/backend_startup.c
@@ -382,6 +382,11 @@ ProcessSSLStartup(Port *port)
{
elog(LOG, "Detected direct SSL handshake");
+ if (!ssl_enable_alpn)
+ {
+ elog(WARNING, "Received direct SSL connection without ssl_enable_alpn enabled");
+ }
+
#ifdef USE_SSL
if (!LoadedSSL || port->laddr.addr.ss_family == AF_UNIX)
{
@@ -396,6 +401,14 @@ ProcessSSLStartup(Port *port)
*/
return STATUS_ERROR;
}
+
+ if (!port->alpn_used)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("Received direct SSL connection request without required ALPN protocol negotiation extension")));
+ return STATUS_ERROR;
+ }
#else
/* SSL not supported by this build */
return STATUS_ERROR;
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index abd9029451f..ad917303770 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1085,6 +1085,15 @@ struct config_bool ConfigureNamesBool[] =
true,
NULL, NULL, NULL
},
+ {
+ {"ssl_enable_alpn", PGC_SIGHUP, CONN_AUTH_SSL,
+ gettext_noop("Respond to TLS ALPN Extension Requests."),
+ NULL,
+ },
+ &ssl_enable_alpn,
+ true,
+ NULL, NULL, NULL
+ },
{
{"fsync", PGC_SIGHUP, WAL_SETTINGS,
gettext_noop("Forces synchronization of updates to disk."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 2244ee52f79..34f0c3b56cf 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -127,6 +127,7 @@
#ssl_dh_params_file = ''
#ssl_passphrase_command = ''
#ssl_passphrase_command_supports_reload = off
+#ssl_enable_alpn = on
#------------------------------------------------------------------------------
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 9b0fa041f73..5607dea8049 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -3814,6 +3814,7 @@ printSSLInfo(void)
const char *protocol;
const char *cipher;
const char *compression;
+ const char *alpn;
if (!PQsslInUse(pset.db))
return; /* no SSL */
@@ -3821,11 +3822,13 @@ printSSLInfo(void)
protocol = PQsslAttribute(pset.db, "protocol");
cipher = PQsslAttribute(pset.db, "cipher");
compression = PQsslAttribute(pset.db, "compression");
+ alpn = PQsslAttribute(pset.db, "alpn");
- printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s)\n"),
+ printf(_("SSL connection (protocol: %s, cipher: %s, compression: %s, ALPN: %s)\n"),
protocol ? protocol : _("unknown"),
cipher ? cipher : _("unknown"),
- (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"));
+ (compression && strcmp(compression, "off") != 0) ? _("on") : _("off"),
+ alpn ? alpn : _("none"));
}
/*
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 86f773999ed..a2414c401bb 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -203,6 +203,7 @@ typedef struct Port
char *peer_cn;
char *peer_dn;
bool peer_cert_valid;
+ bool alpn_used;
/*
* OpenSSL structures. (Keep these last so that the locations of other
diff --git a/src/include/libpq/libpq.h b/src/include/libpq/libpq.h
index a9c89e8179b..91a61e9eacb 100644
--- a/src/include/libpq/libpq.h
+++ b/src/include/libpq/libpq.h
@@ -122,6 +122,7 @@ extern PGDLLIMPORT char *SSLECDHCurve;
extern PGDLLIMPORT bool SSLPreferServerCiphers;
extern PGDLLIMPORT int ssl_min_protocol_version;
extern PGDLLIMPORT int ssl_max_protocol_version;
+extern PGDLLIMPORT bool ssl_enable_alpn;
enum ssl_protocol_versions
{
diff --git a/src/include/libpq/pqcomm.h b/src/include/libpq/pqcomm.h
index 9ae469c86c4..fb93c820530 100644
--- a/src/include/libpq/pqcomm.h
+++ b/src/include/libpq/pqcomm.h
@@ -139,6 +139,25 @@ typedef struct CancelRequestPacket
uint32 cancelAuthCode; /* secret key to authorize cancel */
} CancelRequestPacket;
+/* Application-Layer Protocol Negotiation is required for direct connections
+ * to avoid protocol confusion attacks (e.g https://alpaca-attack.com/).
+ *
+ * ALPN is specified in RFC 7301
+ *
+ * This string should be registered at:
+ * https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids
+ *
+ * OpenSSL uses this wire-format for the list of alpn protocols even in the
+ * API. Both server and client take the same format parameter but the client
+ * actually sends it to the server as-is and the server it specifies the
+ * preference order to use to choose the one selected to send back.
+ *
+ * c.f. https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_alpn_select_cb.html
+ *
+ * The #define can be used to initialize a char[] vector to use directly in the API
+ */
+#define PG_ALPN_PROTOCOL "TBD-pgsql"
+#define PG_ALPN_PROTOCOL_VECTOR { 9, 'T','B','D','-','p','g','s','q','l' }
/*
* A client can also start by sending a SSL or GSSAPI negotiation request to
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index a559c80e258..620a2abbfa3 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -313,6 +313,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"SSL-SNI", "", 1,
offsetof(struct pg_conn, sslsni)},
+ {"sslalpn", "PGSSLALPN", "1", NULL,
+ "SSL-ALPN", "", 1,
+ offsetof(struct pg_conn, sslalpn)},
+
{"requirepeer", "PGREQUIREPEER", NULL, NULL,
"Require-Peer", "", 10,
offsetof(struct pg_conn, requirepeer)},
@@ -4566,6 +4570,7 @@ freePGconn(PGconn *conn)
free(conn->sslcrldir);
free(conn->sslcompression);
free(conn->sslsni);
+ free(conn->sslalpn);
free(conn->requirepeer);
free(conn->require_auth);
free(conn->ssl_min_protocol_version);
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index bf45a8edc31..9aad2ac605c 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -885,6 +885,9 @@ destroy_ssl_system(void)
#endif
}
+/* See pqcomm.h comments on OpenSSL implementation of ALPN (RFC 7301) */
+static unsigned char alpn_protos[] = PG_ALPN_PROTOCOL_VECTOR;
+
/*
* Create per-connection SSL object, and load the client certificate,
* private key, and trusted CA certs.
@@ -1234,6 +1237,22 @@ initialize_SSL(PGconn *conn)
}
}
+ if (conn->sslalpn && conn->sslalpn[0] == '1')
+ {
+ int retval;
+
+ retval = SSL_set_alpn_protos(conn->ssl, alpn_protos, sizeof(alpn_protos));
+
+ if (retval != 0)
+ {
+ char *err = SSLerrmessage(ERR_get_error());
+
+ libpq_append_conn_error(conn, "could not set ssl alpn extension: %s", err);
+ SSLerrfree(err);
+ return -1;
+ }
+ }
+
/*
* Read the SSL key. If a key is specified, treat it as an engine:key
* combination if there is colon present - we don't support files with
@@ -1753,6 +1772,7 @@ PQsslAttributeNames(PGconn *conn)
"cipher",
"compression",
"protocol",
+ "alpn",
NULL
};
static const char *const empty_attrs[] = {NULL};
@@ -1807,6 +1827,21 @@ PQsslAttribute(PGconn *conn, const char *attribute_name)
if (strcmp(attribute_name, "protocol") == 0)
return SSL_get_version(conn->ssl);
+ if (strcmp(attribute_name, "alpn") == 0)
+ {
+ const unsigned char *data;
+ unsigned int len;
+ static char alpn_str[256]; /* alpn doesn't support longer than 255
+ * bytes */
+
+ SSL_get0_alpn_selected(conn->ssl, &data, &len);
+ if (data == NULL || len == 0 || len > sizeof(alpn_str) - 1)
+ return NULL;
+ memcpy(alpn_str, data, len);
+ alpn_str[len] = 0;
+ return alpn_str;
+ }
+
return NULL; /* unknown attribute */
}
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 78a39dccfce..def49a68af3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -399,6 +399,7 @@ struct pg_conn
char *sslcrl; /* certificate revocation list filename */
char *sslcrldir; /* certificate revocation list directory name */
char *sslsni; /* use SSL SNI extension (0 or 1) */
+ char *sslalpn; /* use SSL ALPN extension (0 or 1) */
char *requirepeer; /* required peer credentials for local sockets */
char *gssencmode; /* GSS mode (require,prefer,disable) */
char *krbsrvname; /* Kerberos service name */
--
2.39.2
v9-0005-Add-tests-for-sslnegotiation.patchtext/x-patch; charset=UTF-8; name=v9-0005-Add-tests-for-sslnegotiation.patchDownload
From 5d744b1f29502a1e47064b1a929b96e909191f89 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Wed, 10 Jan 2024 10:25:08 +0200
Subject: [PATCH v9 5/6] Add tests for sslnegotiation
Author: Heikki Linnakangas, Matthias van de Meent
---
.../t/001_negotiate_encryption.pl | 25 +++++++++++++------
1 file changed, 18 insertions(+), 7 deletions(-)
diff --git a/src/test/libpq_encryption/t/001_negotiate_encryption.pl b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
index b8646c5bc97..b8b4cd2726d 100644
--- a/src/test/libpq_encryption/t/001_negotiate_encryption.pl
+++ b/src/test/libpq_encryption/t/001_negotiate_encryption.pl
@@ -231,13 +231,13 @@ sub resolve_connection_type
# First test with SSL disabled in the server
-# Test the cube of parameters: user, sslmode, and gssencmode
+# Test the cube of parameters: user, sslmode, sslnegotiation and gssencmode
sub test_modes
{
local $Test::Builder::Level = $Test::Builder::Level + 1;
my ($pg_node, $node_conf,
- $test_users, $ssl_modes, $gss_modes) = @_;
+ $test_users, $ssl_modes, $ssl_negotiations, $gss_modes) = @_;
foreach my $test_user (@{$test_users})
{
@@ -253,13 +253,18 @@ sub test_modes
gssmode=>$gssencmode,
);
my $res = resolve_connection_type(\%params);
- connect_test($pg_node, "user=$test_user sslmode=$client_mode gssencmode=$gssencmode", $res);
+ # Negotiation type doesn't matter for supported connection types
+ foreach my $negotiation (@{$ssl_negotiations})
+ {
+ connect_test($pg_node, "user=$test_user sslmode=$client_mode sslnegotiation=$negotiation gssencmode=$gssencmode", $res);
+ }
}
}
}
}
my $sslmodes = ['disable', 'allow', 'prefer', 'require'];
+my $sslnegotiations = ['postgres', 'direct', 'requiredirect'];
my $gssencmodes = ['disable', 'prefer', 'require'];
my $server_config = {
@@ -270,7 +275,7 @@ my $server_config = {
note("Running tests with SSL and GSS disabled in server");
test_modes($node, $server_config,
['testuser'],
- $sslmodes, $gssencmodes);
+ $sslmodes, $sslnegotiations, $gssencmodes);
# Enable SSL in the server
SKIP:
@@ -284,7 +289,7 @@ SKIP:
note("Running tests with SSL enabled in server");
test_modes($node, $server_config,
['testuser', 'ssluser', 'nossluser'],
- $sslmodes, ['disable']);
+ $sslmodes, $sslnegotiations, ['disable']);
$node->adjust_conf('postgresql.conf', 'ssl', 'off');
$node->reload;
@@ -306,7 +311,7 @@ SKIP:
note("Running tests with GSS enabled in server");
test_modes($node, $server_config,
['testuser', 'gssuser', 'nogssuser'],
- $sslmodes, $gssencmodes);
+ $sslmodes, $sslnegotiations, $gssencmodes);
# Check that logs match the expected 'no pg_hba.conf entry' line, too, as
# that is not tested by test_modes.
@@ -320,6 +325,10 @@ SKIP:
# with no encryption.
connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer', 'plain',
'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer sslnegotiation=direct', 'plain',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
+ connect_test($node, 'user=nogssuser sslmode=prefer gssencmode=prefer sslnegotiation=requiredirect', 'plain',
+ 'no pg_hba.conf entry for host "127.0.0.1", user "nogssuser", database "postgres", GSS encryption');
}
# Server supports both SSL and GSSAPI
@@ -329,6 +338,8 @@ SKIP:
# SSL is still disabled
connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer sslnegotiation=direct', 'gss');
+ connect_test($node, 'user=testuser sslmode=prefer gssencmode=prefer sslnegotiation=requiredirect', 'gss');
# Enable SSL
$node->adjust_conf('postgresql.conf', 'ssl', 'on');
@@ -338,7 +349,7 @@ SKIP:
note("Running tests with both GSS and SSL enabled in server");
test_modes($node, $server_config,
['testuser', 'gssuser', 'ssluser', 'nogssuser', 'nossluser'],
- $sslmodes, $gssencmodes);
+ $sslmodes, $sslnegotiations, $gssencmodes);
# Test case that server supports GSSAPI, but it's not allowed for
# this user. Special cased because we check output
--
2.39.2
v9-0006-WIP-refactor-state-machine-in-libpq.patchtext/x-patch; charset=UTF-8; name=v9-0006-WIP-refactor-state-machine-in-libpq.patchDownload
From 8254d98b20bad0eb87769da825f02a9d6b126a58 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date: Tue, 5 Mar 2024 16:03:52 +0200
Subject: [PATCH v9 6/6] WIP: refactor state machine in libpq
---
src/interfaces/libpq/fe-connect.c | 506 +++++++++++++----------
src/interfaces/libpq/fe-secure-openssl.c | 12 +-
src/interfaces/libpq/libpq-fe.h | 3 +-
src/interfaces/libpq/libpq-int.h | 18 +-
4 files changed, 315 insertions(+), 224 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 620a2abbfa3..edc324dad05 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -396,6 +396,12 @@ static const char uri_designator[] = "postgresql://";
static const char short_uri_designator[] = "postgres://";
static bool connectOptions1(PGconn *conn, const char *conninfo);
+static bool init_allowed_encryption_methods(PGconn *conn);
+#if defined(USE_SSL) || defined(USE_GSS)
+static int encryption_negotiation_failed(PGconn *conn);
+#endif
+static bool connection_failed(PGconn *conn);
+static bool select_next_encryption_method(PGconn *conn, bool negotiation_failure);
static PGPing internal_ping(PGconn *conn);
static void pqFreeCommandQueue(PGcmdQueueEntry *queue);
static bool fillPGconn(PGconn *conn, PQconninfoOption *connOptions);
@@ -1727,21 +1733,6 @@ pqConnectOptions2(PGconn *conn)
conn->gssencmode);
return false;
}
-#endif
-#ifdef USE_SSL
-
- /*
- * GSS is incompatible with direct SSL connections so it requires the
- * default postgres style connection ssl negotiation
- */
- if (strcmp(conn->gssencmode, "require") == 0 &&
- strcmp(conn->sslnegotiation, "postgres") != 0)
- {
- conn->status = CONNECTION_BAD;
- libpq_append_conn_error(conn, "gssencmode value \"%s\" invalid when Direct SSL negotiation is enabled",
- conn->gssencmode);
- return false;
- }
#endif
}
else
@@ -2840,16 +2831,9 @@ keep_going: /* We will come back to here until there is
*/
conn->pversion = PG_PROTOCOL(3, 0);
conn->send_appname = true;
-#ifdef USE_SSL
- /* initialize these values based on SSL mode */
- conn->allow_ssl_try = (conn->sslmode[0] != 'd'); /* "disable" */
- conn->wait_ssl_try = (conn->sslmode[0] == 'a'); /* "allow" */
- /* direct ssl is incompatible with "allow" or "disabled" ssl */
- conn->allow_direct_ssl_try = conn->allow_ssl_try && !conn->wait_ssl_try && (conn->sslnegotiation[0] != 'p');
-#endif
-#ifdef ENABLE_GSS
- conn->try_gss = (conn->gssencmode[0] != 'd'); /* "disable" */
-#endif
+ conn->failed_enc_methods = 0;
+ conn->current_enc_method = 0;
+ conn->allowed_enc_methods = 0;
reset_connection_state_machine = false;
need_new_connection = true;
}
@@ -2875,6 +2859,34 @@ keep_going: /* We will come back to here until there is
need_new_connection = false;
}
+ /* Decide what to do next, if SSL or GSS negotiation fails */
+#define ENCRYPTION_NEGOTIATION_FAILED() \
+ do { \
+ switch (encryption_negotiation_failed(conn)) \
+ { \
+ case 0: \
+ goto error_return; \
+ case 1: \
+ conn->status = CONNECTION_MADE; \
+ return PGRES_POLLING_WRITING; \
+ case 2: \
+ need_new_connection = true; \
+ goto keep_going; \
+ } \
+ } while(0);
+
+ /* Decide what to do next, if connection fails */
+#define CONNECTION_FAILED() \
+ do { \
+ if (connection_failed(conn)) \
+ { \
+ need_new_connection = true; \
+ goto keep_going; \
+ } \
+ else \
+ goto error_return; \
+ } while(0);
+
/* Now try to advance the state machine for this connection */
switch (conn->status)
{
@@ -3189,18 +3201,6 @@ keep_going: /* We will come back to here until there is
goto error_return;
}
- /*
- * Make sure we can write before advancing to next step.
- */
- conn->status = CONNECTION_MADE;
- return PGRES_POLLING_WRITING;
- }
-
- case CONNECTION_MADE:
- {
- char *startpacket;
- int packetlen;
-
/*
* Implement requirepeer check, if requested and it's a
* Unix-domain socket.
@@ -3249,30 +3249,31 @@ keep_going: /* We will come back to here until there is
#endif /* WIN32 */
}
- if (conn->raddr.addr.ss_family == AF_UNIX)
- {
- /* Don't request SSL or GSSAPI over Unix sockets */
-#ifdef USE_SSL
- conn->allow_ssl_try = false;
-#endif
-#ifdef ENABLE_GSS
- conn->try_gss = false;
-#endif
- }
+ /* Choose encryption method to try first */
+ if (!init_allowed_encryption_methods(conn))
+ goto error_return;
+
+ /*
+ * Make sure we can write before advancing to next step.
+ */
+ conn->status = CONNECTION_MADE;
+ return PGRES_POLLING_WRITING;
+ }
+
+ case CONNECTION_MADE:
+ {
+ char *startpacket;
+ int packetlen;
#ifdef ENABLE_GSS
/*
- * If GSSAPI encryption is enabled, then call
- * pg_GSS_have_cred_cache() which will return true if we can
- * acquire credentials (and give us a handle to use in
- * conn->gcred), and then send a packet to the server asking
- * for GSSAPI Encryption (and skip past SSL negotiation and
- * regular startup below).
+ * If GSSAPI encryption is enabled, send a packet to the
+ * server asking for GSSAPI Encryption and proceed with GSSAPI
+ * handshake. We will come back here after GSSAPI encryption
+ * has been established, with conn->gctx set.
*/
- if (conn->try_gss && !conn->gctx)
- conn->try_gss = pg_GSS_have_cred_cache(&conn->gcred);
- if (conn->try_gss && !conn->gctx)
+ if (conn->current_enc_method == ENC_GSSAPI && !conn->gctx)
{
ProtocolVersion pv = pg_hton32(NEGOTIATE_GSS_CODE);
@@ -3287,12 +3288,6 @@ keep_going: /* We will come back to here until there is
conn->status = CONNECTION_GSS_STARTUP;
return PGRES_POLLING_READING;
}
- else if (!conn->gctx && conn->gssencmode[0] == 'r')
- {
- libpq_append_conn_error(conn,
- "GSSAPI encryption required but was impossible (possibly no credential cache, no server support, or using a local socket)");
- goto error_return;
- }
#endif
#ifdef USE_SSL
@@ -3308,39 +3303,22 @@ keep_going: /* We will come back to here until there is
goto error_return;
/*
- * If SSL is enabled and direct SSL connections are enabled
- * and we haven't already established an SSL connection (or
- * already tried a direct connection and failed or succeeded)
- * then try just enabling SSL directly.
- *
- * If we fail then we'll either fail the connection (if
- * sslnegotiation is set to requiredirect or turn
- * allow_direct_ssl_try to false
+ * If direct SSL is enabled, jump right into SSL handshake. We
+ * will come back here after SSL encryption has been
+ * established, with ssl_in_use set.
*/
- if (conn->allow_ssl_try
- && !conn->wait_ssl_try
- && conn->allow_direct_ssl_try
- && !conn->ssl_in_use
-#ifdef ENABLE_GSS
- && !conn->gssenc
-#endif
- )
+ if (conn->current_enc_method == ENC_DIRECT_SSL && !conn->ssl_in_use)
{
conn->status = CONNECTION_SSL_STARTUP;
return PGRES_POLLING_WRITING;
}
/*
- * If SSL is enabled and we haven't already got encryption of
- * some sort running, request SSL instead of sending the
- * startup message.
+ * If negotiated SSL is enabled, request SSL and proceed with
+ * SSL handshake. We will come back here after SSL encryption
+ * has been established, with ssl_in_use set.
*/
- if (conn->allow_ssl_try && !conn->wait_ssl_try &&
- !conn->ssl_in_use
-#ifdef ENABLE_GSS
- && !conn->gssenc
-#endif
- )
+ if (conn->current_enc_method == ENC_NEGOTIATED_SSL && !conn->ssl_in_use)
{
ProtocolVersion pv;
@@ -3388,8 +3366,11 @@ keep_going: /* We will come back to here until there is
}
/*
- * Build the startup packet.
+ * We have now established encryption, or we are happy to
+ * proceed without.
*/
+
+ /* Build the startup packet. */
startpacket = pqBuildStartupPacket3(conn, &packetlen,
EnvironmentOptions);
if (!startpacket)
@@ -3430,10 +3411,9 @@ keep_going: /* We will come back to here until there is
/*
* On first time through, get the postmaster's response to our
* SSL negotiation packet. If we are trying a direct ssl
- * connection skip reading the negotiation packet and go
- * straight to initiating an ssl connection.
+ * connection, go straight to initiating ssl.
*/
- if (!conn->ssl_in_use && !conn->allow_direct_ssl_try)
+ if (!conn->ssl_in_use && conn->current_enc_method == ENC_NEGOTIATED_SSL)
{
/*
* We use pqReadData here since it has the logic to
@@ -3463,34 +3443,14 @@ keep_going: /* We will come back to here until there is
{
/* mark byte consumed */
conn->inStart = conn->inCursor;
-
- /*
- * Set up global SSL state if required. The crypto
- * state has already been set if libpq took care of
- * doing that, so there is no need to make that happen
- * again.
- */
- if (pqsecure_initialize(conn, true, false) != 0)
- goto error_return;
}
else if (SSLok == 'N')
{
/* mark byte consumed */
conn->inStart = conn->inCursor;
/* OK to do without SSL? */
- if (conn->sslmode[0] == 'r' || /* "require" */
- conn->sslmode[0] == 'v') /* "verify-ca" or
- * "verify-full" */
- {
- /* Require SSL, but server does not want it */
- libpq_append_conn_error(conn, "server does not support SSL, but SSL was required");
- goto error_return;
- }
- /* Otherwise, proceed with normal startup */
- conn->allow_ssl_try = false;
/* We can proceed using this connection */
- conn->status = CONNECTION_MADE;
- return PGRES_POLLING_WRITING;
+ ENCRYPTION_NEGOTIATION_FAILED();
}
else if (SSLok == 'E')
{
@@ -3515,6 +3475,14 @@ keep_going: /* We will come back to here until there is
}
}
+ /*
+ * Set up global SSL state if required. The crypto state has
+ * already been set if libpq took care of doing that, so there
+ * is no need to make that happen again.
+ */
+ if (pqsecure_initialize(conn, true, false) != 0)
+ goto error_return;
+
/*
* Begin or continue the SSL negotiation process.
*/
@@ -3543,32 +3511,7 @@ keep_going: /* We will come back to here until there is
* Failed direct ssl connection, possibly try a new
* connection with postgres negotiation
*/
- if (conn->allow_direct_ssl_try)
- {
- /* if it's requiredirect then it's a hard failure */
- if (conn->sslnegotiation[0] == 'r')
- goto error_return;
- /* otherwise only retry using postgres connection */
- conn->allow_direct_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
-
- /*
- * Failed ... if sslmode is "prefer" then do a non-SSL
- * retry
- */
- if (conn->sslmode[0] == 'p' /* "prefer" */
- && conn->allow_ssl_try /* redundant? */
- && !conn->wait_ssl_try) /* redundant? */
- {
- /* only retry once */
- conn->allow_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
- /* Else it's a hard failure */
- goto error_return;
+ CONNECTION_FAILED();
}
/* Else, return POLLING_READING or POLLING_WRITING status */
return pollres;
@@ -3587,7 +3530,7 @@ keep_going: /* We will come back to here until there is
* If we haven't yet, get the postmaster's response to our
* negotiation packet
*/
- if (conn->try_gss && !conn->gctx)
+ if (!conn->gctx)
{
char gss_ok;
int rdresult = pqReadData(conn);
@@ -3611,9 +3554,7 @@ keep_going: /* We will come back to here until there is
* error message on retry). Server gets fussy if we
* don't hang up the socket, though.
*/
- conn->try_gss = false;
- need_new_connection = true;
- goto keep_going;
+ CONNECTION_FAILED();
}
/* mark byte consumed */
@@ -3621,17 +3562,8 @@ keep_going: /* We will come back to here until there is
if (gss_ok == 'N')
{
- /* Server doesn't want GSSAPI; fall back if we can */
- if (conn->gssencmode[0] == 'r')
- {
- libpq_append_conn_error(conn, "server doesn't support GSSAPI encryption, but it was required");
- goto error_return;
- }
-
- conn->try_gss = false;
/* We can proceed using this connection */
- conn->status = CONNECTION_MADE;
- return PGRES_POLLING_WRITING;
+ ENCRYPTION_NEGOTIATION_FAILED();
}
else if (gss_ok != 'G')
{
@@ -3663,18 +3595,7 @@ keep_going: /* We will come back to here until there is
}
else if (pollres == PGRES_POLLING_FAILED)
{
- if (conn->gssencmode[0] == 'p')
- {
- /*
- * We failed, but we can retry on "prefer". Have to
- * drop the current connection to do so, though.
- */
- conn->try_gss = false;
- need_new_connection = true;
- goto keep_going;
- }
- /* Else it's a hard failure */
- goto error_return;
+ CONNECTION_FAILED();
}
/* Else, return POLLING_READING or POLLING_WRITING status */
return pollres;
@@ -3850,55 +3771,7 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
-#ifdef ENABLE_GSS
-
- /*
- * If gssencmode is "prefer" and we're using GSSAPI, retry
- * without it.
- */
- if (conn->gssenc && conn->gssencmode[0] == 'p')
- {
- /* only retry once */
- conn->try_gss = false;
- need_new_connection = true;
- goto keep_going;
- }
-#endif
-
-#ifdef USE_SSL
-
- /*
- * if sslmode is "allow" and we haven't tried an SSL
- * connection already, then retry with an SSL connection
- */
- if (conn->sslmode[0] == 'a' /* "allow" */
- && !conn->ssl_in_use
- && conn->allow_ssl_try
- && conn->wait_ssl_try)
- {
- /* only retry once */
- conn->wait_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
-
- /*
- * if sslmode is "prefer" and we're in an SSL connection,
- * then do a non-SSL retry
- */
- if (conn->sslmode[0] == 'p' /* "prefer" */
- && conn->ssl_in_use
- && conn->allow_ssl_try /* redundant? */
- && !conn->wait_ssl_try) /* redundant? */
- {
- /* only retry once */
- conn->allow_ssl_try = false;
- need_new_connection = true;
- goto keep_going;
- }
-#endif
-
- goto error_return;
+ CONNECTION_FAILED();
}
else if (beresp == PqMsg_NegotiateProtocolVersion)
{
@@ -4344,6 +4217,213 @@ error_return:
return PGRES_POLLING_FAILED;
}
+static bool
+init_allowed_encryption_methods(PGconn *conn)
+{
+ if (conn->raddr.addr.ss_family == AF_UNIX)
+ {
+ /* Don't request SSL or GSSAPI over Unix sockets */
+ conn->allowed_enc_methods &= ~(ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL | ENC_GSSAPI);
+
+ /* to give a better error message */
+
+ /*
+ * XXX: we probably should not do this. sslmode=require works
+ * differently
+ */
+ if (conn->gssencmode[0] == 'r')
+ {
+ libpq_append_conn_error(conn,
+ "GSSAPI encryption required but it is not supported over a local socket)");
+ conn->allowed_enc_methods = 0;
+ conn->current_enc_method = ENC_ERROR;
+ return false;
+ }
+
+ conn->allowed_enc_methods = ENC_PLAINTEXT;
+ conn->current_enc_method = ENC_PLAINTEXT;
+ return true;
+ }
+
+ /* initialize these values based on sslmode and gssencmode */
+ conn->allowed_enc_methods = 0;
+
+#ifdef USE_SSL
+ /* sslmode anything but 'disable', and GSSAPI not required */
+ if (conn->sslmode[0] != 'd' && conn->gssencmode[0] != 'r')
+ {
+ if (conn->sslnegotiation[0] == 'p')
+ conn->allowed_enc_methods |= ENC_NEGOTIATED_SSL;
+ else if (conn->sslnegotiation[0] == 'd')
+ conn->allowed_enc_methods |= ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL;
+ else if (conn->sslnegotiation[0] == 'r')
+ conn->allowed_enc_methods |= ENC_DIRECT_SSL;
+ }
+#endif
+
+#ifdef ENABLE_GSS
+ if (conn->gssencmode[0] != 'd')
+ conn->allowed_enc_methods |= ENC_GSSAPI;
+#endif
+
+ if ((conn->sslmode[0] == 'd' || conn->sslmode[0] == 'p' || conn->sslmode[0] == 'a') &&
+ (conn->gssencmode[0] == 'd' || conn->gssencmode[0] == 'p'))
+ {
+ conn->allowed_enc_methods |= ENC_PLAINTEXT;
+ }
+
+ return select_next_encryption_method(conn, false);
+}
+
+/*
+ * Out-of-line portion of the ENCRYPTION_NEGOTIATION_FAILED() macro in the
+ * PQconnectPoll state machine.
+ *
+ * Return value:
+ * 0: connection failed and we are out of encryption methods to try. return an error
+ * 1: Retry with next connection method. The TCP connection is still valid and in
+ * known state, so we can proceed with the negotiating next method without
+ * reconnecting.
+ * 2: Disconnect, and retry with next connection method.
+ *
+ * conn->current_enc_method is updated to the next method to try.
+ */
+#if defined(USE_SSL) || defined(USE_GSS)
+static int
+encryption_negotiation_failed(PGconn *conn)
+{
+ Assert((conn->failed_enc_methods & conn->current_enc_method) == 0);
+ conn->failed_enc_methods |= conn->current_enc_method;
+
+ if (select_next_encryption_method(conn, true))
+ {
+ if (conn->current_enc_method == ENC_DIRECT_SSL)
+ return 2;
+ else
+ return 1;
+ }
+ else
+ return 0;
+}
+#endif
+
+/*
+ * Out-of-line portion of the CONNECTION_FAILED() macro
+ *
+ * Returns true, if we should retry the connection with different encryption method.
+ * conn->current_enc_method is updated to the next method to try.
+ */
+static bool
+connection_failed(PGconn *conn)
+{
+ Assert((conn->failed_enc_methods & conn->current_enc_method) == 0);
+ conn->failed_enc_methods |= conn->current_enc_method;
+
+ /*
+ * If the server reported an error after the SSL handshake, no point in
+ * retrying with negotiated vs direct SSL.
+ */
+ if ((conn->current_enc_method & (ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL)) != 0 &&
+ conn->ssl_handshake_started)
+ {
+ conn->failed_enc_methods |= (ENC_DIRECT_SSL | ENC_NEGOTIATED_SSL) & conn->allowed_enc_methods;
+ }
+ else
+ conn->failed_enc_methods |= conn->current_enc_method;
+
+ return select_next_encryption_method(conn, false);
+}
+
+/*
+ * Choose the next encryption method to try. If this is a retry,
+ * conn->failed_enc_methods has already been updated. conn->current_enc_method
+ * is updated to the next method to try.
+ */
+static bool
+select_next_encryption_method(PGconn *conn, bool have_valid_connection)
+{
+ int remaining_methods;
+
+ remaining_methods = conn->allowed_enc_methods & ~conn->failed_enc_methods;
+
+ /*
+ * Try GSSAPI before SSL
+ */
+#ifdef ENABLE_GSS
+ if ((remaining_methods & ENC_GSSAPI) != 0)
+ {
+ /*
+ * If GSSAPI encryption is enabled, then call pg_GSS_have_cred_cache()
+ * which will return true if we can acquire credentials (and give us a
+ * handle to use in conn->gcred), and then send a packet to the server
+ * asking for GSSAPI Encryption (and skip past SSL negotiation and
+ * regular startup below).
+ */
+ if (!conn->gctx)
+ {
+ if (!pg_GSS_have_cred_cache(&conn->gcred))
+ {
+ conn->allowed_enc_methods &= ~ENC_GSSAPI;
+ remaining_methods &= ~ENC_GSSAPI;
+
+ if (conn->gssencmode[0] == 'r')
+ {
+ libpq_append_conn_error(conn,
+ "GSSAPI encryption required but no credential cache");
+ }
+ }
+ }
+ if ((remaining_methods & ENC_GSSAPI) != 0)
+ {
+ conn->current_enc_method = ENC_GSSAPI;
+ return true;
+ }
+ }
+#endif
+
+ /* With sslmode=allow, try plaintext connection before SSL. */
+ if (conn->sslmode[0] == 'a' && (remaining_methods & ENC_PLAINTEXT) != 0)
+ {
+ conn->current_enc_method = ENC_PLAINTEXT;
+ return true;
+ }
+
+ /*
+ * Try SSL. If enabled, try direct SSL. Unless we have a valid TCP
+ * connection that failed negotiating GSSAPI encryption; in that case we
+ * prefer to reuse the connection with negotiated SSL, instead of
+ * reconnecting to do direct SSL. The point of direct SSL is to avoid the
+ * roundtrip from the negotiation, but reconnecting would also incur a
+ * roundtrip.
+ */
+ if (have_valid_connection && (remaining_methods & ENC_NEGOTIATED_SSL) != 0)
+ {
+ conn->current_enc_method = ENC_NEGOTIATED_SSL;
+ return true;
+ }
+
+ if ((remaining_methods & ENC_DIRECT_SSL) != 0)
+ {
+ conn->current_enc_method = ENC_DIRECT_SSL;
+ return true;
+ }
+
+ if ((remaining_methods & ENC_NEGOTIATED_SSL) != 0)
+ {
+ conn->current_enc_method = ENC_NEGOTIATED_SSL;
+ return true;
+ }
+
+ if (conn->sslmode[0] != 'a' && (remaining_methods & ENC_PLAINTEXT) != 0)
+ {
+ conn->current_enc_method = ENC_PLAINTEXT;
+ return true;
+ }
+
+ /* No more options */
+ conn->current_enc_method = ENC_ERROR;
+ return false;
+}
/*
* internal_ping
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index 9aad2ac605c..336325b020d 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -1484,6 +1484,7 @@ open_client_SSL(PGconn *conn)
SOCK_ERRNO_SET(0);
ERR_clear_error();
r = SSL_connect(conn->ssl);
+
if (r <= 0)
{
int save_errno = SOCK_ERRNO;
@@ -1587,7 +1588,7 @@ open_client_SSL(PGconn *conn)
/*
* We already checked the server certificate in initialize_SSL() using
- * SSL_CTX_set_verify(), if root.crt exists.
+ * SSL_set_verify(), if root.crt exists.
*/
/* get server certificate */
@@ -1631,6 +1632,7 @@ pgtls_close(PGconn *conn)
SSL_free(conn->ssl);
conn->ssl = NULL;
conn->ssl_in_use = false;
+ conn->ssl_handshake_started = false;
destroy_needed = true;
}
@@ -1654,7 +1656,7 @@ pgtls_close(PGconn *conn)
{
/*
* In the non-SSL case, just remove the crypto callbacks if the
- * connection has then loaded. This code path has no dependency on
+ * connection has loaded them. This code path has no dependency on
* any pending SSL calls.
*/
if (conn->crypto_loaded)
@@ -1860,9 +1862,10 @@ static BIO_METHOD *my_bio_methods;
static int
my_sock_read(BIO *h, char *buf, int size)
{
+ PGconn *conn = (PGconn *) BIO_get_app_data(h);
int res;
- res = pqsecure_raw_read((PGconn *) BIO_get_app_data(h), buf, size);
+ res = pqsecure_raw_read(conn, buf, size);
BIO_clear_retry_flags(h);
if (res < 0)
{
@@ -1884,6 +1887,9 @@ my_sock_read(BIO *h, char *buf, int size)
}
}
+ if (res > 0)
+ conn->ssl_handshake_started = true;
+
return res;
}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index ef6ce9d0c0b..4fdd2cbee8c 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -72,7 +72,7 @@ typedef enum
CONNECTION_AUTH_OK, /* Received authentication; waiting for
* backend startup. */
CONNECTION_SETENV, /* This state is no longer used. */
- CONNECTION_SSL_STARTUP, /* Negotiating SSL. */
+ CONNECTION_SSL_STARTUP, /* Performing SSL handshake. */
CONNECTION_NEEDED, /* Internal state: connect() needed. */
CONNECTION_CHECK_WRITABLE, /* Checking if session is read-write. */
CONNECTION_CONSUME, /* Consuming any extra messages. */
@@ -82,7 +82,6 @@ typedef enum
CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
CONNECTION_ALLOCATED, /* Waiting for connection attempt to be
* started. */
- CONNECTION_DIRECT_SSL_STARTUP /* Starting SSL without PG Negotiation. */
} ConnStatusType;
typedef enum
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index def49a68af3..9c5bd898fd4 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -76,6 +76,7 @@ typedef struct
#include <openssl/ssl.h>
#include <openssl/err.h>
+
#ifndef OPENSSL_NO_ENGINE
#define USE_SSL_ENGINE
#endif
@@ -231,6 +232,12 @@ typedef enum
PGASYNC_PIPELINE_IDLE, /* "Idle" between commands in pipeline mode */
} PGAsyncStatusType;
+#define ENC_ERROR 0
+#define ENC_DIRECT_SSL 0x01
+#define ENC_GSSAPI 0x02
+#define ENC_NEGOTIATED_SSL 0x04
+#define ENC_PLAINTEXT 0x08
+
/* Target server type (decoded value of target_session_attrs) */
typedef enum
{
@@ -550,17 +557,17 @@ struct pg_conn
void *sasl_state;
int scram_sha_256_iterations;
+ uint8 allowed_enc_methods;
+ uint8 failed_enc_methods;
+ uint8 current_enc_method;
+
/* SSL structures */
bool ssl_in_use;
+ bool ssl_handshake_started;
bool ssl_cert_requested; /* Did the server ask us for a cert? */
bool ssl_cert_sent; /* Did we send one in reply? */
#ifdef USE_SSL
- bool allow_direct_ssl_try; /* Try to make a direct SSL connection
- * without an "SSL negotiation packet" */
- bool allow_ssl_try; /* Allowed to try SSL negotiation */
- bool wait_ssl_try; /* Delay SSL negotiation until after
- * attempting normal connection */
#ifdef USE_OPENSSL
SSL *ssl; /* SSL status, if have SSL connection */
X509 *peer; /* X509 cert of server */
@@ -583,7 +590,6 @@ struct pg_conn
gss_name_t gtarg_nam; /* GSS target name */
/* The following are encryption-only */
- bool try_gss; /* GSS attempting permitted */
bool gssenc; /* GSS encryption is usable */
gss_cred_id_t gcred; /* GSS credential temp storage. */
--
2.39.2
On Thu, 28 Mar 2024, 13:37 Heikki Linnakangas, <hlinnaka@iki.fi> wrote:
On 28/03/2024 13:15, Matthias van de Meent wrote:
On Tue, 5 Mar 2024 at 15:08, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
I hope I didn't joggle your elbow reviewing this, Jacob, but I spent
some time rebase and fix various little things:With the recent changes to backend startup committed by you, this
patchset has gotten major apply failures.Could you provide a new version of the patchset so that it can be
reviewed in the context of current HEAD?Here you are.
Sorry for the delay. I've run some tests and didn't find any specific
issues in the patchset.
I did get sidetracked on trying to further improve the test suite,
where I was trying to find out how to use Test::More::subtests, but
have now decided it's not worth the lost time now vs adding this as a
feature in 17.
Some remaining comments:
patches 0001/0002: not reviewed in detail.
Patch 0003:
The read size in secure_raw_read is capped to port->raw_buf_remaining
if the raw buf has any data. While the user will probably call into
this function again, I think that's a waste of cycles.
pq_buffer_has_data now doesn't have any protections against
desynchronized state between PqRecvLength and PqRecvPointer. An
Assert(PqRecvLength >= PqRecvPointer) to that value would be
appreciated.
(in backend_startup.c)
+ elog(LOG, "Detected direct SSL handshake");
I think this should be gated at a lower log level, or a GUC, as this
wouls easily DOS a logfile by bulk sending of SSL handshake bytes.
0004:
backend_startup.c
+ if (!ssl_enable_alpn) + { + elog(WARNING, "Received direct SSL connection without ssl_enable_alpn enabled");
This is too verbose, too.
+ if (!port->alpn_used) + { + ereport(COMMERROR, + (errcode(ERRCODE_PROTOCOL_VIOLATION), + errmsg("Received direct SSL connection request without required ALPN protocol negotiation extension")));
If ssl_enable_alpn is disabled, we shouln't report a COMMERROR when
the client does indeed not have alpn enabled.
0005:
As mentioned above, I'd have loved to use subtests here for the cube()
of tests, but I got in too much of a rabbit hole to get that done.
0006:
In CONNECTION_FAILED, we use connection_failed() to select whether we
need a new connection or stop trying altogether, but that function's
description states:
+ * Out-of-line portion of the CONNECTION_FAILED() macro + * + * Returns true, if we should retry the connection with different encryption method.
Which to me reads like we should reuse the connection, and try a
different method on that same connection. Maybe we can improve the
wording to something like
+ * Returns true, if we should reconnect with a different encryption method.
to make the reconnect part more clear.
In select_next_encryption_method, there are several copies of this pattern:
if ((remaining_methods & ENC_METHOD) != 0)
{
conn->current_enc_method = ENC_METHOD;
return true;
}
I think a helper macro would reduce the verbosity of the scaffolding,
like in the attached SELECT_NEXT_METHOD.diff.txt.
Kind regards,
Matthias van de Meent
Attachments:
SELECT_NEXT_METHOD.diff.txttext/plain; charset=US-ASCII; name=SELECT_NEXT_METHOD.diff.txtDownload
src/interfaces/libpq/fe-connect.c | 51 +++++++++++++++------------------------
1 file changed, 20 insertions(+), 31 deletions(-)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index edc324dad0..ef95b07978 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -4344,6 +4344,15 @@ select_next_encryption_method(PGconn *conn, bool have_valid_connection)
{
int remaining_methods;
+#define SELECT_NEXT_METHOD(method) \
+ do { \
+ if ((remaining_methods & method) != 0) \
+ { \
+ conn->current_enc_method = method; \
+ return true; \
+ } \
+ } while (false)
+
remaining_methods = conn->allowed_enc_methods & ~conn->failed_enc_methods;
/*
@@ -4373,20 +4382,14 @@ select_next_encryption_method(PGconn *conn, bool have_valid_connection)
}
}
}
- if ((remaining_methods & ENC_GSSAPI) != 0)
- {
- conn->current_enc_method = ENC_GSSAPI;
- return true;
- }
}
+
+ SELECT_NEXT_METHOD(ENC_GSSAPI);
#endif
/* With sslmode=allow, try plaintext connection before SSL. */
- if (conn->sslmode[0] == 'a' && (remaining_methods & ENC_PLAINTEXT) != 0)
- {
- conn->current_enc_method = ENC_PLAINTEXT;
- return true;
- }
+ if (conn->sslmode[0] == 'a')
+ SELECT_NEXT_METHOD(ENC_PLAINTEXT);
/*
* Try SSL. If enabled, try direct SSL. Unless we have a valid TCP
@@ -4396,33 +4399,19 @@ select_next_encryption_method(PGconn *conn, bool have_valid_connection)
* roundtrip from the negotiation, but reconnecting would also incur a
* roundtrip.
*/
- if (have_valid_connection && (remaining_methods & ENC_NEGOTIATED_SSL) != 0)
- {
- conn->current_enc_method = ENC_NEGOTIATED_SSL;
- return true;
- }
-
- if ((remaining_methods & ENC_DIRECT_SSL) != 0)
- {
- conn->current_enc_method = ENC_DIRECT_SSL;
- return true;
- }
+ if (have_valid_connection)
+ SELECT_NEXT_METHOD(ENC_NEGOTIATED_SSL);
- if ((remaining_methods & ENC_NEGOTIATED_SSL) != 0)
- {
- conn->current_enc_method = ENC_NEGOTIATED_SSL;
- return true;
- }
+ SELECT_NEXT_METHOD(ENC_DIRECT_SSL);
+ SELECT_NEXT_METHOD(ENC_NEGOTIATED_SSL);
- if (conn->sslmode[0] != 'a' && (remaining_methods & ENC_PLAINTEXT) != 0)
- {
- conn->current_enc_method = ENC_PLAINTEXT;
- return true;
- }
+ if (conn->sslmode[0] != 'a')
+ SELECT_NEXT_METHOD(ENC_PLAINTEXT);
/* No more options */
conn->current_enc_method = ENC_ERROR;
return false;
+#undef SELECT_NEXT_METHOD
}
/*
Committed this. Thank you to everyone involved!
On 04/04/2024 14:08, Matthias van de Meent wrote:
Patch 0003:
The read size in secure_raw_read is capped to port->raw_buf_remaining
if the raw buf has any data. While the user will probably call into
this function again, I think that's a waste of cycles.
Hmm, yeah, I suppose we could read more data in the same call. It seems
simpler not to. The case that "raw_buf_remaining > 0" is a very rare.
pq_buffer_has_data now doesn't have any protections against
desynchronized state between PqRecvLength and PqRecvPointer. An
Assert(PqRecvLength >= PqRecvPointer) to that value would be
appreciated.
Added.
0006:
In CONNECTION_FAILED, we use connection_failed() to select whether we
need a new connection or stop trying altogether, but that function's
description states:+ * Out-of-line portion of the CONNECTION_FAILED() macro + * + * Returns true, if we should retry the connection with different encryption method.Which to me reads like we should reuse the connection, and try a
different method on that same connection. Maybe we can improve the
wording to something like
+ * Returns true, if we should reconnect with a different encryption method.
to make the reconnect part more clear.
Changed to "Returns true, if we should reconnect and retry with a
different encryption method".
In select_next_encryption_method, there are several copies of this pattern:
if ((remaining_methods & ENC_METHOD) != 0)
{
conn->current_enc_method = ENC_METHOD;
return true;
}I think a helper macro would reduce the verbosity of the scaffolding,
like in the attached SELECT_NEXT_METHOD.diff.txt.
Applied.
In addition to the above, I made heavy changes to the tests. I wanted to
test not just the outcome (SSL, GSSAPI, plaintext, or fail), but also
the steps and reconnections needed to get there. To facilitate that, I
rewrote how the expected outcome was represented in the test script. It
now uses a table-driven approach, with a line for each test iteration,
ie. for each different combination of options that are tested.
I then added some more logging, so that whenever the server receives an
SSLRequest or GSSENCRequest packet, it logs a line. That's controlled by
a new not-in-sample GUC ("trace_connection_negotiation"), intended only
for the test and debugging. The test scrapes the log for the lines that
it prints, and the expected output includes a compact trace of expected
events. For example, the expected output for "user=testuser
gssencmode=prefer sslmode=prefer sslnegotiation=direct", when GSS and
SSL are both disabled in the server, looks like this:
# USER GSSENCMODE SSLMODE SSLNEGOTIATION EVENTS -> OUTCOME
testuser prefer prefer direct connect,
directsslreject, reconnect, sslreject, authok -> plain
That means, we expect libpq to first try direct SSL, which is rejected
by the server. It should then reconnect and attempt traditional
negotiated SSL, which is also rejected. Finally, it should try plaintext
authentication, without reconnecting, which succeeds.
That actually revealed a couple of slightly bogus behaviors with the
current code. Here's one example:
# XXX: libpq retries the connection unnecessarily in this case:
nogssuser require allow connect, gssaccept, authfail,
reconnect, gssaccept, authfail -> fail
That means, with "gssencmode=require sslmode=allow", if the server
accepts the GSS encryption but refuses the connection at authentication,
libpq will reconnect and go through the same motions again. The second
attempt is pointless, we know it's going to fail. The refactoring to the
libpq state machine fixed that issue as a side-effect.
I removed the server ssl_enable_alpn and libpq sslalpn options. The idea
was that they could be useful for testing, but we didn't actually have
any tests that would use them, and you get the same result by testing
with an older server or client version. I'm open to adding them back if
we also add tests that benefit from them, but they were pretty pointless
as they were.
One important open item now is that we need to register a proper ALPN
protocol ID with IANA.
--
Heikki Linnakangas
Neon (https://neon.tech)
Heikki Linnakangas <hlinnaka@iki.fi> writes:
Committed this. Thank you to everyone involved!
Looks like perlcritic isn't too happy with the test code:
koel and crake say
./src/test/libpq_encryption/t/001_negotiate_encryption.pl: Return value of flagged function ignored - chmod at line 138, column 2. See pages 208,278 of PBP. ([InputOutput::RequireCheckedSyscalls] Severity: 5)
./src/test/libpq_encryption/t/001_negotiate_encryption.pl: Return value of flagged function ignored - open at line 184, column 1. See pages 208,278 of PBP. ([InputOutput::RequireCheckedSyscalls] Severity: 5)
regards, tom lane
On 08/04/2024 04:28, Tom Lane wrote:
Heikki Linnakangas <hlinnaka@iki.fi> writes:
Committed this. Thank you to everyone involved!
Looks like perlcritic isn't too happy with the test code:
koel and crake say./src/test/libpq_encryption/t/001_negotiate_encryption.pl: Return value of flagged function ignored - chmod at line 138, column 2. See pages 208,278 of PBP. ([InputOutput::RequireCheckedSyscalls] Severity: 5)
./src/test/libpq_encryption/t/001_negotiate_encryption.pl: Return value of flagged function ignored - open at line 184, column 1. See pages 208,278 of PBP. ([InputOutput::RequireCheckedSyscalls] Severity: 5)
Fixed, thanks.
I'll make a note in my personal TODO list to add perlcritic to cirrus CI
if possible. I rely heavily on that nowadays to catch issues before the
buildfarm.
--
Heikki Linnakangas
Neon (https://neon.tech)
On 08/04/2024 04:25, Heikki Linnakangas wrote:
One important open item now is that we need to register a proper ALPN
protocol ID with IANA.
I sent a request for that:
https://mailarchive.ietf.org/arch/msg/tls-reg-review/9LWPzQfOpbc8dTT7vc9ahNeNaiw/
--
Heikki Linnakangas
Neon (https://neon.tech)
On 08.04.24 10:38, Heikki Linnakangas wrote:
On 08/04/2024 04:25, Heikki Linnakangas wrote:
One important open item now is that we need to register a proper ALPN
protocol ID with IANA.I sent a request for that:
https://mailarchive.ietf.org/arch/msg/tls-reg-review/9LWPzQfOpbc8dTT7vc9ahNeNaiw/
Why did you ask for "pgsql"? The IANA protocol name for port 5432 is
"postgres". This seems confusing.
On 01.03.24 22:49, Jacob Champion wrote:
If we're interested in ALPN negotiation in the future, we may also
want to look at GREASE [1] to keep those options open in the presence
of third-party implementations. Unfortunately OpenSSL doesn't do this
automatically yet.If we don't have a reason not to, it'd be good to follow the strictest
recommendations from [2] to avoid cross-protocol attacks. (For anyone
currently running web servers and Postgres on the same host, they
really don't want browsers "talking" to their Postgres servers.) That
would mean checking the negotiated ALPN on both the server and client
side, and failing if it's not what we expect.
I've been reading up on ALPN. There is another thread that is
discussing PostgreSQL protocol version negotiation, and ALPN also has
"protocol negotiation" in the name and there is some discussion in this
thread about the granularity oft the protocol names.
I'm concerned that there appears to be some confusion over whether ALPN
is a performance feature or a security feature. RFC 7301 appears to be
pretty clear that it's for performance, not for security.
Looking at the ALPACA attack, I'm not convinced that it's very relevant
for PostgreSQL. It's basically just a case of, you connected to the
wrong server. And web browsers routinely open additional connections
based on what data they have previously received, and they liberally
send along session cookies to those new connections, so I understand
that this can be a problem. But I don't see how ALPN is a good defense.
It can help only if all other possible services other than http
implement it and say, you're a web browser, go away. And what if the
rogue server is in fact a web server, then it doesn't help at all. I
guess there could be some common configurations where there is a web
server, and ftp server, and some mail servers running on the same TLS
end point. But in how many cases is there also a PostgreSQL server
running on the same end point? The page about ALPACA also suggests SNI
as a mitigation, which seems more sensible, because the burden is then
on the client to do the right thing, and not on all other servers to
send away clients doing the wrong thing. And of course libpq already
supports SNI.
For the protocol negotiation aspect, how does this work if the wrapped
protocol already has a version negotiation system? For example, various
HTTP versions are registered as separate protocols for ALPN. What if
ALPN says it's HTTP/1.0 but the actual HTTP requests specify 1.1, or
vice versa? What is the actual mechanism where the performance benefits
(saving round-trips) are created? I haven't caught up with HTTP 2 and
so on, so maybe there are additional things at play there, but it is not
fully explained in the RFCs. I suppose PostgreSQL would keep its
internal protocol version negotiation in any case, but then what do we
need ALPN on top for?
On Wed, Apr 24, 2024 at 1:57 PM Peter Eisentraut <peter@eisentraut.org> wrote:
I'm concerned that there appears to be some confusion over whether ALPN
is a performance feature or a security feature. RFC 7301 appears to be
pretty clear that it's for performance, not for security.
It was also designed to give benefits for more complex topologies
(proxies, cert selection, etc.), but yeah, this is a mitigation
technique that just uses what is already widely implemented.
Looking at the ALPACA attack, I'm not convinced that it's very relevant
for PostgreSQL. It's basically just a case of, you connected to the
wrong server.
I think that's an oversimplification. This prevents active MITM, where
an adversary has connected you to the wrong server.
But I don't see how ALPN is a good defense.
It can help only if all other possible services other than http
implement it and say, you're a web browser, go away.
Why? An ALPACA-aware client will fail the connection if the server
doesn't advertise the correct protocol. An ALPACA-aware server will
fail the handshake if the client doesn't advertise the correct
protocol. They protect themselves, and their peers, without needing
their peers to understand.
And what if the
rogue server is in fact a web server, then it doesn't help at all.
It's not a rogue server; the attack is using other friendly services
against you. If you're able to set up an attacker-controlled server,
using the same certificate as the valid server, on a host covered by
the cert, I think it's game over for many other reasons.
If you mean that you can't prevent an attacker from redirecting one
web server's traffic to another (friendly) web server that's running
on the same host, that's correct. Web admins who care would need to
implement countermeasures, like Origin header filtering or something?
I don't think we have a similar concept to that -- it'd be nice! --
but we don't need to have one in order to provide protection for the
other network protocols we exist next to.
I
guess there could be some common configurations where there is a web
server, and ftp server, and some mail servers running on the same TLS
end point. But in how many cases is there also a PostgreSQL server
running on the same end point?
Not only have I seen those cohosted, I've deployed such setups myself.
Isn't that basically cPanel's MO, and a standard setup for <shared web
hosting provider here>? (It's been a while and I don't have a setup
handy to double-check, sorry; feel free to push back if they don't do
that anymore.)
A quick search for "running web server and Postgres on the same host"
seems to yield plenty of conversations. Some of those conversations
say "don't do it", but of course others do not :) Some actively
encourage it for simplicity.
The page about ALPACA also suggests SNI
as a mitigation, which seems more sensible, because the burden is then
on the client to do the right thing, and not on all other servers to
send away clients doing the wrong thing. And of course libpq already
supports SNI.
That mitigates a different attack. From the ALPACA site [1]https://alpaca-attack.com/libs.html:
Implementing these [ALPN] countermeasures is effective in preventing cross-protocol attacks irregardless of hostnames and ports used for application servers.
...
Implementing these [SNI] countermeasures is effective in preventing same-protocol attacks on servers with different hostnames, as well as cross-protocol attacks on servers with different hostnames even if the ALPN countermeasures can not be implemented.
SNI is super useful; it's just not always enough. And a strict SNI
check would also be good to do, but it doesn't seem imperative to tie
it to this feature, since same-protocol attacks were already possible
AFAICT. It's the cross-protocol attacks that are new, made possible by
the new handshake.
For the protocol negotiation aspect, how does this work if the wrapped
protocol already has a version negotiation system? For example, various
HTTP versions are registered as separate protocols for ALPN. What if
ALPN says it's HTTP/1.0 but the actual HTTP requests specify 1.1, or
vice versa?
If a client or server incorrectly negotiates a protocol and then
starts speaking a different one, then it's just protocol-dependent
whether that works or not. HTTP/1.0 and HTTP/1.1 would still be
cross-compatible in some cases. The others, not so much.
What is the actual mechanism where the performance benefits
(saving round-trips) are created?
The negotiation gets done as part of the TLS handshake, which had to
be done anyway.
I haven't caught up with HTTP 2 and
so on, so maybe there are additional things at play there, but it is not
fully explained in the RFCs.
Practically speaking, HTTP/2 is negotiated via ALPN in the real world,
at least last I checked. I don't think browsers ever supported the
plaintext h2c:// scheme. There's also an in-band `Upgrade: h2c` path
defined that does not use ALPN at all, but again I don't think any
browsers use it.
I suppose PostgreSQL would keep its
internal protocol version negotiation in any case, but then what do we
need ALPN on top for?
That is entirely up to us. If there's a 4.0 protocol that's completely
incompatible at the network level (multiplexing? QUIC?) then issuing a
new ALPN would probably be useful.
Thanks,
--Jacob
On 24/04/2024 23:51, Peter Eisentraut wrote:
On 08.04.24 10:38, Heikki Linnakangas wrote:
On 08/04/2024 04:25, Heikki Linnakangas wrote:
One important open item now is that we need to register a proper ALPN
protocol ID with IANA.I sent a request for that:
https://mailarchive.ietf.org/arch/msg/tls-reg-review/9LWPzQfOpbc8dTT7vc9ahNeNaiw/Why did you ask for "pgsql"? The IANA protocol name for port 5432 is
"postgres". This seems confusing.
Oh, I was not aware of that. According to [1]https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt, it's actually
"postgresql". The ALPN registration has not been approved yet, so I'll
reply on the ietf thread to point that out.
[1]: https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt
https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt
--
Heikki Linnakangas
Neon (https://neon.tech)