pgsql: Add support for OAUTHBEARER SASL mechanism

Started by Daniel Gustafsson11 months ago9 messages

dgustafsson@postgresql.org

11 months ago

Add support for OAUTHBEARER SASL mechanism

This commit implements OAUTHBEARER, RFC 7628, and OAuth 2.0 Device
Authorization Grants, RFC 8628. In order to use this there is a
new pg_hba auth method called oauth. When speaking to a OAuth-
enabled server, it looks a bit like this:

$ psql 'host=example.org oauth_issuer=... oauth_client_id=...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG

Device authorization is currently the only supported flow so the
OAuth issuer must support that in order for users to authenticate.
Third-party clients may however extend this and provide their own
flows. The built-in device authorization flow is currently not
supported on Windows.

In order for validation to happen server side a new framework for
plugging in OAuth validation modules is added. As validation is
implementation specific, with no default specified in the standard,
PostgreSQL does not ship with one built-in. Each pg_hba entry can
specify a specific validator or be left blank for the validator
installed as default.

This adds a requirement on libcurl for the client side support,
which is optional to build, but the server side has no additional
build requirements. In order to run the tests, Python is required
as this adds a https server written in Python. Tests are gated
behind PG_TEST_EXTRA as they open ports.

This patch has been a multi-year project with many contributors
involved with reviews and in-depth discussions: Michael Paquier,
Heikki Linnakangas, Zhihong Yu, Mahendrakar Srinivasarao, Andrey
Chudnovsky and Stephen Frost to name a few. While Jacob Champion
is the main author there have been some levels of hacking by others.
Daniel Gustafsson contributed the validation module and various bits
and pieces; Thomas Munro wrote the client side support for kqueue.

Author: Jacob Champion <jacob.champion@enterprisedb.com>
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
Co-authored-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Antonin Houska <ah@cybertec.at>
Reviewed-by: Kashif Zeeshan <kashi.zeeshan@gmail.com>
Discussion: /messages/by-id/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/b3f0be788afc17d2206e1ae1c731d8aeda1f2f59

Christoph Berg

myon@debian.org

10 months ago

In reply to: Daniel Gustafsson (#1)

Re: pgsql: Add support for OAUTHBEARER SASL mechanism

Add support for OAUTHBEARER SASL mechanism

Debian still has this experimental port with a GNU userland and a
FreeBSD kernel called kfreebsd. I don't expect anyone to particularly
care about it, but it found an actual bug:

/build/reproducible-path/postgresql-18-18~~devel.20250331/build/../src/interfaces/libpq/fe-auth-oauth-curl.c: In function ‘register_socket’:
/build/reproducible-path/postgresql-18-18~~devel.20250331/build/../src/interfaces/libpq/fe-auth-oauth-curl.c:1317:20: error: ‘actx’ undeclared (first use in this function); did you mean ‘ctx’?
1317 | actx_error(actx, "libpq does not support multiplexer sockets on this platform");
| ^~~~

This should not be a compile-time error; actx is not defined outside
the #ifdef blocks there:

/*
* Adds and removes sockets from the multiplexer set, as directed by the
* libcurl multi handle.
*/
static int
register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
void *socketp)
{
#ifdef HAVE_SYS_EPOLL_H
struct async_ctx *actx = ctx;
...
#endif
#ifdef HAVE_SYS_EVENT_H
struct async_ctx *actx = ctx;
...
#endif

actx_error(actx, "libpq does not support multiplexer sockets on this platform");
return -1;
}

https://buildd.debian.org/status/fetch.php?pkg=postgresql-18&arch=hurd-amd64&ver=18%7E%7Edevel.20250331-1&stamp=1743455288&raw=0

Christoph

Christoph Berg

myon@debian.org

10 months ago

In reply to: Christoph Berg (#2)

Re: pgsql: Add support for OAUTHBEARER SASL mechanism

Re: To Daniel Gustafsson

Add support for OAUTHBEARER SASL mechanism

Debian still has this experimental port with a GNU userland and a
FreeBSD kernel called kfreebsd.

Sorry this part was nonsense, kfreebsd was actually terminated and
obviously I didn't even read the port's name. The failing port (still
experimental and care-is-optional) is hurd-amd64.

The bug is the same, though.

Christoph

Jacob Champion

jacob.champion@enterprisedb.com

10 months ago

In reply to: Christoph Berg (#2)

Re: pgsql: Add support for OAUTHBEARER SASL mechanism

On Mon, Mar 31, 2025 at 2:54 PM Christoph Berg <myon@debian.org> wrote:

Add support for OAUTHBEARER SASL mechanism

Debian still has this experimental port with a GNU userland and a
FreeBSD kernel called kfreebsd. I don't expect anyone to particularly
care about it, but it found an actual bug:

/build/reproducible-path/postgresql-18-18~~devel.20250331/build/../src/interfaces/libpq/fe-auth-oauth-curl.c: In function ‘register_socket’:
/build/reproducible-path/postgresql-18-18~~devel.20250331/build/../src/interfaces/libpq/fe-auth-oauth-curl.c:1317:20: error: ‘actx’ undeclared (first use in this function); did you mean ‘ctx’?
1317 | actx_error(actx, "libpq does not support multiplexer sockets on this platform");
| ^~~~

This should not be a compile-time error; actx is not defined outside
the #ifdef blocks there:

Ah, sorry about that. Thank you for reporting it!

(That means that Windows builds --with-libcurl are similarly broken, I
think. Not that Windows packagers will want to use --with-libcurl --
it doesn't do anything -- but it should build.)

I don't have hurd-amd64 to test, but I'm working on a patch that will
build and pass tests if I manually munge pg_config.h. We were skipping
the useless tests via a $windows_os check; I think I should use
check_pg_config() instead.

We could change how this works a bit for the proposed libpq-oauth.so
plugin, and only build it if we have a workable implementation. I do
like having these other platforms compile the Curl code, though, since
we'd prefer to keep the build clean for a future Windows
implementation...

--Jacob

Jacob Champion

jacob.champion@enterprisedb.com

10 months ago

In reply to: Jacob Champion (#4)

1 attachment(s)

Re: pgsql: Add support for OAUTHBEARER SASL mechanism

On Mon, Mar 31, 2025 at 4:09 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:

I don't have hurd-amd64 to test, but I'm working on a patch that will
build and pass tests if I manually munge pg_config.h. We were skipping
the useless tests via a $windows_os check; I think I should use
check_pg_config() instead.

Proposed fix attached.

Thanks,
--Jacob

Attachments:

0001-oauth-Fix-build-on-platforms-without-epoll-kqueue.patchapplication/octet-stream; name=0001-oauth-Fix-build-on-platforms-without-epoll-kqueue.patchDownload

From a6bce74459370e8b81654328d93a8ea4601274ed Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon, 31 Mar 2025 16:07:33 -0700
Subject: [PATCH] oauth: Fix build on platforms without epoll/kqueue

register_socket() missed a variable declaration if neither
HAVE_SYS_EPOLL_H nor HAVE_SYS_EVENT_H was defined.

While we're fixing that, adjust the tests to check pg_config.h for one
of the multiplexer implementations, rather than assuming that Windows is
the only platform without support. (Christoph reported this on
hurd-amd64, an experimental Debian.)

Reported-by: Christoph Berg <myon@debian.org>
Discussion: https://postgr.es/m/Z-sPFl27Y0ZC-VBl%40msg.df7cb.de
---
 src/interfaces/libpq/fe-auth-oauth-curl.c        | 4 ++--
 src/test/modules/oauth_validator/t/001_server.pl | 6 ++++--
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c
index 9e0e8a9f2be..cd9c0323bb6 100644
--- a/src/interfaces/libpq/fe-auth-oauth-curl.c
+++ b/src/interfaces/libpq/fe-auth-oauth-curl.c
@@ -1172,8 +1172,9 @@ static int
 register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
 				void *socketp)
 {
-#ifdef HAVE_SYS_EPOLL_H
 	struct async_ctx *actx = ctx;
+
+#ifdef HAVE_SYS_EPOLL_H
 	struct epoll_event ev = {0};
 	int			res;
 	int			op = EPOLL_CTL_ADD;
@@ -1231,7 +1232,6 @@ register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
 	return 0;
 #endif
 #ifdef HAVE_SYS_EVENT_H
-	struct async_ctx *actx = ctx;
 	struct kevent ev[2] = {0};
 	struct kevent ev_out[2];
 	struct timespec timeout = {0};
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
index 30295364ebd..d88994abc24 100644
--- a/src/test/modules/oauth_validator/t/001_server.pl
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -26,9 +26,11 @@ if (!$ENV{PG_TEST_EXTRA} || $ENV{PG_TEST_EXTRA} !~ /\boauth\b/)
 	  'Potentially unsafe test oauth not enabled in PG_TEST_EXTRA';
 }
 
-if ($windows_os)
+unless (check_pg_config("#define HAVE_SYS_EVENT_H 1")
+	or check_pg_config("#define HAVE_SYS_EPOLL_H 1"))
 {
-	plan skip_all => 'OAuth server-side tests are not supported on Windows';
+	plan skip_all =>
+	  'OAuth server-side tests are not supported on this platform';
 }
 
 if ($ENV{with_libcurl} ne 'yes')
-- 
2.34.1

Christoph Berg

myon@debian.org

10 months ago

In reply to: Jacob Champion (#4)

Re: pgsql: Add support for OAUTHBEARER SASL mechanism

Re: Jacob Champion

(That means that Windows builds --with-libcurl are similarly broken, I
think. Not that Windows packagers will want to use --with-libcurl --
it doesn't do anything -- but it should build.)

Does --with-libcurl still do anything useful if this feature test
fails? From what you are saying, the answer is "no", and I can see
more "not on this platform" error messages in other callbacks.

This should be documented in doc/src/sgml/installation.sgml.

We could change how this works a bit for the proposed libpq-oauth.so
plugin, and only build it if we have a workable implementation. I do
like having these other platforms compile the Curl code, though, since
we'd prefer to keep the build clean for a future Windows
implementation...

I would prefer to get an error from configure if the feature doesn't
do anything on my platform. The current way is confusing. If future
users of libcurl change that, the configure test can still be changed.

With the libpq-oauth split, this makes even more sense because
building a library that always throws an error isn't very useful.
(Don't build that file at all if the feature doesn't work.)

Since oauth/curl have some security implications, would it make more
sense to call the switch --enable-oauth (-Doauth) so users could
control better what features their libpq is going to have? Perhaps
some other feature (pg_service as URL?) is going to need libcurl as
well, but it should be configurable separately.

Christoph

Daniel Gustafsson

daniel@yesql.se

10 months ago

In reply to: Christoph Berg (#6)

Re: pgsql: Add support for OAUTHBEARER SASL mechanism

On 1 Apr 2025, at 15:03, Christoph Berg <myon@debian.org> wrote:

With the libpq-oauth split, this makes even more sense because
building a library that always throws an error isn't very useful.
(Don't build that file at all if the feature doesn't work.)

After the split, configure/meson should fail if the libcurl dependency isn't
satisfied or if the platform isn't supported.

Since oauth/curl have some security implications, would it make more
sense to call the switch --enable-oauth (-Doauth) so users could
control better what features their libpq is going to have? Perhaps
some other feature (pg_service as URL?) is going to need libcurl as
well, but it should be configurable separately.

Perhaps --with-oauth-client for the opt-in libpq-oauth?

--
Daniel Gustafsson

Daniel Gustafsson

dgustafsson@postgresql.org

10 months ago

In reply to: Jacob Champion (#5)

Re: pgsql: Add support for OAUTHBEARER SASL mechanism

On 1 Apr 2025, at 02:06, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

On Mon, Mar 31, 2025 at 4:09 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:

I don't have hurd-amd64 to test, but I'm working on a patch that will
build and pass tests if I manually munge pg_config.h. We were skipping
the useless tests via a $windows_os check; I think I should use
check_pg_config() instead.

Proposed fix attached.

Thanks, I agree that this is the right fix. While this is all subject to
change, I will go ahead with this patch in the meantime to make the tree
properly buildable.

--
Daniel Gustafsson

Jacob Champion

jacob.champion@enterprisedb.com

10 months ago

In reply to: Daniel Gustafsson (#7)

Re: pgsql: Add support for OAUTHBEARER SASL mechanism

On Tue, Apr 1, 2025 at 6:12 AM Daniel Gustafsson <daniel@yesql.se> wrote:

On 1 Apr 2025, at 15:03, Christoph Berg <myon@debian.org> wrote:

With the libpq-oauth split, this makes even more sense because
building a library that always throws an error isn't very useful.
(Don't build that file at all if the feature doesn't work.)

After the split, configure/meson should fail if the libcurl dependency isn't
satisfied or if the platform isn't supported.

Yeah, after sleeping on it I agree. If I want a "canary" buildfarm
animal to opt into compilation on unsupported platforms, I can instead
look into a manual #define or something; it doesn't have to be a
supported configure-time thing.

Since oauth/curl have some security implications, would it make more
sense to call the switch --enable-oauth (-Doauth) so users could
control better what features their libpq is going to have? Perhaps
some other feature (pg_service as URL?) is going to need libcurl as
well, but it should be configurable separately.

Perhaps --with-oauth-client for the opt-in libpq-oauth?

It started as -Doauth way back when, but was changed as part of the
discussion at [1]/messages/by-id/6bde5f56-9e7a-4148-b81c-eb6532cb3651@eisentraut.org. Peter, do you have any objections to switching back
to an OAuth-related name?

--Jacob

[1]: /messages/by-id/6bde5f56-9e7a-4148-b81c-eb6532cb3651@eisentraut.org