dblink: Add SCRAM pass-through authentication
Hi,
The attached patch enables SCRAM authentication for dblink connections when
using dblink_fdw without requiring a plain-text password on user mapping
properties. The implementation is very similar to what was implemented on
postgres_fdw [0]/messages/by-id/27b29a35-9b96-46a9-bc1a-914140869dac@gmail.com.
To make it more closer to what was implemented on postgres_fdw a refactor was
needed on get connection routines. A connect_pg_server function was created to
centralize all the logic to actually open the connection with the foreign
server and then replace the duplicated code on dblink_get_conn and
dblink_connect to just call this new function. The main reason for this
refactor was to centralize the SCRAM logic in a single function, instead of
checking if SCRAM is enabled on both functions.
A new is_valid_dblink_fdw_option function was also created to check for valid
dblink fdw options when creating a server with CREATE SERVER command. The
is_valid_dblink_option function only checks for valid libpq options, and the
use_scram_passthrough option is only valid on CREATE SERVER options.
The documentation was also changed to include a new Foreign Data Wrapper
section to describe the SCRAM pass-through feature.
Thoughts?
[0]: /messages/by-id/27b29a35-9b96-46a9-bc1a-914140869dac@gmail.com
--
Matheus Alcantara
Attachments:
v1-0002-dblink-Add-SCRAM-pass-through-authentication.patchapplication/octet-stream; name=v1-0002-dblink-Add-SCRAM-pass-through-authentication.patchDownload
From 51fb5cfb2bc07c70d558b16ded33c43ebecb6575 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 20 Jan 2025 15:34:10 -0300
Subject: [PATCH v1 2/2] dblink: Add SCRAM pass-through authentication
This enables SCRAM authentication for dblink_fdw when connecting to a
foreign server using the same approach as it was implemented for
postgres_fdw on 761c79508e.
---
contrib/dblink/Makefile | 1 +
contrib/dblink/dblink.c | 111 ++++++++++++++++++++++-
contrib/dblink/meson.build | 5 ++
contrib/dblink/t/001_auth_scram.pl | 137 +++++++++++++++++++++++++++++
doc/src/sgml/dblink.sgml | 81 ++++++++++++++++-
5 files changed, 330 insertions(+), 5 deletions(-)
create mode 100644 contrib/dblink/t/001_auth_scram.pl
diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index d4c7ed625a..fde0b49ddb 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -13,6 +13,7 @@ PGFILEDESC = "dblink - connect to other PostgreSQL databases"
REGRESS = dblink
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
+TAP_TESTS = 1
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 4af8cbd1d0..68d9dbcc6f 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -43,6 +43,8 @@
#include "catalog/pg_foreign_server.h"
#include "catalog/pg_type.h"
#include "catalog/pg_user_mapping.h"
+#include "commands/defrem.h"
+#include "common/base64.h"
#include "executor/spi.h"
#include "foreign/foreign.h"
#include "funcapi.h"
@@ -127,6 +129,16 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+
+static void
+appendSCRAMKeysInfo(StringInfo buf);
+
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context);
+
static PGconn *
connect_pg_server(char *conname_or_str, remoteConn *rconn);
@@ -1907,7 +1919,7 @@ dblink_fdw_validator(PG_FUNCTION_ARGS)
{
DefElem *def = (DefElem *) lfirst(cell);
- if (!is_valid_dblink_option(options, def->defname, context))
+ if (!is_valid_dblink_fdw_option(options, def->defname, context))
{
/*
* Unknown option, or invalid option for the context specified, so
@@ -2765,6 +2777,9 @@ get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
+ appendSCRAMKeysInfo(&buf);
+
foreach(cell, fdw->options)
{
DefElem *def = lfirst(cell);
@@ -2946,6 +2961,21 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
+
+/*
+ * Same as is_valid_dblink_option but also check for only dblink_fdw specific
+ * options.
+ */
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context)
+{
+ if (strcmp(option, "use_scram_passthrough") == 0)
+ return true;
+
+ return is_valid_dblink_option(options, option, context);
+}
+
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3016,6 +3046,69 @@ restoreLocalGucs(int nestlevel)
AtEOXact_GUC(true, nestlevel);
}
+/*
+ * Append SCRAM client key and server key information from the global
+ * MyProcPort into the given StringInfo buffer.
+ */
+static void
+appendSCRAMKeysInfo(StringInfo buf)
+{
+ int len;
+ int encoded_len;
+ char *client_key;
+ char *server_key;
+
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
+ /* don't forget the zero-terminator */
+ client_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ClientKey,
+ sizeof(MyProcPort->scram_ClientKey),
+ client_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM client key");
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ServerKey));
+ /* don't forget the zero-terminator */
+ server_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ServerKey,
+ sizeof(MyProcPort->scram_ServerKey),
+ server_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM server key");
+
+ appendStringInfo(buf, "scram_client_key='%s'", client_key);
+ appendStringInfo(buf, "scram_server_key='%s'", server_key);
+
+ pfree(client_key);
+ pfree(server_key);
+}
+
+
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user)
+{
+ ListCell *cell;
+
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ foreach(cell, user->options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ return false;
+}
+
/*
* Connect to remote server. If connstr_or_srvname maps to a foreign server,
* the associated properties and user mapping properties is also used to open
@@ -3031,6 +3124,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn)
char *srvname;
Oid serverid;
UserMapping *user_mapping;
+ bool useScramPassthrough = false;
Oid userid = GetUserId();
static const PQconninfoOption *options = NULL;
@@ -3061,13 +3155,19 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn)
{
serverid = foreign_server->serverid;
user_mapping = GetUserMapping(userid, serverid);
+ useScramPassthrough = MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping);
connstr = get_connect_string(foreign_server, user_mapping);
}
else
connstr = connstr_or_srvname;
- dblink_connstr_check(connstr);
+ /*
+ * Verify the set of connection parameters only if scram pass-through
+ * is not being used because the password is not necessary.
+ */
+ if (!useScramPassthrough)
+ dblink_connstr_check(connstr);
/* first time, allocate or get the custom wait event */
if (dblink_we_connect == 0)
@@ -3087,7 +3187,12 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn)
errdetail_internal("%s", msg)));
}
- dblink_security_check(conn, rconn, connstr);
+ /*
+ * Perform post-connection security checks only if scram pass-through
+ * is not being used because the password is not necessary.
+ */
+ if (!useScramPassthrough)
+ dblink_security_check(conn, rconn, connstr);
/* attempt to set client encoding to match server encoding, if needed */
if (PQclientEncoding(conn) != GetDatabaseEncoding())
diff --git a/contrib/dblink/meson.build b/contrib/dblink/meson.build
index 3ab7866828..dfd8eb6877 100644
--- a/contrib/dblink/meson.build
+++ b/contrib/dblink/meson.build
@@ -36,4 +36,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/001_auth_scram.pl',
+ ],
+ },
}
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
new file mode 100644
index 0000000000..00fd6d8583
--- /dev/null
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -0,0 +1,137 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+# Test SCRAM authentication when opening a new connection with a foreign
+# server.
+#
+# The test is executed by testing the SCRAM authentifcation on a looplback
+# connection on the same server and with different servers.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $hostaddr = '127.0.0.1';
+my $user = "user01";
+
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
+my $fdw_server = "db1_fdw";
+my $fdw_server2 = "db2_fdw";
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+my $node2 = PostgreSQL::Test::Cluster->new('node2');
+
+$node1->init;
+$node2->init;
+
+$node1->start;
+$node2->start;
+
+# Test setup
+
+$node1->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$node2->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$ENV{PGPASSWORD} = "pass";
+
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db0');
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db1');
+$node2->safe_psql('postgres', qq'CREATE DATABASE $db2');
+
+setup_table($node1, $db1, "t");
+setup_table($node2, $db2, "t2");
+
+$node1->safe_psql($db0, 'CREATE EXTENSION IF NOT EXISTS dblink');
+setup_fdw_server($node1, $db0, $fdw_server, $node1, $db1);
+setup_fdw_server($node1, $db0, $fdw_server2, $node2, $db2);
+
+setup_user_mapping($node1, $db0, $fdw_server);
+setup_user_mapping($node1, $db0, $fdw_server2);
+
+# Make the user have the same SCRAM key on both servers. Forcing to have the
+# same iteration and salt.
+my $rolpassword = $node1->safe_psql('postgres',
+ qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';");
+$node2->safe_psql('postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'");
+
+setup_pghba($node1);
+setup_pghba($node2);
+
+# End of test setup
+
+test_fdw_auth($node1, $db0, "t", $fdw_server,
+ "SCRAM auth on the same database cluster must succeed");
+
+test_fdw_auth($node1, $db0, "t2", $fdw_server2,
+ "SCRAM auth on a different database cluster must succeed");
+
+# Helper functions
+
+sub test_fdw_auth
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $db, $tbl, $fdw, $testname) = @_;
+ my $connstr = $node->connstr($db) . qq' user=$user';
+
+ my $ret = $node->safe_psql(
+ $db,
+ qq"SELECT count(1) FROM dblink('$fdw', 'SELECT * FROM $tbl') AS $tbl(a int, b int)",
+ connstr => $connstr);
+
+ is($ret, '10', $testname);
+}
+
+sub setup_pghba
+{
+ my ($node) = @_;
+
+ unlink($node->data_dir . '/pg_hba.conf');
+ $node->append_conf(
+ 'pg_hba.conf', qq{
+ local all all scram-sha-256
+ host all all $hostaddr/32 scram-sha-256
+ host all all ::1/128 scram-sha-256
+ });
+
+ $node->restart;
+}
+
+sub setup_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\');'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\') '
+ );
+}
+
+sub setup_table
+{
+ my ($node, $db, $tbl) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE TABLE $tbl AS SELECT g as a, g + 1 as b FROM generate_series(1,10) g(g)'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON SCHEMA public TO $user');
+ $node->safe_psql($db, qq'GRANT SELECT ON $tbl TO $user');
+}
+
+done_testing();
+
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index 81f35986c8..f6e1009c02 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,6 +136,82 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
+<refsect1>
+ <title>Foreign Data Wrapper</title>
+
+ <para>
+ A Foreign Data Wrapper can be used as a connection name parameter. The foreign
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ </para>
+
+ <para>
+ The authentication with the foreign server can be via password on USER
+ MAPPING or using SCRAM pass-through. The
+ <literal>use_scram_passthrough</literal> on CREATE SERVER options controls
+ whether <filename>dblink_fdw</filename> will use the SCRAM pass-through
+ authentication to connect to the foreign server. With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
+ </para>
+
+ <para>
+ To use SCRAM pass-through authentication:
+ <itemizedlist>
+ <listitem>
+ <para>
+ The remote server must request SCRAM authentication. (If desired,
+ enforce this on the client side (FDW side) with the option
+ <literal>require_auth</literal>.) If another authentication method is
+ requested by the server, then that one will be used normally.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The remote server can be of any PostgreSQL version that supports SCRAM.
+ Support for <literal>use_scram_passthrough</literal> is only required on
+ the client side (FDW side).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The user mapping password is not used. (It could be set to support other
+ authentication methods, but that would arguably violate the point of this
+ feature, which is to avoid storing plain-text passwords.)
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The server running <filename>dblink_fdw</filename> and the remote server
+ must have identical SCRAM secrets (encrypted passwords) for the user being
+ used on <filename>dblink_fdw</filename> to authenticate on the foreign
+ server (same salt and iterations, not merely the same password).
+ </para>
+
+ <para>
+ As a corollary, if FDW connections to multiple hosts are to be made, for
+ example for partitioned foreign tables/sharding, then all hosts must have
+ identical SCRAM secrets for the users involved.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The current session on the PostgreSQL instance that makes the outgoing FDW
+ connections also must also use SCRAM authentication for its incoming client
+ connection. (Hence <quote>pass-through</quote>: SCRAM must be used going in
+ and out.) This is a technical requirement of the SCRAM protocol.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+
+ </refsect1>
+
<refsect1>
<title>Notes</title>
@@ -181,8 +257,9 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection must require password authentication for this to work properly
--- Otherwise, you will receive the following error from dblink_connect():
+-- Note: local connection that don't use SCRAM pass-through require password
+-- authentication for this to work properly. Otherwise, you will receive
+-- the following error from dblink_connect():
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
--
2.39.5 (Apple Git-154)
v1-0001-dblink-refactor-get-connection-routines.patchapplication/octet-stream; name=v1-0001-dblink-refactor-get-connection-routines.patchDownload
From 08d79f0e5ba1b0af044c498d4e752196b48485c1 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 20 Jan 2025 15:25:51 -0300
Subject: [PATCH v1 1/2] dblink: refactor get connection routines
Refactor dblink_get_conn and dblink_connect to move the logic of
actually opening the connection to the new connect_pg_server function
which them can be re-used on both functions.
This is a pre-work for a next commit that will add support for scram
pass-through authentication to dblink which will be able to implement
most of the logic into the connect_pg_server function which now already
have all necessary data information.
---
contrib/dblink/dblink.c | 176 +++++++++++++++++++++-------------------
1 file changed, 92 insertions(+), 84 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index bed2dee3d7..4af8cbd1d0 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -117,7 +117,7 @@ static bool dblink_connstr_has_pw(const char *connstr);
static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
-static char *get_connect_string(const char *servername);
+static char *get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping);
static char *escape_param_str(const char *str);
static void validate_pkattnums(Relation rel,
int2vector *pkattnums_arg, int32 pknumatts_arg,
@@ -127,13 +127,15 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static PGconn *
+connect_pg_server(char *conname_or_str, remoteConn *rconn);
+
/* Global */
static remoteConn *pconn = NULL;
static HTAB *remoteConnHash = NULL;
/* custom wait event values, retrieved from shared memory */
static uint32 dblink_we_connect = 0;
-static uint32 dblink_we_get_conn = 0;
static uint32 dblink_we_get_result = 0;
/*
@@ -201,33 +203,7 @@ dblink_get_conn(char *conname_or_str,
}
else
{
- const char *connstr;
-
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
- dblink_connstr_check(connstr);
-
- /* first time, allocate or get the custom wait event */
- if (dblink_we_get_conn == 0)
- dblink_we_get_conn = WaitEventExtensionNew("DblinkGetConnect");
-
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_get_conn);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- char *msg = pchomp(PQerrorMessage(conn));
-
- libpqsrv_disconnect(conn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
- dblink_security_check(conn, rconn, connstr);
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn);
freeconn = true;
conname = NULL;
}
@@ -272,9 +248,7 @@ Datum
dblink_connect(PG_FUNCTION_ARGS)
{
char *conname_or_str = NULL;
- char *connstr = NULL;
char *connname = NULL;
- char *msg;
PGconn *conn = NULL;
remoteConn *rconn = NULL;
@@ -297,40 +271,7 @@ dblink_connect(PG_FUNCTION_ARGS)
rconn->newXactForCursor = false;
}
- /* first check for valid foreign data server */
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
-
- /* check password in connection string if not superuser */
- dblink_connstr_check(connstr);
-
- /* first time, allocate or get the custom wait event */
- if (dblink_we_connect == 0)
- dblink_we_connect = WaitEventExtensionNew("DblinkConnect");
-
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_connect);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- msg = pchomp(PQerrorMessage(conn));
- libpqsrv_disconnect(conn);
- if (rconn)
- pfree(rconn);
-
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
-
- /* check password actually used if not superuser */
- dblink_security_check(conn, rconn, connstr);
-
- /* attempt to set client encoding to match server encoding, if needed */
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn);
if (connname)
{
@@ -2784,15 +2725,17 @@ dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
* Obtain connection string for a foreign server
*/
static char *
-get_connect_string(const char *servername)
+get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
{
- ForeignServer *foreign_server = NULL;
- UserMapping *user_mapping;
ListCell *cell;
StringInfoData buf;
ForeignDataWrapper *fdw;
AclResult aclresult;
- char *srvname;
+
+ /* first gather the server connstr options */
+ Oid serverid = foreign_server->serverid;
+ Oid fdwid = foreign_server->fdwid;
+ Oid userid = GetUserId();
static const PQconninfoOption *options = NULL;
@@ -2815,18 +2758,6 @@ get_connect_string(const char *servername)
errdetail("Could not get libpq's default connection options.")));
}
- /* first gather the server connstr options */
- srvname = pstrdup(servername);
- truncate_identifier(srvname, strlen(srvname), false);
- foreign_server = GetForeignServerByName(srvname, true);
-
- if (foreign_server)
- {
- Oid serverid = foreign_server->serverid;
- Oid fdwid = foreign_server->fdwid;
- Oid userid = GetUserId();
-
- user_mapping = GetUserMapping(userid, serverid);
fdw = GetForeignDataWrapper(fdwid);
/* Check permissions, user must have usage on the server. */
@@ -2863,9 +2794,6 @@ get_connect_string(const char *servername)
}
return buf.data;
- }
- else
- return NULL;
}
/*
@@ -3087,3 +3015,83 @@ restoreLocalGucs(int nestlevel)
if (nestlevel > 0)
AtEOXact_GUC(true, nestlevel);
}
+
+/*
+ * Connect to remote server. If connstr_or_srvname maps to a foreign server,
+ * the associated properties and user mapping properties is also used to open
+ * the connection. Otherwise a connection will be open using the raw
+ * connstr_or_srvname value.
+ */
+static PGconn *
+connect_pg_server(char *connstr_or_srvname, remoteConn *rconn)
+{
+ PGconn *conn;
+ ForeignServer *foreign_server = NULL;
+ const char *connstr;
+ char *srvname;
+ Oid serverid;
+ UserMapping *user_mapping;
+ Oid userid = GetUserId();
+
+ static const PQconninfoOption *options = NULL;
+
+ /*
+ * Get list of valid libpq options.
+ *
+ * To avoid unnecessary work, we get the list once and use it throughout
+ * the lifetime of this backend process. We don't need to care about
+ * memory context issues, because PQconndefaults allocates with malloc.
+ */
+ if (!options)
+ {
+ options = PQconndefaults();
+ if (!options) /* assume reason for failure is OOM */
+ ereport(ERROR,
+ (errcode(ERRCODE_FDW_OUT_OF_MEMORY),
+ errmsg("out of memory"),
+ errdetail("Could not get libpq's default connection options.")));
+ }
+
+ /* first gather the server connstr options */
+ srvname = pstrdup(connstr_or_srvname);
+ truncate_identifier(srvname, strlen(srvname), false);
+ foreign_server = GetForeignServerByName(srvname, true);
+
+ if (foreign_server)
+ {
+ serverid = foreign_server->serverid;
+ user_mapping = GetUserMapping(userid, serverid);
+
+ connstr = get_connect_string(foreign_server, user_mapping);
+ }
+ else
+ connstr = connstr_or_srvname;
+
+ dblink_connstr_check(connstr);
+
+ /* first time, allocate or get the custom wait event */
+ if (dblink_we_connect == 0)
+ dblink_we_connect = WaitEventExtensionNew("DblinkConnectPgServer");
+
+ /* OK to make connection */
+ conn = libpqsrv_connect(connstr, dblink_we_connect);
+
+ if (!conn || PQstatus(conn) != CONNECTION_OK)
+ {
+ char *msg = pchomp(PQerrorMessage(conn));
+
+ libpqsrv_disconnect(conn);
+ ereport(ERROR,
+ (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
+ errmsg("could not establish connection"),
+ errdetail_internal("%s", msg)));
+ }
+
+ dblink_security_check(conn, rconn, connstr);
+
+ /* attempt to set client encoding to match server encoding, if needed */
+ if (PQclientEncoding(conn) != GetDatabaseEncoding())
+ PQsetClientEncoding(conn, GetDatabaseEncodingName());
+
+ return conn;
+}
--
2.39.5 (Apple Git-154)
On Wed, Jan 22, 2025 at 6:10 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:
The attached patch enables SCRAM authentication for dblink connections when
using dblink_fdw without requiring a plain-text password on user mapping
properties. The implementation is very similar to what was implemented on
postgres_fdw [0].
Not a full review, but I wanted to highlight one aspect of the patch:
- dblink_connstr_check(connstr); + /* + * Verify the set of connection parameters only if scram pass-through is + * not being used because the password is not necessary. + */ + if (!useScramPassthrough) + dblink_connstr_check(connstr);
and
- dblink_security_check(conn, rconn, connstr); + /* + * Perform post-connection security checks only if scram pass-through is + * not being used because the password is not necessary. + */ + if (!useScramPassthrough) + dblink_security_check(conn, rconn, connstr);
These don't seem right to me. SCRAM passthrough should be considered
as _part_ of the connstr/security checks, but I think it should not
_bypass_ those checks. We have to enforce the use of the SCRAM
credentials on the remote for safety, similarly to GSS delegation.
(This might be a good place for `require_auth=scram-sha-256`?)
I've attached a failing test to better illustrate what I mean.
It looks like the postgres_fdw patch that landed also performs a
bypass -- I think that may need an open item to fix? CC'd Peter.
Thanks!
--Jacob
Attachments:
sec-test.diff.txttext/plain; charset=US-ASCII; name=sec-test.diff.txtDownload
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
index 00fd6d85833..58c924f9aff 100644
--- a/contrib/dblink/t/001_auth_scram.pl
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -56,8 +56,21 @@ my $rolpassword = $node1->safe_psql('postgres',
qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';");
$node2->safe_psql('postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'");
-setup_pghba($node1);
-setup_pghba($node2);
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all scram-sha-256
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all scram-sha-256
+});
+
+$node1->restart;
+$node2->restart;
# End of test setup
@@ -67,6 +80,45 @@ test_fdw_auth($node1, $db0, "t", $fdw_server,
test_fdw_auth($node1, $db0, "t2", $fdw_server2,
"SCRAM auth on a different database cluster must succeed");
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all trust
+});
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on a different cluster');
+like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error from loopback trust (different cluster)');
+
# Helper functions
sub test_fdw_auth
@@ -84,21 +136,6 @@ sub test_fdw_auth
is($ret, '10', $testname);
}
-sub setup_pghba
-{
- my ($node) = @_;
-
- unlink($node->data_dir . '/pg_hba.conf');
- $node->append_conf(
- 'pg_hba.conf', qq{
- local all all scram-sha-256
- host all all $hostaddr/32 scram-sha-256
- host all all ::1/128 scram-sha-256
- });
-
- $node->restart;
-}
-
sub setup_user_mapping
{
my ($node, $db, $fdw) = @_;
Hi, thanks for reviewing this patch!
Em seg., 10 de fev. de 2025 às 20:19, Jacob Champion
<jacob.champion@enterprisedb.com> escreveu:
On Wed, Jan 22, 2025 at 6:10 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:The attached patch enables SCRAM authentication for dblink connections when
using dblink_fdw without requiring a plain-text password on user mapping
properties. The implementation is very similar to what was implemented on
postgres_fdw [0].Not a full review, but I wanted to highlight one aspect of the patch:
- dblink_connstr_check(connstr); + /* + * Verify the set of connection parameters only if scram pass-through is + * not being used because the password is not necessary. + */ + if (!useScramPassthrough) + dblink_connstr_check(connstr);and
- dblink_security_check(conn, rconn, connstr); + /* + * Perform post-connection security checks only if scram pass-through is + * not being used because the password is not necessary. + */ + if (!useScramPassthrough) + dblink_security_check(conn, rconn, connstr);These don't seem right to me. SCRAM passthrough should be considered
as _part_ of the connstr/security checks, but I think it should not
_bypass_ those checks. We have to enforce the use of the SCRAM
credentials on the remote for safety, similarly to GSS delegation.
(This might be a good place for `require_auth=scram-sha-256`?)
Currently dblink_connstr_check and dblink_security_check only check if the
password is present on connection options, in case of not superuser. I added
this logic because the password is not required for SCRAM but I agree with you
that it sounds strange. Maybe these functions could check if the SCRAM is
being used and then skip the password validation if needed internally?
I also agree that we should enforce the use of the SCRAM on the remote for
safety. To do this I think that we could set require_auth=scram-sha-256 on
connection options if SCRAM pass-through is being used, with this we will get a
connection error. WYT?
I've attached a failing test to better illustrate what I mean.
Thanks for this! I'm attaching a v2 patch that includes the
require_auth=scram-sha-256 option and also your test case, which I've just
changed to the expected error message.
It looks like the postgres_fdw patch that landed also performs a
bypass -- I think that may need an open item to fix? CC'd Peter.
I can create a new patch to fix this on postgres_fdw too once we define the
approach to this here on dblink.
--
Matheus Alcantara
Attachments:
v2-0002-dblink-Add-SCRAM-pass-through-authentication.patchapplication/octet-stream; name=v2-0002-dblink-Add-SCRAM-pass-through-authentication.patchDownload
From ebdb5cc081f9894c0c4fa4849fb80ecca2493f3c Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Tue, 11 Feb 2025 10:26:29 -0300
Subject: [PATCH v2 2/2] dblink: Add SCRAM pass-through authentication
This enables SCRAM authentication for dblink_fdw when connecting to a
foreign server using the same approach as it was implemented for
postgres_fdw on 761c79508e.
---
contrib/dblink/Makefile | 1 +
contrib/dblink/dblink.c | 113 ++++++++++++++++++-
contrib/dblink/meson.build | 5 +
contrib/dblink/t/001_auth_scram.pl | 174 +++++++++++++++++++++++++++++
doc/src/sgml/dblink.sgml | 81 +++++++++++++-
5 files changed, 369 insertions(+), 5 deletions(-)
create mode 100644 contrib/dblink/t/001_auth_scram.pl
diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index d4c7ed625ab..fde0b49ddbb 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -13,6 +13,7 @@ PGFILEDESC = "dblink - connect to other PostgreSQL databases"
REGRESS = dblink
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
+TAP_TESTS = 1
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index e02c0f4d730..0f29fefa4bb 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -43,6 +43,8 @@
#include "catalog/pg_foreign_server.h"
#include "catalog/pg_type.h"
#include "catalog/pg_user_mapping.h"
+#include "commands/defrem.h"
+#include "common/base64.h"
#include "executor/spi.h"
#include "foreign/foreign.h"
#include "funcapi.h"
@@ -127,6 +129,16 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static bool
+ UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+
+static void
+ appendSCRAMKeysInfo(StringInfo buf);
+
+static bool
+ is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context);
+
static PGconn *connect_pg_server(char *conname_or_str, remoteConn *rconn);
/* Global */
@@ -1906,7 +1918,7 @@ dblink_fdw_validator(PG_FUNCTION_ARGS)
{
DefElem *def = (DefElem *) lfirst(cell);
- if (!is_valid_dblink_option(options, def->defname, context))
+ if (!is_valid_dblink_fdw_option(options, def->defname, context))
{
/*
* Unknown option, or invalid option for the context specified, so
@@ -2764,6 +2776,9 @@ get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
+ appendSCRAMKeysInfo(&buf);
+
foreach(cell, fdw->options)
{
DefElem *def = lfirst(cell);
@@ -2945,6 +2960,22 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
+
+/*
+ * Same as is_valid_dblink_option but also check for only dblink_fdw specific
+ * options.
+ */
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context)
+{
+ if (strcmp(option, "use_scram_passthrough") == 0)
+ return true;
+
+ return is_valid_dblink_option(options, option, context);
+}
+
+
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3015,6 +3046,70 @@ restoreLocalGucs(int nestlevel)
AtEOXact_GUC(true, nestlevel);
}
+/*
+ * Append SCRAM client key and server key information from the global
+ * MyProcPort into the given StringInfo buffer.
+ */
+static void
+appendSCRAMKeysInfo(StringInfo buf)
+{
+ int len;
+ int encoded_len;
+ char *client_key;
+ char *server_key;
+
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
+ /* don't forget the zero-terminator */
+ client_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ClientKey,
+ sizeof(MyProcPort->scram_ClientKey),
+ client_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM client key");
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ServerKey));
+ /* don't forget the zero-terminator */
+ server_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ServerKey,
+ sizeof(MyProcPort->scram_ServerKey),
+ server_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM server key");
+
+ appendStringInfo(buf, "scram_client_key='%s'", client_key);
+ appendStringInfo(buf, "scram_server_key='%s'", server_key);
+ appendStringInfo(buf, "require_auth='scram-sha-256'");
+
+ pfree(client_key);
+ pfree(server_key);
+}
+
+
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user)
+{
+ ListCell *cell;
+
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ foreach(cell, user->options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ return false;
+}
+
/*
* Connect to remote server. If connstr_or_srvname maps to a foreign server,
* the associated properties and user mapping properties is also used to open
@@ -3031,6 +3126,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn)
Oid serverid;
UserMapping *user_mapping;
Oid userid = GetUserId();
+ bool useScramPassthrough = false;
static const PQconninfoOption *options = NULL;
@@ -3060,13 +3156,19 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn)
{
serverid = foreign_server->serverid;
user_mapping = GetUserMapping(userid, serverid);
+ useScramPassthrough = MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping);
connstr = get_connect_string(foreign_server, user_mapping);
}
else
connstr = connstr_or_srvname;
- dblink_connstr_check(connstr);
+ /*
+ * Verify the set of connection parameters only if scram pass-through is
+ * not being used because the password is not necessary.
+ */
+ if (!useScramPassthrough)
+ dblink_connstr_check(connstr);
/* first time, allocate or get the custom wait event */
if (dblink_we_connect == 0)
@@ -3086,7 +3188,12 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn)
errdetail_internal("%s", msg)));
}
- dblink_security_check(conn, rconn, connstr);
+ /*
+ * Perform post-connection security checks only if scram pass-through is
+ * not being used because the password is not necessary.
+ */
+ if (!useScramPassthrough)
+ dblink_security_check(conn, rconn, connstr);
/* attempt to set client encoding to match server encoding, if needed */
if (PQclientEncoding(conn) != GetDatabaseEncoding())
diff --git a/contrib/dblink/meson.build b/contrib/dblink/meson.build
index 3ab78668288..dfd8eb6877e 100644
--- a/contrib/dblink/meson.build
+++ b/contrib/dblink/meson.build
@@ -36,4 +36,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/001_auth_scram.pl',
+ ],
+ },
}
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
new file mode 100644
index 00000000000..cb672e2b99f
--- /dev/null
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -0,0 +1,174 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+# Test SCRAM authentication when opening a new connection with a foreign
+# server.
+#
+# The test is executed by testing the SCRAM authentifcation on a looplback
+# connection on the same server and with different servers.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $hostaddr = '127.0.0.1';
+my $user = "user01";
+
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
+my $fdw_server = "db1_fdw";
+my $fdw_server2 = "db2_fdw";
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+my $node2 = PostgreSQL::Test::Cluster->new('node2');
+
+$node1->init;
+$node2->init;
+
+$node1->start;
+$node2->start;
+
+# Test setup
+
+$node1->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$node2->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$ENV{PGPASSWORD} = "pass";
+
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db0');
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db1');
+$node2->safe_psql('postgres', qq'CREATE DATABASE $db2');
+
+setup_table($node1, $db1, "t");
+setup_table($node2, $db2, "t2");
+
+$node1->safe_psql($db0, 'CREATE EXTENSION IF NOT EXISTS dblink');
+setup_fdw_server($node1, $db0, $fdw_server, $node1, $db1);
+setup_fdw_server($node1, $db0, $fdw_server2, $node2, $db2);
+
+setup_user_mapping($node1, $db0, $fdw_server);
+setup_user_mapping($node1, $db0, $fdw_server2);
+
+# Make the user have the same SCRAM key on both servers. Forcing to have the
+# same iteration and salt.
+my $rolpassword = $node1->safe_psql('postgres',
+ qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';");
+$node2->safe_psql('postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'");
+
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all scram-sha-256
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all scram-sha-256
+});
+
+$node1->restart;
+$node2->restart;
+
+# End of test setup
+
+test_fdw_auth($node1, $db0, "t", $fdw_server,
+ "SCRAM auth on the same database cluster must succeed");
+
+test_fdw_auth($node1, $db0, "t2", $fdw_server2,
+ "SCRAM auth on a different database cluster must succeed");
+
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all trust
+});
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on a different cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (different cluster)');
+
+# Helper functions
+
+sub test_fdw_auth
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $db, $tbl, $fdw, $testname) = @_;
+ my $connstr = $node->connstr($db) . qq' user=$user';
+
+ my $ret = $node->safe_psql(
+ $db,
+ qq"SELECT count(1) FROM dblink('$fdw', 'SELECT * FROM $tbl') AS $tbl(a int, b int)",
+ connstr => $connstr);
+
+ is($ret, '10', $testname);
+}
+
+sub setup_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\');'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\') '
+ );
+}
+
+sub setup_table
+{
+ my ($node, $db, $tbl) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE TABLE $tbl AS SELECT g as a, g + 1 as b FROM generate_series(1,10) g(g)'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON SCHEMA public TO $user');
+ $node->safe_psql($db, qq'GRANT SELECT ON $tbl TO $user');
+}
+
+done_testing();
+
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index 81f35986c88..f6e1009c028 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,6 +136,82 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
+<refsect1>
+ <title>Foreign Data Wrapper</title>
+
+ <para>
+ A Foreign Data Wrapper can be used as a connection name parameter. The foreign
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ </para>
+
+ <para>
+ The authentication with the foreign server can be via password on USER
+ MAPPING or using SCRAM pass-through. The
+ <literal>use_scram_passthrough</literal> on CREATE SERVER options controls
+ whether <filename>dblink_fdw</filename> will use the SCRAM pass-through
+ authentication to connect to the foreign server. With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
+ </para>
+
+ <para>
+ To use SCRAM pass-through authentication:
+ <itemizedlist>
+ <listitem>
+ <para>
+ The remote server must request SCRAM authentication. (If desired,
+ enforce this on the client side (FDW side) with the option
+ <literal>require_auth</literal>.) If another authentication method is
+ requested by the server, then that one will be used normally.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The remote server can be of any PostgreSQL version that supports SCRAM.
+ Support for <literal>use_scram_passthrough</literal> is only required on
+ the client side (FDW side).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The user mapping password is not used. (It could be set to support other
+ authentication methods, but that would arguably violate the point of this
+ feature, which is to avoid storing plain-text passwords.)
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The server running <filename>dblink_fdw</filename> and the remote server
+ must have identical SCRAM secrets (encrypted passwords) for the user being
+ used on <filename>dblink_fdw</filename> to authenticate on the foreign
+ server (same salt and iterations, not merely the same password).
+ </para>
+
+ <para>
+ As a corollary, if FDW connections to multiple hosts are to be made, for
+ example for partitioned foreign tables/sharding, then all hosts must have
+ identical SCRAM secrets for the users involved.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The current session on the PostgreSQL instance that makes the outgoing FDW
+ connections also must also use SCRAM authentication for its incoming client
+ connection. (Hence <quote>pass-through</quote>: SCRAM must be used going in
+ and out.) This is a technical requirement of the SCRAM protocol.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+
+ </refsect1>
+
<refsect1>
<title>Notes</title>
@@ -181,8 +257,9 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection must require password authentication for this to work properly
--- Otherwise, you will receive the following error from dblink_connect():
+-- Note: local connection that don't use SCRAM pass-through require password
+-- authentication for this to work properly. Otherwise, you will receive
+-- the following error from dblink_connect():
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
--
2.39.5 (Apple Git-154)
v2-0001-dblink-refactor-get-connection-routines.patchapplication/octet-stream; name=v2-0001-dblink-refactor-get-connection-routines.patchDownload
From d7ab6346413998f6b37351cd9df224c678503144 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 20 Jan 2025 15:25:51 -0300
Subject: [PATCH v2 1/2] dblink: refactor get connection routines
Refactor dblink_get_conn and dblink_connect to move the logic of
actually opening the connection to the new connect_pg_server function
which them can be re-used on both functions.
This is a pre-work for a next commit that will add support for scram
pass-through authentication to dblink which will be able to implement
most of the logic into the connect_pg_server function which now already
have all necessary data information.
---
contrib/dblink/dblink.c | 229 +++++++++++++++++++++-------------------
1 file changed, 118 insertions(+), 111 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index bed2dee3d72..e02c0f4d730 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -117,7 +117,7 @@ static bool dblink_connstr_has_pw(const char *connstr);
static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
-static char *get_connect_string(const char *servername);
+static char *get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping);
static char *escape_param_str(const char *str);
static void validate_pkattnums(Relation rel,
int2vector *pkattnums_arg, int32 pknumatts_arg,
@@ -127,13 +127,14 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static PGconn *connect_pg_server(char *conname_or_str, remoteConn *rconn);
+
/* Global */
static remoteConn *pconn = NULL;
static HTAB *remoteConnHash = NULL;
/* custom wait event values, retrieved from shared memory */
static uint32 dblink_we_connect = 0;
-static uint32 dblink_we_get_conn = 0;
static uint32 dblink_we_get_result = 0;
/*
@@ -201,33 +202,7 @@ dblink_get_conn(char *conname_or_str,
}
else
{
- const char *connstr;
-
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
- dblink_connstr_check(connstr);
-
- /* first time, allocate or get the custom wait event */
- if (dblink_we_get_conn == 0)
- dblink_we_get_conn = WaitEventExtensionNew("DblinkGetConnect");
-
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_get_conn);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- char *msg = pchomp(PQerrorMessage(conn));
-
- libpqsrv_disconnect(conn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
- dblink_security_check(conn, rconn, connstr);
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn);
freeconn = true;
conname = NULL;
}
@@ -272,9 +247,7 @@ Datum
dblink_connect(PG_FUNCTION_ARGS)
{
char *conname_or_str = NULL;
- char *connstr = NULL;
char *connname = NULL;
- char *msg;
PGconn *conn = NULL;
remoteConn *rconn = NULL;
@@ -297,40 +270,7 @@ dblink_connect(PG_FUNCTION_ARGS)
rconn->newXactForCursor = false;
}
- /* first check for valid foreign data server */
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
-
- /* check password in connection string if not superuser */
- dblink_connstr_check(connstr);
-
- /* first time, allocate or get the custom wait event */
- if (dblink_we_connect == 0)
- dblink_we_connect = WaitEventExtensionNew("DblinkConnect");
-
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_connect);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- msg = pchomp(PQerrorMessage(conn));
- libpqsrv_disconnect(conn);
- if (rconn)
- pfree(rconn);
-
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
-
- /* check password actually used if not superuser */
- dblink_security_check(conn, rconn, connstr);
-
- /* attempt to set client encoding to match server encoding, if needed */
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn);
if (connname)
{
@@ -2784,15 +2724,17 @@ dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
* Obtain connection string for a foreign server
*/
static char *
-get_connect_string(const char *servername)
+get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
{
- ForeignServer *foreign_server = NULL;
- UserMapping *user_mapping;
ListCell *cell;
StringInfoData buf;
ForeignDataWrapper *fdw;
AclResult aclresult;
- char *srvname;
+
+ /* first gather the server connstr options */
+ Oid serverid = foreign_server->serverid;
+ Oid fdwid = foreign_server->fdwid;
+ Oid userid = GetUserId();
static const PQconninfoOption *options = NULL;
@@ -2815,57 +2757,42 @@ get_connect_string(const char *servername)
errdetail("Could not get libpq's default connection options.")));
}
- /* first gather the server connstr options */
- srvname = pstrdup(servername);
- truncate_identifier(srvname, strlen(srvname), false);
- foreign_server = GetForeignServerByName(srvname, true);
-
- if (foreign_server)
- {
- Oid serverid = foreign_server->serverid;
- Oid fdwid = foreign_server->fdwid;
- Oid userid = GetUserId();
-
- user_mapping = GetUserMapping(userid, serverid);
- fdw = GetForeignDataWrapper(fdwid);
-
- /* Check permissions, user must have usage on the server. */
- aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
- if (aclresult != ACLCHECK_OK)
- aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ fdw = GetForeignDataWrapper(fdwid);
- foreach(cell, fdw->options)
- {
- DefElem *def = lfirst(cell);
+ /* Check permissions, user must have usage on the server. */
+ aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
+ if (aclresult != ACLCHECK_OK)
+ aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
- if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, fdw->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, foreign_server->options)
- {
- DefElem *def = lfirst(cell);
+ if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, user_mapping->options)
- {
+ if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- DefElem *def = lfirst(cell);
+ foreach(cell, user_mapping->options)
+ {
- if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ DefElem *def = lfirst(cell);
- return buf.data;
+ if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
}
- else
- return NULL;
+
+ return buf.data;
}
/*
@@ -3087,3 +3014,83 @@ restoreLocalGucs(int nestlevel)
if (nestlevel > 0)
AtEOXact_GUC(true, nestlevel);
}
+
+/*
+ * Connect to remote server. If connstr_or_srvname maps to a foreign server,
+ * the associated properties and user mapping properties is also used to open
+ * the connection. Otherwise a connection will be open using the raw
+ * connstr_or_srvname value.
+ */
+static PGconn *
+connect_pg_server(char *connstr_or_srvname, remoteConn *rconn)
+{
+ PGconn *conn;
+ ForeignServer *foreign_server = NULL;
+ const char *connstr;
+ char *srvname;
+ Oid serverid;
+ UserMapping *user_mapping;
+ Oid userid = GetUserId();
+
+ static const PQconninfoOption *options = NULL;
+
+ /*
+ * Get list of valid libpq options.
+ *
+ * To avoid unnecessary work, we get the list once and use it throughout
+ * the lifetime of this backend process. We don't need to care about
+ * memory context issues, because PQconndefaults allocates with malloc.
+ */
+ if (!options)
+ {
+ options = PQconndefaults();
+ if (!options) /* assume reason for failure is OOM */
+ ereport(ERROR,
+ (errcode(ERRCODE_FDW_OUT_OF_MEMORY),
+ errmsg("out of memory"),
+ errdetail("Could not get libpq's default connection options.")));
+ }
+
+ /* first gather the server connstr options */
+ srvname = pstrdup(connstr_or_srvname);
+ truncate_identifier(srvname, strlen(srvname), false);
+ foreign_server = GetForeignServerByName(srvname, true);
+
+ if (foreign_server)
+ {
+ serverid = foreign_server->serverid;
+ user_mapping = GetUserMapping(userid, serverid);
+
+ connstr = get_connect_string(foreign_server, user_mapping);
+ }
+ else
+ connstr = connstr_or_srvname;
+
+ dblink_connstr_check(connstr);
+
+ /* first time, allocate or get the custom wait event */
+ if (dblink_we_connect == 0)
+ dblink_we_connect = WaitEventExtensionNew("DblinkConnectPgServer");
+
+ /* OK to make connection */
+ conn = libpqsrv_connect(connstr, dblink_we_connect);
+
+ if (!conn || PQstatus(conn) != CONNECTION_OK)
+ {
+ char *msg = pchomp(PQerrorMessage(conn));
+
+ libpqsrv_disconnect(conn);
+ ereport(ERROR,
+ (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
+ errmsg("could not establish connection"),
+ errdetail_internal("%s", msg)));
+ }
+
+ dblink_security_check(conn, rconn, connstr);
+
+ /* attempt to set client encoding to match server encoding, if needed */
+ if (PQclientEncoding(conn) != GetDatabaseEncoding())
+ PQsetClientEncoding(conn, GetDatabaseEncodingName());
+
+ return conn;
+}
--
2.39.5 (Apple Git-154)
On Wed, Feb 12, 2025 at 11:54 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:
Currently dblink_connstr_check and dblink_security_check only check if the
password is present on connection options, in case of not superuser.
They also check for GSS delegation. I think SCRAM passthrough should
ideally be considered a second form of credentials delegation, from
the perspective of those functions.
I added
this logic because the password is not required for SCRAM but I agree with you
that it sounds strange. Maybe these functions could check if the SCRAM is
being used and then skip the password validation if needed internally?
As long as the end result is to enforce that the credentials must come
from the end user, I think that would be fine in theory.
I also agree that we should enforce the use of the SCRAM on the remote for
safety. To do this I think that we could set require_auth=scram-sha-256 on
connection options if SCRAM pass-through is being used, with this we will get a
connection error. WYT?
We would need to verify that the user mapping can't overwrite that
with its own (less trusted) `require_auth` setting. (I think that
should be true already, but I'm not 100% sure.)
Hardcoding to scram-sha-256 would also prohibit the use of GSS or
standard password auth, now that I think about it. The docs currently
have a note about being able to choose... Should we add the other
permitted authentication types, i.e. `password,md5,scram-sha-256,gss`?
Or should we prohibit the use of other auth types if you've set
use_scram_passthrough? Or maybe there's an easier way to enforce the
use of the SCRAM keys, that better matches the current logic in
dblink_security_check?
I can create a new patch to fix this on postgres_fdw too once we define the
approach to this here on dblink.
Sounds good to me.
Thanks,
--Jacob
On Thu, Feb 13, 2025 at 8:25 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
I also agree that we should enforce the use of the SCRAM on the remote for
safety. To do this I think that we could set require_auth=scram-sha-256 on
connection options if SCRAM pass-through is being used, with this we will get a
connection error. WYT?We would need to verify that the user mapping can't overwrite that
with its own (less trusted) `require_auth` setting. (I think that
should be true already, but I'm not 100% sure.)
Yeah, this is true. The user mapping and the fdw options can overwrite this.
I'll work on a fix for this.
Hardcoding to scram-sha-256 would also prohibit the use of GSS or
standard password auth, now that I think about it. The docs currently
have a note about being able to choose... Should we add the other
permitted authentication types, i.e. `password,md5,scram-sha-256,gss`?
Or should we prohibit the use of other auth types if you've set
use_scram_passthrough? Or maybe there's an easier way to enforce the
use of the SCRAM keys, that better matches the current logic in
dblink_security_check?
But would it be possible to use SCRAM pass-through feature using another auth
method? We need the scram keys (that is saved on MyProcPort during
scram_exchange) to perform the pass-through which I think that we would not
have with another auth type? That being said I think that we could prohibit the
usage of other auth types when use_scram_passthrough is set, what do you think?
--
Matheus Alcantara
On Wed, Feb 19, 2025 at 12:02 PM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:
Hardcoding to scram-sha-256 would also prohibit the use of GSS or
standard password auth, now that I think about it. The docs currently
have a note about being able to choose... Should we add the other
permitted authentication types, i.e. `password,md5,scram-sha-256,gss`?
Or should we prohibit the use of other auth types if you've set
use_scram_passthrough? Or maybe there's an easier way to enforce the
use of the SCRAM keys, that better matches the current logic in
dblink_security_check?But would it be possible to use SCRAM pass-through feature using another auth
method?
No, but if you want the same foreign server to be accessible by users
who log in with different authentication types, forcing a single
require_auth setting will defeat that. I don't have a strong opinion
about how important maintaining that functionality is, but the code
seems to allow it today.
--
Some thoughts on v2-0001:
I like the conceptual simplification of get_connect_string().
+ /* first time, allocate or get the custom wait event */ + if (dblink_we_connect == 0) + dblink_we_connect = WaitEventExtensionNew("DblinkConnectPgServer");
Replacing two existing wait events with one new one is a user-facing
change (see the documented list of events at [1]https://www.postgresql.org/docs/current/dblink.html). Maybe we want that,
but it hasn't been explained. I think that change should be made
independently of a refactoring patch (or else defended in the commit
message).
+ if (foreign_server) + { + serverid = foreign_server->serverid; + user_mapping = GetUserMapping(userid, serverid); + + connstr = get_connect_string(foreign_server, user_mapping); + }
Is there any other valid value for user_mapping that a caller can
choose? If not, I'd like to see the GetUserMapping call pushed back
down into get_connect_string(), unless there's another reason for
pulling it up that I'm missing.
+ static const PQconninfoOption *options = NULL; + + /* + * Get list of valid libpq options. + * + * To avoid unnecessary work, we get the list once and use it throughout + * the lifetime of this backend process. We don't need to care about + * memory context issues, because PQconndefaults allocates with malloc. + */ + if (!options) + { + options = PQconndefaults(); + if (!options) /* assume reason for failure is OOM */ + ereport(ERROR, + (errcode(ERRCODE_FDW_OUT_OF_MEMORY), + errmsg("out of memory"), + errdetail("Could not get libpq's default connection options."))); + }
It looks like `options` might be an unused variable in connect_pg_server()?
- if (PQstatus(conn) == CONNECTION_BAD) + if (!conn || PQstatus(conn) != CONNECTION_OK)
I don't think this should be changed in a refactoring patch.
PQstatus() can handle a NULL conn pointer.
- if (rconn)
- pfree(rconn);
Looks like this code in dblink_connect() was dropped.
Thanks!
--Jacob
Hi, thanks for all the reviews. Attached v3 with some fixes.
They also check for GSS delegation. I think SCRAM passthrough should
ideally be considered a second form of credentials delegation, from
the perspective of those functions.
Changed dblink_connstr_check and dblink_security_check to receive information
if scram passthrough is being used and then perform the validations. I just
don't know if the order that I put the checks on these functions is the better
or not, any input is welcome.
We would need to verify that the user mapping can't overwrite that
with its own (less trusted) `require_auth` setting. (I think that
should be true already, but I'm not 100% sure.)
When adding the require_auth on user mapping options we get an error `invalid
option "require_auth"`, but it was possible to add the require_auth on foreign
data wrapper server options. This new patch also add a validation on
dblink_connstr_check and dblink_security_check to ensure that require_auth is
is present on connection properties with scram-sha-256 as a value (we check only
the connstr so I think that perform the validation only on the first security
check function dblink_connstr_check would be enough, since I don't think that
the connstr will be changed after this check, but I added on both functions
just for sanity).
Hardcoding to scram-sha-256 would also prohibit the use of GSS or
standard password auth, now that I think about it. The docs currently
have a note about being able to choose... Should we add the other
permitted authentication types, i.e. `password,md5,scram-sha-256,gss`?
Or should we prohibit the use of other auth types if you've set
use_scram_passthrough? Or maybe there's an easier way to enforce the
use of the SCRAM keys, that better matches the current logic in
dblink_security_check?But would it be possible to use SCRAM pass-through feature using another auth
method?No, but if you want the same foreign server to be accessible by users
who log in with different authentication types, forcing a single
require_auth setting will defeat that. I don't have a strong opinion
about how important maintaining that functionality is, but the code
seems to allow it today.
The hard coded require_auth=scram-sha-256 will only be added if
use_scram_passthrough is set and also if the client that connect into the
server also uses the scram auth type, so that we can have the scram keys. So in
case the user try to authenticate using a different auth type on the same
foreign server that has use_scram_passthrough the require_auth=scram-sha-256
will not be added because MyProcPort->has_scram_keys is false.
--
Some thoughts on v2-0001:
I like the conceptual simplification of get_connect_string().
+ /* first time, allocate or get the custom wait event */ + if (dblink_we_connect == 0) + dblink_we_connect = WaitEventExtensionNew("DblinkConnectPgServer");Replacing two existing wait events with one new one is a user-facing
change (see the documented list of events at [1]). Maybe we want that,
but it hasn't been explained. I think that change should be made
independently of a refactoring patch (or else defended in the commit
message).
You're right, I've fixed it by changing the connect_pg_server to
receive the wait
event info.
+ if (foreign_server) + { + serverid = foreign_server->serverid; + user_mapping = GetUserMapping(userid, serverid); + + connstr = get_connect_string(foreign_server, user_mapping); + }Is there any other valid value for user_mapping that a caller can
choose? If not, I'd like to see the GetUserMapping call pushed back
down into get_connect_string(), unless there's another reason for
pulling it up that I'm missing.
I agree, I've just declared outside of get_connect_string because on 0002 we
also need the user mapping for UseScramPassthrough function, so I think that it
would make the review more easier.
+ useScramPassthrough = MyProcPort->has_scram_keys &&
UseScramPassthrough(foreign_server, user_mapping);
+ static const PQconninfoOption *options = NULL; + + /* + * Get list of valid libpq options. + * + * To avoid unnecessary work, we get the list once and use it throughout + * the lifetime of this backend process. We don't need to care about + * memory context issues, because PQconndefaults allocates with malloc. + */ + if (!options) + { + options = PQconndefaults(); + if (!options) /* assume reason for failure is OOM */ + ereport(ERROR, + (errcode(ERRCODE_FDW_OUT_OF_MEMORY), + errmsg("out of memory"), + errdetail("Could not get libpq's default connection options."))); + }It looks like `options` might be an unused variable in connect_pg_server()?
It's right. Fixed.
- if (PQstatus(conn) == CONNECTION_BAD) + if (!conn || PQstatus(conn) != CONNECTION_OK)I don't think this should be changed in a refactoring patch.
PQstatus() can handle a NULL conn pointer.
Fixed
- if (rconn)
- pfree(rconn);Looks like this code in dblink_connect() was dropped.
Oops, fixed on connect_pg_server since this logic was moved to this function.
## Some other changes
I also added a new TAP test case to ensure that we return an error if
require_auth is overwritten with another value.
## Questions:
- The new dblink_connstr_has_scam_require_auth function is very similar with
dblink_connstr_has_pw, we may create a common function for these or let it
duplicated?
--
Matheus Alcantara
Attachments:
v3-0001-dblink-refactor-get-connection-routines.patchapplication/octet-stream; name=v3-0001-dblink-refactor-get-connection-routines.patchDownload
From 1c49e71cbe9aa952932695da8b4ab800ba68d8db Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 20 Jan 2025 15:25:51 -0300
Subject: [PATCH v3 1/2] dblink: refactor get connection routines
Refactor dblink_get_conn and dblink_connect to move the logic of
actually opening the connection to the new connect_pg_server function
which them can be re-used on both functions.
This is a pre-work for a next commit that will add support for scram
pass-through authentication to dblink which will be able to implement
most of the logic into the connect_pg_server function which now already
have all necessary data information.
---
contrib/dblink/dblink.c | 200 ++++++++++++++++++++--------------------
1 file changed, 98 insertions(+), 102 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index bed2dee3d72..412734dc34f 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -117,7 +117,7 @@ static bool dblink_connstr_has_pw(const char *connstr);
static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
-static char *get_connect_string(const char *servername);
+static char *get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping);
static char *escape_param_str(const char *str);
static void validate_pkattnums(Relation rel,
int2vector *pkattnums_arg, int32 pknumatts_arg,
@@ -127,6 +127,8 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
+
/* Global */
static remoteConn *pconn = NULL;
static HTAB *remoteConnHash = NULL;
@@ -201,33 +203,11 @@ dblink_get_conn(char *conname_or_str,
}
else
{
- const char *connstr;
-
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_get_conn == 0)
dblink_we_get_conn = WaitEventExtensionNew("DblinkGetConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_get_conn);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- char *msg = pchomp(PQerrorMessage(conn));
-
- libpqsrv_disconnect(conn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
- dblink_security_check(conn, rconn, connstr);
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_get_conn);
freeconn = true;
conname = NULL;
}
@@ -272,9 +252,7 @@ Datum
dblink_connect(PG_FUNCTION_ARGS)
{
char *conname_or_str = NULL;
- char *connstr = NULL;
char *connname = NULL;
- char *msg;
PGconn *conn = NULL;
remoteConn *rconn = NULL;
@@ -297,40 +275,11 @@ dblink_connect(PG_FUNCTION_ARGS)
rconn->newXactForCursor = false;
}
- /* first check for valid foreign data server */
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
-
- /* check password in connection string if not superuser */
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_connect == 0)
dblink_we_connect = WaitEventExtensionNew("DblinkConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_connect);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- msg = pchomp(PQerrorMessage(conn));
- libpqsrv_disconnect(conn);
- if (rconn)
- pfree(rconn);
-
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
-
- /* check password actually used if not superuser */
- dblink_security_check(conn, rconn, connstr);
-
- /* attempt to set client encoding to match server encoding, if needed */
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_connect);
if (connname)
{
@@ -2784,15 +2733,17 @@ dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
* Obtain connection string for a foreign server
*/
static char *
-get_connect_string(const char *servername)
+get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
{
- ForeignServer *foreign_server = NULL;
- UserMapping *user_mapping;
ListCell *cell;
StringInfoData buf;
ForeignDataWrapper *fdw;
AclResult aclresult;
- char *srvname;
+
+ /* first gather the server connstr options */
+ Oid serverid = foreign_server->serverid;
+ Oid fdwid = foreign_server->fdwid;
+ Oid userid = GetUserId();
static const PQconninfoOption *options = NULL;
@@ -2815,57 +2766,42 @@ get_connect_string(const char *servername)
errdetail("Could not get libpq's default connection options.")));
}
- /* first gather the server connstr options */
- srvname = pstrdup(servername);
- truncate_identifier(srvname, strlen(srvname), false);
- foreign_server = GetForeignServerByName(srvname, true);
+ fdw = GetForeignDataWrapper(fdwid);
- if (foreign_server)
- {
- Oid serverid = foreign_server->serverid;
- Oid fdwid = foreign_server->fdwid;
- Oid userid = GetUserId();
-
- user_mapping = GetUserMapping(userid, serverid);
- fdw = GetForeignDataWrapper(fdwid);
-
- /* Check permissions, user must have usage on the server. */
- aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
- if (aclresult != ACLCHECK_OK)
- aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
-
- foreach(cell, fdw->options)
- {
- DefElem *def = lfirst(cell);
+ /* Check permissions, user must have usage on the server. */
+ aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
+ if (aclresult != ACLCHECK_OK)
+ aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
- if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, fdw->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, foreign_server->options)
- {
- DefElem *def = lfirst(cell);
+ if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, user_mapping->options)
- {
+ if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- DefElem *def = lfirst(cell);
+ foreach(cell, user_mapping->options)
+ {
- if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ DefElem *def = lfirst(cell);
- return buf.data;
+ if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
}
- else
- return NULL;
+
+ return buf.data;
}
/*
@@ -3087,3 +3023,63 @@ restoreLocalGucs(int nestlevel)
if (nestlevel > 0)
AtEOXact_GUC(true, nestlevel);
}
+
+/*
+ * Connect to remote server. If connstr_or_srvname maps to a foreign server,
+ * the associated properties and user mapping properties is also used to open
+ * the connection. Otherwise a connection will be open using the raw
+ * connstr_or_srvname value.
+ */
+static PGconn *
+connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info)
+{
+ PGconn *conn;
+ ForeignServer *foreign_server = NULL;
+ const char *connstr;
+ char *srvname;
+ Oid serverid;
+ UserMapping *user_mapping;
+ Oid userid = GetUserId();
+
+ /* first gather the server connstr options */
+ srvname = pstrdup(connstr_or_srvname);
+ truncate_identifier(srvname, strlen(srvname), false);
+ foreign_server = GetForeignServerByName(srvname, true);
+
+ if (foreign_server)
+ {
+ serverid = foreign_server->serverid;
+ user_mapping = GetUserMapping(userid, serverid);
+
+ connstr = get_connect_string(foreign_server, user_mapping);
+ }
+ else
+ connstr = connstr_or_srvname;
+
+ dblink_connstr_check(connstr);
+
+ /* OK to make connection */
+ conn = libpqsrv_connect(connstr, wait_event_info);
+
+ if (PQstatus(conn) == CONNECTION_BAD)
+ {
+ char *msg = pchomp(PQerrorMessage(conn));
+
+ libpqsrv_disconnect(conn);
+ if (rconn)
+ pfree(rconn);
+
+ ereport(ERROR,
+ (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
+ errmsg("could not establish connection"),
+ errdetail_internal("%s", msg)));
+ }
+
+ dblink_security_check(conn, rconn, connstr);
+
+ /* attempt to set client encoding to match server encoding, if needed */
+ if (PQclientEncoding(conn) != GetDatabaseEncoding())
+ PQsetClientEncoding(conn, GetDatabaseEncodingName());
+
+ return conn;
+}
--
2.39.5 (Apple Git-154)
v3-0002-dblink-Add-SCRAM-pass-through-authentication.patchapplication/octet-stream; name=v3-0002-dblink-Add-SCRAM-pass-through-authentication.patchDownload
From ffde1ab0f710a1b7a91195fefdb4aad74eccbf40 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Thu, 20 Feb 2025 15:12:48 -0300
Subject: [PATCH v3 2/2] dblink: Add SCRAM pass-through authentication
This enables SCRAM authentication for dblink_fdw when connecting to a
foreign server using the same approach as it was implemented for
postgres_fdw on 761c79508e.
---
contrib/dblink/Makefile | 1 +
contrib/dblink/dblink.c | 185 ++++++++++++++++++++++--
contrib/dblink/meson.build | 5 +
contrib/dblink/t/001_auth_scram.pl | 220 +++++++++++++++++++++++++++++
doc/src/sgml/dblink.sgml | 81 ++++++++++-
5 files changed, 477 insertions(+), 15 deletions(-)
create mode 100644 contrib/dblink/t/001_auth_scram.pl
diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index d4c7ed625ab..fde0b49ddbb 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -13,6 +13,7 @@ PGFILEDESC = "dblink - connect to other PostgreSQL databases"
REGRESS = dblink
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
+TAP_TESTS = 1
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 412734dc34f..fc7fb56b03e 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -43,6 +43,8 @@
#include "catalog/pg_foreign_server.h"
#include "catalog/pg_type.h"
#include "catalog/pg_user_mapping.h"
+#include "commands/defrem.h"
+#include "common/base64.h"
#include "executor/spi.h"
#include "foreign/foreign.h"
#include "funcapi.h"
@@ -112,9 +114,9 @@ static int get_attnum_pk_pos(int *pkattnums, int pknumatts, int key);
static HeapTuple get_tuple_of_interest(Relation rel, int *pkattnums, int pknumatts, char **src_pkattvals);
static Relation get_rel_from_relname(text *relname_text, LOCKMODE lockmode, AclMode aclmode);
static char *generate_relation_name(Relation rel);
-static void dblink_connstr_check(const char *connstr);
+static void dblink_connstr_check(const char *connstr, bool useScramPassthrough);
static bool dblink_connstr_has_pw(const char *connstr);
-static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
+static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr, bool useScramPassthrough);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
static char *get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping);
@@ -129,6 +131,20 @@ static void restoreLocalGucs(int nestlevel);
static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
+static bool
+ UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+
+static void
+ appendSCRAMKeysInfo(StringInfo buf);
+
+static bool
+ is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context);
+
+
+static bool
+ dblink_connstr_has_scram_require_auth(const char *connstr);
+
/* Global */
static remoteConn *pconn = NULL;
static HTAB *remoteConnHash = NULL;
@@ -1915,7 +1931,7 @@ dblink_fdw_validator(PG_FUNCTION_ARGS)
{
DefElem *def = (DefElem *) lfirst(cell);
- if (!is_valid_dblink_option(options, def->defname, context))
+ if (!is_valid_dblink_fdw_option(options, def->defname, context))
{
/*
* Unknown option, or invalid option for the context specified, so
@@ -2547,14 +2563,57 @@ deleteConnection(const char *name)
errmsg("undefined connection name")));
}
+bool
+dblink_connstr_has_scram_require_auth(const char *connstr)
+{
+ PQconninfoOption *options;
+ PQconninfoOption *option;
+ bool result = false;
+
+ options = PQconninfoParse(connstr, NULL);
+ if (options)
+ {
+ for (option = options; option->keyword != NULL; option++)
+ {
+ if (strcmp(option->keyword, "require_auth") == 0)
+ {
+ /*
+ * Continue iterating even if we found to make sure that there
+ * is no other declaration of require_auth that can overwrite
+ * the first.
+ */
+ if (option->val != NULL && strcmp(option->val, "scram-sha-256") == 0)
+ result = true;
+ else
+ result = false;
+ }
+ }
+ PQconninfoFree(options);
+ }
+
+ return result;
+}
+
/*
* We need to make sure that the connection made used credentials
* which were provided by the user, so check what credentials were
* used to connect and then make sure that they came from the user.
*/
static void
-dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
+dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr, bool useScramPassthrough)
{
+
+ if (useScramPassthrough)
+ {
+ if (dblink_connstr_has_scram_require_auth(connstr))
+ return;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_S_R_E_PROHIBITED_SQL_STATEMENT_ATTEMPTED),
+ errmsg("use_scram_passthrough can only be used with require_auth='scram-sha-256'"),
+ errhint("Ensure that there is not invalid require_auth on foreign server options")));
+ }
+
/* Superuser bypasses security check */
if (superuser())
return;
@@ -2615,16 +2674,29 @@ dblink_connstr_has_pw(const char *connstr)
}
/*
- * For non-superusers, insist that the connstr specify a password, except
- * if GSSAPI credentials have been delegated (and we check that they are used
- * for the connection in dblink_security_check later). This prevents a
- * password or GSSAPI credentials from being picked up from .pgpass, a
- * service file, the environment, etc. We don't want the postgres user's
- * passwords or Kerberos credentials to be accessible to non-superusers.
+ * For non-superusers, insist that the connstr specify a password, except if
+ * GSSAPI credentials have been delegated (and we check that they are used for
+ * the connection in dblink_security_check later) or if scram pass-through is
+ * being used. This prevents a password or GSSAPI credentials from being
+ * picked up from .pgpass, a service file, the environment, etc. We don't want
+ * the postgres user's passwords or Kerberos credentials to be accessible to
+ * non-superusers. In case of scram pass-through insist that the connstr
+ * specify require_auth=scram-sha-256 for a secure connection.
*/
static void
-dblink_connstr_check(const char *connstr)
+dblink_connstr_check(const char *connstr, bool useScramPassthrough)
{
+ if (useScramPassthrough)
+ {
+ if (dblink_connstr_has_scram_require_auth(connstr))
+ return;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_S_R_E_PROHIBITED_SQL_STATEMENT_ATTEMPTED),
+ errmsg("use_scram_passthrough can only be used with require_auth='scram-sha-256'"),
+ errhint("Ensure that there is not invalid require_auth on foreign server options")));
+ }
+
if (superuser())
return;
@@ -2773,6 +2845,9 @@ get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
+ appendSCRAMKeysInfo(&buf);
+
foreach(cell, fdw->options)
{
DefElem *def = lfirst(cell);
@@ -2954,6 +3029,22 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
+
+/*
+ * Same as is_valid_dblink_option but also check for only dblink_fdw specific
+ * options.
+ */
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context)
+{
+ if (strcmp(option, "use_scram_passthrough") == 0)
+ return true;
+
+ return is_valid_dblink_option(options, option, context);
+}
+
+
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3024,6 +3115,70 @@ restoreLocalGucs(int nestlevel)
AtEOXact_GUC(true, nestlevel);
}
+/*
+ * Append SCRAM client key and server key information from the global
+ * MyProcPort into the given StringInfo buffer.
+ */
+static void
+appendSCRAMKeysInfo(StringInfo buf)
+{
+ int len;
+ int encoded_len;
+ char *client_key;
+ char *server_key;
+
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
+ /* don't forget the zero-terminator */
+ client_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ClientKey,
+ sizeof(MyProcPort->scram_ClientKey),
+ client_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM client key");
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ServerKey));
+ /* don't forget the zero-terminator */
+ server_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ServerKey,
+ sizeof(MyProcPort->scram_ServerKey),
+ server_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM server key");
+
+ appendStringInfo(buf, "scram_client_key='%s'", client_key);
+ appendStringInfo(buf, "scram_server_key='%s'", server_key);
+ appendStringInfo(buf, "require_auth='scram-sha-256'");
+
+ pfree(client_key);
+ pfree(server_key);
+}
+
+
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user)
+{
+ ListCell *cell;
+
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ foreach(cell, user->options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ return false;
+}
+
/*
* Connect to remote server. If connstr_or_srvname maps to a foreign server,
* the associated properties and user mapping properties is also used to open
@@ -3040,6 +3195,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
Oid serverid;
UserMapping *user_mapping;
Oid userid = GetUserId();
+ bool useScramPassthrough = false;
/* first gather the server connstr options */
srvname = pstrdup(connstr_or_srvname);
@@ -3050,13 +3206,15 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
{
serverid = foreign_server->serverid;
user_mapping = GetUserMapping(userid, serverid);
+ useScramPassthrough = MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping);
connstr = get_connect_string(foreign_server, user_mapping);
}
else
connstr = connstr_or_srvname;
- dblink_connstr_check(connstr);
+ /* Verify the set of connection parameters. */
+ dblink_connstr_check(connstr, useScramPassthrough);
/* OK to make connection */
conn = libpqsrv_connect(connstr, wait_event_info);
@@ -3075,7 +3233,8 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
errdetail_internal("%s", msg)));
}
- dblink_security_check(conn, rconn, connstr);
+ /* Perform post-connection security checks. */
+ dblink_security_check(conn, rconn, connstr, useScramPassthrough);
/* attempt to set client encoding to match server encoding, if needed */
if (PQclientEncoding(conn) != GetDatabaseEncoding())
diff --git a/contrib/dblink/meson.build b/contrib/dblink/meson.build
index 3ab78668288..dfd8eb6877e 100644
--- a/contrib/dblink/meson.build
+++ b/contrib/dblink/meson.build
@@ -36,4 +36,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/001_auth_scram.pl',
+ ],
+ },
}
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
new file mode 100644
index 00000000000..ac646d2a54d
--- /dev/null
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -0,0 +1,220 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+# Test SCRAM authentication when opening a new connection with a foreign
+# server.
+#
+# The test is executed by testing the SCRAM authentifcation on a looplback
+# connection on the same server and with different servers.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $hostaddr = '127.0.0.1';
+my $user = "user01";
+
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
+my $fdw_server = "db1_fdw";
+my $fdw_server2 = "db2_fdw";
+my $fdw_invalid_server = "db2_fdw_invalid"; # For invalid fdw options
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+my $node2 = PostgreSQL::Test::Cluster->new('node2');
+
+$node1->init;
+$node2->init;
+
+$node1->start;
+$node2->start;
+
+# Test setup
+
+$node1->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$node2->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$ENV{PGPASSWORD} = "pass";
+
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db0');
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db1');
+$node2->safe_psql('postgres', qq'CREATE DATABASE $db2');
+
+setup_table($node1, $db1, "t");
+setup_table($node2, $db2, "t2");
+
+$node1->safe_psql($db0, 'CREATE EXTENSION IF NOT EXISTS dblink');
+setup_fdw_server($node1, $db0, $fdw_server, $node1, $db1);
+setup_fdw_server($node1, $db0, $fdw_server2, $node2, $db2);
+setup_invalid_fdw_server($node1, $db0, $fdw_invalid_server, $node2, $db2);
+
+setup_user_mapping($node1, $db0, $fdw_server);
+setup_user_mapping($node1, $db0, $fdw_server2);
+setup_user_mapping($node1, $db0, $fdw_invalid_server);
+
+# Make the user have the same SCRAM key on both servers. Forcing to have the
+# same iteration and salt.
+my $rolpassword = $node1->safe_psql('postgres',
+ qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';");
+$node2->safe_psql('postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'");
+
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all scram-sha-256
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all scram-sha-256
+});
+
+$node1->restart;
+$node2->restart;
+
+# End of test setup
+
+test_fdw_auth($node1, $db0, "t", $fdw_server,
+ "SCRAM auth on the same database cluster must succeed");
+
+test_fdw_auth($node1, $db0, "t2", $fdw_server2,
+ "SCRAM auth on a different database cluster must succeed");
+
+test_fdw_auth_with_invalid_overwritten_require_auth($fdw_invalid_server);
+
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all trust
+});
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on a different cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (different cluster)');
+
+# Helper functions
+
+sub test_fdw_auth
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $db, $tbl, $fdw, $testname) = @_;
+ my $connstr = $node->connstr($db) . qq' user=$user';
+
+ my $ret = $node->safe_psql(
+ $db,
+ qq"SELECT count(1) FROM dblink('$fdw', 'SELECT * FROM $tbl') AS $tbl(a int, b int)",
+ connstr => $connstr);
+
+ is($ret, '10', $testname);
+}
+
+sub test_fdw_auth_with_invalid_overwritten_require_auth
+{
+
+ my ($fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "select * from dblink('$fdw', 'select * from t') as t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+ is($ret, 3, 'loopback trust fails when overwriting require_auth');
+ like(
+ $stderr,
+ qr/use_scram_passthrough can only be used with require_auth='scram-sha-256'/,
+ 'expected error when connecting to a fdw overwriting the require_auth');
+}
+
+sub setup_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\');'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_invalid_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', require_auth \'none\');'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\') '
+ );
+}
+
+sub setup_invalid_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\', require_auth \'none\') '
+ );
+}
+
+sub setup_table
+{
+ my ($node, $db, $tbl) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE TABLE $tbl AS SELECT g as a, g + 1 as b FROM generate_series(1,10) g(g)'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON SCHEMA public TO $user');
+ $node->safe_psql($db, qq'GRANT SELECT ON $tbl TO $user');
+}
+
+done_testing();
+
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index 81f35986c88..f6e1009c028 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,6 +136,82 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
+<refsect1>
+ <title>Foreign Data Wrapper</title>
+
+ <para>
+ A Foreign Data Wrapper can be used as a connection name parameter. The foreign
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ </para>
+
+ <para>
+ The authentication with the foreign server can be via password on USER
+ MAPPING or using SCRAM pass-through. The
+ <literal>use_scram_passthrough</literal> on CREATE SERVER options controls
+ whether <filename>dblink_fdw</filename> will use the SCRAM pass-through
+ authentication to connect to the foreign server. With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
+ </para>
+
+ <para>
+ To use SCRAM pass-through authentication:
+ <itemizedlist>
+ <listitem>
+ <para>
+ The remote server must request SCRAM authentication. (If desired,
+ enforce this on the client side (FDW side) with the option
+ <literal>require_auth</literal>.) If another authentication method is
+ requested by the server, then that one will be used normally.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The remote server can be of any PostgreSQL version that supports SCRAM.
+ Support for <literal>use_scram_passthrough</literal> is only required on
+ the client side (FDW side).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The user mapping password is not used. (It could be set to support other
+ authentication methods, but that would arguably violate the point of this
+ feature, which is to avoid storing plain-text passwords.)
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The server running <filename>dblink_fdw</filename> and the remote server
+ must have identical SCRAM secrets (encrypted passwords) for the user being
+ used on <filename>dblink_fdw</filename> to authenticate on the foreign
+ server (same salt and iterations, not merely the same password).
+ </para>
+
+ <para>
+ As a corollary, if FDW connections to multiple hosts are to be made, for
+ example for partitioned foreign tables/sharding, then all hosts must have
+ identical SCRAM secrets for the users involved.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The current session on the PostgreSQL instance that makes the outgoing FDW
+ connections also must also use SCRAM authentication for its incoming client
+ connection. (Hence <quote>pass-through</quote>: SCRAM must be used going in
+ and out.) This is a technical requirement of the SCRAM protocol.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+
+ </refsect1>
+
<refsect1>
<title>Notes</title>
@@ -181,8 +257,9 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection must require password authentication for this to work properly
--- Otherwise, you will receive the following error from dblink_connect():
+-- Note: local connection that don't use SCRAM pass-through require password
+-- authentication for this to work properly. Otherwise, you will receive
+-- the following error from dblink_connect():
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
--
2.39.5 (Apple Git-154)
On Fri, Feb 21, 2025 at 6:48 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:
Hi, thanks for all the reviews. Attached v3 with some fixes.
Thanks! I keep getting pulled away from my review of 0002, so I'll
just comment on 0001 to get things moving again; sorry for the delay.
I agree, I've just declared outside of get_connect_string because on 0002 we
also need the user mapping for UseScramPassthrough function, so I think that it
would make the review more easier.
Ah, I missed that part of 0002. Works for me.
- if (rconn)
- pfree(rconn);Looks like this code in dblink_connect() was dropped.
Oops, fixed on connect_pg_server since this logic was moved to this function.
I think this fix may break the other usage in dblink_get_conn(), where
rconn comes from a hash entry. Maybe dblink_connect() should instead
put a PG_CATCH/pfree/PG_RE_THROW around the call to
connect_pg_server(), to ensure that the rconn allocation in
TopMemoryContext doesn't get leaked?
## Questions:
- The new dblink_connstr_has_scam_require_auth function is very similar with
dblink_connstr_has_pw, we may create a common function for these or let it
duplicated?
My preference would be to wait for a third [1]https://en.wikipedia.org/wiki/Rule_of_three_(computer_programming), but at the very least
I think the new function should go right next to the old one, to
highlight the similarity.
I have attached some stylistic suggestions, plus pgindent/pgperltidy
tweaks, as fixup commits 0002 and 0004.
Thanks,
--Jacob
[1]: https://en.wikipedia.org/wiki/Rule_of_three_(computer_programming)
Attachments:
v4-0001-dblink-refactor-get-connection-routines.patchapplication/octet-stream; name=v4-0001-dblink-refactor-get-connection-routines.patchDownload
From cf60bec55cace65088e1b49e65bab2dad081a8c0 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 20 Jan 2025 15:25:51 -0300
Subject: [PATCH v4 1/4] dblink: refactor get connection routines
Refactor dblink_get_conn and dblink_connect to move the logic of
actually opening the connection to the new connect_pg_server function
which them can be re-used on both functions.
This is a pre-work for a next commit that will add support for scram
pass-through authentication to dblink which will be able to implement
most of the logic into the connect_pg_server function which now already
have all necessary data information.
---
contrib/dblink/dblink.c | 200 ++++++++++++++++++++--------------------
1 file changed, 98 insertions(+), 102 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index bed2dee3d72..412734dc34f 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -117,7 +117,7 @@ static bool dblink_connstr_has_pw(const char *connstr);
static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
-static char *get_connect_string(const char *servername);
+static char *get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping);
static char *escape_param_str(const char *str);
static void validate_pkattnums(Relation rel,
int2vector *pkattnums_arg, int32 pknumatts_arg,
@@ -127,6 +127,8 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
+
/* Global */
static remoteConn *pconn = NULL;
static HTAB *remoteConnHash = NULL;
@@ -201,33 +203,11 @@ dblink_get_conn(char *conname_or_str,
}
else
{
- const char *connstr;
-
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_get_conn == 0)
dblink_we_get_conn = WaitEventExtensionNew("DblinkGetConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_get_conn);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- char *msg = pchomp(PQerrorMessage(conn));
-
- libpqsrv_disconnect(conn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
- dblink_security_check(conn, rconn, connstr);
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_get_conn);
freeconn = true;
conname = NULL;
}
@@ -272,9 +252,7 @@ Datum
dblink_connect(PG_FUNCTION_ARGS)
{
char *conname_or_str = NULL;
- char *connstr = NULL;
char *connname = NULL;
- char *msg;
PGconn *conn = NULL;
remoteConn *rconn = NULL;
@@ -297,40 +275,11 @@ dblink_connect(PG_FUNCTION_ARGS)
rconn->newXactForCursor = false;
}
- /* first check for valid foreign data server */
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
-
- /* check password in connection string if not superuser */
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_connect == 0)
dblink_we_connect = WaitEventExtensionNew("DblinkConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_connect);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- msg = pchomp(PQerrorMessage(conn));
- libpqsrv_disconnect(conn);
- if (rconn)
- pfree(rconn);
-
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
-
- /* check password actually used if not superuser */
- dblink_security_check(conn, rconn, connstr);
-
- /* attempt to set client encoding to match server encoding, if needed */
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_connect);
if (connname)
{
@@ -2784,15 +2733,17 @@ dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
* Obtain connection string for a foreign server
*/
static char *
-get_connect_string(const char *servername)
+get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
{
- ForeignServer *foreign_server = NULL;
- UserMapping *user_mapping;
ListCell *cell;
StringInfoData buf;
ForeignDataWrapper *fdw;
AclResult aclresult;
- char *srvname;
+
+ /* first gather the server connstr options */
+ Oid serverid = foreign_server->serverid;
+ Oid fdwid = foreign_server->fdwid;
+ Oid userid = GetUserId();
static const PQconninfoOption *options = NULL;
@@ -2815,57 +2766,42 @@ get_connect_string(const char *servername)
errdetail("Could not get libpq's default connection options.")));
}
- /* first gather the server connstr options */
- srvname = pstrdup(servername);
- truncate_identifier(srvname, strlen(srvname), false);
- foreign_server = GetForeignServerByName(srvname, true);
+ fdw = GetForeignDataWrapper(fdwid);
- if (foreign_server)
- {
- Oid serverid = foreign_server->serverid;
- Oid fdwid = foreign_server->fdwid;
- Oid userid = GetUserId();
-
- user_mapping = GetUserMapping(userid, serverid);
- fdw = GetForeignDataWrapper(fdwid);
-
- /* Check permissions, user must have usage on the server. */
- aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
- if (aclresult != ACLCHECK_OK)
- aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
-
- foreach(cell, fdw->options)
- {
- DefElem *def = lfirst(cell);
+ /* Check permissions, user must have usage on the server. */
+ aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
+ if (aclresult != ACLCHECK_OK)
+ aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
- if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, fdw->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, foreign_server->options)
- {
- DefElem *def = lfirst(cell);
+ if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, user_mapping->options)
- {
+ if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- DefElem *def = lfirst(cell);
+ foreach(cell, user_mapping->options)
+ {
- if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ DefElem *def = lfirst(cell);
- return buf.data;
+ if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
}
- else
- return NULL;
+
+ return buf.data;
}
/*
@@ -3087,3 +3023,63 @@ restoreLocalGucs(int nestlevel)
if (nestlevel > 0)
AtEOXact_GUC(true, nestlevel);
}
+
+/*
+ * Connect to remote server. If connstr_or_srvname maps to a foreign server,
+ * the associated properties and user mapping properties is also used to open
+ * the connection. Otherwise a connection will be open using the raw
+ * connstr_or_srvname value.
+ */
+static PGconn *
+connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info)
+{
+ PGconn *conn;
+ ForeignServer *foreign_server = NULL;
+ const char *connstr;
+ char *srvname;
+ Oid serverid;
+ UserMapping *user_mapping;
+ Oid userid = GetUserId();
+
+ /* first gather the server connstr options */
+ srvname = pstrdup(connstr_or_srvname);
+ truncate_identifier(srvname, strlen(srvname), false);
+ foreign_server = GetForeignServerByName(srvname, true);
+
+ if (foreign_server)
+ {
+ serverid = foreign_server->serverid;
+ user_mapping = GetUserMapping(userid, serverid);
+
+ connstr = get_connect_string(foreign_server, user_mapping);
+ }
+ else
+ connstr = connstr_or_srvname;
+
+ dblink_connstr_check(connstr);
+
+ /* OK to make connection */
+ conn = libpqsrv_connect(connstr, wait_event_info);
+
+ if (PQstatus(conn) == CONNECTION_BAD)
+ {
+ char *msg = pchomp(PQerrorMessage(conn));
+
+ libpqsrv_disconnect(conn);
+ if (rconn)
+ pfree(rconn);
+
+ ereport(ERROR,
+ (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
+ errmsg("could not establish connection"),
+ errdetail_internal("%s", msg)));
+ }
+
+ dblink_security_check(conn, rconn, connstr);
+
+ /* attempt to set client encoding to match server encoding, if needed */
+ if (PQclientEncoding(conn) != GetDatabaseEncoding())
+ PQsetClientEncoding(conn, GetDatabaseEncodingName());
+
+ return conn;
+}
--
2.34.1
v4-0002-fixup-dblink-refactor-get-connection-routines.patchapplication/octet-stream; name=v4-0002-fixup-dblink-refactor-get-connection-routines.patchDownload
From 0b37afe2e170da072e0393493553f4f26e8a8a18 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon, 3 Mar 2025 08:25:52 -0800
Subject: [PATCH v4 2/4] fixup! dblink: refactor get connection routines
---
contrib/dblink/dblink.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 412734dc34f..523548a02b6 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -126,7 +126,6 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
const char *option, Oid context);
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
-
static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
/* Global */
--
2.34.1
v4-0003-dblink-Add-SCRAM-pass-through-authentication.patchapplication/octet-stream; name=v4-0003-dblink-Add-SCRAM-pass-through-authentication.patchDownload
From d7b37951f37780253da8b2dde9afaf2945692397 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Thu, 20 Feb 2025 15:12:48 -0300
Subject: [PATCH v4 3/4] dblink: Add SCRAM pass-through authentication
This enables SCRAM authentication for dblink_fdw when connecting to a
foreign server using the same approach as it was implemented for
postgres_fdw on 761c79508e.
---
contrib/dblink/Makefile | 1 +
contrib/dblink/dblink.c | 185 ++++++++++++++++++++++--
contrib/dblink/meson.build | 5 +
contrib/dblink/t/001_auth_scram.pl | 220 +++++++++++++++++++++++++++++
doc/src/sgml/dblink.sgml | 81 ++++++++++-
5 files changed, 477 insertions(+), 15 deletions(-)
create mode 100644 contrib/dblink/t/001_auth_scram.pl
diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index d4c7ed625ab..fde0b49ddbb 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -13,6 +13,7 @@ PGFILEDESC = "dblink - connect to other PostgreSQL databases"
REGRESS = dblink
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
+TAP_TESTS = 1
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 523548a02b6..31e6c0e935c 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -43,6 +43,8 @@
#include "catalog/pg_foreign_server.h"
#include "catalog/pg_type.h"
#include "catalog/pg_user_mapping.h"
+#include "commands/defrem.h"
+#include "common/base64.h"
#include "executor/spi.h"
#include "foreign/foreign.h"
#include "funcapi.h"
@@ -112,9 +114,9 @@ static int get_attnum_pk_pos(int *pkattnums, int pknumatts, int key);
static HeapTuple get_tuple_of_interest(Relation rel, int *pkattnums, int pknumatts, char **src_pkattvals);
static Relation get_rel_from_relname(text *relname_text, LOCKMODE lockmode, AclMode aclmode);
static char *generate_relation_name(Relation rel);
-static void dblink_connstr_check(const char *connstr);
+static void dblink_connstr_check(const char *connstr, bool useScramPassthrough);
static bool dblink_connstr_has_pw(const char *connstr);
-static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
+static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr, bool useScramPassthrough);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
static char *get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping);
@@ -128,6 +130,20 @@ static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
+static bool
+ UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+
+static void
+ appendSCRAMKeysInfo(StringInfo buf);
+
+static bool
+ is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context);
+
+
+static bool
+ dblink_connstr_has_scram_require_auth(const char *connstr);
+
/* Global */
static remoteConn *pconn = NULL;
static HTAB *remoteConnHash = NULL;
@@ -1914,7 +1930,7 @@ dblink_fdw_validator(PG_FUNCTION_ARGS)
{
DefElem *def = (DefElem *) lfirst(cell);
- if (!is_valid_dblink_option(options, def->defname, context))
+ if (!is_valid_dblink_fdw_option(options, def->defname, context))
{
/*
* Unknown option, or invalid option for the context specified, so
@@ -2546,14 +2562,57 @@ deleteConnection(const char *name)
errmsg("undefined connection name")));
}
+bool
+dblink_connstr_has_scram_require_auth(const char *connstr)
+{
+ PQconninfoOption *options;
+ PQconninfoOption *option;
+ bool result = false;
+
+ options = PQconninfoParse(connstr, NULL);
+ if (options)
+ {
+ for (option = options; option->keyword != NULL; option++)
+ {
+ if (strcmp(option->keyword, "require_auth") == 0)
+ {
+ /*
+ * Continue iterating even if we found to make sure that there
+ * is no other declaration of require_auth that can overwrite
+ * the first.
+ */
+ if (option->val != NULL && strcmp(option->val, "scram-sha-256") == 0)
+ result = true;
+ else
+ result = false;
+ }
+ }
+ PQconninfoFree(options);
+ }
+
+ return result;
+}
+
/*
* We need to make sure that the connection made used credentials
* which were provided by the user, so check what credentials were
* used to connect and then make sure that they came from the user.
*/
static void
-dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
+dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr, bool useScramPassthrough)
{
+
+ if (useScramPassthrough)
+ {
+ if (dblink_connstr_has_scram_require_auth(connstr))
+ return;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_S_R_E_PROHIBITED_SQL_STATEMENT_ATTEMPTED),
+ errmsg("use_scram_passthrough can only be used with require_auth='scram-sha-256'"),
+ errhint("Ensure that there is not invalid require_auth on foreign server options")));
+ }
+
/* Superuser bypasses security check */
if (superuser())
return;
@@ -2614,16 +2673,29 @@ dblink_connstr_has_pw(const char *connstr)
}
/*
- * For non-superusers, insist that the connstr specify a password, except
- * if GSSAPI credentials have been delegated (and we check that they are used
- * for the connection in dblink_security_check later). This prevents a
- * password or GSSAPI credentials from being picked up from .pgpass, a
- * service file, the environment, etc. We don't want the postgres user's
- * passwords or Kerberos credentials to be accessible to non-superusers.
+ * For non-superusers, insist that the connstr specify a password, except if
+ * GSSAPI credentials have been delegated (and we check that they are used for
+ * the connection in dblink_security_check later) or if scram pass-through is
+ * being used. This prevents a password or GSSAPI credentials from being
+ * picked up from .pgpass, a service file, the environment, etc. We don't want
+ * the postgres user's passwords or Kerberos credentials to be accessible to
+ * non-superusers. In case of scram pass-through insist that the connstr
+ * specify require_auth=scram-sha-256 for a secure connection.
*/
static void
-dblink_connstr_check(const char *connstr)
+dblink_connstr_check(const char *connstr, bool useScramPassthrough)
{
+ if (useScramPassthrough)
+ {
+ if (dblink_connstr_has_scram_require_auth(connstr))
+ return;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_S_R_E_PROHIBITED_SQL_STATEMENT_ATTEMPTED),
+ errmsg("use_scram_passthrough can only be used with require_auth='scram-sha-256'"),
+ errhint("Ensure that there is not invalid require_auth on foreign server options")));
+ }
+
if (superuser())
return;
@@ -2772,6 +2844,9 @@ get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
+ appendSCRAMKeysInfo(&buf);
+
foreach(cell, fdw->options)
{
DefElem *def = lfirst(cell);
@@ -2953,6 +3028,22 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
+
+/*
+ * Same as is_valid_dblink_option but also check for only dblink_fdw specific
+ * options.
+ */
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context)
+{
+ if (strcmp(option, "use_scram_passthrough") == 0)
+ return true;
+
+ return is_valid_dblink_option(options, option, context);
+}
+
+
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3023,6 +3114,70 @@ restoreLocalGucs(int nestlevel)
AtEOXact_GUC(true, nestlevel);
}
+/*
+ * Append SCRAM client key and server key information from the global
+ * MyProcPort into the given StringInfo buffer.
+ */
+static void
+appendSCRAMKeysInfo(StringInfo buf)
+{
+ int len;
+ int encoded_len;
+ char *client_key;
+ char *server_key;
+
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
+ /* don't forget the zero-terminator */
+ client_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ClientKey,
+ sizeof(MyProcPort->scram_ClientKey),
+ client_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM client key");
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ServerKey));
+ /* don't forget the zero-terminator */
+ server_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ServerKey,
+ sizeof(MyProcPort->scram_ServerKey),
+ server_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM server key");
+
+ appendStringInfo(buf, "scram_client_key='%s'", client_key);
+ appendStringInfo(buf, "scram_server_key='%s'", server_key);
+ appendStringInfo(buf, "require_auth='scram-sha-256'");
+
+ pfree(client_key);
+ pfree(server_key);
+}
+
+
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user)
+{
+ ListCell *cell;
+
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ foreach(cell, user->options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ return false;
+}
+
/*
* Connect to remote server. If connstr_or_srvname maps to a foreign server,
* the associated properties and user mapping properties is also used to open
@@ -3039,6 +3194,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
Oid serverid;
UserMapping *user_mapping;
Oid userid = GetUserId();
+ bool useScramPassthrough = false;
/* first gather the server connstr options */
srvname = pstrdup(connstr_or_srvname);
@@ -3049,13 +3205,15 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
{
serverid = foreign_server->serverid;
user_mapping = GetUserMapping(userid, serverid);
+ useScramPassthrough = MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping);
connstr = get_connect_string(foreign_server, user_mapping);
}
else
connstr = connstr_or_srvname;
- dblink_connstr_check(connstr);
+ /* Verify the set of connection parameters. */
+ dblink_connstr_check(connstr, useScramPassthrough);
/* OK to make connection */
conn = libpqsrv_connect(connstr, wait_event_info);
@@ -3074,7 +3232,8 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
errdetail_internal("%s", msg)));
}
- dblink_security_check(conn, rconn, connstr);
+ /* Perform post-connection security checks. */
+ dblink_security_check(conn, rconn, connstr, useScramPassthrough);
/* attempt to set client encoding to match server encoding, if needed */
if (PQclientEncoding(conn) != GetDatabaseEncoding())
diff --git a/contrib/dblink/meson.build b/contrib/dblink/meson.build
index 3ab78668288..dfd8eb6877e 100644
--- a/contrib/dblink/meson.build
+++ b/contrib/dblink/meson.build
@@ -36,4 +36,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/001_auth_scram.pl',
+ ],
+ },
}
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
new file mode 100644
index 00000000000..ac646d2a54d
--- /dev/null
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -0,0 +1,220 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+# Test SCRAM authentication when opening a new connection with a foreign
+# server.
+#
+# The test is executed by testing the SCRAM authentifcation on a looplback
+# connection on the same server and with different servers.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $hostaddr = '127.0.0.1';
+my $user = "user01";
+
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
+my $fdw_server = "db1_fdw";
+my $fdw_server2 = "db2_fdw";
+my $fdw_invalid_server = "db2_fdw_invalid"; # For invalid fdw options
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+my $node2 = PostgreSQL::Test::Cluster->new('node2');
+
+$node1->init;
+$node2->init;
+
+$node1->start;
+$node2->start;
+
+# Test setup
+
+$node1->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$node2->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$ENV{PGPASSWORD} = "pass";
+
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db0');
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db1');
+$node2->safe_psql('postgres', qq'CREATE DATABASE $db2');
+
+setup_table($node1, $db1, "t");
+setup_table($node2, $db2, "t2");
+
+$node1->safe_psql($db0, 'CREATE EXTENSION IF NOT EXISTS dblink');
+setup_fdw_server($node1, $db0, $fdw_server, $node1, $db1);
+setup_fdw_server($node1, $db0, $fdw_server2, $node2, $db2);
+setup_invalid_fdw_server($node1, $db0, $fdw_invalid_server, $node2, $db2);
+
+setup_user_mapping($node1, $db0, $fdw_server);
+setup_user_mapping($node1, $db0, $fdw_server2);
+setup_user_mapping($node1, $db0, $fdw_invalid_server);
+
+# Make the user have the same SCRAM key on both servers. Forcing to have the
+# same iteration and salt.
+my $rolpassword = $node1->safe_psql('postgres',
+ qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';");
+$node2->safe_psql('postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'");
+
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all scram-sha-256
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all scram-sha-256
+});
+
+$node1->restart;
+$node2->restart;
+
+# End of test setup
+
+test_fdw_auth($node1, $db0, "t", $fdw_server,
+ "SCRAM auth on the same database cluster must succeed");
+
+test_fdw_auth($node1, $db0, "t2", $fdw_server2,
+ "SCRAM auth on a different database cluster must succeed");
+
+test_fdw_auth_with_invalid_overwritten_require_auth($fdw_invalid_server);
+
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all trust
+});
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on a different cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (different cluster)');
+
+# Helper functions
+
+sub test_fdw_auth
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $db, $tbl, $fdw, $testname) = @_;
+ my $connstr = $node->connstr($db) . qq' user=$user';
+
+ my $ret = $node->safe_psql(
+ $db,
+ qq"SELECT count(1) FROM dblink('$fdw', 'SELECT * FROM $tbl') AS $tbl(a int, b int)",
+ connstr => $connstr);
+
+ is($ret, '10', $testname);
+}
+
+sub test_fdw_auth_with_invalid_overwritten_require_auth
+{
+
+ my ($fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "select * from dblink('$fdw', 'select * from t') as t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+ is($ret, 3, 'loopback trust fails when overwriting require_auth');
+ like(
+ $stderr,
+ qr/use_scram_passthrough can only be used with require_auth='scram-sha-256'/,
+ 'expected error when connecting to a fdw overwriting the require_auth');
+}
+
+sub setup_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\');'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_invalid_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', require_auth \'none\');'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\') '
+ );
+}
+
+sub setup_invalid_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\', require_auth \'none\') '
+ );
+}
+
+sub setup_table
+{
+ my ($node, $db, $tbl) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE TABLE $tbl AS SELECT g as a, g + 1 as b FROM generate_series(1,10) g(g)'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON SCHEMA public TO $user');
+ $node->safe_psql($db, qq'GRANT SELECT ON $tbl TO $user');
+}
+
+done_testing();
+
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index 81f35986c88..f6e1009c028 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,6 +136,82 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
+<refsect1>
+ <title>Foreign Data Wrapper</title>
+
+ <para>
+ A Foreign Data Wrapper can be used as a connection name parameter. The foreign
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ </para>
+
+ <para>
+ The authentication with the foreign server can be via password on USER
+ MAPPING or using SCRAM pass-through. The
+ <literal>use_scram_passthrough</literal> on CREATE SERVER options controls
+ whether <filename>dblink_fdw</filename> will use the SCRAM pass-through
+ authentication to connect to the foreign server. With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
+ </para>
+
+ <para>
+ To use SCRAM pass-through authentication:
+ <itemizedlist>
+ <listitem>
+ <para>
+ The remote server must request SCRAM authentication. (If desired,
+ enforce this on the client side (FDW side) with the option
+ <literal>require_auth</literal>.) If another authentication method is
+ requested by the server, then that one will be used normally.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The remote server can be of any PostgreSQL version that supports SCRAM.
+ Support for <literal>use_scram_passthrough</literal> is only required on
+ the client side (FDW side).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The user mapping password is not used. (It could be set to support other
+ authentication methods, but that would arguably violate the point of this
+ feature, which is to avoid storing plain-text passwords.)
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The server running <filename>dblink_fdw</filename> and the remote server
+ must have identical SCRAM secrets (encrypted passwords) for the user being
+ used on <filename>dblink_fdw</filename> to authenticate on the foreign
+ server (same salt and iterations, not merely the same password).
+ </para>
+
+ <para>
+ As a corollary, if FDW connections to multiple hosts are to be made, for
+ example for partitioned foreign tables/sharding, then all hosts must have
+ identical SCRAM secrets for the users involved.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The current session on the PostgreSQL instance that makes the outgoing FDW
+ connections also must also use SCRAM authentication for its incoming client
+ connection. (Hence <quote>pass-through</quote>: SCRAM must be used going in
+ and out.) This is a technical requirement of the SCRAM protocol.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+
+ </refsect1>
+
<refsect1>
<title>Notes</title>
@@ -181,8 +257,9 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection must require password authentication for this to work properly
--- Otherwise, you will receive the following error from dblink_connect():
+-- Note: local connection that don't use SCRAM pass-through require password
+-- authentication for this to work properly. Otherwise, you will receive
+-- the following error from dblink_connect():
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
--
2.34.1
v4-0004-fixup-dblink-Add-SCRAM-pass-through-authenticatio.patchapplication/octet-stream; name=v4-0004-fixup-dblink-Add-SCRAM-pass-through-authenticatio.patchDownload
From d64971da6e717086eb64287112c118a05e588680 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon, 3 Mar 2025 08:25:52 -0800
Subject: [PATCH v4 4/4] fixup! dblink: Add SCRAM pass-through authentication
---
contrib/dblink/dblink.c | 19 +++++--------------
contrib/dblink/t/001_auth_scram.pl | 14 +++++++-------
doc/src/sgml/dblink.sgml | 4 ++--
3 files changed, 14 insertions(+), 23 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 31e6c0e935c..6908d70f718 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -129,20 +129,11 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
-
-static bool
- UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
-
-static void
- appendSCRAMKeysInfo(StringInfo buf);
-
-static bool
- is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+static bool UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+static void appendSCRAMKeysInfo(StringInfo buf);
+static bool is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
Oid context);
-
-
-static bool
- dblink_connstr_has_scram_require_auth(const char *connstr);
+static bool dblink_connstr_has_scram_require_auth(const char *connstr);
/* Global */
static remoteConn *pconn = NULL;
@@ -2584,7 +2575,7 @@ dblink_connstr_has_scram_require_auth(const char *connstr)
if (option->val != NULL && strcmp(option->val, "scram-sha-256") == 0)
result = true;
else
- result = false;
+ result = false;
}
}
PQconninfoFree(options);
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
index ac646d2a54d..15058ac7c9a 100644
--- a/contrib/dblink/t/001_auth_scram.pl
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -3,7 +3,7 @@
# Test SCRAM authentication when opening a new connection with a foreign
# server.
#
-# The test is executed by testing the SCRAM authentifcation on a looplback
+# The test is executed by testing the SCRAM authentifcation on a loopback
# connection on the same server and with different servers.
use strict;
@@ -12,15 +12,14 @@ use PostgreSQL::Test::Utils;
use PostgreSQL::Test::Cluster;
use Test::More;
-my $hostaddr = '127.0.0.1';
my $user = "user01";
-my $db0 = "db0"; # For node1
-my $db1 = "db1"; # For node1
-my $db2 = "db2"; # For node2
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
my $fdw_server = "db1_fdw";
my $fdw_server2 = "db2_fdw";
-my $fdw_invalid_server = "db2_fdw_invalid"; # For invalid fdw options
+my $fdw_invalid_server = "db2_fdw_invalid"; # For invalid fdw options
my $node1 = PostgreSQL::Test::Cluster->new('node1');
my $node2 = PostgreSQL::Test::Cluster->new('node2');
@@ -155,7 +154,8 @@ sub test_fdw_auth_with_invalid_overwritten_require_auth
like(
$stderr,
qr/use_scram_passthrough can only be used with require_auth='scram-sha-256'/,
- 'expected error when connecting to a fdw overwriting the require_auth');
+ 'expected error when connecting to a fdw overwriting the require_auth'
+ );
}
sub setup_user_mapping
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index f6e1009c028..e3b4129ae26 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -141,7 +141,7 @@ dblink_connect(text connname, text connstr) returns text
<para>
A Foreign Data Wrapper can be used as a connection name parameter. The foreign
- server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
</para>
<para>
@@ -257,7 +257,7 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection that don't use SCRAM pass-through require password
+-- Note: local connection that don't use SCRAM pass-through require password
-- authentication for this to work properly. Otherwise, you will receive
-- the following error from dblink_connect():
-- ERROR: password is required
--
2.34.1
On Mon, Mar 3, 2025 at 9:01 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
I keep getting pulled away from my review of 0002
Here's a review of v3-0002:
+dblink_connstr_check(const char *connstr, bool useScramPassthrough) { + if (useScramPassthrough) + { + if (dblink_connstr_has_scram_require_auth(connstr)) + return;
Can a comment be added somewhere to state that the security of this
check relies on scram_server_key and scram_client_key not coming from
the environment or any mapping options? The fact that those two
options are declared 1) without envvar names and 2) as debug options
is doing a lot of heavy security lifting, but it's hard to see that
from this part of the code.
Alternatively, this check could also verify that
scram_client_key/server_key are set in the connection string
explicitly.
It is still strange to me that we don't fall through to check other
potential safe options (see comment on the dblink_security_check,
below).
+ ... + } + if (superuser()) return;
For the additions to dblink_connstr_check/security_check, I think the
superuser checks should remain at the top. Superusers can still do
what they want.
+dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr, bool useScramPassthrough) { + + if (useScramPassthrough) + { + if (dblink_connstr_has_scram_require_auth(connstr)) + return;
I don't think this check should be the same as the connstr check
above, but I don't have a concrete example of what it should do
instead. require_auth is taking care of the accidental trust config...
Should there be an API that checks that the server and client key were
used during the SCRAM exchange, similar to PQconnectionUsedPassword()?
Would that even add anything?
This division between "connstr check" and "security check" is easier
to describe when we allow a variety of safe options, and check to see
if any of them have been used. use_scram_passthrough locks it down to
one possible option, making this division a little muddier.
+ appendStringInfo(buf, "scram_client_key='%s'", client_key); + appendStringInfo(buf, "scram_server_key='%s'", server_key); + appendStringInfo(buf, "require_auth='scram-sha-256'");
These should have spaces between them; i.e. "scram_client_key='%s' ".
Thanks,
--Jacob
On 11.02.25 00:19, Jacob Champion wrote:
These don't seem right to me. SCRAM passthrough should be considered
as_part_ of the connstr/security checks, but I think it should not
_bypass_ those checks. We have to enforce the use of the SCRAM
credentials on the remote for safety, similarly to GSS delegation.
(This might be a good place for `require_auth=scram-sha-256`?)I've attached a failing test to better illustrate what I mean.
It looks like the postgres_fdw patch that landed also performs a
bypass -- I think that may need an open item to fix? CC'd Peter.
AFAICT, in pgfdw_security_check(), if SCRAM has been used for the
outgoing server connection, then PQconnectionUsedPassword() is true, and
then this check should fail if no "password" parameter was given. That
check should be expanded to allow alternatively passing the SCRAM key
component parameters.
But that would mean the check is too restrictive, while you are
apparently claiming that the check is not restrictive enough?
(Also, this would appear to mean the current SCRAM pass-through code in
postgres_fdw should mostly not work, but the tests work, so I'm confused.)
On Thu, Mar 6, 2025 at 12:33 PM Peter Eisentraut <peter@eisentraut.org> wrote:
AFAICT, in pgfdw_security_check(), if SCRAM has been used for the
outgoing server connection, then PQconnectionUsedPassword() is true, and
then this check should fail if no "password" parameter was given. That
check should be expanded to allow alternatively passing the SCRAM key
component parameters.
pgfdw_security_check() is currently not called if SCRAM passthrough is
in use, though:
/*
* Perform post-connection security checks only if scram pass-through
* is not being used because the password is not necessary.
*/
if (!(MyProcPort->has_scram_keys && UseScramPassthrough(server, user)))
pgfdw_security_check(keywords, values, user, conn);
--Jacob
On 06.03.25 22:58, Jacob Champion wrote:
On Thu, Mar 6, 2025 at 12:33 PM Peter Eisentraut <peter@eisentraut.org> wrote:
AFAICT, in pgfdw_security_check(), if SCRAM has been used for the
outgoing server connection, then PQconnectionUsedPassword() is true, and
then this check should fail if no "password" parameter was given. That
check should be expanded to allow alternatively passing the SCRAM key
component parameters.pgfdw_security_check() is currently not called if SCRAM passthrough is
in use, though:/*
* Perform post-connection security checks only if scram pass-through
* is not being used because the password is not necessary.
*/
if (!(MyProcPort->has_scram_keys && UseScramPassthrough(server, user)))
pgfdw_security_check(keywords, values, user, conn);
Right. How about the attached? It checks as an alternative to a
password whether the SCRAM keys were provided. That should get us back
to the same level of checking?
Attachments:
0001-WIP-postgres_fdw-Fix-SCRAM-pass-through-security.patch.nocfbottext/plain; charset=UTF-8; name=0001-WIP-postgres_fdw-Fix-SCRAM-pass-through-security.patch.nocfbotDownload
From daa5ff65007ed1cef49020191b50abf226228d95 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <peter@eisentraut.org>
Date: Fri, 7 Mar 2025 17:20:33 +0100
Subject: [PATCH] WIP: postgres_fdw: Fix SCRAM pass-through security
---
contrib/postgres_fdw/connection.c | 41 ++++++++++++++++++++-----------
1 file changed, 27 insertions(+), 14 deletions(-)
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 8ef9702c05c..5a069b54078 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -447,12 +447,23 @@ pgfdw_security_check(const char **keywords, const char **values, UserMapping *us
/* Connected via PW, with PW required true, and provided non-empty PW. */
if (PQconnectionUsedPassword(conn))
{
- /* ok if params contain a non-empty password */
+ bool has_scram_server_key = false;
+ bool has_scram_client_key = false;
+
+ /* ok if params contain a non-empty password or SCRAM keys */
for (int i = 0; keywords[i] != NULL; i++)
{
if (strcmp(keywords[i], "password") == 0 && values[i][0] != '\0')
return;
+
+ if (strcmp(keywords[i], "scram_client_key") == 0 && values[i][0] != '\0')
+ has_scram_client_key = true;
+ if (strcmp(keywords[i], "scram_server_key") == 0 && values[i][0] != '\0')
+ has_scram_server_key = true;
}
+
+ if (has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys)
+ return;
}
ereport(ERROR,
@@ -586,12 +597,8 @@ connect_pg_server(ForeignServer *server, UserMapping *user)
keywords[n] = values[n] = NULL;
- /*
- * Verify the set of connection parameters only if scram pass-through
- * is not being used because the password is not necessary.
- */
- if (!(MyProcPort->has_scram_keys && UseScramPassthrough(server, user)))
- check_conn_params(keywords, values, user);
+ /* verify the set of connection parameters */
+ check_conn_params(keywords, values, user);
/* first time, allocate or get the custom wait event */
if (pgfdw_we_connect == 0)
@@ -609,12 +616,8 @@ connect_pg_server(ForeignServer *server, UserMapping *user)
server->servername),
errdetail_internal("%s", pchomp(PQerrorMessage(conn)))));
- /*
- * Perform post-connection security checks only if scram pass-through
- * is not being used because the password is not necessary.
- */
- if (!(MyProcPort->has_scram_keys && UseScramPassthrough(server, user)))
- pgfdw_security_check(keywords, values, user, conn);
+ /* Perform post-connection security checks */
+ pgfdw_security_check(keywords, values, user, conn);
/* Prepare new session for use */
configure_remote_session(conn);
@@ -703,6 +706,8 @@ static void
check_conn_params(const char **keywords, const char **values, UserMapping *user)
{
int i;
+ bool has_scram_server_key = false;
+ bool has_scram_client_key = false;
/* no check required if superuser */
if (superuser_arg(user->userid))
@@ -714,13 +719,21 @@ check_conn_params(const char **keywords, const char **values, UserMapping *user)
return;
#endif
- /* ok if params contain a non-empty password */
+ /* ok if params contain a non-empty password or SCRAM keys */
for (i = 0; keywords[i] != NULL; i++)
{
if (strcmp(keywords[i], "password") == 0 && values[i][0] != '\0')
return;
+
+ if (strcmp(keywords[i], "scram_client_key") == 0 && values[i][0] != '\0')
+ has_scram_client_key = true;
+ if (strcmp(keywords[i], "scram_server_key") == 0 && values[i][0] != '\0')
+ has_scram_server_key = true;
}
+ if (has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys)
+ return;
+
/* ok if the superuser explicitly said so at user mapping creation time */
if (!UserMappingPasswordRequired(user))
return;
--
2.48.1
On Fri, Mar 7, 2025 at 8:22 AM Peter Eisentraut <peter@eisentraut.org> wrote:
Right. How about the attached? It checks as an alternative to a
password whether the SCRAM keys were provided. That should get us back
to the same level of checking?
Yes, I think so. Attached is a set of tests to illustrate, mirroring
the dblink tests added upthread; they fail without this patch.
I like that this solution addresses some of the concerns from my dblink review.
--
Not part of this patchset, but I think the errmsg in
pgfdw_security_check() is confusing:
ERROR: password or GSSAPI delegated credentials required
DETAIL: Non-superuser cannot connect if the server does not
request a password or...
HINT: Target server's authentication method must be changed or...
For the user to have gotten past check_conn_params, they *have*
provided a password/credentials. But the server didn't ask for it (or
at least, not the right one). The detail and hint messages are correct
here, but I'd argue the error message itself is not.
Thanks!
--Jacob
Attachments:
fdw-test.diff.txttext/plain; charset=US-ASCII; name=fdw-test.diff.txtDownload
diff --git a/contrib/postgres_fdw/t/001_auth_scram.pl b/contrib/postgres_fdw/t/001_auth_scram.pl
index 047840cc914..60d46ebc665 100644
--- a/contrib/postgres_fdw/t/001_auth_scram.pl
+++ b/contrib/postgres_fdw/t/001_auth_scram.pl
@@ -68,6 +68,45 @@ test_fdw_auth($node1, $db0, "t2", $fdw_server2,
test_auth($node2, $db2, "t2",
"SCRAM auth directly on foreign server should still succeed");
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all trust
+});
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT count(1) FROM t",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT count(1) FROM t2",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on a different cluster');
+like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error from loopback trust (different cluster)');
+
# Helper functions
sub test_auth
On Mon, Mar 10, 2025 at 11:25 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
On Fri, Mar 7, 2025 at 8:22 AM Peter Eisentraut <peter@eisentraut.org> wrote:
Right. How about the attached? It checks as an alternative to a
password whether the SCRAM keys were provided. That should get us back
to the same level of checking?Yes, I think so. Attached is a set of tests to illustrate, mirroring
the dblink tests added upthread; they fail without this patch.
In an offline discussion with Peter and Matheus, we figured out that
this is still not enough. The latest patch checks that a password was
used, but it doesn't ensure that the password material came from the
SCRAM keys. Attached is an updated test to illustrate.
Thanks,
--Jacob
Attachments:
fdw-test-v2.diff.txttext/plain; charset=US-ASCII; name=fdw-test-v2.diff.txtDownload
commit 4a41754eaa41f2db285e68ff8140d6932c299358
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon Mar 10 11:18:27 2025 -0700
WIP
diff --git a/contrib/postgres_fdw/t/001_auth_scram.pl b/contrib/postgres_fdw/t/001_auth_scram.pl
index 047840cc914..464492948b4 100644
--- a/contrib/postgres_fdw/t/001_auth_scram.pl
+++ b/contrib/postgres_fdw/t/001_auth_scram.pl
@@ -68,6 +68,45 @@ test_fdw_auth($node1, $db0, "t2", $fdw_server2,
test_auth($node2, $db2, "t2",
"SCRAM auth directly on foreign server should still succeed");
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local all all password
+});
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT count(1) FROM t",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT count(1) FROM t2",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback password fails on a different cluster');
+like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error from loopback password (different cluster)');
+
# Helper functions
sub test_auth
Hi, thanks for all the comments! Attached v5 with some fixes
(I'll answer comments for different emails on this since most of them is
fixed on this new v5 version)
I think this fix may break the other usage in dblink_get_conn(), where
rconn comes from a hash entry. Maybe dblink_connect() should instead
put a PG_CATCH/pfree/PG_RE_THROW around the call to
connect_pg_server(), to ensure that the rconn allocation in
TopMemoryContext doesn't get leaked?
Fixed by wrapping on PG_CATCH/pfree/PG_RE_THROW. I didn't managed to
create a test that use this code path, so let me know if I'm still
missing something.
Can a comment be added somewhere to state that the security of this
check relies on scram_server_key and scram_client_key not coming from
the environment or any mapping options? The fact that those two
options are declared 1) without envvar names and 2) as debug options
is doing a lot of heavy security lifting, but it's hard to see that
from this part of the code.
I've added a code comment on dblink_connstr_has_required_scram_options
function which is called on "connstr check" and "security check". Please
let me know what you think.
Alternatively, this check could also verify that
scram_client_key/server_key are set in the connection string
explicitly.
I've added this validation on dblink_connstr_has_required_scram_options.
For the additions to dblink_connstr_check/security_check, I think the
superuser checks should remain at the top. Superusers can still do
what they want.
Fixed
I don't think this check should be the same as the connstr check
above, but I don't have a concrete example of what it should do
instead. require_auth is taking care of the accidental trust config...
Should there be an API that checks that the server and client key were
used during the SCRAM exchange, similar to PQconnectionUsedPassword()?
Would that even add anything?
I was also thinking about this, maybe we could add a new validation
(similar with PQconnectionUsedPassword, on fe-connect.c) that check if
the scram keys is set on PGconn (we only set these keys if we are
actually using the scram pass-through feature)
+int
+PQconnectionUsedScramKeys(const PGconn *conn)
+{
+ if (conn->scram_client_key && conn->scram_server_key)
+ return true;
+
+ return false;
+}
And then call on dblink_security_check
- if (MyProcPort->has_scram_keys &&
dblink_connstr_has_required_scram_options(connstr))
+ if (MyProcPort->has_scram_keys
+ && dblink_connstr_has_required_scram_options(connstr)
+ && PQconnectionUsedScramKeys(conn))
return;
(Note that I didn't implement this on this new patch version, I'm just
sharing some ideas that I had during development.)
These should have spaces between them; i.e. "scram_client_key='%s' ".
Fixed
On Fri, Mar 7, 2025 at 8:22 AM Peter Eisentraut <peter@eisentraut.org> wrote:
Right. How about the attached? It checks as an alternative to a
password whether the SCRAM keys were provided. That should get us back
to the same level of checking?Yes, I think so. Attached is a set of tests to illustrate, mirroring
the dblink tests added upthread; they fail without this patch.In an offline discussion with Peter and Matheus, we figured out that
this is still not enough. The latest patch checks that a password was
used, but it doesn't ensure that the password material came from the
SCRAM keys. Attached is an updated test to illustrate.
On this new patch version I also changed the "connstr check" and
"security check" to have a validation very similar to what Peter
implemented on 0001-WIP-postgres_fdw-Fix-SCRAM-pass-through-security
patch. I also reproduced this test case that you've created on this new
dblink patch version and we actually fail as expected (but with a
different error message) because here we are adding
require_auth=scram-sha-256.
So, I think that having something similar to what Peter implemented on
his patch and adding require_auth=scram-sha-256 may prevent this kind of
security issue?
--
Matheus Alcantara
Attachments:
v5-0001-dblink-refactor-get-connection-routines.patchapplication/octet-stream; name=v5-0001-dblink-refactor-get-connection-routines.patchDownload
From 75e2461bef523129f0826d562a93757d994af5d6 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 20 Jan 2025 15:25:51 -0300
Subject: [PATCH v5 1/2] dblink: refactor get connection routines
Refactor dblink_get_conn and dblink_connect to move the logic of
actually opening the connection to the new connect_pg_server function
which them can be re-used on both functions.
This is a pre-work for a next commit that will add support for scram
pass-through authentication to dblink which will be able to implement
most of the logic into the connect_pg_server function which now already
have all necessary data information.
---
contrib/dblink/dblink.c | 199 ++++++++++++++++++++--------------------
1 file changed, 101 insertions(+), 98 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index bed2dee3d72..f7641f1b2ba 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -117,7 +117,7 @@ static bool dblink_connstr_has_pw(const char *connstr);
static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
-static char *get_connect_string(const char *servername);
+static char *get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping);
static char *escape_param_str(const char *str);
static void validate_pkattnums(Relation rel,
int2vector *pkattnums_arg, int32 pknumatts_arg,
@@ -126,6 +126,7 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
const char *option, Oid context);
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
/* Global */
static remoteConn *pconn = NULL;
@@ -201,33 +202,11 @@ dblink_get_conn(char *conname_or_str,
}
else
{
- const char *connstr;
-
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_get_conn == 0)
dblink_we_get_conn = WaitEventExtensionNew("DblinkGetConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_get_conn);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- char *msg = pchomp(PQerrorMessage(conn));
-
- libpqsrv_disconnect(conn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
- dblink_security_check(conn, rconn, connstr);
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_get_conn);
freeconn = true;
conname = NULL;
}
@@ -272,9 +251,7 @@ Datum
dblink_connect(PG_FUNCTION_ARGS)
{
char *conname_or_str = NULL;
- char *connstr = NULL;
char *connname = NULL;
- char *msg;
PGconn *conn = NULL;
remoteConn *rconn = NULL;
@@ -297,40 +274,21 @@ dblink_connect(PG_FUNCTION_ARGS)
rconn->newXactForCursor = false;
}
- /* first check for valid foreign data server */
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
-
- /* check password in connection string if not superuser */
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_connect == 0)
dblink_we_connect = WaitEventExtensionNew("DblinkConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_connect);
-
- if (PQstatus(conn) == CONNECTION_BAD)
+ PG_TRY();
+ {
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_connect);
+ }
+ PG_CATCH();
{
- msg = pchomp(PQerrorMessage(conn));
- libpqsrv_disconnect(conn);
if (rconn)
pfree(rconn);
-
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
+ PG_RE_THROW();
}
-
- /* check password actually used if not superuser */
- dblink_security_check(conn, rconn, connstr);
-
- /* attempt to set client encoding to match server encoding, if needed */
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ PG_END_TRY();
if (connname)
{
@@ -2784,15 +2742,17 @@ dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
* Obtain connection string for a foreign server
*/
static char *
-get_connect_string(const char *servername)
+get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
{
- ForeignServer *foreign_server = NULL;
- UserMapping *user_mapping;
ListCell *cell;
StringInfoData buf;
ForeignDataWrapper *fdw;
AclResult aclresult;
- char *srvname;
+
+ /* first gather the server connstr options */
+ Oid serverid = foreign_server->serverid;
+ Oid fdwid = foreign_server->fdwid;
+ Oid userid = GetUserId();
static const PQconninfoOption *options = NULL;
@@ -2815,57 +2775,42 @@ get_connect_string(const char *servername)
errdetail("Could not get libpq's default connection options.")));
}
- /* first gather the server connstr options */
- srvname = pstrdup(servername);
- truncate_identifier(srvname, strlen(srvname), false);
- foreign_server = GetForeignServerByName(srvname, true);
-
- if (foreign_server)
- {
- Oid serverid = foreign_server->serverid;
- Oid fdwid = foreign_server->fdwid;
- Oid userid = GetUserId();
-
- user_mapping = GetUserMapping(userid, serverid);
- fdw = GetForeignDataWrapper(fdwid);
-
- /* Check permissions, user must have usage on the server. */
- aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
- if (aclresult != ACLCHECK_OK)
- aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ fdw = GetForeignDataWrapper(fdwid);
- foreach(cell, fdw->options)
- {
- DefElem *def = lfirst(cell);
+ /* Check permissions, user must have usage on the server. */
+ aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
+ if (aclresult != ACLCHECK_OK)
+ aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
- if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, fdw->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, foreign_server->options)
- {
- DefElem *def = lfirst(cell);
+ if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, user_mapping->options)
- {
+ if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- DefElem *def = lfirst(cell);
+ foreach(cell, user_mapping->options)
+ {
- if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ DefElem *def = lfirst(cell);
- return buf.data;
+ if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
}
- else
- return NULL;
+
+ return buf.data;
}
/*
@@ -3087,3 +3032,61 @@ restoreLocalGucs(int nestlevel)
if (nestlevel > 0)
AtEOXact_GUC(true, nestlevel);
}
+
+/*
+ * Connect to remote server. If connstr_or_srvname maps to a foreign server,
+ * the associated properties and user mapping properties is also used to open
+ * the connection. Otherwise a connection will be open using the raw
+ * connstr_or_srvname value.
+ */
+static PGconn *
+connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info)
+{
+ PGconn *conn;
+ ForeignServer *foreign_server = NULL;
+ const char *connstr;
+ char *srvname;
+ Oid serverid;
+ UserMapping *user_mapping;
+ Oid userid = GetUserId();
+
+ /* first gather the server connstr options */
+ srvname = pstrdup(connstr_or_srvname);
+ truncate_identifier(srvname, strlen(srvname), false);
+ foreign_server = GetForeignServerByName(srvname, true);
+
+ if (foreign_server)
+ {
+ serverid = foreign_server->serverid;
+ user_mapping = GetUserMapping(userid, serverid);
+
+ connstr = get_connect_string(foreign_server, user_mapping);
+ }
+ else
+ connstr = connstr_or_srvname;
+
+ dblink_connstr_check(connstr);
+
+ /* OK to make connection */
+ conn = libpqsrv_connect(connstr, wait_event_info);
+
+ if (PQstatus(conn) == CONNECTION_BAD)
+ {
+ char *msg = pchomp(PQerrorMessage(conn));
+
+ libpqsrv_disconnect(conn);
+
+ ereport(ERROR,
+ (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
+ errmsg("could not establish connection"),
+ errdetail_internal("%s", msg)));
+ }
+
+ dblink_security_check(conn, rconn, connstr);
+
+ /* attempt to set client encoding to match server encoding, if needed */
+ if (PQclientEncoding(conn) != GetDatabaseEncoding())
+ PQsetClientEncoding(conn, GetDatabaseEncodingName());
+
+ return conn;
+}
--
2.39.5 (Apple Git-154)
v5-0002-dblink-Add-SCRAM-pass-through-authentication.patchapplication/octet-stream; name=v5-0002-dblink-Add-SCRAM-pass-through-authentication.patchDownload
From 36382383a2553d31d12269b9f7b7e03de3da2fc2 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Thu, 20 Feb 2025 15:12:48 -0300
Subject: [PATCH v5 2/2] dblink: Add SCRAM pass-through authentication
This enables SCRAM authentication for dblink_fdw when connecting to a
foreign server using the same approach as it was implemented for
postgres_fdw on 761c79508e.
---
contrib/dblink/Makefile | 1 +
contrib/dblink/dblink.c | 186 ++++++++++++++++++-
contrib/dblink/meson.build | 5 +
contrib/dblink/t/001_auth_scram.pl | 281 +++++++++++++++++++++++++++++
doc/src/sgml/dblink.sgml | 81 ++++++++-
5 files changed, 544 insertions(+), 10 deletions(-)
create mode 100644 contrib/dblink/t/001_auth_scram.pl
diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index d4c7ed625ab..fde0b49ddbb 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -13,6 +13,7 @@ PGFILEDESC = "dblink - connect to other PostgreSQL databases"
REGRESS = dblink
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
+TAP_TESTS = 1
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index f7641f1b2ba..4e9c08758bd 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -43,6 +43,8 @@
#include "catalog/pg_foreign_server.h"
#include "catalog/pg_type.h"
#include "catalog/pg_user_mapping.h"
+#include "commands/defrem.h"
+#include "common/base64.h"
#include "executor/spi.h"
#include "foreign/foreign.h"
#include "funcapi.h"
@@ -127,6 +129,11 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
+static bool UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+static void appendSCRAMKeysInfo(StringInfo buf);
+static bool is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context);
+static bool dblink_connstr_has_required_scram_options(const char *connstr);
/* Global */
static remoteConn *pconn = NULL;
@@ -1924,7 +1931,7 @@ dblink_fdw_validator(PG_FUNCTION_ARGS)
{
DefElem *def = (DefElem *) lfirst(cell);
- if (!is_valid_dblink_option(options, def->defname, context))
+ if (!is_valid_dblink_fdw_option(options, def->defname, context))
{
/*
* Unknown option, or invalid option for the context specified, so
@@ -2556,13 +2563,68 @@ deleteConnection(const char *name)
errmsg("undefined connection name")));
}
+ /*
+ * Ensure that require_auth and scram keys are correctly set on connstr.
+ * SCRAM keys used to pass-through is coming from the initial connection from
+ * the client with the server.
+ *
+ * All required scram options is set by ourself, so we just need to ensure
+ * that these options are not overwritten by the user.
+ *
+ * See appendSCRAMKeysInfo and it's usage for more.
+ */
+bool
+dblink_connstr_has_required_scram_options(const char *connstr)
+{
+ PQconninfoOption *options;
+ PQconninfoOption *option;
+ bool has_scram_server_key = false;
+ bool has_scram_client_key = false;
+ bool has_require_auth = false;
+ bool has_scram_keys = false;
+
+ options = PQconninfoParse(connstr, NULL);
+ if (options)
+ {
+ for (option = options; option->keyword != NULL; option++)
+ {
+ if (strcmp(option->keyword, "require_auth") == 0)
+ {
+ /*
+ * Continue iterating even if we found to make sure that there
+ * is no other declaration of require_auth that can overwrite
+ * the first.
+ */
+ if (option->val != NULL && strcmp(option->val, "scram-sha-256") == 0)
+ has_require_auth = true;
+ else
+ has_require_auth = false;
+ }
+
+ if (strcmp(option->keyword, "scram_client_key") == 0 && option->val[0] != '\0')
+ has_scram_client_key = true;
+ if (strcmp(option->keyword, "scram_server_key") == 0 && option->val[0] != '\0')
+ has_scram_server_key = true;
+ }
+ PQconninfoFree(options);
+ }
+
+ has_scram_keys = has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys;
+
+ if (has_scram_keys && has_require_auth)
+ return true;
+
+ return false;
+}
+
/*
* We need to make sure that the connection made used credentials
* which were provided by the user, so check what credentials were
* used to connect and then make sure that they came from the user.
*/
static void
-dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
+dblink_security_check(PGconn *conn, remoteConn *rconn,
+ const char *connstr)
{
/* Superuser bypasses security check */
if (superuser())
@@ -2572,6 +2634,18 @@ dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
if (PQconnectionUsedPassword(conn) && dblink_connstr_has_pw(connstr))
return;
+ /*
+ * Password was not used to connect, check if scram pass-through is in
+ * use.
+ *
+ * If dblink_connstr_has_required_scram_options is true we assume that
+ * UseScramPassthrough is also true because the required scram keys is
+ * only added if UseScramPassthrough is set, and the user is not allowed
+ * to add the scram keys on fdw and user mapping options.
+ */
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
#ifdef ENABLE_GSS
/* If GSSAPI creds used to connect, make sure it was one delegated */
if (PQconnectionUsedGSSAPI(conn) && be_gssapi_get_delegation(MyProcPort))
@@ -2624,12 +2698,14 @@ dblink_connstr_has_pw(const char *connstr)
}
/*
- * For non-superusers, insist that the connstr specify a password, except
- * if GSSAPI credentials have been delegated (and we check that they are used
- * for the connection in dblink_security_check later). This prevents a
- * password or GSSAPI credentials from being picked up from .pgpass, a
- * service file, the environment, etc. We don't want the postgres user's
- * passwords or Kerberos credentials to be accessible to non-superusers.
+ * For non-superusers, insist that the connstr specify a password, except if
+ * GSSAPI credentials have been delegated (and we check that they are used for
+ * the connection in dblink_security_check later) or if scram pass-through is
+ * being used. This prevents a password or GSSAPI credentials from being
+ * picked up from .pgpass, a service file, the environment, etc. We don't want
+ * the postgres user's passwords or Kerberos credentials to be accessible to
+ * non-superusers. In case of scram pass-through insist that the connstr
+ * has the required scram pass-through options.
*/
static void
dblink_connstr_check(const char *connstr)
@@ -2640,6 +2716,10 @@ dblink_connstr_check(const char *connstr)
if (dblink_connstr_has_pw(connstr))
return;
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
+
#ifdef ENABLE_GSS
if (be_gssapi_get_delegation(MyProcPort))
return;
@@ -2782,6 +2862,14 @@ get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ /*
+ * First append hardcoded options needed for SCRAM pass-through, so if the
+ * user overwrite these options we can ereport on dblink_connstr_check and
+ * dblink_security_check.
+ */
+ if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
+ appendSCRAMKeysInfo(&buf);
+
foreach(cell, fdw->options)
{
DefElem *def = lfirst(cell);
@@ -2963,6 +3051,22 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
+
+/*
+ * Same as is_valid_dblink_option but also check for only dblink_fdw specific
+ * options.
+ */
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context)
+{
+ if (strcmp(option, "use_scram_passthrough") == 0)
+ return true;
+
+ return is_valid_dblink_option(options, option, context);
+}
+
+
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3033,6 +3137,70 @@ restoreLocalGucs(int nestlevel)
AtEOXact_GUC(true, nestlevel);
}
+/*
+ * Append SCRAM client key and server key information from the global
+ * MyProcPort into the given StringInfo buffer.
+ */
+static void
+appendSCRAMKeysInfo(StringInfo buf)
+{
+ int len;
+ int encoded_len;
+ char *client_key;
+ char *server_key;
+
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
+ /* don't forget the zero-terminator */
+ client_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ClientKey,
+ sizeof(MyProcPort->scram_ClientKey),
+ client_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM client key");
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ServerKey));
+ /* don't forget the zero-terminator */
+ server_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ServerKey,
+ sizeof(MyProcPort->scram_ServerKey),
+ server_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM server key");
+
+ appendStringInfo(buf, "scram_client_key='%s' ", client_key);
+ appendStringInfo(buf, "scram_server_key='%s' ", server_key);
+ appendStringInfo(buf, "require_auth='scram-sha-256' ");
+
+ pfree(client_key);
+ pfree(server_key);
+}
+
+
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user)
+{
+ ListCell *cell;
+
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ foreach(cell, user->options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ return false;
+}
+
/*
* Connect to remote server. If connstr_or_srvname maps to a foreign server,
* the associated properties and user mapping properties is also used to open
@@ -3065,6 +3233,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
else
connstr = connstr_or_srvname;
+ /* Verify the set of connection parameters. */
dblink_connstr_check(connstr);
/* OK to make connection */
@@ -3082,6 +3251,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
errdetail_internal("%s", msg)));
}
+ /* Perform post-connection security checks. */
dblink_security_check(conn, rconn, connstr);
/* attempt to set client encoding to match server encoding, if needed */
diff --git a/contrib/dblink/meson.build b/contrib/dblink/meson.build
index 3ab78668288..dfd8eb6877e 100644
--- a/contrib/dblink/meson.build
+++ b/contrib/dblink/meson.build
@@ -36,4 +36,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/001_auth_scram.pl',
+ ],
+ },
}
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
new file mode 100644
index 00000000000..c6442a055a3
--- /dev/null
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -0,0 +1,281 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+# Test SCRAM authentication when opening a new connection with a foreign
+# server.
+#
+# The test is executed by testing the SCRAM authentifcation on a loopback
+# connection on the same server and with different servers.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $user = "user01";
+
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
+my $fdw_server = "db1_fdw";
+my $fdw_server2 = "db2_fdw";
+my $fdw_invalid_server = "db2_fdw_invalid"; # For invalid fdw options
+my $fdw_invalid_server2 =
+ "db2_fdw_invalid2"; # For invalid scram keys fdw options
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+my $node2 = PostgreSQL::Test::Cluster->new('node2');
+
+$node1->init;
+$node2->init;
+
+$node1->start;
+$node2->start;
+
+# Test setup
+
+$node1->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$node2->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$ENV{PGPASSWORD} = "pass";
+
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db0');
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db1');
+$node2->safe_psql('postgres', qq'CREATE DATABASE $db2');
+
+setup_table($node1, $db1, "t");
+setup_table($node2, $db2, "t2");
+
+$node1->safe_psql($db0, 'CREATE EXTENSION IF NOT EXISTS dblink');
+setup_fdw_server($node1, $db0, $fdw_server, $node1, $db1);
+setup_fdw_server($node1, $db0, $fdw_server2, $node2, $db2);
+setup_invalid_fdw_server($node1, $db0, $fdw_invalid_server, $node2, $db2);
+setup_fdw_server($node1, $db0, $fdw_invalid_server2, $node2, $db2);
+
+setup_user_mapping($node1, $db0, $fdw_server);
+setup_user_mapping($node1, $db0, $fdw_server2);
+setup_user_mapping($node1, $db0, $fdw_invalid_server);
+
+# Make the user have the same SCRAM key on both servers. Forcing to have the
+# same iteration and salt.
+my $rolpassword = $node1->safe_psql('postgres',
+ qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';");
+$node2->safe_psql('postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'");
+
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all scram-sha-256
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all scram-sha-256
+});
+
+$node1->restart;
+$node2->restart;
+
+# End of test setup
+
+test_scram_keys_is_not_overwritten($node1, $db0, $fdw_invalid_server2);
+
+test_fdw_auth($node1, $db0, "t", $fdw_server,
+ "SCRAM auth on the same database cluster must succeed");
+
+test_fdw_auth($node1, $db0, "t2", $fdw_server2,
+ "SCRAM auth on a different database cluster must succeed");
+
+test_fdw_auth_with_invalid_overwritten_require_auth($fdw_invalid_server);
+
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all trust
+});
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on a different cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (different cluster)');
+
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local all all password
+});
+
+$node1->restart;
+$node2->restart;
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/failed: authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback password fails on a different cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server requested a cleartext password/,
+ 'expected error from loopback password (different cluster)');
+
+
+# Helper functions
+
+sub test_fdw_auth
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $db, $tbl, $fdw, $testname) = @_;
+ my $connstr = $node->connstr($db) . qq' user=$user';
+
+ my $ret = $node->safe_psql(
+ $db,
+ qq"SELECT count(1) FROM dblink('$fdw', 'SELECT * FROM $tbl') AS $tbl(a int, b int)",
+ connstr => $connstr);
+
+ is($ret, '10', $testname);
+}
+
+sub test_fdw_auth_with_invalid_overwritten_require_auth
+{
+
+ my ($fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "select * from dblink('$fdw', 'select * from t') as t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+ is($ret, 3, 'loopback trust fails when overwriting require_auth');
+ like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error when connecting to a fdw overwriting the require_auth'
+ );
+}
+
+sub test_scram_keys_is_not_overwritten
+{
+ my ($node, $db, $fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_client_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret, 3, 'user mapping creation fails when using scram_client_key');
+ is( $stderr,
+ "psql:<stdin>:1: ERROR: invalid option \"scram_client_key\"",
+ 'user mapping creation fails when using scram_client_key');
+
+ my ($ret2, $stdout2, $stderr2) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_server_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret2, 3, 'user mapping creation fails when using scram_server_key');
+ is( $stderr2,
+ "psql:<stdin>:1: ERROR: invalid option \"scram_server_key\"",
+ 'user mapping creation fails when using scram_server_key');
+}
+
+sub setup_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\');'
+ );
+}
+
+sub setup_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_invalid_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\', require_auth \'none\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_table
+{
+ my ($node, $db, $tbl) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE TABLE $tbl AS SELECT g as a, g + 1 as b FROM generate_series(1,10) g(g)'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON SCHEMA public TO $user');
+ $node->safe_psql($db, qq'GRANT SELECT ON $tbl TO $user');
+}
+
+done_testing();
+
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index 81f35986c88..e3b4129ae26 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,6 +136,82 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
+<refsect1>
+ <title>Foreign Data Wrapper</title>
+
+ <para>
+ A Foreign Data Wrapper can be used as a connection name parameter. The foreign
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ </para>
+
+ <para>
+ The authentication with the foreign server can be via password on USER
+ MAPPING or using SCRAM pass-through. The
+ <literal>use_scram_passthrough</literal> on CREATE SERVER options controls
+ whether <filename>dblink_fdw</filename> will use the SCRAM pass-through
+ authentication to connect to the foreign server. With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
+ </para>
+
+ <para>
+ To use SCRAM pass-through authentication:
+ <itemizedlist>
+ <listitem>
+ <para>
+ The remote server must request SCRAM authentication. (If desired,
+ enforce this on the client side (FDW side) with the option
+ <literal>require_auth</literal>.) If another authentication method is
+ requested by the server, then that one will be used normally.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The remote server can be of any PostgreSQL version that supports SCRAM.
+ Support for <literal>use_scram_passthrough</literal> is only required on
+ the client side (FDW side).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The user mapping password is not used. (It could be set to support other
+ authentication methods, but that would arguably violate the point of this
+ feature, which is to avoid storing plain-text passwords.)
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The server running <filename>dblink_fdw</filename> and the remote server
+ must have identical SCRAM secrets (encrypted passwords) for the user being
+ used on <filename>dblink_fdw</filename> to authenticate on the foreign
+ server (same salt and iterations, not merely the same password).
+ </para>
+
+ <para>
+ As a corollary, if FDW connections to multiple hosts are to be made, for
+ example for partitioned foreign tables/sharding, then all hosts must have
+ identical SCRAM secrets for the users involved.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The current session on the PostgreSQL instance that makes the outgoing FDW
+ connections also must also use SCRAM authentication for its incoming client
+ connection. (Hence <quote>pass-through</quote>: SCRAM must be used going in
+ and out.) This is a technical requirement of the SCRAM protocol.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+
+ </refsect1>
+
<refsect1>
<title>Notes</title>
@@ -181,8 +257,9 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection must require password authentication for this to work properly
--- Otherwise, you will receive the following error from dblink_connect():
+-- Note: local connection that don't use SCRAM pass-through require password
+-- authentication for this to work properly. Otherwise, you will receive
+-- the following error from dblink_connect():
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
--
2.39.5 (Apple Git-154)
On Thu, Mar 13, 2025 at 6:59 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:
Fixed by wrapping on PG_CATCH/pfree/PG_RE_THROW. I didn't managed to
create a test that use this code path, so let me know if I'm still
missing something.
Thanks! Looks like the regression suite has one test that takes that
path (found by adding an Assert(false) to the PG_CATCH branch):
SET SESSION AUTHORIZATION regress_dblink_user;
-- should fail
SELECT dblink_connect('myconn', 'fdtest');
PG_CATCH();
{
if (rconn)
pfree(rconn);
A comment in this branch might be nice, to draw attention to the fact
that rconn is allocated in the TopMemoryContext and we can't leak it.
I've added a code comment on dblink_connstr_has_required_scram_options
function which is called on "connstr check" and "security check". Please
let me know what you think.
That comment does not seem to match the code now:
+ * All required scram options is set by ourself, so we just need to ensure + * that these options are not overwritten by the user.
But later, there's no provision to detect if the keys have been overwritten:
+ if (strcmp(option->keyword, "scram_client_key") == 0 && option->val[0] != '\0') + has_scram_client_key = true; + if (strcmp(option->keyword, "scram_server_key") == 0 && option->val[0] != '\0') + has_scram_server_key = true;
This needs to match the handling directly above it, if we want to
claim that we'll detect duplicates.
I was also thinking about this, maybe we could add a new validation
(similar with PQconnectionUsedPassword, on fe-connect.c) that check if
the scram keys is set on PGconn (we only set these keys if we are
actually using the scram pass-through feature)+int +PQconnectionUsedScramKeys(const PGconn *conn) +{ + if (conn->scram_client_key && conn->scram_server_key) + return true; + + return false; +}
If we implement this, it needs to check that the keys were actually
sent during scram_exchange(). Having them set on the PGconn doesn't
mean that we used them for authentication.
So, I think that having something similar to what Peter implemented on
his patch and adding require_auth=scram-sha-256 may prevent this kind of
security issue?
Right. I think it'll come down to how Peter feels about putting that
into the solution, vs. PQconnectionUsedScramKeys() or some third
option.
--
Miscellaneous patch review:
-dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr) +dblink_security_check(PGconn *conn, remoteConn *rconn, + const char *connstr)
nit: this whitespace change is not necessary now that
useScramPassthrough is no longer in the signature.
Speaking of which, does get_connect_string() still need to take
user_mapping as an argument?
+ if (has_scram_keys && has_require_auth) + return true; + + return false;
nit: this is equivalent to `return (has_scram_keys && has_require_auth);`
+ my ($ret2, $stdout2, $stderr2) = $node->psql(
Declaring a second set of return values is probably unnecessary; the
previous ones can be reused.
+ is( $stderr, + "psql:<stdin>:1: ERROR: invalid option \"scram_client_key\"", + 'user mapping creation fails when using scram_client_key');
I think the two new tests like this should be using like() rather than
is() so that they can match only the important part of the error. I
don't think we want to pin the "psql:<stdin>" stuff in this test.
+($ret, $stdout, $stderr) = $node1->psql( + $db0, + "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)", + connstr => $node1->connstr($db0) . " user=$user"); + +is($ret, 3, 'loopback trust fails on the same cluster'); +like( + $stderr, + qr/failed: authentication method requirement "scram-sha-256" failed: server did not complete authentication/, + 'expected error from loopback trust (same cluster)');
Is this the same as the previous loopback-trust test? If so I think it
can be removed (or the two sections merged completely).
Thanks!
--Jacob
On Thu, Mar 13, 2025 at 4:54 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
On Thu, Mar 13, 2025 at 6:59 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:Fixed by wrapping on PG_CATCH/pfree/PG_RE_THROW. I didn't managed to
create a test that use this code path, so let me know if I'm still
missing something.Thanks! Looks like the regression suite has one test that takes that
path (found by adding an Assert(false) to the PG_CATCH branch):SET SESSION AUTHORIZATION regress_dblink_user;
-- should fail
SELECT dblink_connect('myconn', 'fdtest');PG_CATCH();
{
if (rconn)
pfree(rconn);A comment in this branch might be nice, to draw attention to the fact
that rconn is allocated in the TopMemoryContext and we can't leak it.
Fixed
I've added a code comment on dblink_connstr_has_required_scram_options
function which is called on "connstr check" and "security check". Please
let me know what you think.That comment does not seem to match the code now:
+ * All required scram options is set by ourself, so we just need to ensure + * that these options are not overwritten by the user.But later, there's no provision to detect if the keys have been overwritten:
+ if (strcmp(option->keyword, "scram_client_key") == 0 && option->val[0] != '\0') + has_scram_client_key = true; + if (strcmp(option->keyword, "scram_server_key") == 0 && option->val[0] != '\0') + has_scram_server_key = true;This needs to match the handling directly above it, if we want to
claim that we'll detect duplicates.
I thought about this; The problem is that at this point, the scram keys
on connection options are base64 encoded (see appendSCRAMKeysInfo), so
we can't compare with the values stored on MyProcPort. I don't know if
decoding the option or encoding the keys on MyProcPort from/to base64 is
a way to go, what do you think?
I've implemented this check in this way because we don't allow adding
the scram keys on user mapping or foreign server options, so the user
can't actually overwrite the scram keys, unless there is the possibility
of filling in these scram keys options in other places besides user
mapping and foreign server options that I am missing?
I was also thinking about this, maybe we could add a new validation
(similar with PQconnectionUsedPassword, on fe-connect.c) that check if
the scram keys is set on PGconn (we only set these keys if we are
actually using the scram pass-through feature)+int +PQconnectionUsedScramKeys(const PGconn *conn) +{ + if (conn->scram_client_key && conn->scram_server_key) + return true; + + return false; +}If we implement this, it needs to check that the keys were actually
sent during scram_exchange(). Having them set on the PGconn doesn't
mean that we used them for authentication.
We use the client key and server key on calculate_client_proof and
verify_server_signature respective during memcpy, it would be too hack
to add new fields on pg_conn like scram_client_key_in_use and
scram_server_key_in_use, set them to true on these functions and then
validate that both are true on PQconnectionUsedScramKeys?
--
Miscellaneous patch review:
-dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr) +dblink_security_check(PGconn *conn, remoteConn *rconn, + const char *connstr)nit: this whitespace change is not necessary now that
useScramPassthrough is no longer in the signature.
Fixed
Speaking of which, does get_connect_string() still need to take
user_mapping as an argument?
Yes, because we need to check if the use_scram_passthrough option is set
on foreign server or user mapping options:
if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server,
user_mapping))
appendSCRAMKeysInfo(&buf);
+ if (has_scram_keys && has_require_auth) + return true; + + return false;nit: this is equivalent to `return (has_scram_keys && has_require_auth);`
Fixed
+ my ($ret2, $stdout2, $stderr2) = $node->psql(
Declaring a second set of return values is probably unnecessary; the
previous ones can be reused.
Fixed
+ is( $stderr, + "psql:<stdin>:1: ERROR: invalid option \"scram_client_key\"", + 'user mapping creation fails when using scram_client_key');I think the two new tests like this should be using like() rather than
is() so that they can match only the important part of the error. I
don't think we want to pin the "psql:<stdin>" stuff in this test.
Yes, having "psq:<stdin>" is weird, fixed.
+($ret, $stdout, $stderr) = $node1->psql( + $db0, + "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)", + connstr => $node1->connstr($db0) . " user=$user"); + +is($ret, 3, 'loopback trust fails on the same cluster'); +like( + $stderr, + qr/failed: authentication method requirement "scram-sha-256" failed: server did not complete authentication/, + 'expected error from loopback trust (same cluster)');Is this the same as the previous loopback-trust test? If so I think it
can be removed (or the two sections merged completely).
The only difference is using "trust" vs "password" on $db2 pg_hba.conf,
but the expectation of the test is the same. I've just removed the test
using "trust". Good catch, I've made a small confusion on these tests.
Thanks for all the comments!
--
Matheus Alcantara
Attachments:
v6-0001-dblink-refactor-get-connection-routines.patchapplication/octet-stream; name=v6-0001-dblink-refactor-get-connection-routines.patchDownload
From 3784df7896650fb81dfe72a4f103dcceb0c8b84d Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 20 Jan 2025 15:25:51 -0300
Subject: [PATCH v6 1/2] dblink: refactor get connection routines
Refactor dblink_get_conn and dblink_connect to move the logic of
actually opening the connection to the new connect_pg_server function
which them can be re-used on both functions.
This is a pre-work for a next commit that will add support for scram
pass-through authentication to dblink which will be able to implement
most of the logic into the connect_pg_server function which now already
have all necessary data information.
---
contrib/dblink/dblink.c | 202 +++++++++++++++++++++-------------------
1 file changed, 105 insertions(+), 97 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 58c1a6221c8..a705d32423f 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -117,7 +117,7 @@ static bool dblink_connstr_has_pw(const char *connstr);
static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
-static char *get_connect_string(const char *servername);
+static char *get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping);
static char *escape_param_str(const char *str);
static void validate_pkattnums(Relation rel,
int2vector *pkattnums_arg, int32 pknumatts_arg,
@@ -126,6 +126,7 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
const char *option, Oid context);
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
/* Global */
static remoteConn *pconn = NULL;
@@ -199,33 +200,11 @@ dblink_get_conn(char *conname_or_str,
}
else
{
- const char *connstr;
-
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_get_conn == 0)
dblink_we_get_conn = WaitEventExtensionNew("DblinkGetConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_get_conn);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- char *msg = pchomp(PQerrorMessage(conn));
-
- libpqsrv_disconnect(conn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
- dblink_security_check(conn, rconn, connstr);
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_get_conn);
freeconn = true;
conname = NULL;
}
@@ -270,9 +249,7 @@ Datum
dblink_connect(PG_FUNCTION_ARGS)
{
char *conname_or_str = NULL;
- char *connstr = NULL;
char *connname = NULL;
- char *msg;
PGconn *conn = NULL;
remoteConn *rconn = NULL;
@@ -295,40 +272,26 @@ dblink_connect(PG_FUNCTION_ARGS)
rconn->newXactForCursor = false;
}
- /* first check for valid foreign data server */
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
-
- /* check password in connection string if not superuser */
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_connect == 0)
dblink_we_connect = WaitEventExtensionNew("DblinkConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_connect);
-
- if (PQstatus(conn) == CONNECTION_BAD)
+ PG_TRY();
{
- msg = pchomp(PQerrorMessage(conn));
- libpqsrv_disconnect(conn);
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_connect);
+ }
+ PG_CATCH();
+ {
+ /*
+ * If not NULL, rconn is allocated on TopMemoryContext, so we need to
+ * pfree to avoid memory leaks.
+ */
if (rconn)
pfree(rconn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
+ PG_RE_THROW();
}
-
- /* check password actually used if not superuser */
- dblink_security_check(conn, rconn, connstr);
-
- /* attempt to set client encoding to match server encoding, if needed */
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ PG_END_TRY();
if (connname)
{
@@ -2782,15 +2745,17 @@ dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
* Obtain connection string for a foreign server
*/
static char *
-get_connect_string(const char *servername)
+get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
{
- ForeignServer *foreign_server = NULL;
- UserMapping *user_mapping;
ListCell *cell;
StringInfoData buf;
ForeignDataWrapper *fdw;
AclResult aclresult;
- char *srvname;
+
+ /* first gather the server connstr options */
+ Oid serverid = foreign_server->serverid;
+ Oid fdwid = foreign_server->fdwid;
+ Oid userid = GetUserId();
static const PQconninfoOption *options = NULL;
@@ -2813,57 +2778,42 @@ get_connect_string(const char *servername)
errdetail("Could not get libpq's default connection options.")));
}
- /* first gather the server connstr options */
- srvname = pstrdup(servername);
- truncate_identifier(srvname, strlen(srvname), false);
- foreign_server = GetForeignServerByName(srvname, true);
-
- if (foreign_server)
- {
- Oid serverid = foreign_server->serverid;
- Oid fdwid = foreign_server->fdwid;
- Oid userid = GetUserId();
-
- user_mapping = GetUserMapping(userid, serverid);
- fdw = GetForeignDataWrapper(fdwid);
-
- /* Check permissions, user must have usage on the server. */
- aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
- if (aclresult != ACLCHECK_OK)
- aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ fdw = GetForeignDataWrapper(fdwid);
- foreach(cell, fdw->options)
- {
- DefElem *def = lfirst(cell);
+ /* Check permissions, user must have usage on the server. */
+ aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
+ if (aclresult != ACLCHECK_OK)
+ aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
- if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, fdw->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, foreign_server->options)
- {
- DefElem *def = lfirst(cell);
+ if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, user_mapping->options)
- {
+ if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- DefElem *def = lfirst(cell);
+ foreach(cell, user_mapping->options)
+ {
- if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ DefElem *def = lfirst(cell);
- return buf.data;
+ if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
}
- else
- return NULL;
+
+ return buf.data;
}
/*
@@ -3085,3 +3035,61 @@ restoreLocalGucs(int nestlevel)
if (nestlevel > 0)
AtEOXact_GUC(true, nestlevel);
}
+
+/*
+ * Connect to remote server. If connstr_or_srvname maps to a foreign server,
+ * the associated properties and user mapping properties is also used to open
+ * the connection. Otherwise a connection will be open using the raw
+ * connstr_or_srvname value.
+ */
+static PGconn *
+connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info)
+{
+ PGconn *conn;
+ ForeignServer *foreign_server = NULL;
+ const char *connstr;
+ char *srvname;
+ Oid serverid;
+ UserMapping *user_mapping;
+ Oid userid = GetUserId();
+
+ /* first gather the server connstr options */
+ srvname = pstrdup(connstr_or_srvname);
+ truncate_identifier(srvname, strlen(srvname), false);
+ foreign_server = GetForeignServerByName(srvname, true);
+
+ if (foreign_server)
+ {
+ serverid = foreign_server->serverid;
+ user_mapping = GetUserMapping(userid, serverid);
+
+ connstr = get_connect_string(foreign_server, user_mapping);
+ }
+ else
+ connstr = connstr_or_srvname;
+
+ dblink_connstr_check(connstr);
+
+ /* OK to make connection */
+ conn = libpqsrv_connect(connstr, wait_event_info);
+
+ if (PQstatus(conn) == CONNECTION_BAD)
+ {
+ char *msg = pchomp(PQerrorMessage(conn));
+
+ libpqsrv_disconnect(conn);
+
+ ereport(ERROR,
+ (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
+ errmsg("could not establish connection"),
+ errdetail_internal("%s", msg)));
+ }
+
+ dblink_security_check(conn, rconn, connstr);
+
+ /* attempt to set client encoding to match server encoding, if needed */
+ if (PQclientEncoding(conn) != GetDatabaseEncoding())
+ PQsetClientEncoding(conn, GetDatabaseEncodingName());
+
+ return conn;
+}
--
2.39.5 (Apple Git-154)
v6-0002-dblink-Add-SCRAM-pass-through-authentication.patchapplication/octet-stream; name=v6-0002-dblink-Add-SCRAM-pass-through-authentication.patchDownload
From 01b6bd5c1c3569f77379668a991806aa0081193e Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Thu, 20 Feb 2025 15:12:48 -0300
Subject: [PATCH v6 2/2] dblink: Add SCRAM pass-through authentication
This enables SCRAM authentication for dblink_fdw when connecting to a
foreign server using the same approach as it was implemented for
postgres_fdw on 761c79508e.
---
contrib/dblink/Makefile | 1 +
contrib/dblink/dblink.c | 183 +++++++++++++++++++++-
contrib/dblink/meson.build | 5 +
contrib/dblink/t/001_auth_scram.pl | 242 +++++++++++++++++++++++++++++
doc/src/sgml/dblink.sgml | 81 +++++++++-
5 files changed, 502 insertions(+), 10 deletions(-)
create mode 100644 contrib/dblink/t/001_auth_scram.pl
diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index d4c7ed625ab..fde0b49ddbb 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -13,6 +13,7 @@ PGFILEDESC = "dblink - connect to other PostgreSQL databases"
REGRESS = dblink
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
+TAP_TESTS = 1
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index a705d32423f..235e1a4cb64 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -43,6 +43,8 @@
#include "catalog/pg_foreign_server.h"
#include "catalog/pg_type.h"
#include "catalog/pg_user_mapping.h"
+#include "commands/defrem.h"
+#include "common/base64.h"
#include "executor/spi.h"
#include "foreign/foreign.h"
#include "funcapi.h"
@@ -127,6 +129,11 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
+static bool UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+static void appendSCRAMKeysInfo(StringInfo buf);
+static bool is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context);
+static bool dblink_connstr_has_required_scram_options(const char *connstr);
/* Global */
static remoteConn *pconn = NULL;
@@ -1927,7 +1934,7 @@ dblink_fdw_validator(PG_FUNCTION_ARGS)
{
DefElem *def = (DefElem *) lfirst(cell);
- if (!is_valid_dblink_option(options, def->defname, context))
+ if (!is_valid_dblink_fdw_option(options, def->defname, context))
{
/*
* Unknown option, or invalid option for the context specified, so
@@ -2559,13 +2566,65 @@ deleteConnection(const char *name)
errmsg("undefined connection name")));
}
+ /*
+ * Ensure that require_auth and scram keys are correctly set on connstr.
+ * SCRAM keys used to pass-through is coming from the initial connection from
+ * the client with the server.
+ *
+ * All required scram options is set by ourself, so we just need to ensure
+ * that these options are not overwritten by the user.
+ *
+ * See appendSCRAMKeysInfo and it's usage for more.
+ */
+bool
+dblink_connstr_has_required_scram_options(const char *connstr)
+{
+ PQconninfoOption *options;
+ PQconninfoOption *option;
+ bool has_scram_server_key = false;
+ bool has_scram_client_key = false;
+ bool has_require_auth = false;
+ bool has_scram_keys = false;
+
+ options = PQconninfoParse(connstr, NULL);
+ if (options)
+ {
+ for (option = options; option->keyword != NULL; option++)
+ {
+ if (strcmp(option->keyword, "require_auth") == 0)
+ {
+ /*
+ * Continue iterating even if we found to make sure that there
+ * is no other declaration of require_auth that can overwrite
+ * the first.
+ */
+ if (option->val != NULL && strcmp(option->val, "scram-sha-256") == 0)
+ has_require_auth = true;
+ else
+ has_require_auth = false;
+ }
+
+ if (strcmp(option->keyword, "scram_client_key") == 0 && option->val[0] != '\0')
+ has_scram_client_key = true;
+ if (strcmp(option->keyword, "scram_server_key") == 0 && option->val[0] != '\0')
+ has_scram_server_key = true;
+ }
+ PQconninfoFree(options);
+ }
+
+ has_scram_keys = has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys;
+
+ return (has_scram_keys && has_require_auth);
+}
+
/*
* We need to make sure that the connection made used credentials
* which were provided by the user, so check what credentials were
* used to connect and then make sure that they came from the user.
*/
static void
-dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
+dblink_security_check(PGconn *conn, remoteConn *rconn,
+ const char *connstr)
{
/* Superuser bypasses security check */
if (superuser())
@@ -2575,6 +2634,18 @@ dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
if (PQconnectionUsedPassword(conn) && dblink_connstr_has_pw(connstr))
return;
+ /*
+ * Password was not used to connect, check if scram pass-through is in
+ * use.
+ *
+ * If dblink_connstr_has_required_scram_options is true we assume that
+ * UseScramPassthrough is also true because the required scram keys is
+ * only added if UseScramPassthrough is set, and the user is not allowed
+ * to add the scram keys on fdw and user mapping options.
+ */
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
#ifdef ENABLE_GSS
/* If GSSAPI creds used to connect, make sure it was one delegated */
if (PQconnectionUsedGSSAPI(conn) && be_gssapi_get_delegation(MyProcPort))
@@ -2627,12 +2698,14 @@ dblink_connstr_has_pw(const char *connstr)
}
/*
- * For non-superusers, insist that the connstr specify a password, except
- * if GSSAPI credentials have been delegated (and we check that they are used
- * for the connection in dblink_security_check later). This prevents a
- * password or GSSAPI credentials from being picked up from .pgpass, a
- * service file, the environment, etc. We don't want the postgres user's
- * passwords or Kerberos credentials to be accessible to non-superusers.
+ * For non-superusers, insist that the connstr specify a password, except if
+ * GSSAPI credentials have been delegated (and we check that they are used for
+ * the connection in dblink_security_check later) or if scram pass-through is
+ * being used. This prevents a password or GSSAPI credentials from being
+ * picked up from .pgpass, a service file, the environment, etc. We don't want
+ * the postgres user's passwords or Kerberos credentials to be accessible to
+ * non-superusers. In case of scram pass-through insist that the connstr
+ * has the required scram pass-through options.
*/
static void
dblink_connstr_check(const char *connstr)
@@ -2643,6 +2716,10 @@ dblink_connstr_check(const char *connstr)
if (dblink_connstr_has_pw(connstr))
return;
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
+
#ifdef ENABLE_GSS
if (be_gssapi_get_delegation(MyProcPort))
return;
@@ -2785,6 +2862,14 @@ get_connect_string(ForeignServer *foreign_server, UserMapping *user_mapping)
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ /*
+ * First append hardcoded options needed for SCRAM pass-through, so if the
+ * user overwrite these options we can ereport on dblink_connstr_check and
+ * dblink_security_check.
+ */
+ if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
+ appendSCRAMKeysInfo(&buf);
+
foreach(cell, fdw->options)
{
DefElem *def = lfirst(cell);
@@ -2966,6 +3051,22 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
+
+/*
+ * Same as is_valid_dblink_option but also check for only dblink_fdw specific
+ * options.
+ */
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context)
+{
+ if (strcmp(option, "use_scram_passthrough") == 0)
+ return true;
+
+ return is_valid_dblink_option(options, option, context);
+}
+
+
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3036,6 +3137,70 @@ restoreLocalGucs(int nestlevel)
AtEOXact_GUC(true, nestlevel);
}
+/*
+ * Append SCRAM client key and server key information from the global
+ * MyProcPort into the given StringInfo buffer.
+ */
+static void
+appendSCRAMKeysInfo(StringInfo buf)
+{
+ int len;
+ int encoded_len;
+ char *client_key;
+ char *server_key;
+
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
+ /* don't forget the zero-terminator */
+ client_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ClientKey,
+ sizeof(MyProcPort->scram_ClientKey),
+ client_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM client key");
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ServerKey));
+ /* don't forget the zero-terminator */
+ server_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ServerKey,
+ sizeof(MyProcPort->scram_ServerKey),
+ server_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM server key");
+
+ appendStringInfo(buf, "scram_client_key='%s' ", client_key);
+ appendStringInfo(buf, "scram_server_key='%s' ", server_key);
+ appendStringInfo(buf, "require_auth='scram-sha-256' ");
+
+ pfree(client_key);
+ pfree(server_key);
+}
+
+
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user)
+{
+ ListCell *cell;
+
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ foreach(cell, user->options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ return false;
+}
+
/*
* Connect to remote server. If connstr_or_srvname maps to a foreign server,
* the associated properties and user mapping properties is also used to open
@@ -3068,6 +3233,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
else
connstr = connstr_or_srvname;
+ /* Verify the set of connection parameters. */
dblink_connstr_check(connstr);
/* OK to make connection */
@@ -3085,6 +3251,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
errdetail_internal("%s", msg)));
}
+ /* Perform post-connection security checks. */
dblink_security_check(conn, rconn, connstr);
/* attempt to set client encoding to match server encoding, if needed */
diff --git a/contrib/dblink/meson.build b/contrib/dblink/meson.build
index 3ab78668288..dfd8eb6877e 100644
--- a/contrib/dblink/meson.build
+++ b/contrib/dblink/meson.build
@@ -36,4 +36,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/001_auth_scram.pl',
+ ],
+ },
}
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
new file mode 100644
index 00000000000..e0f1326e50a
--- /dev/null
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -0,0 +1,242 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+# Test SCRAM authentication when opening a new connection with a foreign
+# server.
+#
+# The test is executed by testing the SCRAM authentifcation on a loopback
+# connection on the same server and with different servers.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $user = "user01";
+
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
+my $fdw_server = "db1_fdw";
+my $fdw_server2 = "db2_fdw";
+my $fdw_invalid_server = "db2_fdw_invalid"; # For invalid fdw options
+my $fdw_invalid_server2 =
+ "db2_fdw_invalid2"; # For invalid scram keys fdw options
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+my $node2 = PostgreSQL::Test::Cluster->new('node2');
+
+$node1->init;
+$node2->init;
+
+$node1->start;
+$node2->start;
+
+# Test setup
+
+$node1->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$node2->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$ENV{PGPASSWORD} = "pass";
+
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db0');
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db1');
+$node2->safe_psql('postgres', qq'CREATE DATABASE $db2');
+
+setup_table($node1, $db1, "t");
+setup_table($node2, $db2, "t2");
+
+$node1->safe_psql($db0, 'CREATE EXTENSION IF NOT EXISTS dblink');
+setup_fdw_server($node1, $db0, $fdw_server, $node1, $db1);
+setup_fdw_server($node1, $db0, $fdw_server2, $node2, $db2);
+setup_invalid_fdw_server($node1, $db0, $fdw_invalid_server, $node2, $db2);
+setup_fdw_server($node1, $db0, $fdw_invalid_server2, $node2, $db2);
+
+setup_user_mapping($node1, $db0, $fdw_server);
+setup_user_mapping($node1, $db0, $fdw_server2);
+setup_user_mapping($node1, $db0, $fdw_invalid_server);
+
+# Make the user have the same SCRAM key on both servers. Forcing to have the
+# same iteration and salt.
+my $rolpassword = $node1->safe_psql('postgres',
+ qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';");
+$node2->safe_psql('postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'");
+
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all scram-sha-256
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all scram-sha-256
+});
+
+$node1->restart;
+$node2->restart;
+
+# End of test setup
+
+test_scram_keys_is_not_overwritten($node1, $db0, $fdw_invalid_server2);
+
+test_fdw_auth($node1, $db0, "t", $fdw_server,
+ "SCRAM auth on the same database cluster must succeed");
+
+test_fdw_auth($node1, $db0, "t2", $fdw_server2,
+ "SCRAM auth on a different database cluster must succeed");
+
+test_fdw_auth_with_invalid_overwritten_require_auth($fdw_invalid_server);
+
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local all all password
+});
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/failed: authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback password fails on a different cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server requested a cleartext password/,
+ 'expected error from loopback password (different cluster)');
+
+
+# Helper functions
+
+sub test_fdw_auth
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $db, $tbl, $fdw, $testname) = @_;
+ my $connstr = $node->connstr($db) . qq' user=$user';
+
+ my $ret = $node->safe_psql(
+ $db,
+ qq"SELECT count(1) FROM dblink('$fdw', 'SELECT * FROM $tbl') AS $tbl(a int, b int)",
+ connstr => $connstr);
+
+ is($ret, '10', $testname);
+}
+
+sub test_fdw_auth_with_invalid_overwritten_require_auth
+{
+
+ my ($fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "select * from dblink('$fdw', 'select * from t') as t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+ is($ret, 3, 'loopback trust fails when overwriting require_auth');
+ like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error when connecting to a fdw overwriting the require_auth'
+ );
+}
+
+sub test_scram_keys_is_not_overwritten
+{
+ my ($node, $db, $fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_client_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret, 3, 'user mapping creation fails when using scram_client_key');
+ like( $stderr,
+ qr/ERROR: invalid option "scram_client_key"/,
+ 'user mapping creation fails when using scram_client_key');
+
+ ($ret, $stdout, $stderr) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_server_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret, 3, 'user mapping creation fails when using scram_server_key');
+ like($stderr,
+ qr/ERROR: invalid option "scram_server_key"/,
+ 'user mapping creation fails when using scram_server_key');
+}
+
+sub setup_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\');'
+ );
+}
+
+sub setup_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_invalid_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\', require_auth \'none\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_table
+{
+ my ($node, $db, $tbl) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE TABLE $tbl AS SELECT g as a, g + 1 as b FROM generate_series(1,10) g(g)'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON SCHEMA public TO $user');
+ $node->safe_psql($db, qq'GRANT SELECT ON $tbl TO $user');
+}
+
+done_testing();
+
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index 81f35986c88..e3b4129ae26 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,6 +136,82 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
+<refsect1>
+ <title>Foreign Data Wrapper</title>
+
+ <para>
+ A Foreign Data Wrapper can be used as a connection name parameter. The foreign
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ </para>
+
+ <para>
+ The authentication with the foreign server can be via password on USER
+ MAPPING or using SCRAM pass-through. The
+ <literal>use_scram_passthrough</literal> on CREATE SERVER options controls
+ whether <filename>dblink_fdw</filename> will use the SCRAM pass-through
+ authentication to connect to the foreign server. With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
+ </para>
+
+ <para>
+ To use SCRAM pass-through authentication:
+ <itemizedlist>
+ <listitem>
+ <para>
+ The remote server must request SCRAM authentication. (If desired,
+ enforce this on the client side (FDW side) with the option
+ <literal>require_auth</literal>.) If another authentication method is
+ requested by the server, then that one will be used normally.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The remote server can be of any PostgreSQL version that supports SCRAM.
+ Support for <literal>use_scram_passthrough</literal> is only required on
+ the client side (FDW side).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The user mapping password is not used. (It could be set to support other
+ authentication methods, but that would arguably violate the point of this
+ feature, which is to avoid storing plain-text passwords.)
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The server running <filename>dblink_fdw</filename> and the remote server
+ must have identical SCRAM secrets (encrypted passwords) for the user being
+ used on <filename>dblink_fdw</filename> to authenticate on the foreign
+ server (same salt and iterations, not merely the same password).
+ </para>
+
+ <para>
+ As a corollary, if FDW connections to multiple hosts are to be made, for
+ example for partitioned foreign tables/sharding, then all hosts must have
+ identical SCRAM secrets for the users involved.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The current session on the PostgreSQL instance that makes the outgoing FDW
+ connections also must also use SCRAM authentication for its incoming client
+ connection. (Hence <quote>pass-through</quote>: SCRAM must be used going in
+ and out.) This is a technical requirement of the SCRAM protocol.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+
+ </refsect1>
+
<refsect1>
<title>Notes</title>
@@ -181,8 +257,9 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection must require password authentication for this to work properly
--- Otherwise, you will receive the following error from dblink_connect():
+-- Note: local connection that don't use SCRAM pass-through require password
+-- authentication for this to work properly. Otherwise, you will receive
+-- the following error from dblink_connect():
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
--
2.39.5 (Apple Git-154)
On Thu, Mar 13, 2025 at 2:38 PM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:
I thought about this; The problem is that at this point, the scram keys
on connection options are base64 encoded (see appendSCRAMKeysInfo), so
we can't compare with the values stored on MyProcPort.
I don't think that's necessary -- the important part is to check
whether they've been unset (empty string).
I've implemented this check in this way because we don't allow adding
the scram keys on user mapping or foreign server options, so the user
can't actually overwrite the scram keys, unless there is the possibility
of filling in these scram keys options in other places besides user
mapping and foreign server options that I am missing?
Understood, but there are two separate comments that claim the code
does something that it doesn't:
+ * All required scram options is set by ourself, so we just need to ensure
+ * that these options are not overwritten by the user.
and
+ * First append hardcoded options needed for SCRAM pass-through, so if the
+ * user overwrite these options we can ereport on dblink_connstr_check and
+ * dblink_security_check.
If the check functions aren't going to check those because it's
unnecessary, then that's fine, but then the comments should be
adjusted.
If we implement this, it needs to check that the keys were actually
sent during scram_exchange(). Having them set on the PGconn doesn't
mean that we used them for authentication.We use the client key and server key on calculate_client_proof and
verify_server_signature respective during memcpy, it would be too hack
to add new fields on pg_conn like scram_client_key_in_use and
scram_server_key_in_use, set them to true on these functions and then
validate that both are true on PQconnectionUsedScramKeys?
I think that's probably a question for Peter: whether or not that
additional API is something we want to support.
-dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr) +dblink_security_check(PGconn *conn, remoteConn *rconn, + const char *connstr)nit: this whitespace change is not necessary now that
useScramPassthrough is no longer in the signature.Fixed
(This diff is still present in v6-0002.)
Speaking of which, does get_connect_string() still need to take
user_mapping as an argument?Yes, because we need to check if the use_scram_passthrough option is set
on foreign server or user mapping options:
if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server,
user_mapping))
appendSCRAMKeysInfo(&buf);
I was referring to the discussion upthread [1]/messages/by-id/CAFY6G8f=jRUAP5yiFRZkHmqstCiRkeeD5Zf2ixVf6HMmjBCgfg@mail.gmail.com; you'd mentioned that
the only reason that get_connect_string() didn't call GetUserMapping()
itself was because we needed that mapping later on for
UseScramPassthrough(). But that's no longer true for this patch,
because the later call to UseScramPassthrough() has been removed. So I
think we can move GetUserMapping() back down, and remove that part of
the refactoring from patch 0001.
Thanks!
--Jacob
[1]: /messages/by-id/CAFY6G8f=jRUAP5yiFRZkHmqstCiRkeeD5Zf2ixVf6HMmjBCgfg@mail.gmail.com
On Mon, Mar 17, 2025 at 1:49 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
On Thu, Mar 13, 2025 at 2:38 PM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:I thought about this; The problem is that at this point, the scram keys
on connection options are base64 encoded (see appendSCRAMKeysInfo), so
we can't compare with the values stored on MyProcPort.I don't think that's necessary -- the important part is to check
whether they've been unset (empty string).I've implemented this check in this way because we don't allow adding
the scram keys on user mapping or foreign server options, so the user
can't actually overwrite the scram keys, unless there is the possibility
of filling in these scram keys options in other places besides user
mapping and foreign server options that I am missing?Understood, but there are two separate comments that claim the code
does something that it doesn't:+ * All required scram options is set by ourself, so we just need to ensure + * that these options are not overwritten by the user.and
+ * First append hardcoded options needed for SCRAM pass-through, so if the + * user overwrite these options we can ereport on dblink_connstr_check and + * dblink_security_check.If the check functions aren't going to check those because it's
unnecessary, then that's fine, but then the comments should be
adjusted.
Ok, now I get all of your points. I've misinterpreted your comments,
sorry about that. I've changed on v7 to validate the scram keys using
the same approach implemented for require_auth, so that now we correctly
check for overwritten scram keys on connection options. I think that the
code comments should be correct now?
If we implement this, it needs to check that the keys were actually
sent during scram_exchange(). Having them set on the PGconn doesn't
mean that we used them for authentication.We use the client key and server key on calculate_client_proof and
verify_server_signature respective during memcpy, it would be too hack
to add new fields on pg_conn like scram_client_key_in_use and
scram_server_key_in_use, set them to true on these functions and then
validate that both are true on PQconnectionUsedScramKeys?I think that's probably a question for Peter: whether or not that
additional API is something we want to support.
Ok
-dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr) +dblink_security_check(PGconn *conn, remoteConn *rconn, + const char *connstr)nit: this whitespace change is not necessary now that
useScramPassthrough is no longer in the signature.Fixed
(This diff is still present in v6-0002.)
Sorry, I think that now is fixed.
Speaking of which, does get_connect_string() still need to take
user_mapping as an argument?Yes, because we need to check if the use_scram_passthrough option is set
on foreign server or user mapping options:
if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server,
user_mapping))
appendSCRAMKeysInfo(&buf);I was referring to the discussion upthread [1]; you'd mentioned that
the only reason that get_connect_string() didn't call GetUserMapping()
itself was because we needed that mapping later on for
UseScramPassthrough(). But that's no longer true for this patch,
because the later call to UseScramPassthrough() has been removed. So I
think we can move GetUserMapping() back down, and remove that part of
the refactoring from patch 0001.
Ok, it totally makes sense. Fixed on v7.
--
Matheus Alcantara
Attachments:
v7-0002-dblink-Add-SCRAM-pass-through-authentication.patchapplication/octet-stream; name=v7-0002-dblink-Add-SCRAM-pass-through-authentication.patchDownload
From f6e07bb98884d9032184c80528e6c986e904180b Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Thu, 20 Feb 2025 15:12:48 -0300
Subject: [PATCH v7 2/2] dblink: Add SCRAM pass-through authentication
This enables SCRAM authentication for dblink_fdw when connecting to a
foreign server using the same approach as it was implemented for
postgres_fdw on 761c79508e.
---
contrib/dblink/Makefile | 1 +
contrib/dblink/dblink.c | 193 ++++++++++++++++++++++-
contrib/dblink/meson.build | 5 +
contrib/dblink/t/001_auth_scram.pl | 242 +++++++++++++++++++++++++++++
doc/src/sgml/dblink.sgml | 81 +++++++++-
5 files changed, 513 insertions(+), 9 deletions(-)
create mode 100644 contrib/dblink/t/001_auth_scram.pl
diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index d4c7ed625ab..fde0b49ddbb 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -13,6 +13,7 @@ PGFILEDESC = "dblink - connect to other PostgreSQL databases"
REGRESS = dblink
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
+TAP_TESTS = 1
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 5bfb1f0db1f..1d54b12372a 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -43,6 +43,8 @@
#include "catalog/pg_foreign_server.h"
#include "catalog/pg_type.h"
#include "catalog/pg_user_mapping.h"
+#include "commands/defrem.h"
+#include "common/base64.h"
#include "executor/spi.h"
#include "foreign/foreign.h"
#include "funcapi.h"
@@ -127,6 +129,11 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
+static bool UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+static void appendSCRAMKeysInfo(StringInfo buf);
+static bool is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context);
+static bool dblink_connstr_has_required_scram_options(const char *connstr);
/* Global */
static remoteConn *pconn = NULL;
@@ -1927,7 +1934,7 @@ dblink_fdw_validator(PG_FUNCTION_ARGS)
{
DefElem *def = (DefElem *) lfirst(cell);
- if (!is_valid_dblink_option(options, def->defname, context))
+ if (!is_valid_dblink_fdw_option(options, def->defname, context))
{
/*
* Unknown option, or invalid option for the context specified, so
@@ -2559,6 +2566,68 @@ deleteConnection(const char *name)
errmsg("undefined connection name")));
}
+ /*
+ * Ensure that require_auth and scram keys are correctly set on connstr.
+ * SCRAM keys used to pass-through is coming from the initial connection from
+ * the client with the server.
+ *
+ * All required scram options is set by ourself, so we just need to ensure
+ * that these options are not overwritten by the user.
+ *
+ * See appendSCRAMKeysInfo and it's usage for more.
+ */
+bool
+dblink_connstr_has_required_scram_options(const char *connstr)
+{
+ PQconninfoOption *options;
+ PQconninfoOption *option;
+ bool has_scram_server_key = false;
+ bool has_scram_client_key = false;
+ bool has_require_auth = false;
+ bool has_scram_keys = false;
+
+ options = PQconninfoParse(connstr, NULL);
+ if (options)
+ {
+ /*
+ * We continue iterating even if we found the keys that we need to
+ * validate to make sure that there is no other declaration of these
+ * keys that can overwrite the first.
+ */
+ for (option = options; option->keyword != NULL; option++)
+ {
+ if (strcmp(option->keyword, "require_auth") == 0)
+ {
+ if (option->val != NULL && strcmp(option->val, "scram-sha-256") == 0)
+ has_require_auth = true;
+ else
+ has_require_auth = false;
+ }
+
+ if (strcmp(option->keyword, "scram_client_key") == 0)
+ {
+ if (option->val != NULL && option->val[0] != '\0')
+ has_scram_client_key = true;
+ else
+ has_scram_client_key = false;
+ }
+
+ if (strcmp(option->keyword, "scram_server_key") == 0)
+ {
+ if (option->val != NULL && option->val[0] != '\0')
+ has_scram_server_key = true;
+ else
+ has_scram_server_key = false;
+ }
+ }
+ PQconninfoFree(options);
+ }
+
+ has_scram_keys = has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys;
+
+ return (has_scram_keys && has_require_auth);
+}
+
/*
* We need to make sure that the connection made used credentials
* which were provided by the user, so check what credentials were
@@ -2575,6 +2644,18 @@ dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
if (PQconnectionUsedPassword(conn) && dblink_connstr_has_pw(connstr))
return;
+ /*
+ * Password was not used to connect, check if scram pass-through is in
+ * use.
+ *
+ * If dblink_connstr_has_required_scram_options is true we assume that
+ * UseScramPassthrough is also true because the required scram keys is
+ * only added if UseScramPassthrough is set, and the user is not allowed
+ * to add the scram keys on fdw and user mapping options.
+ */
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
#ifdef ENABLE_GSS
/* If GSSAPI creds used to connect, make sure it was one delegated */
if (PQconnectionUsedGSSAPI(conn) && be_gssapi_get_delegation(MyProcPort))
@@ -2627,12 +2708,14 @@ dblink_connstr_has_pw(const char *connstr)
}
/*
- * For non-superusers, insist that the connstr specify a password, except
- * if GSSAPI credentials have been delegated (and we check that they are used
- * for the connection in dblink_security_check later). This prevents a
- * password or GSSAPI credentials from being picked up from .pgpass, a
- * service file, the environment, etc. We don't want the postgres user's
- * passwords or Kerberos credentials to be accessible to non-superusers.
+ * For non-superusers, insist that the connstr specify a password, except if
+ * GSSAPI credentials have been delegated (and we check that they are used for
+ * the connection in dblink_security_check later) or if scram pass-through is
+ * being used. This prevents a password or GSSAPI credentials from being
+ * picked up from .pgpass, a service file, the environment, etc. We don't want
+ * the postgres user's passwords or Kerberos credentials to be accessible to
+ * non-superusers. In case of scram pass-through insist that the connstr
+ * has the required scram pass-through options.
*/
static void
dblink_connstr_check(const char *connstr)
@@ -2643,6 +2726,10 @@ dblink_connstr_check(const char *connstr)
if (dblink_connstr_has_pw(connstr))
return;
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
+
#ifdef ENABLE_GSS
if (be_gssapi_get_delegation(MyProcPort))
return;
@@ -2751,6 +2838,7 @@ get_connect_string(ForeignServer *foreign_server)
StringInfoData buf;
ForeignDataWrapper *fdw;
AclResult aclresult;
+ UserMapping *user_mapping;
/* first gather the server connstr options */
Oid serverid = foreign_server->serverid;
@@ -2779,12 +2867,21 @@ get_connect_string(ForeignServer *foreign_server)
}
fdw = GetForeignDataWrapper(fdwid);
+ user_mapping = GetUserMapping(userid, serverid);
/* Check permissions, user must have usage on the server. */
aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ /*
+ * First append hardcoded options needed for SCRAM pass-through, so if the
+ * user overwrite these options we can ereport on dblink_connstr_check and
+ * dblink_security_check.
+ */
+ if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
+ appendSCRAMKeysInfo(&buf);
+
foreach(cell, fdw->options)
{
DefElem *def = lfirst(cell);
@@ -2966,6 +3063,22 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
+
+/*
+ * Same as is_valid_dblink_option but also check for only dblink_fdw specific
+ * options.
+ */
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context)
+{
+ if (strcmp(option, "use_scram_passthrough") == 0)
+ return true;
+
+ return is_valid_dblink_option(options, option, context);
+}
+
+
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3036,6 +3149,70 @@ restoreLocalGucs(int nestlevel)
AtEOXact_GUC(true, nestlevel);
}
+/*
+ * Append SCRAM client key and server key information from the global
+ * MyProcPort into the given StringInfo buffer.
+ */
+static void
+appendSCRAMKeysInfo(StringInfo buf)
+{
+ int len;
+ int encoded_len;
+ char *client_key;
+ char *server_key;
+
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
+ /* don't forget the zero-terminator */
+ client_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ClientKey,
+ sizeof(MyProcPort->scram_ClientKey),
+ client_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM client key");
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ServerKey));
+ /* don't forget the zero-terminator */
+ server_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ServerKey,
+ sizeof(MyProcPort->scram_ServerKey),
+ server_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM server key");
+
+ appendStringInfo(buf, "scram_client_key='%s' ", client_key);
+ appendStringInfo(buf, "scram_server_key='%s' ", server_key);
+ appendStringInfo(buf, "require_auth='scram-sha-256' ");
+
+ pfree(client_key);
+ pfree(server_key);
+}
+
+
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user)
+{
+ ListCell *cell;
+
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ foreach(cell, user->options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ return false;
+}
+
/*
* Connect to remote server. If connstr_or_srvname maps to a foreign server,
* the associated properties and user mapping properties is also used to open
@@ -3060,6 +3237,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
else
connstr = connstr_or_srvname;
+ /* Verify the set of connection parameters. */
dblink_connstr_check(connstr);
/* OK to make connection */
@@ -3077,6 +3255,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
errdetail_internal("%s", msg)));
}
+ /* Perform post-connection security checks. */
dblink_security_check(conn, rconn, connstr);
/* attempt to set client encoding to match server encoding, if needed */
diff --git a/contrib/dblink/meson.build b/contrib/dblink/meson.build
index 3ab78668288..dfd8eb6877e 100644
--- a/contrib/dblink/meson.build
+++ b/contrib/dblink/meson.build
@@ -36,4 +36,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/001_auth_scram.pl',
+ ],
+ },
}
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
new file mode 100644
index 00000000000..e0f1326e50a
--- /dev/null
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -0,0 +1,242 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+# Test SCRAM authentication when opening a new connection with a foreign
+# server.
+#
+# The test is executed by testing the SCRAM authentifcation on a loopback
+# connection on the same server and with different servers.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $user = "user01";
+
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
+my $fdw_server = "db1_fdw";
+my $fdw_server2 = "db2_fdw";
+my $fdw_invalid_server = "db2_fdw_invalid"; # For invalid fdw options
+my $fdw_invalid_server2 =
+ "db2_fdw_invalid2"; # For invalid scram keys fdw options
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+my $node2 = PostgreSQL::Test::Cluster->new('node2');
+
+$node1->init;
+$node2->init;
+
+$node1->start;
+$node2->start;
+
+# Test setup
+
+$node1->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$node2->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$ENV{PGPASSWORD} = "pass";
+
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db0');
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db1');
+$node2->safe_psql('postgres', qq'CREATE DATABASE $db2');
+
+setup_table($node1, $db1, "t");
+setup_table($node2, $db2, "t2");
+
+$node1->safe_psql($db0, 'CREATE EXTENSION IF NOT EXISTS dblink');
+setup_fdw_server($node1, $db0, $fdw_server, $node1, $db1);
+setup_fdw_server($node1, $db0, $fdw_server2, $node2, $db2);
+setup_invalid_fdw_server($node1, $db0, $fdw_invalid_server, $node2, $db2);
+setup_fdw_server($node1, $db0, $fdw_invalid_server2, $node2, $db2);
+
+setup_user_mapping($node1, $db0, $fdw_server);
+setup_user_mapping($node1, $db0, $fdw_server2);
+setup_user_mapping($node1, $db0, $fdw_invalid_server);
+
+# Make the user have the same SCRAM key on both servers. Forcing to have the
+# same iteration and salt.
+my $rolpassword = $node1->safe_psql('postgres',
+ qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';");
+$node2->safe_psql('postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'");
+
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all scram-sha-256
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all scram-sha-256
+});
+
+$node1->restart;
+$node2->restart;
+
+# End of test setup
+
+test_scram_keys_is_not_overwritten($node1, $db0, $fdw_invalid_server2);
+
+test_fdw_auth($node1, $db0, "t", $fdw_server,
+ "SCRAM auth on the same database cluster must succeed");
+
+test_fdw_auth($node1, $db0, "t2", $fdw_server2,
+ "SCRAM auth on a different database cluster must succeed");
+
+test_fdw_auth_with_invalid_overwritten_require_auth($fdw_invalid_server);
+
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+});
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local all all password
+});
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/failed: authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback password fails on a different cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server requested a cleartext password/,
+ 'expected error from loopback password (different cluster)');
+
+
+# Helper functions
+
+sub test_fdw_auth
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $db, $tbl, $fdw, $testname) = @_;
+ my $connstr = $node->connstr($db) . qq' user=$user';
+
+ my $ret = $node->safe_psql(
+ $db,
+ qq"SELECT count(1) FROM dblink('$fdw', 'SELECT * FROM $tbl') AS $tbl(a int, b int)",
+ connstr => $connstr);
+
+ is($ret, '10', $testname);
+}
+
+sub test_fdw_auth_with_invalid_overwritten_require_auth
+{
+
+ my ($fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "select * from dblink('$fdw', 'select * from t') as t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+ is($ret, 3, 'loopback trust fails when overwriting require_auth');
+ like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error when connecting to a fdw overwriting the require_auth'
+ );
+}
+
+sub test_scram_keys_is_not_overwritten
+{
+ my ($node, $db, $fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_client_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret, 3, 'user mapping creation fails when using scram_client_key');
+ like( $stderr,
+ qr/ERROR: invalid option "scram_client_key"/,
+ 'user mapping creation fails when using scram_client_key');
+
+ ($ret, $stdout, $stderr) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_server_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret, 3, 'user mapping creation fails when using scram_server_key');
+ like($stderr,
+ qr/ERROR: invalid option "scram_server_key"/,
+ 'user mapping creation fails when using scram_server_key');
+}
+
+sub setup_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\');'
+ );
+}
+
+sub setup_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_invalid_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\', require_auth \'none\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_table
+{
+ my ($node, $db, $tbl) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE TABLE $tbl AS SELECT g as a, g + 1 as b FROM generate_series(1,10) g(g)'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON SCHEMA public TO $user');
+ $node->safe_psql($db, qq'GRANT SELECT ON $tbl TO $user');
+}
+
+done_testing();
+
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index 81f35986c88..e3b4129ae26 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,6 +136,82 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
+<refsect1>
+ <title>Foreign Data Wrapper</title>
+
+ <para>
+ A Foreign Data Wrapper can be used as a connection name parameter. The foreign
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ </para>
+
+ <para>
+ The authentication with the foreign server can be via password on USER
+ MAPPING or using SCRAM pass-through. The
+ <literal>use_scram_passthrough</literal> on CREATE SERVER options controls
+ whether <filename>dblink_fdw</filename> will use the SCRAM pass-through
+ authentication to connect to the foreign server. With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
+ </para>
+
+ <para>
+ To use SCRAM pass-through authentication:
+ <itemizedlist>
+ <listitem>
+ <para>
+ The remote server must request SCRAM authentication. (If desired,
+ enforce this on the client side (FDW side) with the option
+ <literal>require_auth</literal>.) If another authentication method is
+ requested by the server, then that one will be used normally.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The remote server can be of any PostgreSQL version that supports SCRAM.
+ Support for <literal>use_scram_passthrough</literal> is only required on
+ the client side (FDW side).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The user mapping password is not used. (It could be set to support other
+ authentication methods, but that would arguably violate the point of this
+ feature, which is to avoid storing plain-text passwords.)
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The server running <filename>dblink_fdw</filename> and the remote server
+ must have identical SCRAM secrets (encrypted passwords) for the user being
+ used on <filename>dblink_fdw</filename> to authenticate on the foreign
+ server (same salt and iterations, not merely the same password).
+ </para>
+
+ <para>
+ As a corollary, if FDW connections to multiple hosts are to be made, for
+ example for partitioned foreign tables/sharding, then all hosts must have
+ identical SCRAM secrets for the users involved.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The current session on the PostgreSQL instance that makes the outgoing FDW
+ connections also must also use SCRAM authentication for its incoming client
+ connection. (Hence <quote>pass-through</quote>: SCRAM must be used going in
+ and out.) This is a technical requirement of the SCRAM protocol.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+
+ </refsect1>
+
<refsect1>
<title>Notes</title>
@@ -181,8 +257,9 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection must require password authentication for this to work properly
--- Otherwise, you will receive the following error from dblink_connect():
+-- Note: local connection that don't use SCRAM pass-through require password
+-- authentication for this to work properly. Otherwise, you will receive
+-- the following error from dblink_connect():
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
--
2.39.5 (Apple Git-154)
v7-0001-dblink-refactor-get-connection-routines.patchapplication/octet-stream; name=v7-0001-dblink-refactor-get-connection-routines.patchDownload
From 6debb32dca9e7cfce0d933b6eecdbc9db67f1483 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 20 Jan 2025 15:25:51 -0300
Subject: [PATCH v7 1/2] dblink: refactor get connection routines
Refactor dblink_get_conn and dblink_connect to move the logic of
actually opening the connection to the new connect_pg_server function
which them can be re-used on both functions.
This is a pre-work for a next commit that will add support for scram
pass-through authentication to dblink which will be able to implement
most of the logic into the connect_pg_server function which now already
have all necessary data information.
---
contrib/dblink/dblink.c | 194 ++++++++++++++++++++--------------------
1 file changed, 97 insertions(+), 97 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 58c1a6221c8..5bfb1f0db1f 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -117,7 +117,7 @@ static bool dblink_connstr_has_pw(const char *connstr);
static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
-static char *get_connect_string(const char *servername);
+static char *get_connect_string(ForeignServer *foreign_server);
static char *escape_param_str(const char *str);
static void validate_pkattnums(Relation rel,
int2vector *pkattnums_arg, int32 pknumatts_arg,
@@ -126,6 +126,7 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
const char *option, Oid context);
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
/* Global */
static remoteConn *pconn = NULL;
@@ -199,33 +200,11 @@ dblink_get_conn(char *conname_or_str,
}
else
{
- const char *connstr;
-
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_get_conn == 0)
dblink_we_get_conn = WaitEventExtensionNew("DblinkGetConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_get_conn);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- char *msg = pchomp(PQerrorMessage(conn));
-
- libpqsrv_disconnect(conn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
- dblink_security_check(conn, rconn, connstr);
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_get_conn);
freeconn = true;
conname = NULL;
}
@@ -270,9 +249,7 @@ Datum
dblink_connect(PG_FUNCTION_ARGS)
{
char *conname_or_str = NULL;
- char *connstr = NULL;
char *connname = NULL;
- char *msg;
PGconn *conn = NULL;
remoteConn *rconn = NULL;
@@ -295,40 +272,26 @@ dblink_connect(PG_FUNCTION_ARGS)
rconn->newXactForCursor = false;
}
- /* first check for valid foreign data server */
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
-
- /* check password in connection string if not superuser */
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_connect == 0)
dblink_we_connect = WaitEventExtensionNew("DblinkConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_connect);
-
- if (PQstatus(conn) == CONNECTION_BAD)
+ PG_TRY();
{
- msg = pchomp(PQerrorMessage(conn));
- libpqsrv_disconnect(conn);
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_connect);
+ }
+ PG_CATCH();
+ {
+ /*
+ * If not NULL, rconn is allocated on TopMemoryContext, so we need to
+ * pfree to avoid memory leaks.
+ */
if (rconn)
pfree(rconn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
+ PG_RE_THROW();
}
-
- /* check password actually used if not superuser */
- dblink_security_check(conn, rconn, connstr);
-
- /* attempt to set client encoding to match server encoding, if needed */
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ PG_END_TRY();
if (connname)
{
@@ -2782,15 +2745,17 @@ dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
* Obtain connection string for a foreign server
*/
static char *
-get_connect_string(const char *servername)
+get_connect_string(ForeignServer *foreign_server)
{
- ForeignServer *foreign_server = NULL;
- UserMapping *user_mapping;
ListCell *cell;
StringInfoData buf;
ForeignDataWrapper *fdw;
AclResult aclresult;
- char *srvname;
+
+ /* first gather the server connstr options */
+ Oid serverid = foreign_server->serverid;
+ Oid fdwid = foreign_server->fdwid;
+ Oid userid = GetUserId();
static const PQconninfoOption *options = NULL;
@@ -2813,57 +2778,42 @@ get_connect_string(const char *servername)
errdetail("Could not get libpq's default connection options.")));
}
- /* first gather the server connstr options */
- srvname = pstrdup(servername);
- truncate_identifier(srvname, strlen(srvname), false);
- foreign_server = GetForeignServerByName(srvname, true);
-
- if (foreign_server)
- {
- Oid serverid = foreign_server->serverid;
- Oid fdwid = foreign_server->fdwid;
- Oid userid = GetUserId();
+ fdw = GetForeignDataWrapper(fdwid);
- user_mapping = GetUserMapping(userid, serverid);
- fdw = GetForeignDataWrapper(fdwid);
-
- /* Check permissions, user must have usage on the server. */
- aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
- if (aclresult != ACLCHECK_OK)
- aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
-
- foreach(cell, fdw->options)
- {
- DefElem *def = lfirst(cell);
+ /* Check permissions, user must have usage on the server. */
+ aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
+ if (aclresult != ACLCHECK_OK)
+ aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
- if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, fdw->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, foreign_server->options)
- {
- DefElem *def = lfirst(cell);
+ if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, user_mapping->options)
- {
+ if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- DefElem *def = lfirst(cell);
+ foreach(cell, user_mapping->options)
+ {
- if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ DefElem *def = lfirst(cell);
- return buf.data;
+ if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
}
- else
- return NULL;
+
+ return buf.data;
}
/*
@@ -3085,3 +3035,53 @@ restoreLocalGucs(int nestlevel)
if (nestlevel > 0)
AtEOXact_GUC(true, nestlevel);
}
+
+/*
+ * Connect to remote server. If connstr_or_srvname maps to a foreign server,
+ * the associated properties and user mapping properties is also used to open
+ * the connection. Otherwise a connection will be open using the raw
+ * connstr_or_srvname value.
+ */
+static PGconn *
+connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info)
+{
+ PGconn *conn;
+ ForeignServer *foreign_server = NULL;
+ const char *connstr;
+ char *srvname;
+
+ /* first gather the server connstr options */
+ srvname = pstrdup(connstr_or_srvname);
+ truncate_identifier(srvname, strlen(srvname), false);
+ foreign_server = GetForeignServerByName(srvname, true);
+
+ if (foreign_server)
+ connstr = get_connect_string(foreign_server);
+ else
+ connstr = connstr_or_srvname;
+
+ dblink_connstr_check(connstr);
+
+ /* OK to make connection */
+ conn = libpqsrv_connect(connstr, wait_event_info);
+
+ if (PQstatus(conn) == CONNECTION_BAD)
+ {
+ char *msg = pchomp(PQerrorMessage(conn));
+
+ libpqsrv_disconnect(conn);
+
+ ereport(ERROR,
+ (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
+ errmsg("could not establish connection"),
+ errdetail_internal("%s", msg)));
+ }
+
+ dblink_security_check(conn, rconn, connstr);
+
+ /* attempt to set client encoding to match server encoding, if needed */
+ if (PQclientEncoding(conn) != GetDatabaseEncoding())
+ PQsetClientEncoding(conn, GetDatabaseEncodingName());
+
+ return conn;
+}
--
2.39.5 (Apple Git-154)
On Mon, Mar 17, 2025 at 11:54 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:
If the check functions aren't going to check those because it's
unnecessary, then that's fine, but then the comments should be
adjusted.Ok, now I get all of your points. I've misinterpreted your comments,
sorry about that. I've changed on v7 to validate the scram keys using
the same approach implemented for require_auth, so that now we correctly
check for overwritten scram keys on connection options. I think that the
code comments should be correct now?
Looks good.
I was referring to the discussion upthread [1]; you'd mentioned that
the only reason that get_connect_string() didn't call GetUserMapping()
itself was because we needed that mapping later on for
UseScramPassthrough(). But that's no longer true for this patch,
because the later call to UseScramPassthrough() has been removed. So I
think we can move GetUserMapping() back down, and remove that part of
the refactoring from patch 0001.Ok, it totally makes sense. Fixed on v7.
The fix is in, but it needs to be part of 0001 rather than 0002;
otherwise 0001 doesn't compile.
--
A pgperltidy run is needed for some of the more recent test changes.
But I'm rapidly running out of feedback, so I think this is very
close.
Thanks!
--Jacob
On Mon, Mar 17, 2025 at 8:26 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
I was referring to the discussion upthread [1]; you'd mentioned that
the only reason that get_connect_string() didn't call GetUserMapping()
itself was because we needed that mapping later on for
UseScramPassthrough(). But that's no longer true for this patch,
because the later call to UseScramPassthrough() has been removed. So I
think we can move GetUserMapping() back down, and remove that part of
the refactoring from patch 0001.Ok, it totally makes sense. Fixed on v7.
The fix is in, but it needs to be part of 0001 rather than 0002;
otherwise 0001 doesn't compile.
Fixed. 0001 and 0002 are compiling now.
--
A pgperltidy run is needed for some of the more recent test changes.
But I'm rapidly running out of feedback, so I think this is very
close.
Fixed
Thanks very much for all the reviews on this patch!
--
Matheus Alcantara
Attachments:
v8-0001-dblink-refactor-get-connection-routines.patchapplication/octet-stream; name=v8-0001-dblink-refactor-get-connection-routines.patchDownload
From 1f8c114b5365eba6fcaa958aad8ef5010a6efbff Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 20 Jan 2025 15:25:51 -0300
Subject: [PATCH v8 1/2] dblink: refactor get connection routines
Refactor dblink_get_conn and dblink_connect to move the logic of
actually opening the connection to the new connect_pg_server function
which them can be re-used on both functions.
This is a pre-work for a next commit that will add support for scram
pass-through authentication to dblink which will be able to implement
most of the logic into the connect_pg_server function which now already
have all necessary data information.
---
contrib/dblink/dblink.c | 194 ++++++++++++++++++++--------------------
1 file changed, 98 insertions(+), 96 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 58c1a6221c8..4497b7a4999 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -117,7 +117,7 @@ static bool dblink_connstr_has_pw(const char *connstr);
static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
-static char *get_connect_string(const char *servername);
+static char *get_connect_string(ForeignServer *foreign_server);
static char *escape_param_str(const char *str);
static void validate_pkattnums(Relation rel,
int2vector *pkattnums_arg, int32 pknumatts_arg,
@@ -126,6 +126,7 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
const char *option, Oid context);
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
/* Global */
static remoteConn *pconn = NULL;
@@ -199,33 +200,11 @@ dblink_get_conn(char *conname_or_str,
}
else
{
- const char *connstr;
-
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_get_conn == 0)
dblink_we_get_conn = WaitEventExtensionNew("DblinkGetConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_get_conn);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- char *msg = pchomp(PQerrorMessage(conn));
-
- libpqsrv_disconnect(conn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
- dblink_security_check(conn, rconn, connstr);
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_get_conn);
freeconn = true;
conname = NULL;
}
@@ -270,9 +249,7 @@ Datum
dblink_connect(PG_FUNCTION_ARGS)
{
char *conname_or_str = NULL;
- char *connstr = NULL;
char *connname = NULL;
- char *msg;
PGconn *conn = NULL;
remoteConn *rconn = NULL;
@@ -295,40 +272,26 @@ dblink_connect(PG_FUNCTION_ARGS)
rconn->newXactForCursor = false;
}
- /* first check for valid foreign data server */
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
-
- /* check password in connection string if not superuser */
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_connect == 0)
dblink_we_connect = WaitEventExtensionNew("DblinkConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_connect);
-
- if (PQstatus(conn) == CONNECTION_BAD)
+ PG_TRY();
{
- msg = pchomp(PQerrorMessage(conn));
- libpqsrv_disconnect(conn);
+ conn = connect_pg_server(conname_or_str, rconn, dblink_we_connect);
+ }
+ PG_CATCH();
+ {
+ /*
+ * If not NULL, rconn is allocated on TopMemoryContext, so we need to
+ * pfree to avoid memory leaks.
+ */
if (rconn)
pfree(rconn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
+ PG_RE_THROW();
}
-
- /* check password actually used if not superuser */
- dblink_security_check(conn, rconn, connstr);
-
- /* attempt to set client encoding to match server encoding, if needed */
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ PG_END_TRY();
if (connname)
{
@@ -2782,15 +2745,18 @@ dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
* Obtain connection string for a foreign server
*/
static char *
-get_connect_string(const char *servername)
+get_connect_string(ForeignServer *foreign_server)
{
- ForeignServer *foreign_server = NULL;
UserMapping *user_mapping;
ListCell *cell;
StringInfoData buf;
ForeignDataWrapper *fdw;
AclResult aclresult;
- char *srvname;
+
+ /* first gather the server connstr options */
+ Oid serverid = foreign_server->serverid;
+ Oid fdwid = foreign_server->fdwid;
+ Oid userid = GetUserId();
static const PQconninfoOption *options = NULL;
@@ -2813,57 +2779,43 @@ get_connect_string(const char *servername)
errdetail("Could not get libpq's default connection options.")));
}
- /* first gather the server connstr options */
- srvname = pstrdup(servername);
- truncate_identifier(srvname, strlen(srvname), false);
- foreign_server = GetForeignServerByName(srvname, true);
-
- if (foreign_server)
- {
- Oid serverid = foreign_server->serverid;
- Oid fdwid = foreign_server->fdwid;
- Oid userid = GetUserId();
+ user_mapping = GetUserMapping(userid, serverid);
+ fdw = GetForeignDataWrapper(fdwid);
- user_mapping = GetUserMapping(userid, serverid);
- fdw = GetForeignDataWrapper(fdwid);
-
- /* Check permissions, user must have usage on the server. */
- aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
- if (aclresult != ACLCHECK_OK)
- aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
-
- foreach(cell, fdw->options)
- {
- DefElem *def = lfirst(cell);
+ /* Check permissions, user must have usage on the server. */
+ aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
+ if (aclresult != ACLCHECK_OK)
+ aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
- if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, fdw->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, foreign_server->options)
- {
- DefElem *def = lfirst(cell);
+ if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, user_mapping->options)
- {
+ if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- DefElem *def = lfirst(cell);
+ foreach(cell, user_mapping->options)
+ {
- if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ DefElem *def = lfirst(cell);
- return buf.data;
+ if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
}
- else
- return NULL;
+
+ return buf.data;
}
/*
@@ -3085,3 +3037,53 @@ restoreLocalGucs(int nestlevel)
if (nestlevel > 0)
AtEOXact_GUC(true, nestlevel);
}
+
+/*
+ * Connect to remote server. If connstr_or_srvname maps to a foreign server,
+ * the associated properties and user mapping properties is also used to open
+ * the connection. Otherwise a connection will be open using the raw
+ * connstr_or_srvname value.
+ */
+static PGconn *
+connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info)
+{
+ PGconn *conn;
+ ForeignServer *foreign_server = NULL;
+ const char *connstr;
+ char *srvname;
+
+ /* first gather the server connstr options */
+ srvname = pstrdup(connstr_or_srvname);
+ truncate_identifier(srvname, strlen(srvname), false);
+ foreign_server = GetForeignServerByName(srvname, true);
+
+ if (foreign_server)
+ connstr = get_connect_string(foreign_server);
+ else
+ connstr = connstr_or_srvname;
+
+ dblink_connstr_check(connstr);
+
+ /* OK to make connection */
+ conn = libpqsrv_connect(connstr, wait_event_info);
+
+ if (PQstatus(conn) == CONNECTION_BAD)
+ {
+ char *msg = pchomp(PQerrorMessage(conn));
+
+ libpqsrv_disconnect(conn);
+
+ ereport(ERROR,
+ (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
+ errmsg("could not establish connection"),
+ errdetail_internal("%s", msg)));
+ }
+
+ dblink_security_check(conn, rconn, connstr);
+
+ /* attempt to set client encoding to match server encoding, if needed */
+ if (PQclientEncoding(conn) != GetDatabaseEncoding())
+ PQsetClientEncoding(conn, GetDatabaseEncodingName());
+
+ return conn;
+}
--
2.39.5 (Apple Git-154)
v8-0002-dblink-Add-SCRAM-pass-through-authentication.patchapplication/octet-stream; name=v8-0002-dblink-Add-SCRAM-pass-through-authentication.patchDownload
From 9596f9a90f557a6e5b75d10d295e1c2f0f234dec Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Thu, 20 Feb 2025 15:12:48 -0300
Subject: [PATCH v8 2/2] dblink: Add SCRAM pass-through authentication
This enables SCRAM authentication for dblink_fdw when connecting to a
foreign server using the same approach as it was implemented for
postgres_fdw on 761c79508e.
---
contrib/dblink/Makefile | 1 +
contrib/dblink/dblink.c | 191 +++++++++++++++++++++-
contrib/dblink/meson.build | 5 +
contrib/dblink/t/001_auth_scram.pl | 249 +++++++++++++++++++++++++++++
doc/src/sgml/dblink.sgml | 81 +++++++++-
5 files changed, 518 insertions(+), 9 deletions(-)
create mode 100644 contrib/dblink/t/001_auth_scram.pl
diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index d4c7ed625ab..fde0b49ddbb 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -13,6 +13,7 @@ PGFILEDESC = "dblink - connect to other PostgreSQL databases"
REGRESS = dblink
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
+TAP_TESTS = 1
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 4497b7a4999..50959c4f5bb 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -43,6 +43,8 @@
#include "catalog/pg_foreign_server.h"
#include "catalog/pg_type.h"
#include "catalog/pg_user_mapping.h"
+#include "commands/defrem.h"
+#include "common/base64.h"
#include "executor/spi.h"
#include "foreign/foreign.h"
#include "funcapi.h"
@@ -127,6 +129,11 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
static PGconn *connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event_info);
+static bool UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+static void appendSCRAMKeysInfo(StringInfo buf);
+static bool is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context);
+static bool dblink_connstr_has_required_scram_options(const char *connstr);
/* Global */
static remoteConn *pconn = NULL;
@@ -1927,7 +1934,7 @@ dblink_fdw_validator(PG_FUNCTION_ARGS)
{
DefElem *def = (DefElem *) lfirst(cell);
- if (!is_valid_dblink_option(options, def->defname, context))
+ if (!is_valid_dblink_fdw_option(options, def->defname, context))
{
/*
* Unknown option, or invalid option for the context specified, so
@@ -2559,6 +2566,68 @@ deleteConnection(const char *name)
errmsg("undefined connection name")));
}
+ /*
+ * Ensure that require_auth and scram keys are correctly set on connstr.
+ * SCRAM keys used to pass-through is coming from the initial connection from
+ * the client with the server.
+ *
+ * All required scram options is set by ourself, so we just need to ensure
+ * that these options are not overwritten by the user.
+ *
+ * See appendSCRAMKeysInfo and it's usage for more.
+ */
+bool
+dblink_connstr_has_required_scram_options(const char *connstr)
+{
+ PQconninfoOption *options;
+ PQconninfoOption *option;
+ bool has_scram_server_key = false;
+ bool has_scram_client_key = false;
+ bool has_require_auth = false;
+ bool has_scram_keys = false;
+
+ options = PQconninfoParse(connstr, NULL);
+ if (options)
+ {
+ /*
+ * We continue iterating even if we found the keys that we need to
+ * validate to make sure that there is no other declaration of these
+ * keys that can overwrite the first.
+ */
+ for (option = options; option->keyword != NULL; option++)
+ {
+ if (strcmp(option->keyword, "require_auth") == 0)
+ {
+ if (option->val != NULL && strcmp(option->val, "scram-sha-256") == 0)
+ has_require_auth = true;
+ else
+ has_require_auth = false;
+ }
+
+ if (strcmp(option->keyword, "scram_client_key") == 0)
+ {
+ if (option->val != NULL && option->val[0] != '\0')
+ has_scram_client_key = true;
+ else
+ has_scram_client_key = false;
+ }
+
+ if (strcmp(option->keyword, "scram_server_key") == 0)
+ {
+ if (option->val != NULL && option->val[0] != '\0')
+ has_scram_server_key = true;
+ else
+ has_scram_server_key = false;
+ }
+ }
+ PQconninfoFree(options);
+ }
+
+ has_scram_keys = has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys;
+
+ return (has_scram_keys && has_require_auth);
+}
+
/*
* We need to make sure that the connection made used credentials
* which were provided by the user, so check what credentials were
@@ -2575,6 +2644,18 @@ dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
if (PQconnectionUsedPassword(conn) && dblink_connstr_has_pw(connstr))
return;
+ /*
+ * Password was not used to connect, check if scram pass-through is in
+ * use.
+ *
+ * If dblink_connstr_has_required_scram_options is true we assume that
+ * UseScramPassthrough is also true because the required scram keys is
+ * only added if UseScramPassthrough is set, and the user is not allowed
+ * to add the scram keys on fdw and user mapping options.
+ */
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
#ifdef ENABLE_GSS
/* If GSSAPI creds used to connect, make sure it was one delegated */
if (PQconnectionUsedGSSAPI(conn) && be_gssapi_get_delegation(MyProcPort))
@@ -2627,12 +2708,14 @@ dblink_connstr_has_pw(const char *connstr)
}
/*
- * For non-superusers, insist that the connstr specify a password, except
- * if GSSAPI credentials have been delegated (and we check that they are used
- * for the connection in dblink_security_check later). This prevents a
- * password or GSSAPI credentials from being picked up from .pgpass, a
- * service file, the environment, etc. We don't want the postgres user's
- * passwords or Kerberos credentials to be accessible to non-superusers.
+ * For non-superusers, insist that the connstr specify a password, except if
+ * GSSAPI credentials have been delegated (and we check that they are used for
+ * the connection in dblink_security_check later) or if scram pass-through is
+ * being used. This prevents a password or GSSAPI credentials from being
+ * picked up from .pgpass, a service file, the environment, etc. We don't want
+ * the postgres user's passwords or Kerberos credentials to be accessible to
+ * non-superusers. In case of scram pass-through insist that the connstr
+ * has the required scram pass-through options.
*/
static void
dblink_connstr_check(const char *connstr)
@@ -2643,6 +2726,10 @@ dblink_connstr_check(const char *connstr)
if (dblink_connstr_has_pw(connstr))
return;
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
+
#ifdef ENABLE_GSS
if (be_gssapi_get_delegation(MyProcPort))
return;
@@ -2787,6 +2874,14 @@ get_connect_string(ForeignServer *foreign_server)
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ /*
+ * First append hardcoded options needed for SCRAM pass-through, so if the
+ * user overwrite these options we can ereport on dblink_connstr_check and
+ * dblink_security_check.
+ */
+ if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
+ appendSCRAMKeysInfo(&buf);
+
foreach(cell, fdw->options)
{
DefElem *def = lfirst(cell);
@@ -2968,6 +3063,22 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
+
+/*
+ * Same as is_valid_dblink_option but also check for only dblink_fdw specific
+ * options.
+ */
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context)
+{
+ if (strcmp(option, "use_scram_passthrough") == 0)
+ return true;
+
+ return is_valid_dblink_option(options, option, context);
+}
+
+
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3038,6 +3149,70 @@ restoreLocalGucs(int nestlevel)
AtEOXact_GUC(true, nestlevel);
}
+/*
+ * Append SCRAM client key and server key information from the global
+ * MyProcPort into the given StringInfo buffer.
+ */
+static void
+appendSCRAMKeysInfo(StringInfo buf)
+{
+ int len;
+ int encoded_len;
+ char *client_key;
+ char *server_key;
+
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
+ /* don't forget the zero-terminator */
+ client_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ClientKey,
+ sizeof(MyProcPort->scram_ClientKey),
+ client_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM client key");
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ServerKey));
+ /* don't forget the zero-terminator */
+ server_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ServerKey,
+ sizeof(MyProcPort->scram_ServerKey),
+ server_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM server key");
+
+ appendStringInfo(buf, "scram_client_key='%s' ", client_key);
+ appendStringInfo(buf, "scram_server_key='%s' ", server_key);
+ appendStringInfo(buf, "require_auth='scram-sha-256' ");
+
+ pfree(client_key);
+ pfree(server_key);
+}
+
+
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user)
+{
+ ListCell *cell;
+
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ foreach(cell, user->options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ return false;
+}
+
/*
* Connect to remote server. If connstr_or_srvname maps to a foreign server,
* the associated properties and user mapping properties is also used to open
@@ -3062,6 +3237,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
else
connstr = connstr_or_srvname;
+ /* Verify the set of connection parameters. */
dblink_connstr_check(connstr);
/* OK to make connection */
@@ -3079,6 +3255,7 @@ connect_pg_server(char *connstr_or_srvname, remoteConn *rconn, uint32 wait_event
errdetail_internal("%s", msg)));
}
+ /* Perform post-connection security checks. */
dblink_security_check(conn, rconn, connstr);
/* attempt to set client encoding to match server encoding, if needed */
diff --git a/contrib/dblink/meson.build b/contrib/dblink/meson.build
index 3ab78668288..dfd8eb6877e 100644
--- a/contrib/dblink/meson.build
+++ b/contrib/dblink/meson.build
@@ -36,4 +36,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/001_auth_scram.pl',
+ ],
+ },
}
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
new file mode 100644
index 00000000000..3a94a27cb0c
--- /dev/null
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -0,0 +1,249 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+# Test SCRAM authentication when opening a new connection with a foreign
+# server.
+#
+# The test is executed by testing the SCRAM authentifcation on a loopback
+# connection on the same server and with different servers.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $user = "user01";
+
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
+my $fdw_server = "db1_fdw";
+my $fdw_server2 = "db2_fdw";
+my $fdw_invalid_server = "db2_fdw_invalid"; # For invalid fdw options
+my $fdw_invalid_server2 =
+ "db2_fdw_invalid2"; # For invalid scram keys fdw options
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+my $node2 = PostgreSQL::Test::Cluster->new('node2');
+
+$node1->init;
+$node2->init;
+
+$node1->start;
+$node2->start;
+
+# Test setup
+
+$node1->safe_psql( 'postgres', qq'CREATE USER $user WITH password \'pass\'' );
+$node2->safe_psql( 'postgres', qq'CREATE USER $user WITH password \'pass\'' );
+$ENV{PGPASSWORD} = "pass";
+
+$node1->safe_psql( 'postgres', qq'CREATE DATABASE $db0' );
+$node1->safe_psql( 'postgres', qq'CREATE DATABASE $db1' );
+$node2->safe_psql( 'postgres', qq'CREATE DATABASE $db2' );
+
+setup_table( $node1, $db1, "t" );
+setup_table( $node2, $db2, "t2" );
+
+$node1->safe_psql( $db0, 'CREATE EXTENSION IF NOT EXISTS dblink' );
+setup_fdw_server( $node1, $db0, $fdw_server, $node1, $db1 );
+setup_fdw_server( $node1, $db0, $fdw_server2, $node2, $db2 );
+setup_invalid_fdw_server( $node1, $db0, $fdw_invalid_server, $node2, $db2 );
+setup_fdw_server( $node1, $db0, $fdw_invalid_server2, $node2, $db2 );
+
+setup_user_mapping( $node1, $db0, $fdw_server );
+setup_user_mapping( $node1, $db0, $fdw_server2 );
+setup_user_mapping( $node1, $db0, $fdw_invalid_server );
+
+# Make the user have the same SCRAM key on both servers. Forcing to have the
+# same iteration and salt.
+my $rolpassword = $node1->safe_psql( 'postgres',
+ qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';" );
+$node2->safe_psql( 'postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'" );
+
+unlink( $node1->data_dir . '/pg_hba.conf' );
+unlink( $node2->data_dir . '/pg_hba.conf' );
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all scram-sha-256
+}
+);
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all scram-sha-256
+}
+);
+
+$node1->restart;
+$node2->restart;
+
+# End of test setup
+
+test_scram_keys_is_not_overwritten( $node1, $db0, $fdw_invalid_server2 );
+
+test_fdw_auth( $node1, $db0, "t", $fdw_server,
+ "SCRAM auth on the same database cluster must succeed" );
+
+test_fdw_auth( $node1, $db0, "t2", $fdw_server2,
+ "SCRAM auth on a different database cluster must succeed" );
+
+test_fdw_auth_with_invalid_overwritten_require_auth($fdw_invalid_server);
+
+# Ensure that trust connections fail without superuser opt-in.
+unlink( $node1->data_dir . '/pg_hba.conf' );
+unlink( $node2->data_dir . '/pg_hba.conf' );
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+}
+);
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local all all password
+}
+);
+
+$node1->restart;
+$node2->restart;
+
+my ( $ret, $stdout, $stderr ) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user"
+);
+
+is( $ret, 3, 'loopback trust fails on the same cluster' );
+like(
+ $stderr,
+qr/failed: authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (same cluster)'
+);
+
+( $ret, $stdout, $stderr ) = $node1->psql(
+ $db0,
+"SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user"
+);
+
+is( $ret, 3, 'loopback password fails on a different cluster' );
+like(
+ $stderr,
+qr/authentication method requirement "scram-sha-256" failed: server requested a cleartext password/,
+ 'expected error from loopback password (different cluster)'
+);
+
+# Helper functions
+
+sub test_fdw_auth {
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ( $node, $db, $tbl, $fdw, $testname ) = @_;
+ my $connstr = $node->connstr($db) . qq' user=$user';
+
+ my $ret = $node->safe_psql(
+ $db,
+qq"SELECT count(1) FROM dblink('$fdw', 'SELECT * FROM $tbl') AS $tbl(a int, b int)",
+ connstr => $connstr
+ );
+
+ is( $ret, '10', $testname );
+}
+
+sub test_fdw_auth_with_invalid_overwritten_require_auth {
+
+ my ($fdw) = @_;
+
+ my ( $ret, $stdout, $stderr ) = $node1->psql(
+ $db0,
+ "select * from dblink('$fdw', 'select * from t') as t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user"
+ );
+
+ is( $ret, 3, 'loopback trust fails when overwriting require_auth' );
+ like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error when connecting to a fdw overwriting the require_auth'
+ );
+}
+
+sub test_scram_keys_is_not_overwritten {
+ my ( $node, $db, $fdw ) = @_;
+
+ my ( $ret, $stdout, $stderr ) = $node->psql(
+ $db,
+qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_client_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user"
+ );
+
+ is( $ret, 3, 'user mapping creation fails when using scram_client_key' );
+ like(
+ $stderr,
+ qr/ERROR: invalid option "scram_client_key"/,
+ 'user mapping creation fails when using scram_client_key'
+ );
+
+ ( $ret, $stdout, $stderr ) = $node->psql(
+ $db,
+qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_server_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user"
+ );
+
+ is( $ret, 3, 'user mapping creation fails when using scram_server_key' );
+ like(
+ $stderr,
+ qr/ERROR: invalid option "scram_server_key"/,
+ 'user mapping creation fails when using scram_server_key'
+ );
+}
+
+sub setup_user_mapping {
+ my ( $node, $db, $fdw ) = @_;
+
+ $node->safe_psql( $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\');'
+ );
+}
+
+sub setup_fdw_server {
+ my ( $node, $db, $fdw, $fdw_node, $dbname ) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\') '
+ );
+ $node->safe_psql( $db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;' );
+ $node->safe_psql( $db, qq'GRANT ALL ON SCHEMA public TO $user' );
+}
+
+sub setup_invalid_fdw_server {
+ my ( $node, $db, $fdw, $fdw_node, $dbname ) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\', require_auth \'none\') '
+ );
+ $node->safe_psql( $db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;' );
+ $node->safe_psql( $db, qq'GRANT ALL ON SCHEMA public TO $user' );
+}
+
+sub setup_table {
+ my ( $node, $db, $tbl ) = @_;
+
+ $node->safe_psql( $db,
+qq'CREATE TABLE $tbl AS SELECT g as a, g + 1 as b FROM generate_series(1,10) g(g)'
+ );
+ $node->safe_psql( $db, qq'GRANT USAGE ON SCHEMA public TO $user' );
+ $node->safe_psql( $db, qq'GRANT SELECT ON $tbl TO $user' );
+}
+
+done_testing();
+
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index 81f35986c88..e3b4129ae26 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,6 +136,82 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
+<refsect1>
+ <title>Foreign Data Wrapper</title>
+
+ <para>
+ A Foreign Data Wrapper can be used as a connection name parameter. The foreign
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ </para>
+
+ <para>
+ The authentication with the foreign server can be via password on USER
+ MAPPING or using SCRAM pass-through. The
+ <literal>use_scram_passthrough</literal> on CREATE SERVER options controls
+ whether <filename>dblink_fdw</filename> will use the SCRAM pass-through
+ authentication to connect to the foreign server. With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
+ </para>
+
+ <para>
+ To use SCRAM pass-through authentication:
+ <itemizedlist>
+ <listitem>
+ <para>
+ The remote server must request SCRAM authentication. (If desired,
+ enforce this on the client side (FDW side) with the option
+ <literal>require_auth</literal>.) If another authentication method is
+ requested by the server, then that one will be used normally.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The remote server can be of any PostgreSQL version that supports SCRAM.
+ Support for <literal>use_scram_passthrough</literal> is only required on
+ the client side (FDW side).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The user mapping password is not used. (It could be set to support other
+ authentication methods, but that would arguably violate the point of this
+ feature, which is to avoid storing plain-text passwords.)
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The server running <filename>dblink_fdw</filename> and the remote server
+ must have identical SCRAM secrets (encrypted passwords) for the user being
+ used on <filename>dblink_fdw</filename> to authenticate on the foreign
+ server (same salt and iterations, not merely the same password).
+ </para>
+
+ <para>
+ As a corollary, if FDW connections to multiple hosts are to be made, for
+ example for partitioned foreign tables/sharding, then all hosts must have
+ identical SCRAM secrets for the users involved.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The current session on the PostgreSQL instance that makes the outgoing FDW
+ connections also must also use SCRAM authentication for its incoming client
+ connection. (Hence <quote>pass-through</quote>: SCRAM must be used going in
+ and out.) This is a technical requirement of the SCRAM protocol.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+
+ </refsect1>
+
<refsect1>
<title>Notes</title>
@@ -181,8 +257,9 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection must require password authentication for this to work properly
--- Otherwise, you will receive the following error from dblink_connect():
+-- Note: local connection that don't use SCRAM pass-through require password
+-- authentication for this to work properly. Otherwise, you will receive
+-- the following error from dblink_connect():
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
--
2.39.5 (Apple Git-154)
On 17.03.25 17:49, Jacob Champion wrote:
If we implement this, it needs to check that the keys were actually
sent during scram_exchange(). Having them set on the PGconn doesn't
mean that we used them for authentication.We use the client key and server key on calculate_client_proof and
verify_server_signature respective during memcpy, it would be too hack
to add new fields on pg_conn like scram_client_key_in_use and
scram_server_key_in_use, set them to true on these functions and then
validate that both are true on PQconnectionUsedScramKeys?I think that's probably a question for Peter: whether or not that
additional API is something we want to support.
So the way I understand this is that the options are:
(1) We add a libpq function like PQconnectionUsedScramKeys() in the
style of PQconnectionUsedPassword() and call that function during the
checks.
(2) We make use_scram_passthrough=true imply require_auth=scram-sha-256.
This is essentially a way to get the info from (1) out of libpq using
existing facilities. But it would preempt certain setups that might
otherwise work. (Which ones? Are they important?)
Why can't we use PQconnectionUsedPassword()? What problems would that
leave? The example test case that Jacob showed earlier involved the
remote server using "trust". We don't want that, of course. But what
we want to make sure is that some kind of authentication happened
between postgres_fdw and the remote server. PQconnectionUsedPassword()
does indicate that.
(Or could we just stick in require_auth=!none to solve this problem once
and for all?)
On Tue, Mar 18, 2025 at 9:35 AM Peter Eisentraut <peter@eisentraut.org> wrote:
So the way I understand this is that the options are:
(1) We add a libpq function like PQconnectionUsedScramKeys() in the
style of PQconnectionUsedPassword() and call that function during the
checks.(2) We make use_scram_passthrough=true imply require_auth=scram-sha-256.
This is essentially a way to get the info from (1) out of libpq using
existing facilities.
Right.
But it would preempt certain setups that might
otherwise work. (Which ones? Are they important?)
If the backend HBA later changes, to require delegated GSS or a
different type of password authentication, the user will have to unset
use_scram_passthrough (or ask the owner of the foreign server to unset
it). Whereas before they could just add a password to their user
mapping or enable delegation to move forward immediately.
I think this is probably not a serious limitation, in practice.
Why can't we use PQconnectionUsedPassword()? What problems would that
leave? The example test case that Jacob showed earlier involved the
remote server using "trust". We don't want that, of course. But what
we want to make sure is that some kind of authentication happened
between postgres_fdw and the remote server. PQconnectionUsedPassword()
does indicate that.
That test also added a "password" HBA case, where the correct password
got pulled from the environment instead of the connection string.
Making sure authentication happens is only one part -- we have to
ensure authentication takes place using the end user's credentials and
not the server's.
So since PQconnectionUsedPassword() can't differentiate between "I
used your SCRAM key" and "I used a password I found lying around on
disk", it's not strong enough for this check.
(Or could we just stick in require_auth=!none to solve this problem once
and for all?)
It solves the trust case nicely (and we should maybe consider it for a
future simplification?), but not the "wrong credentials were used"
case.
Thanks!
--Jacob
On 18.03.25 17:53, Jacob Champion wrote:
On Tue, Mar 18, 2025 at 9:35 AM Peter Eisentraut <peter@eisentraut.org> wrote:
So the way I understand this is that the options are:
(1) We add a libpq function like PQconnectionUsedScramKeys() in the
style of PQconnectionUsedPassword() and call that function during the
checks.(2) We make use_scram_passthrough=true imply require_auth=scram-sha-256.
This is essentially a way to get the info from (1) out of libpq using
existing facilities.Right.
But it would preempt certain setups that might
otherwise work. (Which ones? Are they important?)If the backend HBA later changes, to require delegated GSS or a
different type of password authentication, the user will have to unset
use_scram_passthrough (or ask the owner of the foreign server to unset
it). Whereas before they could just add a password to their user
mapping or enable delegation to move forward immediately.I think this is probably not a serious limitation, in practice.
Yeah, I think option (2) is enough for now. If someone wants to enable
the kinds of things you describe, they can always come back and
implement option (1) later.
On Tue, Mar 18, 2025 at 12:32 PM Peter Eisentraut <peter@eisentraut.org> wrote:
Yeah, I think option (2) is enough for now. If someone wants to enable
the kinds of things you describe, they can always come back and
implement option (1) later.
Sounds good to me.
--
Notes on v8:
- The following documentation pieces need to be adjusted, now that
we've decided that `use_scram_passthrough` will enforce
`require_auth=scram-sha-256`:
+ The remote server must request SCRAM authentication. (If desired, + enforce this on the client side (FDW side) with the option + <literal>require_auth</literal>.) If another authentication method is + requested by the server, then that one will be used normally.
and
+ The user mapping password is not used. (It could be set to support other + authentication methods, but that would arguably violate the point of this + feature, which is to avoid storing plain-text passwords.)
I think they should just be reduced to "The remote server must request
SCRAM authentication." and "The user mapping password is not used."
- In get_connect_string():
+ /* first gather the server connstr options */ + Oid serverid = foreign_server->serverid;
I think this comment belongs elsewhere (connect_pg_server) and should
be deleted from this block.
- Sorry for not realizing this before now, but I couldn't figure out
why connect_pg_server() took the rconn pointer, and it turns out it
just passes it along to dblink_security_check(), which pfree's it
before throwing an error. So that will double-free with the current
refactoring patch (and I'm not sure why assertions aren't catching
that?).
I thought for sure this inconsistency would be a problem on HEAD, but
it turns out that rconn is set to NULL in the code path where it would
be a bug... How confusing.
Now that we handle the pfree() in PG_CATCH instead, that lower-level
pfree should be removed, and then connect_pg_server() doesn't need to
take an rconn pointer at all. For extra credit you could maybe move
the allocation of rconn down below the call to connect_pg_server(),
and get rid of the try/catch?
+ /* Verify the set of connection parameters. */ dblink_connstr_check(connstr); ... + /* Perform post-connection security checks. */ dblink_security_check(conn, rconn, connstr);
- These comment additions probably belong in 0001 rather than 0002.
- As discussed offlist, 0002 needs pgperltidy'd rather than perltidy'd.
- I have attached some additional nitpicky comment edits and
whitespace changes as a diff; pick and choose as desired.
Thanks!
--Jacob
Attachments:
fixup.diff.txttext/plain; charset=US-ASCII; name=fixup.diff.txtDownload
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 50959c4f5bb..d333f1c5429 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -2567,14 +2567,14 @@ deleteConnection(const char *name)
}
/*
- * Ensure that require_auth and scram keys are correctly set on connstr.
- * SCRAM keys used to pass-through is coming from the initial connection from
- * the client with the server.
+ * Ensure that require_auth and SCRAM keys are correctly set on connstr.
+ * SCRAM keys used to pass-through are coming from the initial connection
+ * from the client with the server.
*
- * All required scram options is set by ourself, so we just need to ensure
+ * All required SCRAM options are set by dblink, so we just need to ensure
* that these options are not overwritten by the user.
*
- * See appendSCRAMKeysInfo and it's usage for more.
+ * See appendSCRAMKeysInfo and its usage for more.
*/
bool
dblink_connstr_has_required_scram_options(const char *connstr)
@@ -2645,13 +2645,13 @@ dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
return;
/*
- * Password was not used to connect, check if scram pass-through is in
+ * Password was not used to connect, check if SCRAM pass-through is in
* use.
*
* If dblink_connstr_has_required_scram_options is true we assume that
- * UseScramPassthrough is also true because the required scram keys is
+ * UseScramPassthrough is also true because the required SCRAM keys are
* only added if UseScramPassthrough is set, and the user is not allowed
- * to add the scram keys on fdw and user mapping options.
+ * to add the SCRAM keys on fdw and user mapping options.
*/
if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
return;
@@ -2710,12 +2710,12 @@ dblink_connstr_has_pw(const char *connstr)
/*
* For non-superusers, insist that the connstr specify a password, except if
* GSSAPI credentials have been delegated (and we check that they are used for
- * the connection in dblink_security_check later) or if scram pass-through is
+ * the connection in dblink_security_check later) or if SCRAM pass-through is
* being used. This prevents a password or GSSAPI credentials from being
* picked up from .pgpass, a service file, the environment, etc. We don't want
* the postgres user's passwords or Kerberos credentials to be accessible to
- * non-superusers. In case of scram pass-through insist that the connstr
- * has the required scram pass-through options.
+ * non-superusers. In case of SCRAM pass-through insist that the connstr
+ * has the required SCRAM pass-through options.
*/
static void
dblink_connstr_check(const char *connstr)
@@ -2729,7 +2729,6 @@ dblink_connstr_check(const char *connstr)
if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
return;
-
#ifdef ENABLE_GSS
if (be_gssapi_get_delegation(MyProcPort))
return;
@@ -2876,8 +2875,8 @@ get_connect_string(ForeignServer *foreign_server)
/*
* First append hardcoded options needed for SCRAM pass-through, so if the
- * user overwrite these options we can ereport on dblink_connstr_check and
- * dblink_security_check.
+ * user overwrites these options we can ereport on dblink_connstr_check
+ * and dblink_security_check.
*/
if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
appendSCRAMKeysInfo(&buf);
@@ -3063,7 +3062,6 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
-
/*
* Same as is_valid_dblink_option but also check for only dblink_fdw specific
* options.
@@ -3078,7 +3076,6 @@ is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
return is_valid_dblink_option(options, option, context);
}
-
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3161,7 +3158,6 @@ appendSCRAMKeysInfo(StringInfo buf)
char *client_key;
char *server_key;
-
len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
/* don't forget the zero-terminator */
client_key = palloc0(len + 1);
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index e3b4129ae26..9cbcf645e78 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,7 +136,7 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
-<refsect1>
+ <refsect1>
<title>Foreign Data Wrapper</title>
<para>
@@ -208,8 +208,6 @@ dblink_connect(text connname, text connstr) returns text
</listitem>
</itemizedlist>
</para>
-
-
</refsect1>
<refsect1>
@@ -257,7 +255,7 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection that don't use SCRAM pass-through require password
+-- Note: local connections that don't use SCRAM pass-through require password
-- authentication for this to work properly. Otherwise, you will receive
-- the following error from dblink_connect():
-- ERROR: password is required
On Wed, Mar 19, 2025 at 4:21 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
On Tue, Mar 18, 2025 at 12:32 PM Peter Eisentraut <peter@eisentraut.org> wrote:
Yeah, I think option (2) is enough for now. If someone wants to enable
the kinds of things you describe, they can always come back and
implement option (1) later.Sounds good to me.
Since the security checks are defined I'm attaching 0003 which include
the fix of security checks for postgres_fdw. It implements the
validations very similar to what are being implemented on dblink.
Notes on v8:
- The following documentation pieces need to be adjusted, now that
we've decided that `use_scram_passthrough` will enforce
`require_auth=scram-sha-256`:+ The remote server must request SCRAM authentication. (If desired, + enforce this on the client side (FDW side) with the option + <literal>require_auth</literal>.) If another authentication method is + requested by the server, then that one will be used normally.and
+ The user mapping password is not used. (It could be set to support other + authentication methods, but that would arguably violate the point of this + feature, which is to avoid storing plain-text passwords.)I think they should just be reduced to "The remote server must request
SCRAM authentication." and "The user mapping password is not used."
I've removed the "user mapping password" <listitem> because we already
mentioned above that the password is not used and having just "The user
mapping password is not used." again seems redundant, what do you think?
+ ... With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
- In get_connect_string():
+ /* first gather the server connstr options */ + Oid serverid = foreign_server->serverid;I think this comment belongs elsewhere (connect_pg_server) and should
be deleted from this block.
Removed
- Sorry for not realizing this before now, but I couldn't figure out
why connect_pg_server() took the rconn pointer, and it turns out it
just passes it along to dblink_security_check(), which pfree's it
before throwing an error. So that will double-free with the current
refactoring patch (and I'm not sure why assertions aren't catching
that?).I thought for sure this inconsistency would be a problem on HEAD, but
it turns out that rconn is set to NULL in the code path where it would
be a bug... How confusing.Now that we handle the pfree() in PG_CATCH instead, that lower-level
pfree should be removed, and then connect_pg_server() doesn't need to
take an rconn pointer at all. For extra credit you could maybe move
the allocation of rconn down below the call to connect_pg_server(),
and get rid of the try/catch?
Good catch, thanks, it's much better now! With this change we can also
remove the second if (connname) condition. All fixed on attached.
+ /* Verify the set of connection parameters. */ dblink_connstr_check(connstr); ... + /* Perform post-connection security checks. */ dblink_security_check(conn, rconn, connstr);- These comment additions probably belong in 0001 rather than 0002.
Fixed
- As discussed offlist, 0002 needs pgperltidy'd rather than perltidy'd.
Fixed
- I have attached some additional nitpicky comment edits and
whitespace changes as a diff; pick and choose as desired.
I've squashed into this new version thanks!
--
Matheus Alcantara
Attachments:
v9-0003-postgres_fdw-improve-security-checks.patchapplication/octet-stream; name=v9-0003-postgres_fdw-improve-security-checks.patchDownload
From a8bbbe75034979f15a56815127a2d6439eaecd36 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Wed, 19 Mar 2025 14:16:18 -0300
Subject: [PATCH v9 3/3] postgres_fdw: improve security checks
On adding scram pass-through for dblink [1] thread, it was discussed
that we should not by-pass fdw security check as it was implemented for
postgres_fdw on 761c79508e7.
This commit improve the security check by adding new scram pass-through
checks to ensure that the required scram connection options are not
overwritten by the user mapping or foreign server options.
[1] https://www.postgresql.org/message-id/flat/CAFY6G8ercA1KES%3DE_0__R9QCTR805TTyYr1No8qF8ZxmMg8z2Q%40mail.gmail.com
---
contrib/postgres_fdw/connection.c | 100 ++++++++++++++++++++---
contrib/postgres_fdw/t/001_auth_scram.pl | 44 +++++++++-
doc/src/sgml/postgres-fdw.sgml | 15 +---
3 files changed, 133 insertions(+), 26 deletions(-)
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 8ef9702c05c..b8b30284086 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -185,6 +185,8 @@ static void postgres_fdw_get_connections_internal(FunctionCallInfo fcinfo,
static int pgfdw_conn_check(PGconn *conn);
static bool pgfdw_conn_checkable(void);
+static bool pgfdw_has_required_scram_options(const char **keywords, const char **values);
+
/*
* Get a PGconn which can be used to execute queries on the remote PostgreSQL
* server with the user's authorization. A new connection is established
@@ -455,6 +457,15 @@ pgfdw_security_check(const char **keywords, const char **values, UserMapping *us
}
}
+ /*
+ * Ok if SCRAM pass-through is being used and all required scram options
+ * are set correctly. If pgfdw_has_required_scram_options returns true we
+ * assume that UseScramPassthrough is also true since SCRAM options are
+ * only set when UseScramPassthrough is enabled.
+ */
+ if (MyProcPort->has_scram_keys && pgfdw_has_required_scram_options(keywords, values))
+ return;
+
ereport(ERROR,
(errcode(ERRCODE_S_R_E_PROHIBITED_SQL_STATEMENT_ATTEMPTED),
errmsg("password or GSSAPI delegated credentials required"),
@@ -556,6 +567,7 @@ connect_pg_server(ForeignServer *server, UserMapping *user)
values[n] = GetDatabaseEncodingName();
n++;
+ /* Add required SCRAM pass-through connection options if it's enable. */
if (MyProcPort->has_scram_keys && UseScramPassthrough(server, user))
{
int len;
@@ -582,16 +594,20 @@ connect_pg_server(ForeignServer *server, UserMapping *user)
if (encoded_len < 0)
elog(ERROR, "could not encode SCRAM server key");
n++;
+
+ /*
+ * Require scram-sha-256 to ensure that no other auth method is
+ * used when connecting with foreign server.
+ */
+ keywords[n] = "require_auth";
+ values[n] = "scram-sha-256";
+ n++;
}
keywords[n] = values[n] = NULL;
- /*
- * Verify the set of connection parameters only if scram pass-through
- * is not being used because the password is not necessary.
- */
- if (!(MyProcPort->has_scram_keys && UseScramPassthrough(server, user)))
- check_conn_params(keywords, values, user);
+ /* Verify the set of connection parameters. */
+ check_conn_params(keywords, values, user);
/* first time, allocate or get the custom wait event */
if (pgfdw_we_connect == 0)
@@ -609,12 +625,8 @@ connect_pg_server(ForeignServer *server, UserMapping *user)
server->servername),
errdetail_internal("%s", pchomp(PQerrorMessage(conn)))));
- /*
- * Perform post-connection security checks only if scram pass-through
- * is not being used because the password is not necessary.
- */
- if (!(MyProcPort->has_scram_keys && UseScramPassthrough(server, user)))
- pgfdw_security_check(keywords, values, user, conn);
+ /* Perform post-connection security checks. */
+ pgfdw_security_check(keywords, values, user, conn);
/* Prepare new session for use */
configure_remote_session(conn);
@@ -725,6 +737,15 @@ check_conn_params(const char **keywords, const char **values, UserMapping *user)
if (!UserMappingPasswordRequired(user))
return;
+ /*
+ * Ok if SCRAM pass-through is being used and all required scram options
+ * are set correctly. If pgfdw_has_required_scram_options returns true we
+ * assume that UseScramPassthrough is also true since SCRAM options are
+ * only set when UseScramPassthrough is enabled.
+ */
+ if (MyProcPort->has_scram_keys && pgfdw_has_required_scram_options(keywords, values))
+ return;
+
ereport(ERROR,
(errcode(ERRCODE_S_R_E_PROHIBITED_SQL_STATEMENT_ATTEMPTED),
errmsg("password or GSSAPI delegated credentials required"),
@@ -2487,3 +2508,58 @@ pgfdw_conn_checkable(void)
return false;
#endif
}
+
+ /*
+ * Ensure that require_auth and scram keys are correctly set on values. SCRAM
+ * keys used to pass-through are coming from the initial connection from the
+ * client with the server.
+ *
+ * All required scram options are set by postgres_fdw, so we just need to
+ * ensure that these options are not overwritten by the user.
+ */
+static bool
+pgfdw_has_required_scram_options(const char **keywords, const char **values)
+{
+ int i;
+ bool has_scram_server_key = false;
+ bool has_scram_client_key = false;
+ bool has_require_auth = false;
+ bool has_scram_keys = false;
+
+ /*
+ * Continue iterating even if we found the keys that we need to validate
+ * to make sure that there is no other declaration of these keys that can
+ * overwrite the first.
+ */
+ for (i = 0; keywords[i] != NULL; i++)
+ {
+ if (strcmp(keywords[i], "scram_client_key") == 0)
+ {
+ if (values[i] != NULL && values[i][0] != '\0')
+ has_scram_client_key = true;
+ else
+ has_scram_client_key = false;
+ }
+
+ if (strcmp(keywords[i], "scram_server_key") == 0)
+ {
+ if (values[i] != NULL && values[i][0] != '\0')
+ has_scram_server_key = true;
+ else
+ has_scram_server_key = false;
+ }
+
+ if (strcmp(keywords[i], "require_auth") == 0)
+ {
+ if (values[i] != NULL && strcmp(values[i], "scram-sha-256") == 0)
+ has_require_auth = true;
+ else
+ has_require_auth = false;
+ }
+ }
+
+ has_scram_keys = has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys;
+
+
+ return (has_scram_keys && has_require_auth);
+}
diff --git a/contrib/postgres_fdw/t/001_auth_scram.pl b/contrib/postgres_fdw/t/001_auth_scram.pl
index 047840cc914..c45cbe6946c 100644
--- a/contrib/postgres_fdw/t/001_auth_scram.pl
+++ b/contrib/postgres_fdw/t/001_auth_scram.pl
@@ -68,6 +68,47 @@ test_fdw_auth($node1, $db0, "t2", $fdw_server2,
test_auth($node2, $db2, "t2",
"SCRAM auth directly on foreign server should still succeed");
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+}
+);
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local all all password
+}
+);
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ qq'select count(1) from t',
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/failed: authentication method requirement "scram-sha-256"/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ qq'select count(1) from t2',
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback password fails on a different cluster');
+like(
+ $stderr,
+ qr/failed: authentication method requirement "scram-sha-256"/,
+ 'expected error from loopback password (different cluster)');
+
# Helper functions
sub test_auth
@@ -109,7 +150,8 @@ sub setup_pghba
'pg_hba.conf', qq{
local all all scram-sha-256
host all all $hostaddr/32 scram-sha-256
- });
+ }
+ );
$node->restart;
}
diff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src/sgml/postgres-fdw.sgml
index a7f2f5ca182..c4d0f5ed2a3 100644
--- a/doc/src/sgml/postgres-fdw.sgml
+++ b/doc/src/sgml/postgres-fdw.sgml
@@ -788,10 +788,8 @@ OPTIONS (ADD password_required 'false');
<itemizedlist>
<listitem>
<para>
- The remote server must request SCRAM authentication. (If desired,
- enforce this on the client side (FDW side) with the option
- <literal>require_auth</literal>.) If another authentication method
- is requested by the server, then that one will be used normally.
+ The remote server must request the <literal>scram-sha-256</literal>
+ authentication method; otherwise, the connection will fail.
</para>
</listitem>
@@ -803,15 +801,6 @@ OPTIONS (ADD password_required 'false');
</para>
</listitem>
- <listitem>
- <para>
- The user mapping password is not used. (It could be set to support
- other authentication methods, but that would arguably violate the
- point of this feature, which is to avoid storing plain-text
- passwords.)
- </para>
- </listitem>
-
<listitem>
<para>
The server running <filename>postgres_fdw</filename> and the remote
--
2.39.5 (Apple Git-154)
v9-0002-dblink-Add-SCRAM-pass-through-authentication.patchapplication/octet-stream; name=v9-0002-dblink-Add-SCRAM-pass-through-authentication.patchDownload
From cc58bc5f37efc81e007082d5667ccaa9727fc196 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Thu, 20 Feb 2025 15:12:48 -0300
Subject: [PATCH v9 2/3] dblink: Add SCRAM pass-through authentication
This enables SCRAM authentication for dblink_fdw when connecting to a
foreign server using the same approach as it was implemented for
postgres_fdw on 761c79508e.
---
contrib/dblink/Makefile | 1 +
contrib/dblink/dblink.c | 185 +++++++++++++++++++++-
contrib/dblink/meson.build | 5 +
contrib/dblink/t/001_auth_scram.pl | 246 +++++++++++++++++++++++++++++
doc/src/sgml/dblink.sgml | 69 +++++++-
5 files changed, 497 insertions(+), 9 deletions(-)
create mode 100644 contrib/dblink/t/001_auth_scram.pl
diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index d4c7ed625ab..fde0b49ddbb 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -13,6 +13,7 @@ PGFILEDESC = "dblink - connect to other PostgreSQL databases"
REGRESS = dblink
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
+TAP_TESTS = 1
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 4be8cfdbf74..c7561b2d69b 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -43,6 +43,8 @@
#include "catalog/pg_foreign_server.h"
#include "catalog/pg_type.h"
#include "catalog/pg_user_mapping.h"
+#include "commands/defrem.h"
+#include "common/base64.h"
#include "executor/spi.h"
#include "foreign/foreign.h"
#include "funcapi.h"
@@ -127,6 +129,11 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
static PGconn *connect_pg_server(char *connstr_or_srvname, uint32 wait_event_info);
+static bool UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+static void appendSCRAMKeysInfo(StringInfo buf);
+static bool is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context);
+static bool dblink_connstr_has_required_scram_options(const char *connstr);
/* Global */
static remoteConn *pconn = NULL;
@@ -1907,7 +1914,7 @@ dblink_fdw_validator(PG_FUNCTION_ARGS)
{
DefElem *def = (DefElem *) lfirst(cell);
- if (!is_valid_dblink_option(options, def->defname, context))
+ if (!is_valid_dblink_fdw_option(options, def->defname, context))
{
/*
* Unknown option, or invalid option for the context specified, so
@@ -2539,6 +2546,68 @@ deleteConnection(const char *name)
errmsg("undefined connection name")));
}
+ /*
+ * Ensure that require_auth and SCRAM keys are correctly set on connstr.
+ * SCRAM keys used to pass-through are coming from the initial connection
+ * from the client with the server.
+ *
+ * All required SCRAM options are set by dblink, so we just need to ensure
+ * that these options are not overwritten by the user.
+ *
+ * See appendSCRAMKeysInfo and its usage for more.
+ */
+bool
+dblink_connstr_has_required_scram_options(const char *connstr)
+{
+ PQconninfoOption *options;
+ PQconninfoOption *option;
+ bool has_scram_server_key = false;
+ bool has_scram_client_key = false;
+ bool has_require_auth = false;
+ bool has_scram_keys = false;
+
+ options = PQconninfoParse(connstr, NULL);
+ if (options)
+ {
+ /*
+ * Continue iterating even if we found the keys that we need to
+ * validate to make sure that there is no other declaration of these
+ * keys that can overwrite the first.
+ */
+ for (option = options; option->keyword != NULL; option++)
+ {
+ if (strcmp(option->keyword, "require_auth") == 0)
+ {
+ if (option->val != NULL && strcmp(option->val, "scram-sha-256") == 0)
+ has_require_auth = true;
+ else
+ has_require_auth = false;
+ }
+
+ if (strcmp(option->keyword, "scram_client_key") == 0)
+ {
+ if (option->val != NULL && option->val[0] != '\0')
+ has_scram_client_key = true;
+ else
+ has_scram_client_key = false;
+ }
+
+ if (strcmp(option->keyword, "scram_server_key") == 0)
+ {
+ if (option->val != NULL && option->val[0] != '\0')
+ has_scram_server_key = true;
+ else
+ has_scram_server_key = false;
+ }
+ }
+ PQconninfoFree(options);
+ }
+
+ has_scram_keys = has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys;
+
+ return (has_scram_keys && has_require_auth);
+}
+
/*
* We need to make sure that the connection made used credentials
* which were provided by the user, so check what credentials were
@@ -2555,6 +2624,18 @@ dblink_security_check(PGconn *conn, const char *connstr)
if (PQconnectionUsedPassword(conn) && dblink_connstr_has_pw(connstr))
return;
+ /*
+ * Password was not used to connect, check if SCRAM pass-through is in
+ * use.
+ *
+ * If dblink_connstr_has_required_scram_options is true we assume that
+ * UseScramPassthrough is also true because the required SCRAM keys are
+ * only added if UseScramPassthrough is set, and the user is not allowed
+ * to add the SCRAM keys on fdw and user mapping options.
+ */
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
#ifdef ENABLE_GSS
/* If GSSAPI creds used to connect, make sure it was one delegated */
if (PQconnectionUsedGSSAPI(conn) && be_gssapi_get_delegation(MyProcPort))
@@ -2605,12 +2686,14 @@ dblink_connstr_has_pw(const char *connstr)
}
/*
- * For non-superusers, insist that the connstr specify a password, except
- * if GSSAPI credentials have been delegated (and we check that they are used
- * for the connection in dblink_security_check later). This prevents a
- * password or GSSAPI credentials from being picked up from .pgpass, a
- * service file, the environment, etc. We don't want the postgres user's
- * passwords or Kerberos credentials to be accessible to non-superusers.
+ * For non-superusers, insist that the connstr specify a password, except if
+ * GSSAPI credentials have been delegated (and we check that they are used for
+ * the connection in dblink_security_check later) or if SCRAM pass-through is
+ * being used. This prevents a password or GSSAPI credentials from being
+ * picked up from .pgpass, a service file, the environment, etc. We don't want
+ * the postgres user's passwords or Kerberos credentials to be accessible to
+ * non-superusers. In case of SCRAM pass-through insist that the connstr
+ * has the required SCRAM pass-through options.
*/
static void
dblink_connstr_check(const char *connstr)
@@ -2621,6 +2704,9 @@ dblink_connstr_check(const char *connstr)
if (dblink_connstr_has_pw(connstr))
return;
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
#ifdef ENABLE_GSS
if (be_gssapi_get_delegation(MyProcPort))
return;
@@ -2763,6 +2849,14 @@ get_connect_string(ForeignServer *foreign_server)
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ /*
+ * First append hardcoded options needed for SCRAM pass-through, so if the
+ * user overwrites these options we can ereport on dblink_connstr_check
+ * and dblink_security_check.
+ */
+ if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
+ appendSCRAMKeysInfo(&buf);
+
foreach(cell, fdw->options)
{
DefElem *def = lfirst(cell);
@@ -2944,6 +3038,20 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
+/*
+ * Same as is_valid_dblink_option but also check for only dblink_fdw specific
+ * options.
+ */
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context)
+{
+ if (strcmp(option, "use_scram_passthrough") == 0)
+ return true;
+
+ return is_valid_dblink_option(options, option, context);
+}
+
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3014,6 +3122,69 @@ restoreLocalGucs(int nestlevel)
AtEOXact_GUC(true, nestlevel);
}
+/*
+ * Append SCRAM client key and server key information from the global
+ * MyProcPort into the given StringInfo buffer.
+ */
+static void
+appendSCRAMKeysInfo(StringInfo buf)
+{
+ int len;
+ int encoded_len;
+ char *client_key;
+ char *server_key;
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
+ /* don't forget the zero-terminator */
+ client_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ClientKey,
+ sizeof(MyProcPort->scram_ClientKey),
+ client_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM client key");
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ServerKey));
+ /* don't forget the zero-terminator */
+ server_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ServerKey,
+ sizeof(MyProcPort->scram_ServerKey),
+ server_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM server key");
+
+ appendStringInfo(buf, "scram_client_key='%s' ", client_key);
+ appendStringInfo(buf, "scram_server_key='%s' ", server_key);
+ appendStringInfo(buf, "require_auth='scram-sha-256' ");
+
+ pfree(client_key);
+ pfree(server_key);
+}
+
+
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user)
+{
+ ListCell *cell;
+
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ foreach(cell, user->options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ return false;
+}
+
/*
* Connect to remote server. If connstr_or_srvname maps to a foreign server,
* the associated properties and user mapping properties is also used to open
diff --git a/contrib/dblink/meson.build b/contrib/dblink/meson.build
index 3ab78668288..dfd8eb6877e 100644
--- a/contrib/dblink/meson.build
+++ b/contrib/dblink/meson.build
@@ -36,4 +36,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/001_auth_scram.pl',
+ ],
+ },
}
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
new file mode 100644
index 00000000000..3b25961b664
--- /dev/null
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -0,0 +1,246 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+# Test SCRAM authentication when opening a new connection with a foreign
+# server.
+#
+# The test is executed by testing the SCRAM authentifcation on a loopback
+# connection on the same server and with different servers.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $user = "user01";
+
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
+my $fdw_server = "db1_fdw";
+my $fdw_server2 = "db2_fdw";
+my $fdw_invalid_server = "db2_fdw_invalid"; # For invalid fdw options
+my $fdw_invalid_server2 =
+ "db2_fdw_invalid2"; # For invalid scram keys fdw options
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+my $node2 = PostgreSQL::Test::Cluster->new('node2');
+
+$node1->init;
+$node2->init;
+
+$node1->start;
+$node2->start;
+
+# Test setup
+
+$node1->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$node2->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$ENV{PGPASSWORD} = "pass";
+
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db0');
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db1');
+$node2->safe_psql('postgres', qq'CREATE DATABASE $db2');
+
+setup_table($node1, $db1, "t");
+setup_table($node2, $db2, "t2");
+
+$node1->safe_psql($db0, 'CREATE EXTENSION IF NOT EXISTS dblink');
+setup_fdw_server($node1, $db0, $fdw_server, $node1, $db1);
+setup_fdw_server($node1, $db0, $fdw_server2, $node2, $db2);
+setup_invalid_fdw_server($node1, $db0, $fdw_invalid_server, $node2, $db2);
+setup_fdw_server($node1, $db0, $fdw_invalid_server2, $node2, $db2);
+
+setup_user_mapping($node1, $db0, $fdw_server);
+setup_user_mapping($node1, $db0, $fdw_server2);
+setup_user_mapping($node1, $db0, $fdw_invalid_server);
+
+# Make the user have the same SCRAM key on both servers. Forcing to have the
+# same iteration and salt.
+my $rolpassword = $node1->safe_psql('postgres',
+ qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';");
+$node2->safe_psql('postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'");
+
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all scram-sha-256
+}
+);
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all scram-sha-256
+}
+);
+
+$node1->restart;
+$node2->restart;
+
+# End of test setup
+
+test_scram_keys_is_not_overwritten($node1, $db0, $fdw_invalid_server2);
+
+test_fdw_auth($node1, $db0, "t", $fdw_server,
+ "SCRAM auth on the same database cluster must succeed");
+
+test_fdw_auth($node1, $db0, "t2", $fdw_server2,
+ "SCRAM auth on a different database cluster must succeed");
+
+test_fdw_auth_with_invalid_overwritten_require_auth($fdw_invalid_server);
+
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+}
+);
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local all all password
+}
+);
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/failed: authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback password fails on a different cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server requested a cleartext password/,
+ 'expected error from loopback password (different cluster)');
+
+# Helper functions
+
+sub test_fdw_auth
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $db, $tbl, $fdw, $testname) = @_;
+ my $connstr = $node->connstr($db) . qq' user=$user';
+
+ my $ret = $node->safe_psql(
+ $db,
+ qq"SELECT count(1) FROM dblink('$fdw', 'SELECT * FROM $tbl') AS $tbl(a int, b int)",
+ connstr => $connstr);
+
+ is($ret, '10', $testname);
+}
+
+sub test_fdw_auth_with_invalid_overwritten_require_auth
+{
+
+ my ($fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "select * from dblink('$fdw', 'select * from t') as t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+ is($ret, 3, 'loopback trust fails when overwriting require_auth');
+ like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error when connecting to a fdw overwriting the require_auth'
+ );
+}
+
+sub test_scram_keys_is_not_overwritten
+{
+ my ($node, $db, $fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_client_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret, 3, 'user mapping creation fails when using scram_client_key');
+ like(
+ $stderr,
+ qr/ERROR: invalid option "scram_client_key"/,
+ 'user mapping creation fails when using scram_client_key');
+
+ ($ret, $stdout, $stderr) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_server_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret, 3, 'user mapping creation fails when using scram_server_key');
+ like(
+ $stderr,
+ qr/ERROR: invalid option "scram_server_key"/,
+ 'user mapping creation fails when using scram_server_key');
+}
+
+sub setup_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\');'
+ );
+}
+
+sub setup_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_invalid_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\', require_auth \'none\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_table
+{
+ my ($node, $db, $tbl) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE TABLE $tbl AS SELECT g as a, g + 1 as b FROM generate_series(1,10) g(g)'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON SCHEMA public TO $user');
+ $node->safe_psql($db, qq'GRANT SELECT ON $tbl TO $user');
+}
+
+done_testing();
+
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index 81f35986c88..24a533f9116 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,6 +136,70 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
+ <refsect1>
+ <title>Foreign Data Wrapper</title>
+
+ <para>
+ A Foreign Data Wrapper can be used as a connection name parameter. The foreign
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ </para>
+
+ <para>
+ The authentication with the foreign server can be via password on USER
+ MAPPING or using SCRAM pass-through. The
+ <literal>use_scram_passthrough</literal> on CREATE SERVER options controls
+ whether <filename>dblink_fdw</filename> will use the SCRAM pass-through
+ authentication to connect to the foreign server. With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
+ </para>
+
+ <para>
+ To use SCRAM pass-through authentication:
+ <itemizedlist>
+ <listitem>
+ <para>
+ The remote server must request the <literal>scram-sha-256</literal>
+ authentication method; otherwise, the connection will fail.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The remote server can be of any PostgreSQL version that supports SCRAM.
+ Support for <literal>use_scram_passthrough</literal> is only required on
+ the client side (FDW side).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The server running <filename>dblink_fdw</filename> and the remote server
+ must have identical SCRAM secrets (encrypted passwords) for the user being
+ used on <filename>dblink_fdw</filename> to authenticate on the foreign
+ server (same salt and iterations, not merely the same password).
+ </para>
+
+ <para>
+ As a corollary, if FDW connections to multiple hosts are to be made, for
+ example for partitioned foreign tables/sharding, then all hosts must have
+ identical SCRAM secrets for the users involved.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The current session on the PostgreSQL instance that makes the outgoing FDW
+ connections also must also use SCRAM authentication for its incoming client
+ connection. (Hence <quote>pass-through</quote>: SCRAM must be used going in
+ and out.) This is a technical requirement of the SCRAM protocol.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </refsect1>
+
<refsect1>
<title>Notes</title>
@@ -181,8 +245,9 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection must require password authentication for this to work properly
--- Otherwise, you will receive the following error from dblink_connect():
+-- Note: local connections that don't use SCRAM pass-through require password
+-- authentication for this to work properly. Otherwise, you will receive
+-- the following error from dblink_connect():
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
--
2.39.5 (Apple Git-154)
v9-0001-dblink-refactor-get-connection-routines.patchapplication/octet-stream; name=v9-0001-dblink-refactor-get-connection-routines.patchDownload
From d8abad0b3b93031bb0abf855caddf7ccfb9275df Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 20 Jan 2025 15:25:51 -0300
Subject: [PATCH v9 1/3] dblink: refactor get connection routines
Refactor dblink_get_conn and dblink_connect to move the logic of
actually opening the connection to the new connect_pg_server function
which them can be re-used on both functions.
This is a pre-work for a next commit that will add support for scram
pass-through authentication to dblink which will be able to implement
most of the logic into the connect_pg_server function which now already
have all necessary data information.
Note that some pfree(rconn) calls has been removed on this commit, but
the allocation of rconn was also moved after connect_pg_server which was
the only function that could force the pfree(rconn) call.
---
contrib/dblink/dblink.c | 208 ++++++++++++++++++----------------------
1 file changed, 94 insertions(+), 114 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 58c1a6221c8..4be8cfdbf74 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -114,10 +114,10 @@ static Relation get_rel_from_relname(text *relname_text, LOCKMODE lockmode, AclM
static char *generate_relation_name(Relation rel);
static void dblink_connstr_check(const char *connstr);
static bool dblink_connstr_has_pw(const char *connstr);
-static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
+static void dblink_security_check(PGconn *conn, const char *connstr);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
-static char *get_connect_string(const char *servername);
+static char *get_connect_string(ForeignServer *foreign_server);
static char *escape_param_str(const char *str);
static void validate_pkattnums(Relation rel,
int2vector *pkattnums_arg, int32 pknumatts_arg,
@@ -126,6 +126,7 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
const char *option, Oid context);
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static PGconn *connect_pg_server(char *connstr_or_srvname, uint32 wait_event_info);
/* Global */
static remoteConn *pconn = NULL;
@@ -199,33 +200,11 @@ dblink_get_conn(char *conname_or_str,
}
else
{
- const char *connstr;
-
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_get_conn == 0)
dblink_we_get_conn = WaitEventExtensionNew("DblinkGetConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_get_conn);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- char *msg = pchomp(PQerrorMessage(conn));
-
- libpqsrv_disconnect(conn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
- dblink_security_check(conn, rconn, connstr);
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, dblink_we_get_conn);
freeconn = true;
conname = NULL;
}
@@ -270,9 +249,7 @@ Datum
dblink_connect(PG_FUNCTION_ARGS)
{
char *conname_or_str = NULL;
- char *connstr = NULL;
char *connname = NULL;
- char *msg;
PGconn *conn = NULL;
remoteConn *rconn = NULL;
@@ -286,53 +263,19 @@ dblink_connect(PG_FUNCTION_ARGS)
else if (PG_NARGS() == 1)
conname_or_str = text_to_cstring(PG_GETARG_TEXT_PP(0));
- if (connname)
- {
- rconn = (remoteConn *) MemoryContextAlloc(TopMemoryContext,
- sizeof(remoteConn));
- rconn->conn = NULL;
- rconn->openCursorCount = 0;
- rconn->newXactForCursor = false;
- }
-
- /* first check for valid foreign data server */
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
-
- /* check password in connection string if not superuser */
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_connect == 0)
dblink_we_connect = WaitEventExtensionNew("DblinkConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_connect);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- msg = pchomp(PQerrorMessage(conn));
- libpqsrv_disconnect(conn);
- if (rconn)
- pfree(rconn);
-
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
-
- /* check password actually used if not superuser */
- dblink_security_check(conn, rconn, connstr);
-
- /* attempt to set client encoding to match server encoding, if needed */
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, dblink_we_connect);
if (connname)
{
+ rconn = (remoteConn *) MemoryContextAlloc(TopMemoryContext,
+ sizeof(remoteConn));
rconn->conn = conn;
+ rconn->openCursorCount = 0;
+ rconn->newXactForCursor = false;
createNewConnection(connname, rconn);
}
else
@@ -2602,7 +2545,7 @@ deleteConnection(const char *name)
* used to connect and then make sure that they came from the user.
*/
static void
-dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
+dblink_security_check(PGconn *conn, const char *connstr)
{
/* Superuser bypasses security check */
if (superuser())
@@ -2620,8 +2563,6 @@ dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
/* Otherwise, fail out */
libpqsrv_disconnect(conn);
- if (rconn)
- pfree(rconn);
ereport(ERROR,
(errcode(ERRCODE_S_R_E_PROHIBITED_SQL_STATEMENT_ATTEMPTED),
@@ -2782,15 +2723,16 @@ dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
* Obtain connection string for a foreign server
*/
static char *
-get_connect_string(const char *servername)
+get_connect_string(ForeignServer *foreign_server)
{
- ForeignServer *foreign_server = NULL;
UserMapping *user_mapping;
ListCell *cell;
StringInfoData buf;
ForeignDataWrapper *fdw;
AclResult aclresult;
- char *srvname;
+ Oid serverid = foreign_server->serverid;
+ Oid fdwid = foreign_server->fdwid;
+ Oid userid = GetUserId();
static const PQconninfoOption *options = NULL;
@@ -2813,57 +2755,43 @@ get_connect_string(const char *servername)
errdetail("Could not get libpq's default connection options.")));
}
- /* first gather the server connstr options */
- srvname = pstrdup(servername);
- truncate_identifier(srvname, strlen(srvname), false);
- foreign_server = GetForeignServerByName(srvname, true);
-
- if (foreign_server)
- {
- Oid serverid = foreign_server->serverid;
- Oid fdwid = foreign_server->fdwid;
- Oid userid = GetUserId();
-
- user_mapping = GetUserMapping(userid, serverid);
- fdw = GetForeignDataWrapper(fdwid);
-
- /* Check permissions, user must have usage on the server. */
- aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
- if (aclresult != ACLCHECK_OK)
- aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ user_mapping = GetUserMapping(userid, serverid);
+ fdw = GetForeignDataWrapper(fdwid);
- foreach(cell, fdw->options)
- {
- DefElem *def = lfirst(cell);
+ /* Check permissions, user must have usage on the server. */
+ aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
+ if (aclresult != ACLCHECK_OK)
+ aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
- if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, fdw->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, foreign_server->options)
- {
- DefElem *def = lfirst(cell);
+ if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, user_mapping->options)
- {
+ if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- DefElem *def = lfirst(cell);
+ foreach(cell, user_mapping->options)
+ {
- if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ DefElem *def = lfirst(cell);
- return buf.data;
+ if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
}
- else
- return NULL;
+
+ return buf.data;
}
/*
@@ -3085,3 +3013,55 @@ restoreLocalGucs(int nestlevel)
if (nestlevel > 0)
AtEOXact_GUC(true, nestlevel);
}
+
+/*
+ * Connect to remote server. If connstr_or_srvname maps to a foreign server,
+ * the associated properties and user mapping properties is also used to open
+ * the connection. Otherwise a connection will be open using the raw
+ * connstr_or_srvname value.
+ */
+static PGconn *
+connect_pg_server(char *connstr_or_srvname, uint32 wait_event_info)
+{
+ PGconn *conn;
+ ForeignServer *foreign_server = NULL;
+ const char *connstr;
+ char *srvname;
+
+ /* first gather the server connstr options */
+ srvname = pstrdup(connstr_or_srvname);
+ truncate_identifier(srvname, strlen(srvname), false);
+ foreign_server = GetForeignServerByName(srvname, true);
+
+ if (foreign_server)
+ connstr = get_connect_string(foreign_server);
+ else
+ connstr = connstr_or_srvname;
+
+ /* Verify the set of connection parameters. */
+ dblink_connstr_check(connstr);
+
+ /* OK to make connection */
+ conn = libpqsrv_connect(connstr, wait_event_info);
+
+ if (PQstatus(conn) == CONNECTION_BAD)
+ {
+ char *msg = pchomp(PQerrorMessage(conn));
+
+ libpqsrv_disconnect(conn);
+
+ ereport(ERROR,
+ (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
+ errmsg("could not establish connection"),
+ errdetail_internal("%s", msg)));
+ }
+
+ /* Perform post-connection security checks. */
+ dblink_security_check(conn, connstr);
+
+ /* attempt to set client encoding to match server encoding, if needed */
+ if (PQclientEncoding(conn) != GetDatabaseEncoding())
+ PQsetClientEncoding(conn, GetDatabaseEncodingName());
+
+ return conn;
+}
--
2.39.5 (Apple Git-154)
On Thu, Mar 20, 2025 at 12:54 PM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:
Since the security checks are defined I'm attaching 0003 which include
the fix of security checks for postgres_fdw. It implements the
validations very similar to what are being implemented on dblink.
Comments on 0003:
+ keywords[n] = "require_auth"; + values[n] = "scram-sha-256"; + n++;
The keywords and values arrays need to be lengthened for this.
host all all $hostaddr/32 scram-sha-256 - }); + } + );
Accidental diff?
A few whitespace and comment tweaks are attached as well.
--
I think they should just be reduced to "The remote server must request
SCRAM authentication." and "The user mapping password is not used."I've removed the "user mapping password" <listitem> because we already
mentioned above that the password is not used and having just "The user
mapping password is not used." again seems redundant, what do you think?
Personally, I think it's still useful to call out that the password in
the user mapping is explicitly ignored. The other text motivates the
feature, but it doesn't explain how it interacts with existing user
mappings (most of which will have passwords).
Now that we handle the pfree() in PG_CATCH instead, that lower-level
pfree should be removed, and then connect_pg_server() doesn't need to
take an rconn pointer at all. For extra credit you could maybe move
the allocation of rconn down below the call to connect_pg_server(),
and get rid of the try/catch?Good catch, thanks, it's much better now! With this change we can also
remove the second if (connname) condition. All fixed on attached.
I like that; the patch is a lot easier to follow now.
Thanks,
--Jacob
Attachments:
fixup.diff.txttext/plain; charset=US-ASCII; name=fixup.diff.txtDownload
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index b8b30284086..b33895b4f56 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -184,7 +184,6 @@ static void postgres_fdw_get_connections_internal(FunctionCallInfo fcinfo,
enum pgfdwVersion api_version);
static int pgfdw_conn_check(PGconn *conn);
static bool pgfdw_conn_checkable(void);
-
static bool pgfdw_has_required_scram_options(const char **keywords, const char **values);
/*
@@ -458,7 +457,7 @@ pgfdw_security_check(const char **keywords, const char **values, UserMapping *us
}
/*
- * Ok if SCRAM pass-through is being used and all required scram options
+ * Ok if SCRAM pass-through is being used and all required SCRAM options
* are set correctly. If pgfdw_has_required_scram_options returns true we
* assume that UseScramPassthrough is also true since SCRAM options are
* only set when UseScramPassthrough is enabled.
@@ -567,7 +566,7 @@ connect_pg_server(ForeignServer *server, UserMapping *user)
values[n] = GetDatabaseEncodingName();
n++;
- /* Add required SCRAM pass-through connection options if it's enable. */
+ /* Add required SCRAM pass-through connection options if it's enabled. */
if (MyProcPort->has_scram_keys && UseScramPassthrough(server, user))
{
int len;
@@ -2509,14 +2508,14 @@ pgfdw_conn_checkable(void)
#endif
}
- /*
- * Ensure that require_auth and scram keys are correctly set on values. SCRAM
- * keys used to pass-through are coming from the initial connection from the
- * client with the server.
- *
- * All required scram options are set by postgres_fdw, so we just need to
- * ensure that these options are not overwritten by the user.
- */
+/*
+ * Ensure that require_auth and SCRAM keys are correctly set on values. SCRAM
+ * keys used to pass-through are coming from the initial connection from the
+ * client with the server.
+ *
+ * All required SCRAM options are set by postgres_fdw, so we just need to
+ * ensure that these options are not overwritten by the user.
+ */
static bool
pgfdw_has_required_scram_options(const char **keywords, const char **values)
{
@@ -2560,6 +2559,5 @@ pgfdw_has_required_scram_options(const char **keywords, const char **values)
has_scram_keys = has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys;
-
return (has_scram_keys && has_require_auth);
}
On Thu, Mar 20, 2025 at 9:02 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
On Thu, Mar 20, 2025 at 12:54 PM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:Since the security checks are defined I'm attaching 0003 which include
the fix of security checks for postgres_fdw. It implements the
validations very similar to what are being implemented on dblink.Comments on 0003:
+ keywords[n] = "require_auth"; + values[n] = "scram-sha-256"; + n++;The keywords and values arrays need to be lengthened for this.
Fixed. I've also changed the code comment to mention the scram keys and
required options.
host all all $hostaddr/32 scram-sha-256 - }); + } + );Accidental diff?
Yep, sorry, I made some confusion with dblink formatting. Removed
A few whitespace and comment tweaks are attached as well.
Squashed
--
I think they should just be reduced to "The remote server must request
SCRAM authentication." and "The user mapping password is not used."I've removed the "user mapping password" <listitem> because we already
mentioned above that the password is not used and having just "The user
mapping password is not used." again seems redundant, what do you think?Personally, I think it's still useful to call out that the password in
the user mapping is explicitly ignored. The other text motivates the
feature, but it doesn't explain how it interacts with existing user
mappings (most of which will have passwords).
Fair point. I've changed it to just "The user mapping password is not
used".
--
Matheus Alcantara
Attachments:
v10-0001-dblink-refactor-get-connection-routines.patchapplication/octet-stream; name=v10-0001-dblink-refactor-get-connection-routines.patchDownload
From 02c667cc960a8bf09bf581e7b29d2167915a7ac9 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 20 Jan 2025 15:25:51 -0300
Subject: [PATCH v10 1/3] dblink: refactor get connection routines
Refactor dblink_get_conn and dblink_connect to move the logic of
actually opening the connection to the new connect_pg_server function
which them can be re-used on both functions.
This is a pre-work for a next commit that will add support for scram
pass-through authentication to dblink which will be able to implement
most of the logic into the connect_pg_server function which now already
have all necessary data information.
Note that some pfree(rconn) calls has been removed on this commit, but
the allocation of rconn was also moved after connect_pg_server which was
the only function that could force the pfree(rconn) call.
---
contrib/dblink/dblink.c | 208 ++++++++++++++++++----------------------
1 file changed, 94 insertions(+), 114 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 58c1a6221c8..4be8cfdbf74 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -114,10 +114,10 @@ static Relation get_rel_from_relname(text *relname_text, LOCKMODE lockmode, AclM
static char *generate_relation_name(Relation rel);
static void dblink_connstr_check(const char *connstr);
static bool dblink_connstr_has_pw(const char *connstr);
-static void dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr);
+static void dblink_security_check(PGconn *conn, const char *connstr);
static void dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
bool fail, const char *fmt,...) pg_attribute_printf(5, 6);
-static char *get_connect_string(const char *servername);
+static char *get_connect_string(ForeignServer *foreign_server);
static char *escape_param_str(const char *str);
static void validate_pkattnums(Relation rel,
int2vector *pkattnums_arg, int32 pknumatts_arg,
@@ -126,6 +126,7 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
const char *option, Oid context);
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static PGconn *connect_pg_server(char *connstr_or_srvname, uint32 wait_event_info);
/* Global */
static remoteConn *pconn = NULL;
@@ -199,33 +200,11 @@ dblink_get_conn(char *conname_or_str,
}
else
{
- const char *connstr;
-
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_get_conn == 0)
dblink_we_get_conn = WaitEventExtensionNew("DblinkGetConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_get_conn);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- char *msg = pchomp(PQerrorMessage(conn));
-
- libpqsrv_disconnect(conn);
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
- dblink_security_check(conn, rconn, connstr);
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, dblink_we_get_conn);
freeconn = true;
conname = NULL;
}
@@ -270,9 +249,7 @@ Datum
dblink_connect(PG_FUNCTION_ARGS)
{
char *conname_or_str = NULL;
- char *connstr = NULL;
char *connname = NULL;
- char *msg;
PGconn *conn = NULL;
remoteConn *rconn = NULL;
@@ -286,53 +263,19 @@ dblink_connect(PG_FUNCTION_ARGS)
else if (PG_NARGS() == 1)
conname_or_str = text_to_cstring(PG_GETARG_TEXT_PP(0));
- if (connname)
- {
- rconn = (remoteConn *) MemoryContextAlloc(TopMemoryContext,
- sizeof(remoteConn));
- rconn->conn = NULL;
- rconn->openCursorCount = 0;
- rconn->newXactForCursor = false;
- }
-
- /* first check for valid foreign data server */
- connstr = get_connect_string(conname_or_str);
- if (connstr == NULL)
- connstr = conname_or_str;
-
- /* check password in connection string if not superuser */
- dblink_connstr_check(connstr);
-
/* first time, allocate or get the custom wait event */
if (dblink_we_connect == 0)
dblink_we_connect = WaitEventExtensionNew("DblinkConnect");
- /* OK to make connection */
- conn = libpqsrv_connect(connstr, dblink_we_connect);
-
- if (PQstatus(conn) == CONNECTION_BAD)
- {
- msg = pchomp(PQerrorMessage(conn));
- libpqsrv_disconnect(conn);
- if (rconn)
- pfree(rconn);
-
- ereport(ERROR,
- (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
- errmsg("could not establish connection"),
- errdetail_internal("%s", msg)));
- }
-
- /* check password actually used if not superuser */
- dblink_security_check(conn, rconn, connstr);
-
- /* attempt to set client encoding to match server encoding, if needed */
- if (PQclientEncoding(conn) != GetDatabaseEncoding())
- PQsetClientEncoding(conn, GetDatabaseEncodingName());
+ conn = connect_pg_server(conname_or_str, dblink_we_connect);
if (connname)
{
+ rconn = (remoteConn *) MemoryContextAlloc(TopMemoryContext,
+ sizeof(remoteConn));
rconn->conn = conn;
+ rconn->openCursorCount = 0;
+ rconn->newXactForCursor = false;
createNewConnection(connname, rconn);
}
else
@@ -2602,7 +2545,7 @@ deleteConnection(const char *name)
* used to connect and then make sure that they came from the user.
*/
static void
-dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
+dblink_security_check(PGconn *conn, const char *connstr)
{
/* Superuser bypasses security check */
if (superuser())
@@ -2620,8 +2563,6 @@ dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
/* Otherwise, fail out */
libpqsrv_disconnect(conn);
- if (rconn)
- pfree(rconn);
ereport(ERROR,
(errcode(ERRCODE_S_R_E_PROHIBITED_SQL_STATEMENT_ATTEMPTED),
@@ -2782,15 +2723,16 @@ dblink_res_error(PGconn *conn, const char *conname, PGresult *res,
* Obtain connection string for a foreign server
*/
static char *
-get_connect_string(const char *servername)
+get_connect_string(ForeignServer *foreign_server)
{
- ForeignServer *foreign_server = NULL;
UserMapping *user_mapping;
ListCell *cell;
StringInfoData buf;
ForeignDataWrapper *fdw;
AclResult aclresult;
- char *srvname;
+ Oid serverid = foreign_server->serverid;
+ Oid fdwid = foreign_server->fdwid;
+ Oid userid = GetUserId();
static const PQconninfoOption *options = NULL;
@@ -2813,57 +2755,43 @@ get_connect_string(const char *servername)
errdetail("Could not get libpq's default connection options.")));
}
- /* first gather the server connstr options */
- srvname = pstrdup(servername);
- truncate_identifier(srvname, strlen(srvname), false);
- foreign_server = GetForeignServerByName(srvname, true);
-
- if (foreign_server)
- {
- Oid serverid = foreign_server->serverid;
- Oid fdwid = foreign_server->fdwid;
- Oid userid = GetUserId();
-
- user_mapping = GetUserMapping(userid, serverid);
- fdw = GetForeignDataWrapper(fdwid);
-
- /* Check permissions, user must have usage on the server. */
- aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
- if (aclresult != ACLCHECK_OK)
- aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ user_mapping = GetUserMapping(userid, serverid);
+ fdw = GetForeignDataWrapper(fdwid);
- foreach(cell, fdw->options)
- {
- DefElem *def = lfirst(cell);
+ /* Check permissions, user must have usage on the server. */
+ aclresult = object_aclcheck(ForeignServerRelationId, serverid, userid, ACL_USAGE);
+ if (aclresult != ACLCHECK_OK)
+ aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
- if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, fdw->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, foreign_server->options)
- {
- DefElem *def = lfirst(cell);
+ if (is_valid_dblink_option(options, def->defname, ForeignDataWrapperRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
- foreach(cell, user_mapping->options)
- {
+ if (is_valid_dblink_option(options, def->defname, ForeignServerRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
+ }
- DefElem *def = lfirst(cell);
+ foreach(cell, user_mapping->options)
+ {
- if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
- appendStringInfo(&buf, "%s='%s' ", def->defname,
- escape_param_str(strVal(def->arg)));
- }
+ DefElem *def = lfirst(cell);
- return buf.data;
+ if (is_valid_dblink_option(options, def->defname, UserMappingRelationId))
+ appendStringInfo(&buf, "%s='%s' ", def->defname,
+ escape_param_str(strVal(def->arg)));
}
- else
- return NULL;
+
+ return buf.data;
}
/*
@@ -3085,3 +3013,55 @@ restoreLocalGucs(int nestlevel)
if (nestlevel > 0)
AtEOXact_GUC(true, nestlevel);
}
+
+/*
+ * Connect to remote server. If connstr_or_srvname maps to a foreign server,
+ * the associated properties and user mapping properties is also used to open
+ * the connection. Otherwise a connection will be open using the raw
+ * connstr_or_srvname value.
+ */
+static PGconn *
+connect_pg_server(char *connstr_or_srvname, uint32 wait_event_info)
+{
+ PGconn *conn;
+ ForeignServer *foreign_server = NULL;
+ const char *connstr;
+ char *srvname;
+
+ /* first gather the server connstr options */
+ srvname = pstrdup(connstr_or_srvname);
+ truncate_identifier(srvname, strlen(srvname), false);
+ foreign_server = GetForeignServerByName(srvname, true);
+
+ if (foreign_server)
+ connstr = get_connect_string(foreign_server);
+ else
+ connstr = connstr_or_srvname;
+
+ /* Verify the set of connection parameters. */
+ dblink_connstr_check(connstr);
+
+ /* OK to make connection */
+ conn = libpqsrv_connect(connstr, wait_event_info);
+
+ if (PQstatus(conn) == CONNECTION_BAD)
+ {
+ char *msg = pchomp(PQerrorMessage(conn));
+
+ libpqsrv_disconnect(conn);
+
+ ereport(ERROR,
+ (errcode(ERRCODE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION),
+ errmsg("could not establish connection"),
+ errdetail_internal("%s", msg)));
+ }
+
+ /* Perform post-connection security checks. */
+ dblink_security_check(conn, connstr);
+
+ /* attempt to set client encoding to match server encoding, if needed */
+ if (PQclientEncoding(conn) != GetDatabaseEncoding())
+ PQsetClientEncoding(conn, GetDatabaseEncodingName());
+
+ return conn;
+}
--
2.39.5 (Apple Git-154)
v10-0002-dblink-Add-SCRAM-pass-through-authentication.patchapplication/octet-stream; name=v10-0002-dblink-Add-SCRAM-pass-through-authentication.patchDownload
From 7df4198770385a4f832216358f02b9e0cb471515 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Thu, 20 Feb 2025 15:12:48 -0300
Subject: [PATCH v10 2/3] dblink: Add SCRAM pass-through authentication
This enables SCRAM authentication for dblink_fdw when connecting to a
foreign server using the same approach as it was implemented for
postgres_fdw on 761c79508e.
---
contrib/dblink/Makefile | 1 +
contrib/dblink/dblink.c | 185 +++++++++++++++++++++-
contrib/dblink/meson.build | 5 +
contrib/dblink/t/001_auth_scram.pl | 246 +++++++++++++++++++++++++++++
doc/src/sgml/dblink.sgml | 75 ++++++++-
5 files changed, 503 insertions(+), 9 deletions(-)
create mode 100644 contrib/dblink/t/001_auth_scram.pl
diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index d4c7ed625ab..fde0b49ddbb 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -13,6 +13,7 @@ PGFILEDESC = "dblink - connect to other PostgreSQL databases"
REGRESS = dblink
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
+TAP_TESTS = 1
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 4be8cfdbf74..c7561b2d69b 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -43,6 +43,8 @@
#include "catalog/pg_foreign_server.h"
#include "catalog/pg_type.h"
#include "catalog/pg_user_mapping.h"
+#include "commands/defrem.h"
+#include "common/base64.h"
#include "executor/spi.h"
#include "foreign/foreign.h"
#include "funcapi.h"
@@ -127,6 +129,11 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
static PGconn *connect_pg_server(char *connstr_or_srvname, uint32 wait_event_info);
+static bool UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+static void appendSCRAMKeysInfo(StringInfo buf);
+static bool is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context);
+static bool dblink_connstr_has_required_scram_options(const char *connstr);
/* Global */
static remoteConn *pconn = NULL;
@@ -1907,7 +1914,7 @@ dblink_fdw_validator(PG_FUNCTION_ARGS)
{
DefElem *def = (DefElem *) lfirst(cell);
- if (!is_valid_dblink_option(options, def->defname, context))
+ if (!is_valid_dblink_fdw_option(options, def->defname, context))
{
/*
* Unknown option, or invalid option for the context specified, so
@@ -2539,6 +2546,68 @@ deleteConnection(const char *name)
errmsg("undefined connection name")));
}
+ /*
+ * Ensure that require_auth and SCRAM keys are correctly set on connstr.
+ * SCRAM keys used to pass-through are coming from the initial connection
+ * from the client with the server.
+ *
+ * All required SCRAM options are set by dblink, so we just need to ensure
+ * that these options are not overwritten by the user.
+ *
+ * See appendSCRAMKeysInfo and its usage for more.
+ */
+bool
+dblink_connstr_has_required_scram_options(const char *connstr)
+{
+ PQconninfoOption *options;
+ PQconninfoOption *option;
+ bool has_scram_server_key = false;
+ bool has_scram_client_key = false;
+ bool has_require_auth = false;
+ bool has_scram_keys = false;
+
+ options = PQconninfoParse(connstr, NULL);
+ if (options)
+ {
+ /*
+ * Continue iterating even if we found the keys that we need to
+ * validate to make sure that there is no other declaration of these
+ * keys that can overwrite the first.
+ */
+ for (option = options; option->keyword != NULL; option++)
+ {
+ if (strcmp(option->keyword, "require_auth") == 0)
+ {
+ if (option->val != NULL && strcmp(option->val, "scram-sha-256") == 0)
+ has_require_auth = true;
+ else
+ has_require_auth = false;
+ }
+
+ if (strcmp(option->keyword, "scram_client_key") == 0)
+ {
+ if (option->val != NULL && option->val[0] != '\0')
+ has_scram_client_key = true;
+ else
+ has_scram_client_key = false;
+ }
+
+ if (strcmp(option->keyword, "scram_server_key") == 0)
+ {
+ if (option->val != NULL && option->val[0] != '\0')
+ has_scram_server_key = true;
+ else
+ has_scram_server_key = false;
+ }
+ }
+ PQconninfoFree(options);
+ }
+
+ has_scram_keys = has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys;
+
+ return (has_scram_keys && has_require_auth);
+}
+
/*
* We need to make sure that the connection made used credentials
* which were provided by the user, so check what credentials were
@@ -2555,6 +2624,18 @@ dblink_security_check(PGconn *conn, const char *connstr)
if (PQconnectionUsedPassword(conn) && dblink_connstr_has_pw(connstr))
return;
+ /*
+ * Password was not used to connect, check if SCRAM pass-through is in
+ * use.
+ *
+ * If dblink_connstr_has_required_scram_options is true we assume that
+ * UseScramPassthrough is also true because the required SCRAM keys are
+ * only added if UseScramPassthrough is set, and the user is not allowed
+ * to add the SCRAM keys on fdw and user mapping options.
+ */
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
#ifdef ENABLE_GSS
/* If GSSAPI creds used to connect, make sure it was one delegated */
if (PQconnectionUsedGSSAPI(conn) && be_gssapi_get_delegation(MyProcPort))
@@ -2605,12 +2686,14 @@ dblink_connstr_has_pw(const char *connstr)
}
/*
- * For non-superusers, insist that the connstr specify a password, except
- * if GSSAPI credentials have been delegated (and we check that they are used
- * for the connection in dblink_security_check later). This prevents a
- * password or GSSAPI credentials from being picked up from .pgpass, a
- * service file, the environment, etc. We don't want the postgres user's
- * passwords or Kerberos credentials to be accessible to non-superusers.
+ * For non-superusers, insist that the connstr specify a password, except if
+ * GSSAPI credentials have been delegated (and we check that they are used for
+ * the connection in dblink_security_check later) or if SCRAM pass-through is
+ * being used. This prevents a password or GSSAPI credentials from being
+ * picked up from .pgpass, a service file, the environment, etc. We don't want
+ * the postgres user's passwords or Kerberos credentials to be accessible to
+ * non-superusers. In case of SCRAM pass-through insist that the connstr
+ * has the required SCRAM pass-through options.
*/
static void
dblink_connstr_check(const char *connstr)
@@ -2621,6 +2704,9 @@ dblink_connstr_check(const char *connstr)
if (dblink_connstr_has_pw(connstr))
return;
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
#ifdef ENABLE_GSS
if (be_gssapi_get_delegation(MyProcPort))
return;
@@ -2763,6 +2849,14 @@ get_connect_string(ForeignServer *foreign_server)
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ /*
+ * First append hardcoded options needed for SCRAM pass-through, so if the
+ * user overwrites these options we can ereport on dblink_connstr_check
+ * and dblink_security_check.
+ */
+ if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
+ appendSCRAMKeysInfo(&buf);
+
foreach(cell, fdw->options)
{
DefElem *def = lfirst(cell);
@@ -2944,6 +3038,20 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
+/*
+ * Same as is_valid_dblink_option but also check for only dblink_fdw specific
+ * options.
+ */
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context)
+{
+ if (strcmp(option, "use_scram_passthrough") == 0)
+ return true;
+
+ return is_valid_dblink_option(options, option, context);
+}
+
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3014,6 +3122,69 @@ restoreLocalGucs(int nestlevel)
AtEOXact_GUC(true, nestlevel);
}
+/*
+ * Append SCRAM client key and server key information from the global
+ * MyProcPort into the given StringInfo buffer.
+ */
+static void
+appendSCRAMKeysInfo(StringInfo buf)
+{
+ int len;
+ int encoded_len;
+ char *client_key;
+ char *server_key;
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
+ /* don't forget the zero-terminator */
+ client_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ClientKey,
+ sizeof(MyProcPort->scram_ClientKey),
+ client_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM client key");
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ServerKey));
+ /* don't forget the zero-terminator */
+ server_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ServerKey,
+ sizeof(MyProcPort->scram_ServerKey),
+ server_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM server key");
+
+ appendStringInfo(buf, "scram_client_key='%s' ", client_key);
+ appendStringInfo(buf, "scram_server_key='%s' ", server_key);
+ appendStringInfo(buf, "require_auth='scram-sha-256' ");
+
+ pfree(client_key);
+ pfree(server_key);
+}
+
+
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user)
+{
+ ListCell *cell;
+
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ foreach(cell, user->options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ return false;
+}
+
/*
* Connect to remote server. If connstr_or_srvname maps to a foreign server,
* the associated properties and user mapping properties is also used to open
diff --git a/contrib/dblink/meson.build b/contrib/dblink/meson.build
index 3ab78668288..dfd8eb6877e 100644
--- a/contrib/dblink/meson.build
+++ b/contrib/dblink/meson.build
@@ -36,4 +36,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/001_auth_scram.pl',
+ ],
+ },
}
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
new file mode 100644
index 00000000000..3b25961b664
--- /dev/null
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -0,0 +1,246 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+# Test SCRAM authentication when opening a new connection with a foreign
+# server.
+#
+# The test is executed by testing the SCRAM authentifcation on a loopback
+# connection on the same server and with different servers.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $user = "user01";
+
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
+my $fdw_server = "db1_fdw";
+my $fdw_server2 = "db2_fdw";
+my $fdw_invalid_server = "db2_fdw_invalid"; # For invalid fdw options
+my $fdw_invalid_server2 =
+ "db2_fdw_invalid2"; # For invalid scram keys fdw options
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+my $node2 = PostgreSQL::Test::Cluster->new('node2');
+
+$node1->init;
+$node2->init;
+
+$node1->start;
+$node2->start;
+
+# Test setup
+
+$node1->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$node2->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$ENV{PGPASSWORD} = "pass";
+
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db0');
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db1');
+$node2->safe_psql('postgres', qq'CREATE DATABASE $db2');
+
+setup_table($node1, $db1, "t");
+setup_table($node2, $db2, "t2");
+
+$node1->safe_psql($db0, 'CREATE EXTENSION IF NOT EXISTS dblink');
+setup_fdw_server($node1, $db0, $fdw_server, $node1, $db1);
+setup_fdw_server($node1, $db0, $fdw_server2, $node2, $db2);
+setup_invalid_fdw_server($node1, $db0, $fdw_invalid_server, $node2, $db2);
+setup_fdw_server($node1, $db0, $fdw_invalid_server2, $node2, $db2);
+
+setup_user_mapping($node1, $db0, $fdw_server);
+setup_user_mapping($node1, $db0, $fdw_server2);
+setup_user_mapping($node1, $db0, $fdw_invalid_server);
+
+# Make the user have the same SCRAM key on both servers. Forcing to have the
+# same iteration and salt.
+my $rolpassword = $node1->safe_psql('postgres',
+ qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';");
+$node2->safe_psql('postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'");
+
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all scram-sha-256
+}
+);
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all scram-sha-256
+}
+);
+
+$node1->restart;
+$node2->restart;
+
+# End of test setup
+
+test_scram_keys_is_not_overwritten($node1, $db0, $fdw_invalid_server2);
+
+test_fdw_auth($node1, $db0, "t", $fdw_server,
+ "SCRAM auth on the same database cluster must succeed");
+
+test_fdw_auth($node1, $db0, "t2", $fdw_server2,
+ "SCRAM auth on a different database cluster must succeed");
+
+test_fdw_auth_with_invalid_overwritten_require_auth($fdw_invalid_server);
+
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+}
+);
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local all all password
+}
+);
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/failed: authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback password fails on a different cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server requested a cleartext password/,
+ 'expected error from loopback password (different cluster)');
+
+# Helper functions
+
+sub test_fdw_auth
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $db, $tbl, $fdw, $testname) = @_;
+ my $connstr = $node->connstr($db) . qq' user=$user';
+
+ my $ret = $node->safe_psql(
+ $db,
+ qq"SELECT count(1) FROM dblink('$fdw', 'SELECT * FROM $tbl') AS $tbl(a int, b int)",
+ connstr => $connstr);
+
+ is($ret, '10', $testname);
+}
+
+sub test_fdw_auth_with_invalid_overwritten_require_auth
+{
+
+ my ($fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "select * from dblink('$fdw', 'select * from t') as t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+ is($ret, 3, 'loopback trust fails when overwriting require_auth');
+ like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error when connecting to a fdw overwriting the require_auth'
+ );
+}
+
+sub test_scram_keys_is_not_overwritten
+{
+ my ($node, $db, $fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_client_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret, 3, 'user mapping creation fails when using scram_client_key');
+ like(
+ $stderr,
+ qr/ERROR: invalid option "scram_client_key"/,
+ 'user mapping creation fails when using scram_client_key');
+
+ ($ret, $stdout, $stderr) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_server_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret, 3, 'user mapping creation fails when using scram_server_key');
+ like(
+ $stderr,
+ qr/ERROR: invalid option "scram_server_key"/,
+ 'user mapping creation fails when using scram_server_key');
+}
+
+sub setup_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\');'
+ );
+}
+
+sub setup_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_invalid_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\', require_auth \'none\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_table
+{
+ my ($node, $db, $tbl) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE TABLE $tbl AS SELECT g as a, g + 1 as b FROM generate_series(1,10) g(g)'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON SCHEMA public TO $user');
+ $node->safe_psql($db, qq'GRANT SELECT ON $tbl TO $user');
+}
+
+done_testing();
+
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index 81f35986c88..a575d0eb431 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,6 +136,76 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
+ <refsect1>
+ <title>Foreign Data Wrapper</title>
+
+ <para>
+ A Foreign Data Wrapper can be used as a connection name parameter. The foreign
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ </para>
+
+ <para>
+ The authentication with the foreign server can be via password on USER
+ MAPPING or using SCRAM pass-through. The
+ <literal>use_scram_passthrough</literal> on CREATE SERVER options controls
+ whether <filename>dblink_fdw</filename> will use the SCRAM pass-through
+ authentication to connect to the foreign server. With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
+ </para>
+
+ <para>
+ To use SCRAM pass-through authentication:
+ <itemizedlist>
+ <listitem>
+ <para>
+ The remote server must request the <literal>scram-sha-256</literal>
+ authentication method; otherwise, the connection will fail.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The user mapping password is not used.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The remote server can be of any PostgreSQL version that supports SCRAM.
+ Support for <literal>use_scram_passthrough</literal> is only required on
+ the client side (FDW side).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The server running <filename>dblink_fdw</filename> and the remote server
+ must have identical SCRAM secrets (encrypted passwords) for the user being
+ used on <filename>dblink_fdw</filename> to authenticate on the foreign
+ server (same salt and iterations, not merely the same password).
+ </para>
+
+ <para>
+ As a corollary, if FDW connections to multiple hosts are to be made, for
+ example for partitioned foreign tables/sharding, then all hosts must have
+ identical SCRAM secrets for the users involved.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The current session on the PostgreSQL instance that makes the outgoing FDW
+ connections also must also use SCRAM authentication for its incoming client
+ connection. (Hence <quote>pass-through</quote>: SCRAM must be used going in
+ and out.) This is a technical requirement of the SCRAM protocol.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </refsect1>
+
<refsect1>
<title>Notes</title>
@@ -181,8 +251,9 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection must require password authentication for this to work properly
--- Otherwise, you will receive the following error from dblink_connect():
+-- Note: local connections that don't use SCRAM pass-through require password
+-- authentication for this to work properly. Otherwise, you will receive
+-- the following error from dblink_connect():
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
--
2.39.5 (Apple Git-154)
v10-0003-postgres_fdw-improve-security-checks.patchapplication/octet-stream; name=v10-0003-postgres_fdw-improve-security-checks.patchDownload
From e4667b7e5520bbe7380ea684ebb815522a44fe82 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Wed, 19 Mar 2025 14:16:18 -0300
Subject: [PATCH v10 3/3] postgres_fdw: improve security checks
On adding scram pass-through for dblink [1] thread, it was discussed
that we should not by-pass fdw security check as it was implemented for
postgres_fdw on 761c79508e7.
This commit improve the security check by adding new scram pass-through
checks to ensure that the required scram connection options are not
overwritten by the user mapping or foreign server options.
[1] https://www.postgresql.org/message-id/flat/CAFY6G8ercA1KES%3DE_0__R9QCTR805TTyYr1No8qF8ZxmMg8z2Q%40mail.gmail.com
---
contrib/postgres_fdw/connection.c | 103 ++++++++++++++++++++---
contrib/postgres_fdw/t/001_auth_scram.pl | 41 +++++++++
doc/src/sgml/postgres-fdw.sgml | 11 +--
3 files changed, 133 insertions(+), 22 deletions(-)
diff --git a/contrib/postgres_fdw/connection.c b/contrib/postgres_fdw/connection.c
index 8ef9702c05c..5a6e3e9825a 100644
--- a/contrib/postgres_fdw/connection.c
+++ b/contrib/postgres_fdw/connection.c
@@ -184,6 +184,7 @@ static void postgres_fdw_get_connections_internal(FunctionCallInfo fcinfo,
enum pgfdwVersion api_version);
static int pgfdw_conn_check(PGconn *conn);
static bool pgfdw_conn_checkable(void);
+static bool pgfdw_has_required_scram_options(const char **keywords, const char **values);
/*
* Get a PGconn which can be used to execute queries on the remote PostgreSQL
@@ -455,6 +456,15 @@ pgfdw_security_check(const char **keywords, const char **values, UserMapping *us
}
}
+ /*
+ * Ok if SCRAM pass-through is being used and all required SCRAM options
+ * are set correctly. If pgfdw_has_required_scram_options returns true we
+ * assume that UseScramPassthrough is also true since SCRAM options are
+ * only set when UseScramPassthrough is enabled.
+ */
+ if (MyProcPort->has_scram_keys && pgfdw_has_required_scram_options(keywords, values))
+ return;
+
ereport(ERROR,
(errcode(ERRCODE_S_R_E_PROHIBITED_SQL_STATEMENT_ATTEMPTED),
errmsg("password or GSSAPI delegated credentials required"),
@@ -485,9 +495,10 @@ connect_pg_server(ForeignServer *server, UserMapping *user)
* and UserMapping. (Some of them might not be libpq options, in
* which case we'll just waste a few array slots.) Add 4 extra slots
* for application_name, fallback_application_name, client_encoding,
- * end marker.
+ * end marker, and 3 extra slots for scram keys and required scram
+ * pass-through options.
*/
- n = list_length(server->options) + list_length(user->options) + 4 + 2;
+ n = list_length(server->options) + list_length(user->options) + 4 + 3;
keywords = (const char **) palloc(n * sizeof(char *));
values = (const char **) palloc(n * sizeof(char *));
@@ -556,6 +567,7 @@ connect_pg_server(ForeignServer *server, UserMapping *user)
values[n] = GetDatabaseEncodingName();
n++;
+ /* Add required SCRAM pass-through connection options if it's enabled. */
if (MyProcPort->has_scram_keys && UseScramPassthrough(server, user))
{
int len;
@@ -582,16 +594,20 @@ connect_pg_server(ForeignServer *server, UserMapping *user)
if (encoded_len < 0)
elog(ERROR, "could not encode SCRAM server key");
n++;
+
+ /*
+ * Require scram-sha-256 to ensure that no other auth method is
+ * used when connecting with foreign server.
+ */
+ keywords[n] = "require_auth";
+ values[n] = "scram-sha-256";
+ n++;
}
keywords[n] = values[n] = NULL;
- /*
- * Verify the set of connection parameters only if scram pass-through
- * is not being used because the password is not necessary.
- */
- if (!(MyProcPort->has_scram_keys && UseScramPassthrough(server, user)))
- check_conn_params(keywords, values, user);
+ /* Verify the set of connection parameters. */
+ check_conn_params(keywords, values, user);
/* first time, allocate or get the custom wait event */
if (pgfdw_we_connect == 0)
@@ -609,12 +625,8 @@ connect_pg_server(ForeignServer *server, UserMapping *user)
server->servername),
errdetail_internal("%s", pchomp(PQerrorMessage(conn)))));
- /*
- * Perform post-connection security checks only if scram pass-through
- * is not being used because the password is not necessary.
- */
- if (!(MyProcPort->has_scram_keys && UseScramPassthrough(server, user)))
- pgfdw_security_check(keywords, values, user, conn);
+ /* Perform post-connection security checks. */
+ pgfdw_security_check(keywords, values, user, conn);
/* Prepare new session for use */
configure_remote_session(conn);
@@ -725,6 +737,15 @@ check_conn_params(const char **keywords, const char **values, UserMapping *user)
if (!UserMappingPasswordRequired(user))
return;
+ /*
+ * Ok if SCRAM pass-through is being used and all required scram options
+ * are set correctly. If pgfdw_has_required_scram_options returns true we
+ * assume that UseScramPassthrough is also true since SCRAM options are
+ * only set when UseScramPassthrough is enabled.
+ */
+ if (MyProcPort->has_scram_keys && pgfdw_has_required_scram_options(keywords, values))
+ return;
+
ereport(ERROR,
(errcode(ERRCODE_S_R_E_PROHIBITED_SQL_STATEMENT_ATTEMPTED),
errmsg("password or GSSAPI delegated credentials required"),
@@ -2487,3 +2508,57 @@ pgfdw_conn_checkable(void)
return false;
#endif
}
+
+/*
+ * Ensure that require_auth and SCRAM keys are correctly set on values. SCRAM
+ * keys used to pass-through are coming from the initial connection from the
+ * client with the server.
+ *
+ * All required SCRAM options are set by postgres_fdw, so we just need to
+ * ensure that these options are not overwritten by the user.
+ */
+static bool
+pgfdw_has_required_scram_options(const char **keywords, const char **values)
+{
+ int i;
+ bool has_scram_server_key = false;
+ bool has_scram_client_key = false;
+ bool has_require_auth = false;
+ bool has_scram_keys = false;
+
+ /*
+ * Continue iterating even if we found the keys that we need to validate
+ * to make sure that there is no other declaration of these keys that can
+ * overwrite the first.
+ */
+ for (i = 0; keywords[i] != NULL; i++)
+ {
+ if (strcmp(keywords[i], "scram_client_key") == 0)
+ {
+ if (values[i] != NULL && values[i][0] != '\0')
+ has_scram_client_key = true;
+ else
+ has_scram_client_key = false;
+ }
+
+ if (strcmp(keywords[i], "scram_server_key") == 0)
+ {
+ if (values[i] != NULL && values[i][0] != '\0')
+ has_scram_server_key = true;
+ else
+ has_scram_server_key = false;
+ }
+
+ if (strcmp(keywords[i], "require_auth") == 0)
+ {
+ if (values[i] != NULL && strcmp(values[i], "scram-sha-256") == 0)
+ has_require_auth = true;
+ else
+ has_require_auth = false;
+ }
+ }
+
+ has_scram_keys = has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys;
+
+ return (has_scram_keys && has_require_auth);
+}
diff --git a/contrib/postgres_fdw/t/001_auth_scram.pl b/contrib/postgres_fdw/t/001_auth_scram.pl
index 047840cc914..2cce21b0fdb 100644
--- a/contrib/postgres_fdw/t/001_auth_scram.pl
+++ b/contrib/postgres_fdw/t/001_auth_scram.pl
@@ -68,6 +68,47 @@ test_fdw_auth($node1, $db0, "t2", $fdw_server2,
test_auth($node2, $db2, "t2",
"SCRAM auth directly on foreign server should still succeed");
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+}
+);
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local all all password
+}
+);
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ qq'select count(1) from t',
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/failed: authentication method requirement "scram-sha-256"/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ qq'select count(1) from t2',
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback password fails on a different cluster');
+like(
+ $stderr,
+ qr/failed: authentication method requirement "scram-sha-256"/,
+ 'expected error from loopback password (different cluster)');
+
# Helper functions
sub test_auth
diff --git a/doc/src/sgml/postgres-fdw.sgml b/doc/src/sgml/postgres-fdw.sgml
index a7f2f5ca182..65e36f1f3e4 100644
--- a/doc/src/sgml/postgres-fdw.sgml
+++ b/doc/src/sgml/postgres-fdw.sgml
@@ -788,10 +788,8 @@ OPTIONS (ADD password_required 'false');
<itemizedlist>
<listitem>
<para>
- The remote server must request SCRAM authentication. (If desired,
- enforce this on the client side (FDW side) with the option
- <literal>require_auth</literal>.) If another authentication method
- is requested by the server, then that one will be used normally.
+ The remote server must request the <literal>scram-sha-256</literal>
+ authentication method; otherwise, the connection will fail.
</para>
</listitem>
@@ -805,10 +803,7 @@ OPTIONS (ADD password_required 'false');
<listitem>
<para>
- The user mapping password is not used. (It could be set to support
- other authentication methods, but that would arguably violate the
- point of this feature, which is to avoid storing plain-text
- passwords.)
+ The user mapping password is not used.
</para>
</listitem>
--
2.39.5 (Apple Git-154)
Great, thank you! Looking over v10, I think I've run out of feedback
at this point. Marked Ready for Committer.
--Jacob
On Fri, Mar 21, 2025 at 1:28 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
Great, thank you! Looking over v10, I think I've run out of feedback
at this point. Marked Ready for Committer.
Thanks for all the effort reviewing this patch!
--
Matheus Alcantara
On 21.03.25 19:24, Matheus Alcantara wrote:
On Fri, Mar 21, 2025 at 1:28 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:Great, thank you! Looking over v10, I think I've run out of feedback
at this point. Marked Ready for Committer.Thanks for all the effort reviewing this patch!
I have committed the 0003 patch (the postgres_fdw bug fix).
The dblink feature patch (0002) looks good to me.
I'm a bit confused about the refactoring patch 0001. There are some
details there that don't seem right. For example, you write that the
pfree(rconn) calls are no longer necessary, but AFAICT, it would still
be needed in dblink_get_conn(). Also, there appear to be some possible
behavior changes, or at least it's not fully explained, like
connect_pg_server() doing foreign-server name resolution, which
dblink_get_conn() did not do before.
But it's actually not clear to me how the refactoring in 0001
contributes to making the patch 0002 better, since patch 0002 barely
touches the code touched by 0001.
How would patch 0002 look without 0001 before it? Which code would need
to be duplicated or what other awkward changes are you trying to avoid?
On Mon, Mar 24, 2025 at 1:16 PM Peter Eisentraut <peter@eisentraut.org> wrote:
On 21.03.25 19:24, Matheus Alcantara wrote:
On Fri, Mar 21, 2025 at 1:28 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:Great, thank you! Looking over v10, I think I've run out of feedback
at this point. Marked Ready for Committer.Thanks for all the effort reviewing this patch!
I have committed the 0003 patch (the postgres_fdw bug fix).
Thanks!
I'm a bit confused about the refactoring patch 0001. There are some
details there that don't seem right. For example, you write that the
pfree(rconn) calls are no longer necessary, but AFAICT, it would still
be needed in dblink_get_conn(). Also, there appear to be some possible
behavior changes, or at least it's not fully explained, like
connect_pg_server() doing foreign-server name resolution, which
dblink_get_conn() did not do before.But it's actually not clear to me how the refactoring in 0001
contributes to making the patch 0002 better, since patch 0002 barely
touches the code touched by 0001.How would patch 0002 look without 0001 before it? Which code would need
to be duplicated or what other awkward changes are you trying to avoid?
You are right, I think that the refactor was needed on the initial
versions of the patch because it was referencing the UseScramPassthrough
function in multiple places, so the refactor was needed to accomplish the
parameters of the function.
Since we now assume that the UseScramPassthrough is already checked on
some parts of the code I agree that this refactor is not required
anymore. Attached v11 without the refactor patch.
Thanks!
--
Matheus Alcantara
Attachments:
v11-0001-dblink-Add-SCRAM-pass-through-authentication.patchapplication/octet-stream; name=v11-0001-dblink-Add-SCRAM-pass-through-authentication.patchDownload
From e392e18dfb17cdd6a8093ab6c9f9bfd5d56b03c4 Mon Sep 17 00:00:00 2001
From: Matheus Alcantara <mths.dev@pm.me>
Date: Mon, 24 Mar 2025 16:17:01 -0300
Subject: [PATCH v11] dblink: Add SCRAM pass-through authentication
This enables SCRAM authentication for dblink_fdw when connecting to a
foreign server using the same approach as it was implemented for
postgres_fdw on 761c795.
---
contrib/dblink/Makefile | 1 +
contrib/dblink/dblink.c | 185 +++++++++++++++++++++-
contrib/dblink/meson.build | 5 +
contrib/dblink/t/001_auth_scram.pl | 246 +++++++++++++++++++++++++++++
doc/src/sgml/dblink.sgml | 75 ++++++++-
5 files changed, 503 insertions(+), 9 deletions(-)
create mode 100644 contrib/dblink/t/001_auth_scram.pl
diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index d4c7ed625ab..fde0b49ddbb 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -13,6 +13,7 @@ PGFILEDESC = "dblink - connect to other PostgreSQL databases"
REGRESS = dblink
REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress
+TAP_TESTS = 1
ifdef USE_PGXS
PG_CONFIG = pg_config
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 58c1a6221c8..ce795612fb3 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -43,6 +43,8 @@
#include "catalog/pg_foreign_server.h"
#include "catalog/pg_type.h"
#include "catalog/pg_user_mapping.h"
+#include "commands/defrem.h"
+#include "common/base64.h"
#include "executor/spi.h"
#include "foreign/foreign.h"
#include "funcapi.h"
@@ -126,6 +128,11 @@ static bool is_valid_dblink_option(const PQconninfoOption *options,
const char *option, Oid context);
static int applyRemoteGucs(PGconn *conn);
static void restoreLocalGucs(int nestlevel);
+static bool UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user);
+static void appendSCRAMKeysInfo(StringInfo buf);
+static bool is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context);
+static bool dblink_connstr_has_required_scram_options(const char *connstr);
/* Global */
static remoteConn *pconn = NULL;
@@ -1964,7 +1971,7 @@ dblink_fdw_validator(PG_FUNCTION_ARGS)
{
DefElem *def = (DefElem *) lfirst(cell);
- if (!is_valid_dblink_option(options, def->defname, context))
+ if (!is_valid_dblink_fdw_option(options, def->defname, context))
{
/*
* Unknown option, or invalid option for the context specified, so
@@ -2596,6 +2603,68 @@ deleteConnection(const char *name)
errmsg("undefined connection name")));
}
+ /*
+ * Ensure that require_auth and SCRAM keys are correctly set on connstr.
+ * SCRAM keys used to pass-through are coming from the initial connection
+ * from the client with the server.
+ *
+ * All required SCRAM options are set by dblink, so we just need to ensure
+ * that these options are not overwritten by the user.
+ *
+ * See appendSCRAMKeysInfo and its usage for more.
+ */
+bool
+dblink_connstr_has_required_scram_options(const char *connstr)
+{
+ PQconninfoOption *options;
+ PQconninfoOption *option;
+ bool has_scram_server_key = false;
+ bool has_scram_client_key = false;
+ bool has_require_auth = false;
+ bool has_scram_keys = false;
+
+ options = PQconninfoParse(connstr, NULL);
+ if (options)
+ {
+ /*
+ * Continue iterating even if we found the keys that we need to
+ * validate to make sure that there is no other declaration of these
+ * keys that can overwrite the first.
+ */
+ for (option = options; option->keyword != NULL; option++)
+ {
+ if (strcmp(option->keyword, "require_auth") == 0)
+ {
+ if (option->val != NULL && strcmp(option->val, "scram-sha-256") == 0)
+ has_require_auth = true;
+ else
+ has_require_auth = false;
+ }
+
+ if (strcmp(option->keyword, "scram_client_key") == 0)
+ {
+ if (option->val != NULL && option->val[0] != '\0')
+ has_scram_client_key = true;
+ else
+ has_scram_client_key = false;
+ }
+
+ if (strcmp(option->keyword, "scram_server_key") == 0)
+ {
+ if (option->val != NULL && option->val[0] != '\0')
+ has_scram_server_key = true;
+ else
+ has_scram_server_key = false;
+ }
+ }
+ PQconninfoFree(options);
+ }
+
+ has_scram_keys = has_scram_client_key && has_scram_server_key && MyProcPort->has_scram_keys;
+
+ return (has_scram_keys && has_require_auth);
+}
+
/*
* We need to make sure that the connection made used credentials
* which were provided by the user, so check what credentials were
@@ -2612,6 +2681,18 @@ dblink_security_check(PGconn *conn, remoteConn *rconn, const char *connstr)
if (PQconnectionUsedPassword(conn) && dblink_connstr_has_pw(connstr))
return;
+ /*
+ * Password was not used to connect, check if SCRAM pass-through is in
+ * use.
+ *
+ * If dblink_connstr_has_required_scram_options is true we assume that
+ * UseScramPassthrough is also true because the required SCRAM keys are
+ * only added if UseScramPassthrough is set, and the user is not allowed
+ * to add the SCRAM keys on fdw and user mapping options.
+ */
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
#ifdef ENABLE_GSS
/* If GSSAPI creds used to connect, make sure it was one delegated */
if (PQconnectionUsedGSSAPI(conn) && be_gssapi_get_delegation(MyProcPort))
@@ -2664,12 +2745,14 @@ dblink_connstr_has_pw(const char *connstr)
}
/*
- * For non-superusers, insist that the connstr specify a password, except
- * if GSSAPI credentials have been delegated (and we check that they are used
- * for the connection in dblink_security_check later). This prevents a
- * password or GSSAPI credentials from being picked up from .pgpass, a
- * service file, the environment, etc. We don't want the postgres user's
- * passwords or Kerberos credentials to be accessible to non-superusers.
+ * For non-superusers, insist that the connstr specify a password, except if
+ * GSSAPI credentials have been delegated (and we check that they are used for
+ * the connection in dblink_security_check later) or if SCRAM pass-through is
+ * being used. This prevents a password or GSSAPI credentials from being
+ * picked up from .pgpass, a service file, the environment, etc. We don't want
+ * the postgres user's passwords or Kerberos credentials to be accessible to
+ * non-superusers. In case of SCRAM pass-through insist that the connstr
+ * has the required SCRAM pass-through options.
*/
static void
dblink_connstr_check(const char *connstr)
@@ -2680,6 +2763,9 @@ dblink_connstr_check(const char *connstr)
if (dblink_connstr_has_pw(connstr))
return;
+ if (MyProcPort->has_scram_keys && dblink_connstr_has_required_scram_options(connstr))
+ return;
+
#ifdef ENABLE_GSS
if (be_gssapi_get_delegation(MyProcPort))
return;
@@ -2832,6 +2918,14 @@ get_connect_string(const char *servername)
if (aclresult != ACLCHECK_OK)
aclcheck_error(aclresult, OBJECT_FOREIGN_SERVER, foreign_server->servername);
+ /*
+ * First append hardcoded options needed for SCRAM pass-through, so if
+ * the user overwrites these options we can ereport on
+ * dblink_connstr_check and dblink_security_check.
+ */
+ if (MyProcPort->has_scram_keys && UseScramPassthrough(foreign_server, user_mapping))
+ appendSCRAMKeysInfo(&buf);
+
foreach(cell, fdw->options)
{
DefElem *def = lfirst(cell);
@@ -3016,6 +3110,20 @@ is_valid_dblink_option(const PQconninfoOption *options, const char *option,
return true;
}
+/*
+ * Same as is_valid_dblink_option but also check for only dblink_fdw specific
+ * options.
+ */
+static bool
+is_valid_dblink_fdw_option(const PQconninfoOption *options, const char *option,
+ Oid context)
+{
+ if (strcmp(option, "use_scram_passthrough") == 0)
+ return true;
+
+ return is_valid_dblink_option(options, option, context);
+}
+
/*
* Copy the remote session's values of GUCs that affect datatype I/O
* and apply them locally in a new GUC nesting level. Returns the new
@@ -3085,3 +3193,66 @@ restoreLocalGucs(int nestlevel)
if (nestlevel > 0)
AtEOXact_GUC(true, nestlevel);
}
+
+/*
+ * Append SCRAM client key and server key information from the global
+ * MyProcPort into the given StringInfo buffer.
+ */
+static void
+appendSCRAMKeysInfo(StringInfo buf)
+{
+ int len;
+ int encoded_len;
+ char *client_key;
+ char *server_key;
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ClientKey));
+ /* don't forget the zero-terminator */
+ client_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ClientKey,
+ sizeof(MyProcPort->scram_ClientKey),
+ client_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM client key");
+
+ len = pg_b64_enc_len(sizeof(MyProcPort->scram_ServerKey));
+ /* don't forget the zero-terminator */
+ server_key = palloc0(len + 1);
+ encoded_len = pg_b64_encode((const char *) MyProcPort->scram_ServerKey,
+ sizeof(MyProcPort->scram_ServerKey),
+ server_key, len);
+ if (encoded_len < 0)
+ elog(ERROR, "could not encode SCRAM server key");
+
+ appendStringInfo(buf, "scram_client_key='%s' ", client_key);
+ appendStringInfo(buf, "scram_server_key='%s' ", server_key);
+ appendStringInfo(buf, "require_auth='scram-sha-256' ");
+
+ pfree(client_key);
+ pfree(server_key);
+}
+
+
+static bool
+UseScramPassthrough(ForeignServer *foreign_server, UserMapping *user)
+{
+ ListCell *cell;
+
+ foreach(cell, foreign_server->options)
+ {
+ DefElem *def = lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ foreach(cell, user->options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ if (strcmp(def->defname, "use_scram_passthrough") == 0)
+ return defGetBoolean(def);
+ }
+
+ return false;
+}
diff --git a/contrib/dblink/meson.build b/contrib/dblink/meson.build
index 3ab78668288..dfd8eb6877e 100644
--- a/contrib/dblink/meson.build
+++ b/contrib/dblink/meson.build
@@ -36,4 +36,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/001_auth_scram.pl',
+ ],
+ },
}
diff --git a/contrib/dblink/t/001_auth_scram.pl b/contrib/dblink/t/001_auth_scram.pl
new file mode 100644
index 00000000000..3b25961b664
--- /dev/null
+++ b/contrib/dblink/t/001_auth_scram.pl
@@ -0,0 +1,246 @@
+# Copyright (c) 2024-2025, PostgreSQL Global Development Group
+
+# Test SCRAM authentication when opening a new connection with a foreign
+# server.
+#
+# The test is executed by testing the SCRAM authentifcation on a loopback
+# connection on the same server and with different servers.
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $user = "user01";
+
+my $db0 = "db0"; # For node1
+my $db1 = "db1"; # For node1
+my $db2 = "db2"; # For node2
+my $fdw_server = "db1_fdw";
+my $fdw_server2 = "db2_fdw";
+my $fdw_invalid_server = "db2_fdw_invalid"; # For invalid fdw options
+my $fdw_invalid_server2 =
+ "db2_fdw_invalid2"; # For invalid scram keys fdw options
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+my $node2 = PostgreSQL::Test::Cluster->new('node2');
+
+$node1->init;
+$node2->init;
+
+$node1->start;
+$node2->start;
+
+# Test setup
+
+$node1->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$node2->safe_psql('postgres', qq'CREATE USER $user WITH password \'pass\'');
+$ENV{PGPASSWORD} = "pass";
+
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db0');
+$node1->safe_psql('postgres', qq'CREATE DATABASE $db1');
+$node2->safe_psql('postgres', qq'CREATE DATABASE $db2');
+
+setup_table($node1, $db1, "t");
+setup_table($node2, $db2, "t2");
+
+$node1->safe_psql($db0, 'CREATE EXTENSION IF NOT EXISTS dblink');
+setup_fdw_server($node1, $db0, $fdw_server, $node1, $db1);
+setup_fdw_server($node1, $db0, $fdw_server2, $node2, $db2);
+setup_invalid_fdw_server($node1, $db0, $fdw_invalid_server, $node2, $db2);
+setup_fdw_server($node1, $db0, $fdw_invalid_server2, $node2, $db2);
+
+setup_user_mapping($node1, $db0, $fdw_server);
+setup_user_mapping($node1, $db0, $fdw_server2);
+setup_user_mapping($node1, $db0, $fdw_invalid_server);
+
+# Make the user have the same SCRAM key on both servers. Forcing to have the
+# same iteration and salt.
+my $rolpassword = $node1->safe_psql('postgres',
+ qq"SELECT rolpassword FROM pg_authid WHERE rolname = '$user';");
+$node2->safe_psql('postgres', qq"ALTER ROLE $user PASSWORD '$rolpassword'");
+
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all scram-sha-256
+}
+);
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local db2 all scram-sha-256
+}
+);
+
+$node1->restart;
+$node2->restart;
+
+# End of test setup
+
+test_scram_keys_is_not_overwritten($node1, $db0, $fdw_invalid_server2);
+
+test_fdw_auth($node1, $db0, "t", $fdw_server,
+ "SCRAM auth on the same database cluster must succeed");
+
+test_fdw_auth($node1, $db0, "t2", $fdw_server2,
+ "SCRAM auth on a different database cluster must succeed");
+
+test_fdw_auth_with_invalid_overwritten_require_auth($fdw_invalid_server);
+
+# Ensure that trust connections fail without superuser opt-in.
+unlink($node1->data_dir . '/pg_hba.conf');
+unlink($node2->data_dir . '/pg_hba.conf');
+
+$node1->append_conf(
+ 'pg_hba.conf', qq{
+local db0 all scram-sha-256
+local db1 all trust
+}
+);
+$node2->append_conf(
+ 'pg_hba.conf', qq{
+local all all password
+}
+);
+
+$node1->restart;
+$node2->restart;
+
+my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server', 'SELECT * FROM t') AS t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback trust fails on the same cluster');
+like(
+ $stderr,
+ qr/failed: authentication method requirement "scram-sha-256" failed: server did not complete authentication/,
+ 'expected error from loopback trust (same cluster)');
+
+($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "SELECT * FROM dblink('$fdw_server2', 'SELECT * FROM t2') AS t2(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+is($ret, 3, 'loopback password fails on a different cluster');
+like(
+ $stderr,
+ qr/authentication method requirement "scram-sha-256" failed: server requested a cleartext password/,
+ 'expected error from loopback password (different cluster)');
+
+# Helper functions
+
+sub test_fdw_auth
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $db, $tbl, $fdw, $testname) = @_;
+ my $connstr = $node->connstr($db) . qq' user=$user';
+
+ my $ret = $node->safe_psql(
+ $db,
+ qq"SELECT count(1) FROM dblink('$fdw', 'SELECT * FROM $tbl') AS $tbl(a int, b int)",
+ connstr => $connstr);
+
+ is($ret, '10', $testname);
+}
+
+sub test_fdw_auth_with_invalid_overwritten_require_auth
+{
+
+ my ($fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node1->psql(
+ $db0,
+ "select * from dblink('$fdw', 'select * from t') as t(a int, b int)",
+ connstr => $node1->connstr($db0) . " user=$user");
+
+ is($ret, 3, 'loopback trust fails when overwriting require_auth');
+ like(
+ $stderr,
+ qr/password or GSSAPI delegated credentials required/,
+ 'expected error when connecting to a fdw overwriting the require_auth'
+ );
+}
+
+sub test_scram_keys_is_not_overwritten
+{
+ my ($node, $db, $fdw) = @_;
+
+ my ($ret, $stdout, $stderr) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_client_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret, 3, 'user mapping creation fails when using scram_client_key');
+ like(
+ $stderr,
+ qr/ERROR: invalid option "scram_client_key"/,
+ 'user mapping creation fails when using scram_client_key');
+
+ ($ret, $stdout, $stderr) = $node->psql(
+ $db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\', scram_server_key \'key\');',
+ connstr => $node->connstr($db) . " user=$user");
+
+ is($ret, 3, 'user mapping creation fails when using scram_server_key');
+ like(
+ $stderr,
+ qr/ERROR: invalid option "scram_server_key"/,
+ 'user mapping creation fails when using scram_server_key');
+}
+
+sub setup_user_mapping
+{
+ my ($node, $db, $fdw) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE USER MAPPING FOR $user SERVER $fdw OPTIONS (user \'$user\');'
+ );
+}
+
+sub setup_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_invalid_fdw_server
+{
+ my ($node, $db, $fdw, $fdw_node, $dbname) = @_;
+ my $host = $fdw_node->host;
+ my $port = $fdw_node->port;
+
+ $node->safe_psql(
+ $db, qq'CREATE SERVER $fdw FOREIGN DATA WRAPPER dblink_fdw options (
+ host \'$host\', port \'$port\', dbname \'$dbname\', use_scram_passthrough \'true\', require_auth \'none\') '
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON FOREIGN SERVER $fdw TO $user;');
+ $node->safe_psql($db, qq'GRANT ALL ON SCHEMA public TO $user');
+}
+
+sub setup_table
+{
+ my ($node, $db, $tbl) = @_;
+
+ $node->safe_psql($db,
+ qq'CREATE TABLE $tbl AS SELECT g as a, g + 1 as b FROM generate_series(1,10) g(g)'
+ );
+ $node->safe_psql($db, qq'GRANT USAGE ON SCHEMA public TO $user');
+ $node->safe_psql($db, qq'GRANT SELECT ON $tbl TO $user');
+}
+
+done_testing();
+
diff --git a/doc/src/sgml/dblink.sgml b/doc/src/sgml/dblink.sgml
index 81f35986c88..a575d0eb431 100644
--- a/doc/src/sgml/dblink.sgml
+++ b/doc/src/sgml/dblink.sgml
@@ -136,6 +136,76 @@ dblink_connect(text connname, text connstr) returns text
</para>
</refsect1>
+ <refsect1>
+ <title>Foreign Data Wrapper</title>
+
+ <para>
+ A Foreign Data Wrapper can be used as a connection name parameter. The foreign
+ server can be created using CREATE SERVER and CREATE USER MAPPING commands.
+ </para>
+
+ <para>
+ The authentication with the foreign server can be via password on USER
+ MAPPING or using SCRAM pass-through. The
+ <literal>use_scram_passthrough</literal> on CREATE SERVER options controls
+ whether <filename>dblink_fdw</filename> will use the SCRAM pass-through
+ authentication to connect to the foreign server. With SCRAM pass-through
+ authentication, <filename>dblink_fdw</filename> uses SCRAM-hashed secrets
+ instead of plain-text user passwords to connect to the remote server. This
+ avoids storing plain-text user passwords in PostgreSQL system catalogs.
+ </para>
+
+ <para>
+ To use SCRAM pass-through authentication:
+ <itemizedlist>
+ <listitem>
+ <para>
+ The remote server must request the <literal>scram-sha-256</literal>
+ authentication method; otherwise, the connection will fail.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The user mapping password is not used.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The remote server can be of any PostgreSQL version that supports SCRAM.
+ Support for <literal>use_scram_passthrough</literal> is only required on
+ the client side (FDW side).
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The server running <filename>dblink_fdw</filename> and the remote server
+ must have identical SCRAM secrets (encrypted passwords) for the user being
+ used on <filename>dblink_fdw</filename> to authenticate on the foreign
+ server (same salt and iterations, not merely the same password).
+ </para>
+
+ <para>
+ As a corollary, if FDW connections to multiple hosts are to be made, for
+ example for partitioned foreign tables/sharding, then all hosts must have
+ identical SCRAM secrets for the users involved.
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The current session on the PostgreSQL instance that makes the outgoing FDW
+ connections also must also use SCRAM authentication for its incoming client
+ connection. (Hence <quote>pass-through</quote>: SCRAM must be used going in
+ and out.) This is a technical requirement of the SCRAM protocol.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </refsect1>
+
<refsect1>
<title>Notes</title>
@@ -181,8 +251,9 @@ SELECT dblink_connect('myconn', 'dbname=postgres options=-csearch_path=');
(1 row)
-- FOREIGN DATA WRAPPER functionality
--- Note: local connection must require password authentication for this to work properly
--- Otherwise, you will receive the following error from dblink_connect():
+-- Note: local connections that don't use SCRAM pass-through require password
+-- authentication for this to work properly. Otherwise, you will receive
+-- the following error from dblink_connect():
-- ERROR: password is required
-- DETAIL: Non-superuser cannot connect if the server does not request a password.
-- HINT: Target server's authentication method must be changed.
--
2.39.5 (Apple Git-154)
On 24.03.25 21:33, Matheus Alcantara wrote:
I'm a bit confused about the refactoring patch 0001. There are some
details there that don't seem right. For example, you write that the
pfree(rconn) calls are no longer necessary, but AFAICT, it would still
be needed in dblink_get_conn(). Also, there appear to be some possible
behavior changes, or at least it's not fully explained, like
connect_pg_server() doing foreign-server name resolution, which
dblink_get_conn() did not do before.But it's actually not clear to me how the refactoring in 0001
contributes to making the patch 0002 better, since patch 0002 barely
touches the code touched by 0001.How would patch 0002 look without 0001 before it? Which code would need
to be duplicated or what other awkward changes are you trying to avoid?You are right, I think that the refactor was needed on the initial
versions of the patch because it was referencing the UseScramPassthrough
function in multiple places, so the refactor was needed to accomplish the
parameters of the function.Since we now assume that the UseScramPassthrough is already checked on
some parts of the code I agree that this refactor is not required
anymore. Attached v11 without the refactor patch.
Committed.
I cut down the documentation a bit and instead linked to postgres_fdw
for some of the details. I think that's better than having to maintain
that text in two different places.
On Wed, Mar 26, 2025 at 7:41 AM Peter Eisentraut <peter@eisentraut.org> wrote:
On 24.03.25 21:33, Matheus Alcantara wrote:
I'm a bit confused about the refactoring patch 0001. There are some
details there that don't seem right. For example, you write that the
pfree(rconn) calls are no longer necessary, but AFAICT, it would still
be needed in dblink_get_conn(). Also, there appear to be some possible
behavior changes, or at least it's not fully explained, like
connect_pg_server() doing foreign-server name resolution, which
dblink_get_conn() did not do before.But it's actually not clear to me how the refactoring in 0001
contributes to making the patch 0002 better, since patch 0002 barely
touches the code touched by 0001.How would patch 0002 look without 0001 before it? Which code would need
to be duplicated or what other awkward changes are you trying to avoid?You are right, I think that the refactor was needed on the initial
versions of the patch because it was referencing the UseScramPassthrough
function in multiple places, so the refactor was needed to accomplish the
parameters of the function.Since we now assume that the UseScramPassthrough is already checked on
some parts of the code I agree that this refactor is not required
anymore. Attached v11 without the refactor patch.Committed.
I cut down the documentation a bit and instead linked to postgres_fdw
for some of the details. I think that's better than having to maintain
that text in two different places.
Thanks!
--
Matheus Alcantara