Using AF_UNIX sockets always for tests on Windows

Started by Thomas Munroabout 3 years ago12 messages
#1Thomas Munro
thomas.munro@gmail.com
4 attachment(s)

Hi,

Commit f5580882 established that all supported computers have AF_UNIX.
One of the follow-up consequences that was left unfinished is that we
could simplify our test harness code to make it the same on all
platforms. Currently we have hundreds of lines of C and perl to use
secure TCP connections instead for the benefit of defunct Windows
versions. Here's a patch set for that. These patches and some
discussion of them were buried in the recent
clean-up-after-recently-dropped-obsolete-systems thread[1]/messages/by-id/CA+hUKGJ3LHeP9w5Fgzdr4G8AnEtJ=z=p6hGDEm4qYGEUX5B6fQ@mail.gmail.com, and I
didn't want to lose track of them. I think they would need review and
testing from a Windows-based hacker to make progress. The patches
are:

1. Teach mkdtemp() to make a non-world-accessible directory. This is
required to be able to make a socket that other processes can't
connect to, to match the paranoia level used on Unix. This was
written just by reading documentation, because I am not a Windows
user, so I would be grateful for a second opinion and/or testing from
a Windows hacker, which would involve testing with two different
users. The idea is that Windows' mkdir() is completely ignoring the
permissions (we can see in the mingw headers that it literally throws
away the mode argument), so we shouldn't use that, but native
CreateDirectory() when given a pointer to a SECURITY_ATTRIBUTES with
lpSecurityDesciptor set to NULL should only allow the current user to
access the object (directory). Does this really work, and would it be
better to create some more explicit private-keep-out
SECURITY_ATTRIBUTE, and how would that look?

I'm fairly sure that filesystem permissions must be enough to stop
another OS user from connecting, because it's clear from documentation
that AF_UNIX works on Windows by opening the file and reading some
magic "reparse" data from inside it, so if you can't see into the
containing directory, you can't do it. This patch is the one the rest
are standing on, because the tests should match Unix in their level of
security.

2. Always use AF_UNIX for pg_regress. Remove a bunch of
no-longer-needed sspi auth stuff. Remove comments that worried about
signal handler safety (referring here to real Windows signals, not
fake PostgreSQL signals that are a backend-only concept). By my
reading of the Windows documentation and our code, there is no real
concern here, so the remove_temp() stuff should be fine, as I have
explained in a new comment. But I have not tested this signal safety
claim, not being a Windows user. I added an assertion that should
hold if I am right. If you run this on Windows and interrupt
pg_regress with ^C, does it hold?

3. Use AF_UNIX for TAP tests too.

4. In passing, remove our documentation's claim that Linux's
"abstract" AF_UNIX namespace is available on Windows. It does not
work at all, according to all reports (IMHO it seems like an
inherently insecure interface that other OSes would be unlikely to
adopt).

Note that this thread is not about libpq, which differs from Unix by
defaulting to host=localhost rather than AF_UNIX IIRC. That's a
user-facing policy decision I'm not touching; this thread is just
about cleaning up old test infrastructure of interest to hackers.

[1]: /messages/by-id/CA+hUKGJ3LHeP9w5Fgzdr4G8AnEtJ=z=p6hGDEm4qYGEUX5B6fQ@mail.gmail.com

Attachments:

0001-WIP-Make-mkdtemp-more-secure-on-Windows.patchtext/x-patch; charset=US-ASCII; name=0001-WIP-Make-mkdtemp-more-secure-on-Windows.patchDownload
From 62b1cdbdc848f96eef02ed97f14be9c1f4557539 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 7 Sep 2022 07:35:11 +1200
Subject: [PATCH 1/4] WIP: Make mkdtemp() more secure on Windows.

Our POSIX mkdtemp() implementation in src/port/mkdtemp.c code would
create directories with default permissions on Windows.  Fix, using
native Windows API instead of mkdir().
---
 src/port/mkdtemp.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/src/port/mkdtemp.c b/src/port/mkdtemp.c
index 8809957dcd..8116317435 100644
--- a/src/port/mkdtemp.c
+++ b/src/port/mkdtemp.c
@@ -187,8 +187,20 @@ GETTEMP(char *path, int *doopen, int domkdir)
 		}
 		else if (domkdir)
 		{
+#ifdef WIN32
+			SECURITY_ATTRIBUTES sa = {
+				.nLength = sizeof(SECURITY_ATTRIBUTES),
+				.lpSecurityDescriptor = NULL,
+				.bInheritHandle = false
+			};
+
+			if (CreateDirectory(path, &sa))
+				return 1;
+			_dosmaperr(GetLastError());
+#else
 			if (mkdir(path, 0700) >= 0)
 				return 1;
+#endif
 			if (errno != EEXIST)
 				return 0;
 		}
-- 
2.38.1

0002-WIP-Always-use-Unix-domain-sockets-in-pg_regress-on-.patchtext/x-patch; charset=US-ASCII; name=0002-WIP-Always-use-Unix-domain-sockets-in-pg_regress-on-.patchDownload
From 388719a6fbb45896ff87794ed4bfdbc0f0aac329 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Fri, 19 Aug 2022 11:28:38 +1200
Subject: [PATCH 2/4] WIP: Always use Unix-domain sockets in pg_regress on
 Windows.

Since we can now rely on Unix-domain sockets working on supported
Windows versions (10+), we can remove a source of instability and a
difference between Unix and Windows in pg_regress.

Previously, we thought the socket cleanup code was unsafe, so we made
Unix-domain sockets an option with a "use-at-your-own-risk" note.  On
closer inspection, the concerns about signal handlers don't seem to
apply here.  (initdb.c has similar concerns but needs separate
investigation.)

Previously, commit f6dc6dd5 secured temporary installations using TCP/IP
on Windows, while commit be76a6d3 used file system permissions for Unix
sockets on Unix.  Now that our src/port/mkdtemp.c file creates
non-world-accessible directories on Windows, we can just do the same on
Windows.
---
 src/test/regress/pg_regress.c | 274 ++++------------------------------
 1 file changed, 32 insertions(+), 242 deletions(-)

diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index f308da6c50..dc6b4663c0 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -55,6 +55,20 @@ char	   *host_platform = HOST_TUPLE;
 static char *shellprog = SHELLPROG;
 #endif
 
+/*
+ * The name of the environment variable that controls where we put temporary
+ * files, to override the defaut of "/tmp".
+ */
+#ifdef WIN32
+#define TMPDIR "TMP"
+#else
+#define TMPDIR "TMPDIR"
+#endif
+
+#if defined(WIN32) && defined(USE_ASSERT_CHECKING)
+static DWORD main_thread_id;
+#endif
+
 /*
  * On Windows we use -w in diff switches to avoid problems with inconsistent
  * newline representation.  The actual result files will generally have
@@ -286,9 +300,7 @@ stop_postmaster(void)
  * postmaster exit, so it is indeterminate whether the postmaster has yet to
  * unlink the socket and lock file.  Unlink them here so we can proceed to
  * remove the directory.  Ignore errors; leaking a temporary directory is
- * unimportant.  This can run from a signal handler.  The code is not
- * acceptable in a Windows signal handler (see initdb.c:trapsig()), but
- * on Windows, pg_regress does not use Unix sockets by default.
+ * unimportant.  This can run from a signal handler.
  */
 static void
 remove_temp(void)
@@ -305,6 +317,18 @@ remove_temp(void)
 static void
 signal_remove_temp(SIGNAL_ARGS)
 {
+#ifdef WIN32
+	/*
+	 * In general, it would not be acceptable to call remove_temp() in a
+	 * Windows signal handler.  It is safe in this program though, because
+	 * SIGHUP and SIGPIPE don't really exist and SIGTERM is never raised by the
+	 * system, leaving just SIGINT.  SIGINT doesn't interrupt the main
+	 * execution context on Windows, it runs the handler concurrently in
+	 * another thread.
+	 */
+	Assert(GetCurrentThreadId() != main_thread_id);
+#endif
+
 	remove_temp();
 
 	pqsignal(postgres_signal_arg, SIG_DFL);
@@ -327,7 +351,7 @@ static const char *
 make_temp_sockdir(void)
 {
 	char	   *template = psprintf("%s/pg_regress-XXXXXX",
-									getenv("TMPDIR") ? getenv("TMPDIR") : "/tmp");
+									getenv(TMPDIR) ? getenv(TMPDIR) : "/tmp");
 
 	temp_sockdir = mkdtemp(template);
 	if (temp_sockdir == NULL)
@@ -344,6 +368,10 @@ make_temp_sockdir(void)
 	/* Remove the directory during clean exit. */
 	atexit(remove_temp);
 
+#if defined(WIN32) && defined(USE_ASSERT_CHECKING)
+	main_thread_id = GetCurrentThreadId();
+#endif
+
 	/*
 	 * Remove the directory before dying to the usual signals.  Omit SIGQUIT,
 	 * preserving it as a quick, untidy exit.
@@ -754,211 +782,6 @@ initialize_environment(void)
 	load_resultmap();
 }
 
-#ifdef ENABLE_SSPI
-
-/* support for config_sspi_auth() */
-static const char *
-fmtHba(const char *raw)
-{
-	static char *ret;
-	const char *rp;
-	char	   *wp;
-
-	wp = ret = pg_realloc(ret, 3 + strlen(raw) * 2);
-
-	*wp++ = '"';
-	for (rp = raw; *rp; rp++)
-	{
-		if (*rp == '"')
-			*wp++ = '"';
-		*wp++ = *rp;
-	}
-	*wp++ = '"';
-	*wp++ = '\0';
-
-	return ret;
-}
-
-/*
- * Get account and domain/realm names for the current user.  This is based on
- * pg_SSPI_recvauth().  The returned strings use static storage.
- */
-static void
-current_windows_user(const char **acct, const char **dom)
-{
-	static char accountname[MAXPGPATH];
-	static char domainname[MAXPGPATH];
-	HANDLE		token;
-	TOKEN_USER *tokenuser;
-	DWORD		retlen;
-	DWORD		accountnamesize = sizeof(accountname);
-	DWORD		domainnamesize = sizeof(domainname);
-	SID_NAME_USE accountnameuse;
-
-	if (!OpenProcessToken(GetCurrentProcess(), TOKEN_READ, &token))
-	{
-		fprintf(stderr,
-				_("%s: could not open process token: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-
-	if (!GetTokenInformation(token, TokenUser, NULL, 0, &retlen) && GetLastError() != 122)
-	{
-		fprintf(stderr,
-				_("%s: could not get token information buffer size: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-	tokenuser = pg_malloc(retlen);
-	if (!GetTokenInformation(token, TokenUser, tokenuser, retlen, &retlen))
-	{
-		fprintf(stderr,
-				_("%s: could not get token information: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-
-	if (!LookupAccountSid(NULL, tokenuser->User.Sid, accountname, &accountnamesize,
-						  domainname, &domainnamesize, &accountnameuse))
-	{
-		fprintf(stderr,
-				_("%s: could not look up account SID: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-
-	free(tokenuser);
-
-	*acct = accountname;
-	*dom = domainname;
-}
-
-/*
- * Rewrite pg_hba.conf and pg_ident.conf to use SSPI authentication.  Permit
- * the current OS user to authenticate as the bootstrap superuser and as any
- * user named in a --create-role option.
- *
- * In --config-auth mode, the --user switch can be used to specify the
- * bootstrap superuser's name, otherwise we assume it is the default.
- */
-static void
-config_sspi_auth(const char *pgdata, const char *superuser_name)
-{
-	const char *accountname,
-			   *domainname;
-	char	   *errstr;
-	bool		have_ipv6;
-	char		fname[MAXPGPATH];
-	int			res;
-	FILE	   *hba,
-			   *ident;
-	_stringlist *sl;
-
-	/* Find out the name of the current OS user */
-	current_windows_user(&accountname, &domainname);
-
-	/* Determine the bootstrap superuser's name */
-	if (superuser_name == NULL)
-	{
-		/*
-		 * Compute the default superuser name the same way initdb does.
-		 *
-		 * It's possible that this result always matches "accountname", the
-		 * value SSPI authentication discovers.  But the underlying system
-		 * functions do not clearly guarantee that.
-		 */
-		superuser_name = get_user_name(&errstr);
-		if (superuser_name == NULL)
-		{
-			fprintf(stderr, "%s: %s\n", progname, errstr);
-			exit(2);
-		}
-	}
-
-	/*
-	 * Like initdb.c:setup_config(), determine whether the platform recognizes
-	 * ::1 (IPv6 loopback) as a numeric host address string.
-	 */
-	{
-		struct addrinfo *gai_result;
-		struct addrinfo hints;
-		WSADATA		wsaData;
-
-		hints.ai_flags = AI_NUMERICHOST;
-		hints.ai_family = AF_UNSPEC;
-		hints.ai_socktype = 0;
-		hints.ai_protocol = 0;
-		hints.ai_addrlen = 0;
-		hints.ai_canonname = NULL;
-		hints.ai_addr = NULL;
-		hints.ai_next = NULL;
-
-		have_ipv6 = (WSAStartup(MAKEWORD(2, 2), &wsaData) == 0 &&
-					 getaddrinfo("::1", NULL, &hints, &gai_result) == 0);
-	}
-
-	/* Check a Write outcome and report any error. */
-#define CW(cond)	\
-	do { \
-		if (!(cond)) \
-		{ \
-			fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"), \
-					progname, fname, strerror(errno)); \
-			exit(2); \
-		} \
-	} while (0)
-
-	res = snprintf(fname, sizeof(fname), "%s/pg_hba.conf", pgdata);
-	if (res < 0 || res >= sizeof(fname))
-	{
-		/*
-		 * Truncating this name is a fatal error, because we must not fail to
-		 * overwrite an original trust-authentication pg_hba.conf.
-		 */
-		fprintf(stderr, _("%s: directory name too long\n"), progname);
-		exit(2);
-	}
-	hba = fopen(fname, "w");
-	if (hba == NULL)
-	{
-		fprintf(stderr, _("%s: could not open file \"%s\" for writing: %s\n"),
-				progname, fname, strerror(errno));
-		exit(2);
-	}
-	CW(fputs("# Configuration written by config_sspi_auth()\n", hba) >= 0);
-	CW(fputs("host all all 127.0.0.1/32  sspi include_realm=1 map=regress\n",
-			 hba) >= 0);
-	if (have_ipv6)
-		CW(fputs("host all all ::1/128  sspi include_realm=1 map=regress\n",
-				 hba) >= 0);
-	CW(fclose(hba) == 0);
-
-	snprintf(fname, sizeof(fname), "%s/pg_ident.conf", pgdata);
-	ident = fopen(fname, "w");
-	if (ident == NULL)
-	{
-		fprintf(stderr, _("%s: could not open file \"%s\" for writing: %s\n"),
-				progname, fname, strerror(errno));
-		exit(2);
-	}
-	CW(fputs("# Configuration written by config_sspi_auth()\n", ident) >= 0);
-
-	/*
-	 * Double-quote for the benefit of account names containing whitespace or
-	 * '#'.  Windows forbids the double-quote character itself, so don't
-	 * bother escaping embedded double-quote characters.
-	 */
-	CW(fprintf(ident, "regress  \"%s@%s\"  %s\n",
-			   accountname, domainname, fmtHba(superuser_name)) >= 0);
-	for (sl = extraroles; sl; sl = sl->next)
-		CW(fprintf(ident, "regress  \"%s@%s\"  %s\n",
-				   accountname, domainname, fmtHba(sl->str)) >= 0);
-	CW(fclose(ident) == 0);
-}
-
-#endif							/* ENABLE_SSPI */
-
 /*
  * psql_start_command, psql_add_command, psql_end_command
  *
@@ -2039,7 +1862,6 @@ regression_main(int argc, char *argv[],
 		{NULL, 0, NULL, 0}
 	};
 
-	bool		use_unix_sockets;
 	_stringlist *sl;
 	int			c;
 	int			i;
@@ -2055,20 +1877,6 @@ regression_main(int argc, char *argv[],
 
 	atexit(stop_postmaster);
 
-#if defined(WIN32)
-
-	/*
-	 * We don't use Unix-domain sockets on Windows by default (see comment at
-	 * remove_temp() for a reason).  Override at your own risk.
-	 */
-	use_unix_sockets = getenv("PG_TEST_USE_UNIX_SOCKETS") ? true : false;
-#else
-	use_unix_sockets = true;
-#endif
-
-	if (!use_unix_sockets)
-		hostname = "localhost";
-
 	/*
 	 * We call the initialization function here because that way we can set
 	 * default parameters and let them be overwritten by the commandline.
@@ -2194,13 +2002,7 @@ regression_main(int argc, char *argv[],
 	}
 
 	if (config_auth_datadir)
-	{
-#ifdef ENABLE_SSPI
-		if (!use_unix_sockets)
-			config_sspi_auth(config_auth_datadir, user);
-#endif
 		exit(0);
-	}
 
 	if (temp_instance && !port_specified_by_user)
 
@@ -2319,18 +2121,6 @@ regression_main(int argc, char *argv[],
 
 		fclose(pg_conf);
 
-#ifdef ENABLE_SSPI
-		if (!use_unix_sockets)
-		{
-			/*
-			 * Since we successfully used the same buffer for the much-longer
-			 * "initdb" command, this can't truncate.
-			 */
-			snprintf(buf, sizeof(buf), "%s/data", temp_instance);
-			config_sspi_auth(buf, NULL);
-		}
-#endif
-
 		/*
 		 * Check if there is a postmaster running already.
 		 */
-- 
2.38.1

0003-WIP-Stop-using-TCP-in-TAP-tests-on-Windows.patchtext/x-patch; charset=US-ASCII; name=0003-WIP-Stop-using-TCP-in-TAP-tests-on-Windows.patchDownload
From c4429b2ad41850b8e3a360d1093be8a57014a156 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Fri, 19 Aug 2022 12:00:07 +1200
Subject: [PATCH 3/4] WIP: Stop using TCP in TAP tests on Windows.

Since Unix-domain sockets are available on our minimum target Windows
versions (10+), we can remove a source of instability and a point of
variation between Unix and Windows.
---
 .cirrus.yml                                   |  3 -
 src/bin/pg_ctl/t/001_start_stop.pl            | 13 +--
 src/test/authentication/t/001_password.pl     |  6 --
 src/test/authentication/t/002_saslprep.pl     |  7 --
 src/test/authentication/t/003_peer.pl         |  5 --
 .../authentication/t/004_file_inclusion.pl    |  5 --
 src/test/perl/PostgreSQL/Test/Cluster.pm      | 90 ++++---------------
 src/test/perl/PostgreSQL/Test/Utils.pm        |  9 +-
 8 files changed, 19 insertions(+), 119 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index f31923333e..e4aed541d8 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -502,9 +502,6 @@ WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE
     # git's tar doesn't deal with drive letters, see
     # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net
     TAR: "c:/windows/system32/tar.exe"
-    # Avoids port conflicts between concurrent tap test runs
-    PG_TEST_USE_UNIX_SOCKETS: 1
-    PG_REGRESS_SOCK_DIR: "c:/cirrus/"
 
   sysinfo_script: |
     chcp
diff --git a/src/bin/pg_ctl/t/001_start_stop.pl b/src/bin/pg_ctl/t/001_start_stop.pl
index fdffd76d99..23f942a440 100644
--- a/src/bin/pg_ctl/t/001_start_stop.pl
+++ b/src/bin/pg_ctl/t/001_start_stop.pl
@@ -29,16 +29,9 @@ print $conf "port = $node_port\n";
 print $conf PostgreSQL::Test::Utils::slurp_file($ENV{TEMP_CONFIG})
   if defined $ENV{TEMP_CONFIG};
 
-if ($use_unix_sockets)
-{
-	print $conf "listen_addresses = ''\n";
-	$tempdir_short =~ s!\\!/!g if $PostgreSQL::Test::Utils::windows_os;
-	print $conf "unix_socket_directories = '$tempdir_short'\n";
-}
-else
-{
-	print $conf "listen_addresses = '127.0.0.1'\n";
-}
+print $conf "listen_addresses = ''\n";
+$tempdir_short =~ s!\\!/!g if $PostgreSQL::Test::Utils::windows_os;
+print $conf "unix_socket_directories = '$tempdir_short'\n";
 close $conf;
 my $ctlcmd = [
 	'pg_ctl', 'start', '-D', "$tempdir/data", '-l',
diff --git a/src/test/authentication/t/001_password.pl b/src/test/authentication/t/001_password.pl
index 42d3d4c79b..45d105717a 100644
--- a/src/test/authentication/t/001_password.pl
+++ b/src/test/authentication/t/001_password.pl
@@ -6,18 +6,12 @@
 # - Plain
 # - MD5-encrypted
 # - SCRAM-encrypted
-# This test can only run with Unix-domain sockets.
 
 use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
 use Test::More;
-if (!$use_unix_sockets)
-{
-	plan skip_all =>
-	  "authentication tests cannot run without Unix-domain sockets";
-}
 
 # Delete pg_hba.conf from the given node, add a new entry to it
 # and then execute a reload to refresh it.
diff --git a/src/test/authentication/t/002_saslprep.pl b/src/test/authentication/t/002_saslprep.pl
index 5e87e21ee9..23849632c2 100644
--- a/src/test/authentication/t/002_saslprep.pl
+++ b/src/test/authentication/t/002_saslprep.pl
@@ -2,19 +2,12 @@
 # Copyright (c) 2021-2022, PostgreSQL Global Development Group
 
 # Test password normalization in SCRAM.
-#
-# This test can only run with Unix-domain sockets.
 
 use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
 use Test::More;
-if (!$use_unix_sockets)
-{
-	plan skip_all =>
-	  "authentication tests cannot run without Unix-domain sockets";
-}
 
 # Delete pg_hba.conf from the given node, add a new entry to it
 # and then execute a reload to refresh it.
diff --git a/src/test/authentication/t/003_peer.pl b/src/test/authentication/t/003_peer.pl
index 26c34d05d3..d9d8616e30 100644
--- a/src/test/authentication/t/003_peer.pl
+++ b/src/test/authentication/t/003_peer.pl
@@ -10,11 +10,6 @@ use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
 use Test::More;
-if (!$use_unix_sockets)
-{
-	plan skip_all =>
-	  "authentication tests cannot run without Unix-domain sockets";
-}
 
 # Delete pg_hba.conf from the given node, add a new entry to it
 # and then execute a reload to refresh it.
diff --git a/src/test/authentication/t/004_file_inclusion.pl b/src/test/authentication/t/004_file_inclusion.pl
index c420f3ebca..db4fcd962b 100644
--- a/src/test/authentication/t/004_file_inclusion.pl
+++ b/src/test/authentication/t/004_file_inclusion.pl
@@ -11,11 +11,6 @@ use PostgreSQL::Test::Utils;
 use File::Basename qw(basename);
 use Test::More;
 use Data::Dumper;
-if (!$use_unix_sockets)
-{
-	plan skip_all =>
-	  "authentication tests cannot run without Unix-domain sockets";
-}
 
 # Stores the number of lines created for each file.  hba_rule and ident_rule
 # are used to respectively track pg_hba_file_rules.rule_number and
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index 7411188dc8..fdaed41f7b 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -116,7 +116,7 @@ use PostgreSQL::Test::Utils ();
 use Time::HiRes qw(usleep);
 use Scalar::Util qw(blessed);
 
-our ($use_tcp, $test_localhost, $test_pghost, $last_host_assigned,
+our ($test_pghost,
 	$last_port_assigned, @all_nodes, $died, $portdir);
 
 # the minimum version we believe to be compatible with this package without
@@ -131,21 +131,11 @@ INIT
 
 	# Set PGHOST for backward compatibility.  This doesn't work for own_host
 	# nodes, so prefer to not rely on this when writing new tests.
-	$use_tcp            = !$PostgreSQL::Test::Utils::use_unix_sockets;
-	$test_localhost     = "127.0.0.1";
-	$last_host_assigned = 1;
-	if ($use_tcp)
-	{
-		$test_pghost = $test_localhost;
-	}
-	else
-	{
-		# On windows, replace windows-style \ path separators with / when
-		# putting socket directories either in postgresql.conf or libpq
-		# connection strings, otherwise they are interpreted as escapes.
-		$test_pghost = PostgreSQL::Test::Utils::tempdir_short;
-		$test_pghost =~ s!\\!/!g if $PostgreSQL::Test::Utils::windows_os;
-	}
+	# On windows, replace windows-style \ path separators with / when
+	# putting socket directories either in postgresql.conf or libpq
+	# connection strings, otherwise they are interpreted as escapes.
+	$test_pghost = PostgreSQL::Test::Utils::tempdir_short;
+	$test_pghost =~ s!\\!/!g if $PostgreSQL::Test::Utils::windows_os;
 	$ENV{PGHOST}     = $test_pghost;
 	$ENV{PGDATABASE} = 'postgres';
 
@@ -470,12 +460,6 @@ sub set_replication_conf
 	open my $hba, '>>', "$pgdata/pg_hba.conf";
 	print $hba
 	  "\n# Allow replication (set up by PostgreSQL::Test::Cluster.pm)\n";
-	if ($PostgreSQL::Test::Utils::windows_os
-		&& !$PostgreSQL::Test::Utils::use_unix_sockets)
-	{
-		print $hba
-		  "host replication all $test_localhost/32 sspi include_realm=1 map=regress\n";
-	}
 	close $hba;
 	return;
 }
@@ -568,16 +552,8 @@ sub init
 	}
 
 	print $conf "port = $port\n";
-	if ($use_tcp)
-	{
-		print $conf "unix_socket_directories = ''\n";
-		print $conf "listen_addresses = '$host'\n";
-	}
-	else
-	{
-		print $conf "unix_socket_directories = '$host'\n";
-		print $conf "listen_addresses = ''\n";
-	}
+	print $conf "unix_socket_directories = '$host'\n";
+	print $conf "listen_addresses = ''\n";
 	close $conf;
 
 	chmod($self->group_access ? 0640 : 0600, "$pgdata/postgresql.conf")
@@ -796,15 +772,8 @@ sub init_from_backup
 		qq(
 port = $port
 ));
-	if ($use_tcp)
-	{
-		$self->append_conf('postgresql.conf', "listen_addresses = '$host'");
-	}
-	else
-	{
-		$self->append_conf('postgresql.conf',
-			"unix_socket_directories = '$host'");
-	}
+	$self->append_conf('postgresql.conf',
+		"unix_socket_directories = '$host'");
 	$self->enable_streaming($root_node) if $params{has_streaming};
 	$self->enable_restoring($root_node, $params{standby})
 	  if $params{has_restoring};
@@ -1270,9 +1239,7 @@ sub new
 	else
 	{
 		# When selecting a port, we look for an unassigned TCP port number,
-		# even if we intend to use only Unix-domain sockets.  This is clearly
-		# necessary on $use_tcp (Windows) configurations, and it seems like a
-		# good idea on Unixen as well.
+		# even if we intend to use only Unix-domain sockets.
 		$port = get_free_port();
 	}
 
@@ -1280,17 +1247,8 @@ sub new
 	my $host = $test_pghost;
 	if ($params{own_host})
 	{
-		if ($use_tcp)
-		{
-			$last_host_assigned++;
-			$last_host_assigned > 254 and BAIL_OUT("too many own_host nodes");
-			$host = '127.0.0.' . $last_host_assigned;
-		}
-		else
-		{
-			$host = "$test_pghost/$name"; # Assume $name =~ /^[-_a-zA-Z0-9]+$/
-			mkdir $host;
-		}
+		$host = "$test_pghost/$name"; # Assume $name =~ /^[-_a-zA-Z0-9]+$/
+		mkdir $host;
 	}
 
 	my $testname = basename($0);
@@ -1526,29 +1484,11 @@ sub get_free_port
 		}
 
 		# Check to see if anything else is listening on this TCP port.
-		# Seek a port available for all possible listen_addresses values,
-		# so callers can harness this port for the widest range of purposes.
-		# The 0.0.0.0 test achieves that for MSYS, which automatically sets
-		# SO_EXCLUSIVEADDRUSE.  Testing 0.0.0.0 is insufficient for Windows
-		# native Perl (https://stackoverflow.com/a/14388707), so we also
-		# have to test individual addresses.  Doing that for 127.0.0/24
-		# addresses other than 127.0.0.1 might fail with EADDRNOTAVAIL on
-		# non-Linux, non-Windows kernels.
-		#
-		# Thus, 0.0.0.0 and individual 127.0.0/24 addresses are tested
-		# only on Windows and only when TCP usage is requested.
 		if ($found == 1)
 		{
-			foreach my $addr (qw(127.0.0.1),
-				($use_tcp && $PostgreSQL::Test::Utils::windows_os)
-				  ? qw(127.0.0.2 127.0.0.3 0.0.0.0)
-				  : ())
+			if (!can_bind(qw(127.0.0.1), $port))
 			{
-				if (!can_bind($addr, $port))
-				{
-					$found = 0;
-					last;
-				}
+				$found = 0;
 			}
 			$found = _reserve_port($port) if $found;
 		}
diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm
index b139190cc8..631d8bd362 100644
--- a/src/test/perl/PostgreSQL/Test/Utils.pm
+++ b/src/test/perl/PostgreSQL/Test/Utils.pm
@@ -88,10 +88,9 @@ our @EXPORT = qw(
 
   $windows_os
   $is_msys2
-  $use_unix_sockets
 );
 
-our ($windows_os, $is_msys2, $use_unix_sockets, $timeout_default,
+our ($windows_os, $is_msys2, $timeout_default,
 	$tmp_check, $log_path, $test_logfile);
 
 BEGIN
@@ -153,12 +152,6 @@ BEGIN
 		Win32API::File->import(qw(createFile OsFHandleOpen CloseHandle));
 	}
 
-	# Specifies whether to use Unix sockets for test setups.  On
-	# Windows we don't use them by default since it's not universally
-	# supported, but it can be overridden if desired.
-	$use_unix_sockets =
-	  (!$windows_os || defined $ENV{PG_TEST_USE_UNIX_SOCKETS});
-
 	$timeout_default = $ENV{PG_TEST_TIMEOUT_DEFAULT};
 	$timeout_default = 180
 	  if not defined $timeout_default or $timeout_default eq '';
-- 
2.38.1

0004-Doc-Abstract-AF_UNIX-sockets-don-t-work-on-Windows-a.patchtext/x-patch; charset=US-ASCII; name=0004-Doc-Abstract-AF_UNIX-sockets-don-t-work-on-Windows-a.patchDownload
From e645cc2ab80450b18227931082beefc2bb8ffb0a Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Mon, 15 Aug 2022 10:43:13 +1200
Subject: [PATCH 4/4] Doc: Abstract AF_UNIX sockets don't work on Windows after
 all.

An early release of AF_UNIX in Windows might have supported Linux-style
"abstract" Unix sockets with a system-wide namespace, but they do not
seem to work in current Windows versions and there is no mention of any
of this in the Winsock documentation.  Remove the claim that it works
from our documentation.

Back-patch to 14, where commit c9f0624b landed.

Discussion: https://postgr.es/m/20220813223646.oh2dkjrkj7jn7dpe%40awork3.anarazel.de
---
 doc/src/sgml/config.sgml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 39d1c89e33..55286375dc 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -759,7 +759,7 @@ include_dir 'conf.d'
        <para>
         A value that starts with <literal>@</literal> specifies that a
         Unix-domain socket in the abstract namespace should be created
-        (currently supported on Linux and Windows).  In that case, this value
+        (currently supported on Linux).  In that case, this value
         does not specify a <quote>directory</quote> but a prefix from which
         the actual socket name is computed in the same manner as for the
         file-system namespace.  While the abstract socket name prefix can be
-- 
2.38.1

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Thomas Munro (#1)
Re: Using AF_UNIX sockets always for tests on Windows

Thomas Munro <thomas.munro@gmail.com> writes:

Commit f5580882 established that all supported computers have AF_UNIX.
One of the follow-up consequences that was left unfinished is that we
could simplify our test harness code to make it the same on all
platforms. Currently we have hundreds of lines of C and perl to use
secure TCP connections instead for the benefit of defunct Windows
versions. Here's a patch set for that.

If we remove that, won't we have a whole lot of code that's not
tested at all on any platform, ie all the TCP-socket code?

regards, tom lane

#3Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#2)
Re: Using AF_UNIX sockets always for tests on Windows

Hi,

On 2022-12-01 20:30:36 -0500, Tom Lane wrote:

Thomas Munro <thomas.munro@gmail.com> writes:

Commit f5580882 established that all supported computers have AF_UNIX.
One of the follow-up consequences that was left unfinished is that we
could simplify our test harness code to make it the same on all
platforms. Currently we have hundreds of lines of C and perl to use
secure TCP connections instead for the benefit of defunct Windows
versions. Here's a patch set for that.

If we remove that, won't we have a whole lot of code that's not
tested at all on any platform, ie all the TCP-socket code?

There's some coverage via the auth and ssl tests. But I agree it's an
issue. But to me the fix for that seems to be to add a dedicated test for
that, rather than relying on windows to test our socket code - that's quite a
few separate code paths from the tcp support of other platforms.

Greetings,

Andres Freund

#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#3)
Re: Using AF_UNIX sockets always for tests on Windows

Andres Freund <andres@anarazel.de> writes:

On 2022-12-01 20:30:36 -0500, Tom Lane wrote:

If we remove that, won't we have a whole lot of code that's not
tested at all on any platform, ie all the TCP-socket code?

There's some coverage via the auth and ssl tests. But I agree it's an
issue. But to me the fix for that seems to be to add a dedicated test for
that, rather than relying on windows to test our socket code - that's quite a
few separate code paths from the tcp support of other platforms.

IMO that's not the best way forward, because you'll always have
nagging questions about whether a single-purpose test covers
everything that needs coverage. I think the best place to be in
would be to be able to run the whole test suite using either TCP or
UNIX sockets, on any platform (with stuff like the SSL test
overriding the choice as needed). I doubt ripping out the existing
Windows-only support for TCP-based testing is a good step in that
direction.

regards, tom lane

#5Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#4)
Re: Using AF_UNIX sockets always for tests on Windows

Hi,

On 2022-12-01 20:56:18 -0500, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2022-12-01 20:30:36 -0500, Tom Lane wrote:

If we remove that, won't we have a whole lot of code that's not
tested at all on any platform, ie all the TCP-socket code?

There's some coverage via the auth and ssl tests. But I agree it's an
issue. But to me the fix for that seems to be to add a dedicated test for
that, rather than relying on windows to test our socket code - that's quite a
few separate code paths from the tcp support of other platforms.

IMO that's not the best way forward, because you'll always have
nagging questions about whether a single-purpose test covers
everything that needs coverage.

Still seems better than not having any coverage in our development
environments...

I think the best place to be in would be to be able to run the whole test
suite using either TCP or UNIX sockets, on any platform (with stuff like the
SSL test overriding the choice as needed).

I agree that that's useful. But it seems somewhat independent from the
majority of the proposed changes. To be able to test force-tcp-everywhere we
don't need e.g. code for setting sspi auth in pg_regress etc - it's afaik
just needed so there's a secure way of running tests at all on windows.

I think 0003 should be "trimmed" to only change the default for
$use_unix_sockets on windows and to remove PG_TEST_USE_UNIX_SOCKETS. Whoever
wants to, can then add a new environment variable to force tap tests to use
tcp.

Greetings,

Andres Freund

#6Andrew Dunstan
andrew@dunslane.net
In reply to: Andres Freund (#5)
2 attachment(s)
Re: Using AF_UNIX sockets always for tests on Windows

On 2022-12-01 Th 21:10, Andres Freund wrote:

Hi,

On 2022-12-01 20:56:18 -0500, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2022-12-01 20:30:36 -0500, Tom Lane wrote:

If we remove that, won't we have a whole lot of code that's not
tested at all on any platform, ie all the TCP-socket code?

There's some coverage via the auth and ssl tests. But I agree it's an
issue. But to me the fix for that seems to be to add a dedicated test for
that, rather than relying on windows to test our socket code - that's quite a
few separate code paths from the tcp support of other platforms.

IMO that's not the best way forward, because you'll always have
nagging questions about whether a single-purpose test covers
everything that needs coverage.

Still seems better than not having any coverage in our development
environments...

I think the best place to be in would be to be able to run the whole test
suite using either TCP or UNIX sockets, on any platform (with stuff like the
SSL test overriding the choice as needed).

I agree that that's useful. But it seems somewhat independent from the
majority of the proposed changes. To be able to test force-tcp-everywhere we
don't need e.g. code for setting sspi auth in pg_regress etc - it's afaik
just needed so there's a secure way of running tests at all on windows.

I think 0003 should be "trimmed" to only change the default for
$use_unix_sockets on windows and to remove PG_TEST_USE_UNIX_SOCKETS. Whoever
wants to, can then add a new environment variable to force tap tests to use
tcp.

Not sure if it's useful here, but a few months ago I prepared patches to
remove the config-auth option of pg_regress, which struck me as more
than odd, and replace it with a perl module. I didn't get around to
finishing them, but the patches as of then are attached.

I agree that having some switch that says "run everything with TCP" or
"run (almost) everything with Unix sockets" would be good.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Attachments:

0001-Do-config_auth-in-perl-code-for-TAP-tests-and-vcregr.patchtext/x-patch; charset=UTF-8; name=0001-Do-config_auth-in-perl-code-for-TAP-tests-and-vcregr.patchDownload
From 5c7fad30aa2db2d47d0cc3869f132ca9d14a8814 Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <andrew@dunslane.net>
Date: Tue, 26 Apr 2022 10:49:26 -0400
Subject: [PATCH 1/2] Do config_auth in perl code for TAP tests and
 vcregress.pl.

That means there is no longer any need to call pg_regress --config-auth
to set up Windows authentication. Instead a simple perl function call
does the work.
---
 contrib/basebackup_to_shell/t/001_basic.pl   |   2 +-
 src/bin/pg_basebackup/t/010_pg_basebackup.pl |   2 +-
 src/bin/pg_ctl/t/001_start_stop.pl           |   4 +-
 src/bin/pg_dump/t/010_dump_connstr.pl        |  16 ++-
 src/bin/pg_rewind/t/RewindTest.pm            |   2 +-
 src/test/perl/PostgreSQL/Test/Cluster.pm     |   7 +-
 src/test/perl/PostgreSQL/Test/ConfigAuth.pm  | 115 +++++++++++++++++++
 src/test/recovery/t/001_stream_rep.pl        |   2 +-
 src/tools/msvc/vcregress.pl                  |   8 +-
 9 files changed, 135 insertions(+), 23 deletions(-)
 create mode 100644 src/test/perl/PostgreSQL/Test/ConfigAuth.pm

diff --git a/contrib/basebackup_to_shell/t/001_basic.pl b/contrib/basebackup_to_shell/t/001_basic.pl
index 350d42079a..b5de23b48f 100644
--- a/contrib/basebackup_to_shell/t/001_basic.pl
+++ b/contrib/basebackup_to_shell/t/001_basic.pl
@@ -21,7 +21,7 @@ my $node = PostgreSQL::Test::Cluster->new('primary');
 # Make sure pg_hba.conf is set up to allow connections from backupuser.
 # This is only needed on Windows machines that don't use UNIX sockets.
 $node->init('allows_streaming' => 1,
-			'auth_extra' => [ '--create-role', 'backupuser' ]);
+			'auth_extra' => [ extra_roles => 'backupuser' ]);
 
 $node->append_conf('postgresql.conf',
 				   "shared_preload_libraries = 'basebackup_to_shell'");
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 056fcf3597..a0f4fd7b92 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -29,7 +29,7 @@ umask(0077);
 
 # Initialize node without replication settings
 $node->init(extra => ['--data-checksums'],
-			auth_extra => [ '--create-role', 'backupuser' ]);
+			auth_extra => [ extra_roles => 'backupuser' ]);
 $node->start;
 my $pgdata = $node->data_dir;
 
diff --git a/src/bin/pg_ctl/t/001_start_stop.pl b/src/bin/pg_ctl/t/001_start_stop.pl
index fdffd76d99..c92b06ad85 100644
--- a/src/bin/pg_ctl/t/001_start_stop.pl
+++ b/src/bin/pg_ctl/t/001_start_stop.pl
@@ -4,6 +4,7 @@
 use strict;
 use warnings;
 
+use PostgreSQL::Test::ConfigAuth;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
 use Test::More;
@@ -20,8 +21,7 @@ command_exit_is([ 'pg_ctl', 'start', '-D', "$tempdir/nonexistent" ],
 
 command_ok([ 'pg_ctl', 'initdb', '-D', "$tempdir/data", '-o', '-N' ],
 	'pg_ctl initdb');
-command_ok([ $ENV{PG_REGRESS}, '--config-auth', "$tempdir/data" ],
-	'configure authentication');
+config_auth("$tempdir/data");
 my $node_port = PostgreSQL::Test::Cluster::get_free_port();
 open my $conf, '>>', "$tempdir/data/postgresql.conf";
 print $conf "fsync = off\n";
diff --git a/src/bin/pg_dump/t/010_dump_connstr.pl b/src/bin/pg_dump/t/010_dump_connstr.pl
index 7a745ade0f..576dd18f3a 100644
--- a/src/bin/pg_dump/t/010_dump_connstr.pl
+++ b/src/bin/pg_dump/t/010_dump_connstr.pl
@@ -4,6 +4,7 @@
 use strict;
 use warnings;
 
+use PostgreSQL::Test::ConfigAuth;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
 use Test::More;
@@ -52,13 +53,10 @@ $node->init(extra =>
 	  [ '-U', $src_bootstrap_super, '--locale=C', '--encoding=LATIN1' ]);
 
 # prep pg_hba.conf and pg_ident.conf
-$node->run_log(
-	[
-		$ENV{PG_REGRESS},     '--config-auth',
-		$node->data_dir,      '--user',
-		$src_bootstrap_super, '--create-role',
-		"$username1,$username2,$username3,$username4"
-	]);
+config_auth(
+	$node->data_dir,
+	user => $src_bootstrap_super,
+	extra_roles => "$username1,$username2,$username3,$username4");
 $node->start;
 
 my $backupdir = $node->backup_dir;
@@ -182,7 +180,7 @@ $envar_node->init(
 	extra =>
 	  [ '-U', $dst_bootstrap_super, '--locale=C', '--encoding=LATIN1' ],
 	auth_extra =>
-	  [ '--user', $dst_bootstrap_super, '--create-role', $restore_super ]);
+	  [ user => $dst_bootstrap_super, extra_roles => $restore_super ]);
 $envar_node->start;
 
 # make superuser for restore
@@ -209,7 +207,7 @@ $cmdline_node->init(
 	extra =>
 	  [ '-U', $dst_bootstrap_super, '--locale=C', '--encoding=LATIN1' ],
 	auth_extra =>
-	  [ '--user', $dst_bootstrap_super, '--create-role', $restore_super ]);
+	  [ user => $dst_bootstrap_super, extra_roles => $restore_super ]);
 $cmdline_node->start;
 $cmdline_node->run_log(
 	[ 'createuser', '-U', $dst_bootstrap_super, '-s', $restore_super ]);
diff --git a/src/bin/pg_rewind/t/RewindTest.pm b/src/bin/pg_rewind/t/RewindTest.pm
index 8fd1f4b9de..d4e146324e 100644
--- a/src/bin/pg_rewind/t/RewindTest.pm
+++ b/src/bin/pg_rewind/t/RewindTest.pm
@@ -123,7 +123,7 @@ sub setup_cluster
 	$node_primary->init(
 		allows_streaming => 1,
 		extra            => $extra,
-		auth_extra       => [ '--create-role', 'rewind_user' ]);
+		auth_extra       => [ extra_roles => 'rewind_user' ]);
 
 	# Set wal_keep_size to prevent WAL segment recycling after enforced
 	# checkpoints in the tests.
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index 9a2ada0a10..c9c53d39e1 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -104,6 +104,7 @@ use PostgreSQL::Version;
 use PostgreSQL::Test::RecursiveCopy;
 use Socket;
 use Test::More;
+use PostgreSQL::Test::ConfigAuth;
 use PostgreSQL::Test::Utils ();
 use Time::HiRes qw(usleep);
 use Scalar::Util qw(blessed);
@@ -428,8 +429,7 @@ Initialize a new cluster for testing.
 Authentication is set up so that only the current OS user can access the
 cluster. On Unix, we use Unix domain socket connections, with the socket in
 a directory that's only accessible to the current user to ensure that.
-On Windows, we use SSPI authentication to ensure the same (by pg_regress
---config-auth).
+On Windows, we use SSPI authentication to ensure the same (via config_auth()).
 
 WAL archiving can be enabled on this node by passing the keyword parameter
 has_archiving => 1. This is disabled by default.
@@ -461,8 +461,7 @@ sub init
 
 	PostgreSQL::Test::Utils::system_or_bail('initdb', '-D', $pgdata, '-A', 'trust', '-N',
 		@{ $params{extra} });
-	PostgreSQL::Test::Utils::system_or_bail($ENV{PG_REGRESS}, '--config-auth', $pgdata,
-		@{ $params{auth_extra} });
+	config_auth($pgdata, @{ $params{auth_extra} });
 
 	open my $conf, '>>', "$pgdata/postgresql.conf";
 	print $conf "\n# Added by PostgreSQL::Test::Cluster.pm\n";
diff --git a/src/test/perl/PostgreSQL/Test/ConfigAuth.pm b/src/test/perl/PostgreSQL/Test/ConfigAuth.pm
new file mode 100644
index 0000000000..6aee9d61a7
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/ConfigAuth.pm
@@ -0,0 +1,115 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+
+=pod
+
+=head1 NAME
+
+PostgreSQL::Test::ConfigAuth - module to set up SSPI auth for a data directory
+
+=head1 SYNOPSIS
+
+  use PostgreSQL::Test::ConfigAuth;
+
+  config_auth($datadir);
+  config_auth($datadir, user => $username);
+  config_auth($datadir, user => $username, extra_roles => "$role1,$role2");
+
+=head1 DESCRIPTION
+
+PostgreSQL::Test::ConfigAuth::config_auth() sets up SSPI authentication for a
+data directory. If called in a platform other than on Windows, or if
+PG_TEST_USE_UNIX_SOCKETS is true, it is a noop.
+
+=cut
+
+
+package PostgreSQL::Test::ConfigAuth;
+
+use strict;
+use warnings;
+
+use Config;
+use Exporter qw(import);
+use Socket qw(:addrinfo);
+
+our (@EXPORT, @EXPORT_OK, %EXPORT_TAGS);
+
+@EXPORT      = qw(config_auth);
+%EXPORT_TAGS = ();
+@EXPORT_OK   = ();
+
+sub _have_ipv6
+{
+	my %hints = (flags => AI_NUMERIC_HOST, family => AF_INET6);
+	my ($err, @res) = getaddrinfo("::1", undef, \%hints);
+	return $err ? 0 : 1;
+}
+
+=pod
+
+=head1 METHODS
+
+=over
+
+=item config_auth($datadir, %options);
+
+Set up SSPI authentication on Windows, noop elsewhere. Uses the DOMAIN and
+USERNAME from the environment as the authenticating user.
+
+=over
+
+=item user => $username
+
+The database superuser to authenticate as. The role should exist or be created
+before a connection is attempted. By default the environment variable USERNAME
+is used.
+
+=item extra_roles => "$role1,$role2,..."
+
+Comma separated list of roles that can also be authenticated as. These
+roles are not created - only authentication permission is added.
+
+=back
+
+=back
+
+=cut
+
+
+sub config_auth
+{
+	# only do this on Windows and if not using Unix sockets
+	return 1 unless $Config{osname} eq 'MSWin32' || $Config{osname} eq 'msys';
+	return 1 if $ENV{PG_TEST_USE_UNIX_SOCKETS};
+	my $datadir = shift;
+	my %params = @_;
+	my $username = $ENV{USERNAME};
+	my $domain = $ENV{USERDOMAIN};
+	die "missing username or domain" unless $username && $domain;
+	my $dbsuper = $params{user} || $username;
+	my $extras = $params{extra_roles};
+	my @extras = split(/,/,$extras) if $extras;
+	open (my $hba, '>', "$datadir/pg_hba.conf") ||
+	  die "opening $datadir/pg_hba.conf: $!";
+	my $have_ip6 = _have_ipv6();
+	print $hba
+	  "# Configuration written by PostgreSQL::Test::ConfigAuth\n",
+	  "host all all 127.0.0.1/32  sspi include_realm=1 map=regress\n";
+	print $hba
+	  "host all all ::1/128  sspi include_realm=1 map=regress\n"
+	  if $have_ip6;
+	close($hba);
+	open (my $ident, '>', "$datadir/pg_ident.conf") ||
+	  die "opening $datadir/pg_hba.conf: $!";
+	print $ident
+	  "# Configuration written by PostgreSQL::Test::ConfigAuth\n",
+	  qq{regress  "$username@$domain"  "$dbsuper"\n};
+	foreach my $role (@extras)
+	{
+		print $ident qq{regress  "$username@$domain"  "$role"\n};
+	}
+	close($ident);
+	return 1;
+}
+
+1;
diff --git a/src/test/recovery/t/001_stream_rep.pl b/src/test/recovery/t/001_stream_rep.pl
index 583ee87da8..366769033f 100644
--- a/src/test/recovery/t/001_stream_rep.pl
+++ b/src/test/recovery/t/001_stream_rep.pl
@@ -14,7 +14,7 @@ my $node_primary = PostgreSQL::Test::Cluster->new('primary');
 # and it needs proper authentication configuration.
 $node_primary->init(
 	allows_streaming => 1,
-	auth_extra       => [ '--create-role', 'repl_role' ]);
+	auth_extra       => [ extra_roles => 'repl_role' ]);
 $node_primary->start;
 my $backup_name = 'my_backup';
 
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index fc7aa8b9a3..e8fb1d856d 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -18,6 +18,9 @@ use File::Path qw(rmtree);
 use FindBin;
 use lib $FindBin::RealBin;
 
+use lib "$Findbin::Realbin/../../test/perl";
+use PostgreSQL::Test::ConfigAuth;
+
 use Install qw(Install);
 
 my $startdir = getcwd();
@@ -473,10 +476,7 @@ sub recoverycheck
 # Run "initdb", then reconfigure authentication.
 sub standard_initdb
 {
-	return (
-		system('initdb', '-N') == 0 and system(
-			"$topdir/$Config/pg_regress/pg_regress", '--config-auth',
-			$ENV{PGDATA}) == 0);
+	return (system('initdb', '-N') == 0 and config_auth($ENV{PGDATA}));
 }
 
 # This is similar to appendShellString().  Perl system(@args) bypasses
-- 
2.25.1

0002-Remove-the-config-auth-option-for-pg_regress.patchtext/x-patch; charset=UTF-8; name=0002-Remove-the-config-auth-option-for-pg_regress.patchDownload
From f82b06ed1119bfcd6872458b3a55192098f5dabb Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <andrew@dunslane.net>
Date: Tue, 26 Apr 2022 11:53:22 -0400
Subject: [PATCH 2/2] Remove the --config-auth option for pg_regress.

This is now redundant since TAP tests now know how to do this on their
own. The code which does this for pg_regress itself is kept.
---
 src/test/regress/pg_regress.c | 22 +---------------------
 1 file changed, 1 insertion(+), 21 deletions(-)

diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 982801e029..589a02052f 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -94,7 +94,6 @@ static bool port_specified_by_user = false;
 static char *dlpath = PKGLIBDIR;
 static char *user = NULL;
 static _stringlist *extraroles = NULL;
-static char *config_auth_datadir = NULL;
 
 /* internal variables */
 static const char *progname;
@@ -851,12 +850,7 @@ current_windows_user(const char **acct, const char **dom)
 }
 
 /*
- * Rewrite pg_hba.conf and pg_ident.conf to use SSPI authentication.  Permit
- * the current OS user to authenticate as the bootstrap superuser and as any
- * user named in a --create-role option.
- *
- * In --config-auth mode, the --user switch can be used to specify the
- * bootstrap superuser's name, otherwise we assume it is the default.
+ * Rewrite pg_hba.conf and pg_ident.conf to use SSPI authentication.
  */
 static void
 config_sspi_auth(const char *pgdata, const char *superuser_name)
@@ -1983,7 +1977,6 @@ help(void)
 	printf(_("Options:\n"));
 	printf(_("      --bindir=BINPATH          use BINPATH for programs that are run;\n"));
 	printf(_("                                if empty, use PATH from the environment\n"));
-	printf(_("      --config-auth=DATADIR     update authentication settings for DATADIR\n"));
 	printf(_("      --create-role=ROLE        create the specified role before testing\n"));
 	printf(_("      --dbname=DB               use database DB (default \"regression\")\n"));
 	printf(_("      --debug                   turn on debug mode in programs that are run\n"));
@@ -2050,7 +2043,6 @@ regression_main(int argc, char *argv[],
 		{"use-existing", no_argument, NULL, 20},
 		{"launcher", required_argument, NULL, 21},
 		{"load-extension", required_argument, NULL, 22},
-		{"config-auth", required_argument, NULL, 24},
 		{"max-concurrent-tests", required_argument, NULL, 25},
 		{NULL, 0, NULL, 0}
 	};
@@ -2175,9 +2167,6 @@ regression_main(int argc, char *argv[],
 			case 22:
 				add_stringlist_item(&loadextension, optarg);
 				break;
-			case 24:
-				config_auth_datadir = pg_strdup(optarg);
-				break;
 			case 25:
 				max_concurrent_tests = atoi(optarg);
 				break;
@@ -2198,15 +2187,6 @@ regression_main(int argc, char *argv[],
 		optind++;
 	}
 
-	if (config_auth_datadir)
-	{
-#ifdef ENABLE_SSPI
-		if (!use_unix_sockets)
-			config_sspi_auth(config_auth_datadir, user);
-#endif
-		exit(0);
-	}
-
 	if (temp_instance && !port_specified_by_user)
 
 		/*
-- 
2.25.1

#7vignesh C
vignesh21@gmail.com
In reply to: Andrew Dunstan (#6)
Re: Using AF_UNIX sockets always for tests on Windows

On Fri, 2 Dec 2022 at 18:08, Andrew Dunstan <andrew@dunslane.net> wrote:

On 2022-12-01 Th 21:10, Andres Freund wrote:

Hi,

On 2022-12-01 20:56:18 -0500, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2022-12-01 20:30:36 -0500, Tom Lane wrote:

If we remove that, won't we have a whole lot of code that's not
tested at all on any platform, ie all the TCP-socket code?

There's some coverage via the auth and ssl tests. But I agree it's an
issue. But to me the fix for that seems to be to add a dedicated test for
that, rather than relying on windows to test our socket code - that's quite a
few separate code paths from the tcp support of other platforms.

IMO that's not the best way forward, because you'll always have
nagging questions about whether a single-purpose test covers
everything that needs coverage.

Still seems better than not having any coverage in our development
environments...

I think the best place to be in would be to be able to run the whole test
suite using either TCP or UNIX sockets, on any platform (with stuff like the
SSL test overriding the choice as needed).

I agree that that's useful. But it seems somewhat independent from the
majority of the proposed changes. To be able to test force-tcp-everywhere we
don't need e.g. code for setting sspi auth in pg_regress etc - it's afaik
just needed so there's a secure way of running tests at all on windows.

I think 0003 should be "trimmed" to only change the default for
$use_unix_sockets on windows and to remove PG_TEST_USE_UNIX_SOCKETS. Whoever
wants to, can then add a new environment variable to force tap tests to use
tcp.

Not sure if it's useful here, but a few months ago I prepared patches to
remove the config-auth option of pg_regress, which struck me as more
than odd, and replace it with a perl module. I didn't get around to
finishing them, but the patches as of then are attached.

I agree that having some switch that says "run everything with TCP" or
"run (almost) everything with Unix sockets" would be good.

The patch does not apply on top of HEAD as in [1]http://cfbot.cputube.org/patch_41_4033.log, please post a rebased patch:
=== Applying patches on top of PostgreSQL commit ID
bf03cfd162176d543da79f9398131abc251ddbb9 ===
=== applying patch
./0001-Do-config_auth-in-perl-code-for-TAP-tests-and-vcregr.patch
patching file contrib/basebackup_to_shell/t/001_basic.pl
Hunk #1 FAILED at 21.
1 out of 1 hunk FAILED -- saving rejects to file
contrib/basebackup_to_shell/t/001_basic.pl.rej
patching file src/bin/pg_basebackup/t/010_pg_basebackup.pl
Hunk #1 FAILED at 29.
1 out of 1 hunk FAILED -- saving rejects to file
src/bin/pg_basebackup/t/010_pg_basebackup.pl.rej
Hunk #3 FAILED at 461.
1 out of 3 hunks FAILED -- saving rejects to file
src/test/perl/PostgreSQL/Test/Cluster.pm.rej

[1]: http://cfbot.cputube.org/patch_41_4033.log

Regards,
Vignesh

#8Andrew Dunstan
andrew@dunslane.net
In reply to: vignesh C (#7)
Re: Using AF_UNIX sockets always for tests on Windows

On 2023-01-04 We 07:13, vignesh C wrote:

On Fri, 2 Dec 2022 at 18:08, Andrew Dunstan <andrew@dunslane.net> wrote:

On 2022-12-01 Th 21:10, Andres Freund wrote:

Hi,

On 2022-12-01 20:56:18 -0500, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2022-12-01 20:30:36 -0500, Tom Lane wrote:

If we remove that, won't we have a whole lot of code that's not
tested at all on any platform, ie all the TCP-socket code?

There's some coverage via the auth and ssl tests. But I agree it's an
issue. But to me the fix for that seems to be to add a dedicated test for
that, rather than relying on windows to test our socket code - that's quite a
few separate code paths from the tcp support of other platforms.

IMO that's not the best way forward, because you'll always have
nagging questions about whether a single-purpose test covers
everything that needs coverage.

Still seems better than not having any coverage in our development
environments...

I think the best place to be in would be to be able to run the whole test
suite using either TCP or UNIX sockets, on any platform (with stuff like the
SSL test overriding the choice as needed).

I agree that that's useful. But it seems somewhat independent from the
majority of the proposed changes. To be able to test force-tcp-everywhere we
don't need e.g. code for setting sspi auth in pg_regress etc - it's afaik
just needed so there's a secure way of running tests at all on windows.

I think 0003 should be "trimmed" to only change the default for
$use_unix_sockets on windows and to remove PG_TEST_USE_UNIX_SOCKETS. Whoever
wants to, can then add a new environment variable to force tap tests to use
tcp.

Not sure if it's useful here, but a few months ago I prepared patches to
remove the config-auth option of pg_regress, which struck me as more
than odd, and replace it with a perl module. I didn't get around to
finishing them, but the patches as of then are attached.

I agree that having some switch that says "run everything with TCP" or
"run (almost) everything with Unix sockets" would be good.

The patch does not apply on top of HEAD as in [1], please post a rebased patch:
=== Applying patches on top of PostgreSQL commit ID
bf03cfd162176d543da79f9398131abc251ddbb9 ===
=== applying patch
./0001-Do-config_auth-in-perl-code-for-TAP-tests-and-vcregr.patch
patching file contrib/basebackup_to_shell/t/001_basic.pl
Hunk #1 FAILED at 21.
1 out of 1 hunk FAILED -- saving rejects to file
contrib/basebackup_to_shell/t/001_basic.pl.rej
patching file src/bin/pg_basebackup/t/010_pg_basebackup.pl
Hunk #1 FAILED at 29.
1 out of 1 hunk FAILED -- saving rejects to file
src/bin/pg_basebackup/t/010_pg_basebackup.pl.rej
Hunk #3 FAILED at 461.
1 out of 3 hunks FAILED -- saving rejects to file
src/test/perl/PostgreSQL/Test/Cluster.pm.rej

[1] - http://cfbot.cputube.org/patch_41_4033.log

What I posted was not intended as a replacement for Thomas' patches, or
indeed meant as a CF item at all.

So really we're waiting on Thomas to post a response to Tom's and
Andres' comments upthread.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

#9Juan José Santamaría Flecha
juanjo.santamaria@gmail.com
In reply to: Thomas Munro (#1)
4 attachment(s)
Re: Using AF_UNIX sockets always for tests on Windows

Hello,

On Fri, Dec 2, 2022 at 1:03 AM Thomas Munro <thomas.munro@gmail.com> wrote:

1. Teach mkdtemp() to make a non-world-accessible directory. This is
required to be able to make a socket that other processes can't
connect to, to match the paranoia level used on Unix. This was
written just by reading documentation, because I am not a Windows
user, so I would be grateful for a second opinion and/or testing from
a Windows hacker, which would involve testing with two different
users. The idea is that Windows' mkdir() is completely ignoring the
permissions (we can see in the mingw headers that it literally throws
away the mode argument), so we shouldn't use that, but native
CreateDirectory() when given a pointer to a SECURITY_ATTRIBUTES with
lpSecurityDesciptor set to NULL should only allow the current user to
access the object (directory). Does this really work, and would it be
better to create some more explicit private-keep-out
SECURITY_ATTRIBUTE, and how would that look?

A directory created with a NULL SECURITY_ATTRIBUTES inherits the ACL from
its parent directory [1]https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea. In this case, its parent is the designated
temporary location, which already should have a limited access.

You can create an explicit DACL for that directory, PFA a patch for so.
This is just an example, not something that I'm proposing as a committable
alternative.

I'm fairly sure that filesystem permissions must be enough to stop

another OS user from connecting, because it's clear from documentation
that AF_UNIX works on Windows by opening the file and reading some
magic "reparse" data from inside it, so if you can't see into the
containing directory, you can't do it. This patch is the one the rest
are standing on, because the tests should match Unix in their level of
security.

Yes, this is correct.

Only the first patch is modified, but I'm including all of them so they go

through the cfbot.

[1]: https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea
https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea

Regards,

Juan José Santamaría Flecha

Attachments:

v2-0004-Doc-Abstract-AF_UNIX-sockets-don-t-work-on-Windows-a.patchapplication/octet-stream; name=v2-0004-Doc-Abstract-AF_UNIX-sockets-don-t-work-on-Windows-a.patchDownload
From e645cc2ab80450b18227931082beefc2bb8ffb0a Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Mon, 15 Aug 2022 10:43:13 +1200
Subject: [PATCH 4/4] Doc: Abstract AF_UNIX sockets don't work on Windows after
 all.

An early release of AF_UNIX in Windows might have supported Linux-style
"abstract" Unix sockets with a system-wide namespace, but they do not
seem to work in current Windows versions and there is no mention of any
of this in the Winsock documentation.  Remove the claim that it works
from our documentation.

Back-patch to 14, where commit c9f0624b landed.

Discussion: https://postgr.es/m/20220813223646.oh2dkjrkj7jn7dpe%40awork3.anarazel.de
---
 doc/src/sgml/config.sgml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 39d1c89e33..55286375dc 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -759,7 +759,7 @@ include_dir 'conf.d'
        <para>
         A value that starts with <literal>@</literal> specifies that a
         Unix-domain socket in the abstract namespace should be created
-        (currently supported on Linux and Windows).  In that case, this value
+        (currently supported on Linux).  In that case, this value
         does not specify a <quote>directory</quote> but a prefix from which
         the actual socket name is computed in the same manner as for the
         file-system namespace.  While the abstract socket name prefix can be
-- 
2.38.1

v2-0003-WIP-Stop-using-TCP-in-TAP-tests-on-Windows.patchapplication/octet-stream; name=v2-0003-WIP-Stop-using-TCP-in-TAP-tests-on-Windows.patchDownload
From c4429b2ad41850b8e3a360d1093be8a57014a156 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Fri, 19 Aug 2022 12:00:07 +1200
Subject: [PATCH 3/4] WIP: Stop using TCP in TAP tests on Windows.

Since Unix-domain sockets are available on our minimum target Windows
versions (10+), we can remove a source of instability and a point of
variation between Unix and Windows.
---
 .cirrus.yml                                   |  3 -
 src/bin/pg_ctl/t/001_start_stop.pl            | 13 +--
 src/test/authentication/t/001_password.pl     |  6 --
 src/test/authentication/t/002_saslprep.pl     |  7 --
 src/test/authentication/t/003_peer.pl         |  5 --
 .../authentication/t/004_file_inclusion.pl    |  5 --
 src/test/perl/PostgreSQL/Test/Cluster.pm      | 90 ++++---------------
 src/test/perl/PostgreSQL/Test/Utils.pm        |  9 +-
 8 files changed, 19 insertions(+), 119 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index f31923333e..e4aed541d8 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -502,9 +502,6 @@ WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE
     # git's tar doesn't deal with drive letters, see
     # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net
     TAR: "c:/windows/system32/tar.exe"
-    # Avoids port conflicts between concurrent tap test runs
-    PG_TEST_USE_UNIX_SOCKETS: 1
-    PG_REGRESS_SOCK_DIR: "c:/cirrus/"
 
   sysinfo_script: |
     chcp
diff --git a/src/bin/pg_ctl/t/001_start_stop.pl b/src/bin/pg_ctl/t/001_start_stop.pl
index fdffd76d99..23f942a440 100644
--- a/src/bin/pg_ctl/t/001_start_stop.pl
+++ b/src/bin/pg_ctl/t/001_start_stop.pl
@@ -29,16 +29,9 @@ print $conf "port = $node_port\n";
 print $conf PostgreSQL::Test::Utils::slurp_file($ENV{TEMP_CONFIG})
   if defined $ENV{TEMP_CONFIG};
 
-if ($use_unix_sockets)
-{
-	print $conf "listen_addresses = ''\n";
-	$tempdir_short =~ s!\\!/!g if $PostgreSQL::Test::Utils::windows_os;
-	print $conf "unix_socket_directories = '$tempdir_short'\n";
-}
-else
-{
-	print $conf "listen_addresses = '127.0.0.1'\n";
-}
+print $conf "listen_addresses = ''\n";
+$tempdir_short =~ s!\\!/!g if $PostgreSQL::Test::Utils::windows_os;
+print $conf "unix_socket_directories = '$tempdir_short'\n";
 close $conf;
 my $ctlcmd = [
 	'pg_ctl', 'start', '-D', "$tempdir/data", '-l',
diff --git a/src/test/authentication/t/001_password.pl b/src/test/authentication/t/001_password.pl
index 42d3d4c79b..45d105717a 100644
--- a/src/test/authentication/t/001_password.pl
+++ b/src/test/authentication/t/001_password.pl
@@ -6,18 +6,12 @@
 # - Plain
 # - MD5-encrypted
 # - SCRAM-encrypted
-# This test can only run with Unix-domain sockets.
 
 use strict;
 use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
 use Test::More;
-if (!$use_unix_sockets)
-{
-	plan skip_all =>
-	  "authentication tests cannot run without Unix-domain sockets";
-}
 
 # Delete pg_hba.conf from the given node, add a new entry to it
 # and then execute a reload to refresh it.
diff --git a/src/test/authentication/t/003_peer.pl b/src/test/authentication/t/003_peer.pl
index 26c34d05d3..d9d8616e30 100644
--- a/src/test/authentication/t/003_peer.pl
+++ b/src/test/authentication/t/003_peer.pl
@@ -10,11 +10,6 @@ use warnings;
 use PostgreSQL::Test::Cluster;
 use PostgreSQL::Test::Utils;
 use Test::More;
-if (!$use_unix_sockets)
-{
-	plan skip_all =>
-	  "authentication tests cannot run without Unix-domain sockets";
-}
 
 # Delete pg_hba.conf from the given node, add a new entry to it
 # and then execute a reload to refresh it.
diff --git a/src/test/authentication/t/004_file_inclusion.pl b/src/test/authentication/t/004_file_inclusion.pl
index c420f3ebca..db4fcd962b 100644
--- a/src/test/authentication/t/004_file_inclusion.pl
+++ b/src/test/authentication/t/004_file_inclusion.pl
@@ -11,11 +11,6 @@ use PostgreSQL::Test::Utils;
 use File::Basename qw(basename);
 use Test::More;
 use Data::Dumper;
-if (!$use_unix_sockets)
-{
-	plan skip_all =>
-	  "authentication tests cannot run without Unix-domain sockets";
-}
 
 # Stores the number of lines created for each file.  hba_rule and ident_rule
 # are used to respectively track pg_hba_file_rules.rule_number and
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index 7411188dc8..fdaed41f7b 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -116,7 +116,7 @@ use PostgreSQL::Test::Utils ();
 use Time::HiRes qw(usleep);
 use Scalar::Util qw(blessed);
 
-our ($use_tcp, $test_localhost, $test_pghost, $last_host_assigned,
+our ($test_pghost,
 	$last_port_assigned, @all_nodes, $died, $portdir);
 
 # the minimum version we believe to be compatible with this package without
@@ -131,21 +131,11 @@ INIT
 
 	# Set PGHOST for backward compatibility.  This doesn't work for own_host
 	# nodes, so prefer to not rely on this when writing new tests.
-	$use_tcp            = !$PostgreSQL::Test::Utils::use_unix_sockets;
-	$test_localhost     = "127.0.0.1";
-	$last_host_assigned = 1;
-	if ($use_tcp)
-	{
-		$test_pghost = $test_localhost;
-	}
-	else
-	{
-		# On windows, replace windows-style \ path separators with / when
-		# putting socket directories either in postgresql.conf or libpq
-		# connection strings, otherwise they are interpreted as escapes.
-		$test_pghost = PostgreSQL::Test::Utils::tempdir_short;
-		$test_pghost =~ s!\\!/!g if $PostgreSQL::Test::Utils::windows_os;
-	}
+	# On windows, replace windows-style \ path separators with / when
+	# putting socket directories either in postgresql.conf or libpq
+	# connection strings, otherwise they are interpreted as escapes.
+	$test_pghost = PostgreSQL::Test::Utils::tempdir_short;
+	$test_pghost =~ s!\\!/!g if $PostgreSQL::Test::Utils::windows_os;
 	$ENV{PGHOST}     = $test_pghost;
 	$ENV{PGDATABASE} = 'postgres';
 
@@ -470,12 +460,6 @@ sub set_replication_conf
 	open my $hba, '>>', "$pgdata/pg_hba.conf";
 	print $hba
 	  "\n# Allow replication (set up by PostgreSQL::Test::Cluster.pm)\n";
-	if ($PostgreSQL::Test::Utils::windows_os
-		&& !$PostgreSQL::Test::Utils::use_unix_sockets)
-	{
-		print $hba
-		  "host replication all $test_localhost/32 sspi include_realm=1 map=regress\n";
-	}
 	close $hba;
 	return;
 }
@@ -568,16 +552,8 @@ sub init
 	}
 
 	print $conf "port = $port\n";
-	if ($use_tcp)
-	{
-		print $conf "unix_socket_directories = ''\n";
-		print $conf "listen_addresses = '$host'\n";
-	}
-	else
-	{
-		print $conf "unix_socket_directories = '$host'\n";
-		print $conf "listen_addresses = ''\n";
-	}
+	print $conf "unix_socket_directories = '$host'\n";
+	print $conf "listen_addresses = ''\n";
 	close $conf;
 
 	chmod($self->group_access ? 0640 : 0600, "$pgdata/postgresql.conf")
@@ -796,15 +772,8 @@ sub init_from_backup
 		qq(
 port = $port
 ));
-	if ($use_tcp)
-	{
-		$self->append_conf('postgresql.conf', "listen_addresses = '$host'");
-	}
-	else
-	{
-		$self->append_conf('postgresql.conf',
-			"unix_socket_directories = '$host'");
-	}
+	$self->append_conf('postgresql.conf',
+		"unix_socket_directories = '$host'");
 	$self->enable_streaming($root_node) if $params{has_streaming};
 	$self->enable_restoring($root_node, $params{standby})
 	  if $params{has_restoring};
@@ -1270,9 +1239,7 @@ sub new
 	else
 	{
 		# When selecting a port, we look for an unassigned TCP port number,
-		# even if we intend to use only Unix-domain sockets.  This is clearly
-		# necessary on $use_tcp (Windows) configurations, and it seems like a
-		# good idea on Unixen as well.
+		# even if we intend to use only Unix-domain sockets.
 		$port = get_free_port();
 	}
 
@@ -1280,17 +1247,8 @@ sub new
 	my $host = $test_pghost;
 	if ($params{own_host})
 	{
-		if ($use_tcp)
-		{
-			$last_host_assigned++;
-			$last_host_assigned > 254 and BAIL_OUT("too many own_host nodes");
-			$host = '127.0.0.' . $last_host_assigned;
-		}
-		else
-		{
-			$host = "$test_pghost/$name"; # Assume $name =~ /^[-_a-zA-Z0-9]+$/
-			mkdir $host;
-		}
+		$host = "$test_pghost/$name"; # Assume $name =~ /^[-_a-zA-Z0-9]+$/
+		mkdir $host;
 	}
 
 	my $testname = basename($0);
@@ -1526,29 +1484,11 @@ sub get_free_port
 		}
 
 		# Check to see if anything else is listening on this TCP port.
-		# Seek a port available for all possible listen_addresses values,
-		# so callers can harness this port for the widest range of purposes.
-		# The 0.0.0.0 test achieves that for MSYS, which automatically sets
-		# SO_EXCLUSIVEADDRUSE.  Testing 0.0.0.0 is insufficient for Windows
-		# native Perl (https://stackoverflow.com/a/14388707), so we also
-		# have to test individual addresses.  Doing that for 127.0.0/24
-		# addresses other than 127.0.0.1 might fail with EADDRNOTAVAIL on
-		# non-Linux, non-Windows kernels.
-		#
-		# Thus, 0.0.0.0 and individual 127.0.0/24 addresses are tested
-		# only on Windows and only when TCP usage is requested.
 		if ($found == 1)
 		{
-			foreach my $addr (qw(127.0.0.1),
-				($use_tcp && $PostgreSQL::Test::Utils::windows_os)
-				  ? qw(127.0.0.2 127.0.0.3 0.0.0.0)
-				  : ())
+			if (!can_bind(qw(127.0.0.1), $port))
 			{
-				if (!can_bind($addr, $port))
-				{
-					$found = 0;
-					last;
-				}
+				$found = 0;
 			}
 			$found = _reserve_port($port) if $found;
 		}
diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm
index b139190cc8..631d8bd362 100644
--- a/src/test/perl/PostgreSQL/Test/Utils.pm
+++ b/src/test/perl/PostgreSQL/Test/Utils.pm
@@ -88,10 +88,9 @@ our @EXPORT = qw(
 
   $windows_os
   $is_msys2
-  $use_unix_sockets
 );
 
-our ($windows_os, $is_msys2, $use_unix_sockets, $timeout_default,
+our ($windows_os, $is_msys2, $timeout_default,
 	$tmp_check, $log_path, $test_logfile);
 
 BEGIN
@@ -153,12 +152,6 @@ BEGIN
 		Win32API::File->import(qw(createFile OsFHandleOpen CloseHandle));
 	}
 
-	# Specifies whether to use Unix sockets for test setups.  On
-	# Windows we don't use them by default since it's not universally
-	# supported, but it can be overridden if desired.
-	$use_unix_sockets =
-	  (!$windows_os || defined $ENV{PG_TEST_USE_UNIX_SOCKETS});
-
 	$timeout_default = $ENV{PG_TEST_TIMEOUT_DEFAULT};
 	$timeout_default = 180
 	  if not defined $timeout_default or $timeout_default eq '';
-- 
2.38.1

v2-0002-WIP-Always-use-Unix-domain-sockets-in-pg_regress-on-.patchapplication/octet-stream; name=v2-0002-WIP-Always-use-Unix-domain-sockets-in-pg_regress-on-.patchDownload
From 388719a6fbb45896ff87794ed4bfdbc0f0aac329 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Fri, 19 Aug 2022 11:28:38 +1200
Subject: [PATCH 2/4] WIP: Always use Unix-domain sockets in pg_regress on
 Windows.

Since we can now rely on Unix-domain sockets working on supported
Windows versions (10+), we can remove a source of instability and a
difference between Unix and Windows in pg_regress.

Previously, we thought the socket cleanup code was unsafe, so we made
Unix-domain sockets an option with a "use-at-your-own-risk" note.  On
closer inspection, the concerns about signal handlers don't seem to
apply here.  (initdb.c has similar concerns but needs separate
investigation.)

Previously, commit f6dc6dd5 secured temporary installations using TCP/IP
on Windows, while commit be76a6d3 used file system permissions for Unix
sockets on Unix.  Now that our src/port/mkdtemp.c file creates
non-world-accessible directories on Windows, we can just do the same on
Windows.
---
 src/test/regress/pg_regress.c | 274 ++++------------------------------
 1 file changed, 32 insertions(+), 242 deletions(-)

diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index f308da6c50..dc6b4663c0 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -55,6 +55,20 @@ char	   *host_platform = HOST_TUPLE;
 static char *shellprog = SHELLPROG;
 #endif
 
+/*
+ * The name of the environment variable that controls where we put temporary
+ * files, to override the defaut of "/tmp".
+ */
+#ifdef WIN32
+#define TMPDIR "TMP"
+#else
+#define TMPDIR "TMPDIR"
+#endif
+
+#if defined(WIN32) && defined(USE_ASSERT_CHECKING)
+static DWORD main_thread_id;
+#endif
+
 /*
  * On Windows we use -w in diff switches to avoid problems with inconsistent
  * newline representation.  The actual result files will generally have
@@ -286,9 +300,7 @@ stop_postmaster(void)
  * postmaster exit, so it is indeterminate whether the postmaster has yet to
  * unlink the socket and lock file.  Unlink them here so we can proceed to
  * remove the directory.  Ignore errors; leaking a temporary directory is
- * unimportant.  This can run from a signal handler.  The code is not
- * acceptable in a Windows signal handler (see initdb.c:trapsig()), but
- * on Windows, pg_regress does not use Unix sockets by default.
+ * unimportant.  This can run from a signal handler.
  */
 static void
 remove_temp(void)
@@ -305,6 +317,18 @@ remove_temp(void)
 static void
 signal_remove_temp(SIGNAL_ARGS)
 {
+#ifdef WIN32
+	/*
+	 * In general, it would not be acceptable to call remove_temp() in a
+	 * Windows signal handler.  It is safe in this program though, because
+	 * SIGHUP and SIGPIPE don't really exist and SIGTERM is never raised by the
+	 * system, leaving just SIGINT.  SIGINT doesn't interrupt the main
+	 * execution context on Windows, it runs the handler concurrently in
+	 * another thread.
+	 */
+	Assert(GetCurrentThreadId() != main_thread_id);
+#endif
+
 	remove_temp();
 
 	pqsignal(postgres_signal_arg, SIG_DFL);
@@ -327,7 +351,7 @@ static const char *
 make_temp_sockdir(void)
 {
 	char	   *template = psprintf("%s/pg_regress-XXXXXX",
-									getenv("TMPDIR") ? getenv("TMPDIR") : "/tmp");
+									getenv(TMPDIR) ? getenv(TMPDIR) : "/tmp");
 
 	temp_sockdir = mkdtemp(template);
 	if (temp_sockdir == NULL)
@@ -344,6 +368,10 @@ make_temp_sockdir(void)
 	/* Remove the directory during clean exit. */
 	atexit(remove_temp);
 
+#if defined(WIN32) && defined(USE_ASSERT_CHECKING)
+	main_thread_id = GetCurrentThreadId();
+#endif
+
 	/*
 	 * Remove the directory before dying to the usual signals.  Omit SIGQUIT,
 	 * preserving it as a quick, untidy exit.
@@ -754,211 +782,6 @@ initialize_environment(void)
 	load_resultmap();
 }
 
-#ifdef ENABLE_SSPI
-
-/* support for config_sspi_auth() */
-static const char *
-fmtHba(const char *raw)
-{
-	static char *ret;
-	const char *rp;
-	char	   *wp;
-
-	wp = ret = pg_realloc(ret, 3 + strlen(raw) * 2);
-
-	*wp++ = '"';
-	for (rp = raw; *rp; rp++)
-	{
-		if (*rp == '"')
-			*wp++ = '"';
-		*wp++ = *rp;
-	}
-	*wp++ = '"';
-	*wp++ = '\0';
-
-	return ret;
-}
-
-/*
- * Get account and domain/realm names for the current user.  This is based on
- * pg_SSPI_recvauth().  The returned strings use static storage.
- */
-static void
-current_windows_user(const char **acct, const char **dom)
-{
-	static char accountname[MAXPGPATH];
-	static char domainname[MAXPGPATH];
-	HANDLE		token;
-	TOKEN_USER *tokenuser;
-	DWORD		retlen;
-	DWORD		accountnamesize = sizeof(accountname);
-	DWORD		domainnamesize = sizeof(domainname);
-	SID_NAME_USE accountnameuse;
-
-	if (!OpenProcessToken(GetCurrentProcess(), TOKEN_READ, &token))
-	{
-		fprintf(stderr,
-				_("%s: could not open process token: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-
-	if (!GetTokenInformation(token, TokenUser, NULL, 0, &retlen) && GetLastError() != 122)
-	{
-		fprintf(stderr,
-				_("%s: could not get token information buffer size: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-	tokenuser = pg_malloc(retlen);
-	if (!GetTokenInformation(token, TokenUser, tokenuser, retlen, &retlen))
-	{
-		fprintf(stderr,
-				_("%s: could not get token information: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-
-	if (!LookupAccountSid(NULL, tokenuser->User.Sid, accountname, &accountnamesize,
-						  domainname, &domainnamesize, &accountnameuse))
-	{
-		fprintf(stderr,
-				_("%s: could not look up account SID: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-
-	free(tokenuser);
-
-	*acct = accountname;
-	*dom = domainname;
-}
-
-/*
- * Rewrite pg_hba.conf and pg_ident.conf to use SSPI authentication.  Permit
- * the current OS user to authenticate as the bootstrap superuser and as any
- * user named in a --create-role option.
- *
- * In --config-auth mode, the --user switch can be used to specify the
- * bootstrap superuser's name, otherwise we assume it is the default.
- */
-static void
-config_sspi_auth(const char *pgdata, const char *superuser_name)
-{
-	const char *accountname,
-			   *domainname;
-	char	   *errstr;
-	bool		have_ipv6;
-	char		fname[MAXPGPATH];
-	int			res;
-	FILE	   *hba,
-			   *ident;
-	_stringlist *sl;
-
-	/* Find out the name of the current OS user */
-	current_windows_user(&accountname, &domainname);
-
-	/* Determine the bootstrap superuser's name */
-	if (superuser_name == NULL)
-	{
-		/*
-		 * Compute the default superuser name the same way initdb does.
-		 *
-		 * It's possible that this result always matches "accountname", the
-		 * value SSPI authentication discovers.  But the underlying system
-		 * functions do not clearly guarantee that.
-		 */
-		superuser_name = get_user_name(&errstr);
-		if (superuser_name == NULL)
-		{
-			fprintf(stderr, "%s: %s\n", progname, errstr);
-			exit(2);
-		}
-	}
-
-	/*
-	 * Like initdb.c:setup_config(), determine whether the platform recognizes
-	 * ::1 (IPv6 loopback) as a numeric host address string.
-	 */
-	{
-		struct addrinfo *gai_result;
-		struct addrinfo hints;
-		WSADATA		wsaData;
-
-		hints.ai_flags = AI_NUMERICHOST;
-		hints.ai_family = AF_UNSPEC;
-		hints.ai_socktype = 0;
-		hints.ai_protocol = 0;
-		hints.ai_addrlen = 0;
-		hints.ai_canonname = NULL;
-		hints.ai_addr = NULL;
-		hints.ai_next = NULL;
-
-		have_ipv6 = (WSAStartup(MAKEWORD(2, 2), &wsaData) == 0 &&
-					 getaddrinfo("::1", NULL, &hints, &gai_result) == 0);
-	}
-
-	/* Check a Write outcome and report any error. */
-#define CW(cond)	\
-	do { \
-		if (!(cond)) \
-		{ \
-			fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"), \
-					progname, fname, strerror(errno)); \
-			exit(2); \
-		} \
-	} while (0)
-
-	res = snprintf(fname, sizeof(fname), "%s/pg_hba.conf", pgdata);
-	if (res < 0 || res >= sizeof(fname))
-	{
-		/*
-		 * Truncating this name is a fatal error, because we must not fail to
-		 * overwrite an original trust-authentication pg_hba.conf.
-		 */
-		fprintf(stderr, _("%s: directory name too long\n"), progname);
-		exit(2);
-	}
-	hba = fopen(fname, "w");
-	if (hba == NULL)
-	{
-		fprintf(stderr, _("%s: could not open file \"%s\" for writing: %s\n"),
-				progname, fname, strerror(errno));
-		exit(2);
-	}
-	CW(fputs("# Configuration written by config_sspi_auth()\n", hba) >= 0);
-	CW(fputs("host all all 127.0.0.1/32  sspi include_realm=1 map=regress\n",
-			 hba) >= 0);
-	if (have_ipv6)
-		CW(fputs("host all all ::1/128  sspi include_realm=1 map=regress\n",
-				 hba) >= 0);
-	CW(fclose(hba) == 0);
-
-	snprintf(fname, sizeof(fname), "%s/pg_ident.conf", pgdata);
-	ident = fopen(fname, "w");
-	if (ident == NULL)
-	{
-		fprintf(stderr, _("%s: could not open file \"%s\" for writing: %s\n"),
-				progname, fname, strerror(errno));
-		exit(2);
-	}
-	CW(fputs("# Configuration written by config_sspi_auth()\n", ident) >= 0);
-
-	/*
-	 * Double-quote for the benefit of account names containing whitespace or
-	 * '#'.  Windows forbids the double-quote character itself, so don't
-	 * bother escaping embedded double-quote characters.
-	 */
-	CW(fprintf(ident, "regress  \"%s@%s\"  %s\n",
-			   accountname, domainname, fmtHba(superuser_name)) >= 0);
-	for (sl = extraroles; sl; sl = sl->next)
-		CW(fprintf(ident, "regress  \"%s@%s\"  %s\n",
-				   accountname, domainname, fmtHba(sl->str)) >= 0);
-	CW(fclose(ident) == 0);
-}
-
-#endif							/* ENABLE_SSPI */
-
 /*
  * psql_start_command, psql_add_command, psql_end_command
  *
@@ -2039,7 +1862,6 @@ regression_main(int argc, char *argv[],
 		{NULL, 0, NULL, 0}
 	};
 
-	bool		use_unix_sockets;
 	_stringlist *sl;
 	int			c;
 	int			i;
@@ -2055,20 +1877,6 @@ regression_main(int argc, char *argv[],
 
 	atexit(stop_postmaster);
 
-#if defined(WIN32)
-
-	/*
-	 * We don't use Unix-domain sockets on Windows by default (see comment at
-	 * remove_temp() for a reason).  Override at your own risk.
-	 */
-	use_unix_sockets = getenv("PG_TEST_USE_UNIX_SOCKETS") ? true : false;
-#else
-	use_unix_sockets = true;
-#endif
-
-	if (!use_unix_sockets)
-		hostname = "localhost";
-
 	/*
 	 * We call the initialization function here because that way we can set
 	 * default parameters and let them be overwritten by the commandline.
@@ -2194,13 +2002,7 @@ regression_main(int argc, char *argv[],
 	}
 
 	if (config_auth_datadir)
-	{
-#ifdef ENABLE_SSPI
-		if (!use_unix_sockets)
-			config_sspi_auth(config_auth_datadir, user);
-#endif
 		exit(0);
-	}
 
 	if (temp_instance && !port_specified_by_user)
 
@@ -2319,18 +2121,6 @@ regression_main(int argc, char *argv[],
 
 		fclose(pg_conf);
 
-#ifdef ENABLE_SSPI
-		if (!use_unix_sockets)
-		{
-			/*
-			 * Since we successfully used the same buffer for the much-longer
-			 * "initdb" command, this can't truncate.
-			 */
-			snprintf(buf, sizeof(buf), "%s/data", temp_instance);
-			config_sspi_auth(buf, NULL);
-		}
-#endif
-
 		/*
 		 * Check if there is a postmaster running already.
 		 */
-- 
2.38.1

v2-0001-WIP-Make-mkdtemp-more-secure-on-Windows.patchapplication/octet-stream; name=v2-0001-WIP-Make-mkdtemp-more-secure-on-Windows.patchDownload
From 080d20ca7f56f869106d513c07654e5868ab6e60 Mon Sep 17 00:00:00 2001
From: Juan Jose Santamaria Flecha <juanjo.santamaria@gmail.com>
Date: Mon, 16 Jan 2023 06:27:59 -0500
Subject: [PATCH] [PATCH 1/4] WIP: Make mkdtemp() more secure on Windows.

Our POSIX mkdtemp() implementation in src/port/mkdtemp.c code would
create directories with default permissions on Windows.  Fix, using
native Windows API instead of mkdir().
---
 src/port/mkdtemp.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/src/port/mkdtemp.c b/src/port/mkdtemp.c
index 4578e83..64a0641 100644
--- a/src/port/mkdtemp.c
+++ b/src/port/mkdtemp.c
@@ -80,6 +80,10 @@ __RCSID("$NetBSD: gettemp.c,v 1.17 2014/01/21 19:09:48 seanb Exp $");
 #include <stdlib.h>
 #include <unistd.h>
 
+#ifdef WIN32
+#include <sddl.h>
+#endif
+
 #ifdef NOT_POSTGRESQL
 #if HAVE_NBTOOL_CONFIG_H
 #define GETTEMP		__nbcompat_gettemp
@@ -187,8 +191,50 @@ GETTEMP(char *path, int *doopen, int domkdir)
 		}
 		else if (domkdir)
 		{
+#ifdef WIN32
+			HANDLE				hToken = NULL;
+			DWORD				dwBufferSize = 0;
+			PTOKEN_USER			pTokenUser = NULL;
+			char			   *pSid = NULL;
+			char				pSdd[570];
+			SECURITY_ATTRIBUTES	sa = {sizeof(SECURITY_ATTRIBUTES)};
+
+			/* open the access token associated with the calling process */
+			if (!OpenProcessToken(GetCurrentProcess(), TOKEN_QUERY, &hToken))
+				_dosmaperr(GetLastError());
+
+			/* get the size of the memory buffer needed for the SID */
+			(void)GetTokenInformation(hToken, TokenUser, NULL, 0, &dwBufferSize);
+
+			pTokenUser = (PTOKEN_USER)malloc(dwBufferSize);
+			memset(pTokenUser, 0, dwBufferSize);
+
+			/* retrieve the token information in a TOKEN_USER structure */
+			if (!GetTokenInformation(hToken, TokenUser, pTokenUser, dwBufferSize,
+				&dwBufferSize))
+				_dosmaperr(GetLastError());
+
+			CloseHandle(hToken);
+			ConvertSidToStringSid(pTokenUser->User.Sid, &pSid);
+			free(pTokenUser);
+
+			/*
+			 * Security descriptor for granting access to system, administrators and
+			 * current user.
+			 */
+			sprintf(pSdd, "O:%sD:(A;OICIID;FA;;;SY)(A;OICIID;FA;;;BA)(A;OICIID;FA;;;%s)",
+					pSid, pSid);
+			LocalFree(pSid);
+
+			if (ConvertStringSecurityDescriptorToSecurityDescriptor(pSdd, SDDL_REVISION_1,
+				&sa.lpSecurityDescriptor, NULL) && CreateDirectory(path, &sa))
+				return 1;
+
+			_dosmaperr(GetLastError());
+#else
 			if (mkdir(path, 0700) >= 0)
 				return 1;
+#endif
 			if (errno != EEXIST)
 				return 0;
 		}
-- 
2.11.0

#10Thomas Munro
thomas.munro@gmail.com
In reply to: Juan José Santamaría Flecha (#9)
2 attachment(s)
Re: Using AF_UNIX sockets always for tests on Windows

Thanks Tom, Andres, Juan José, Andrew for your feedback. I agree that
it's a better "OS harmonisation" outcome if we can choose both ways on
both OSes. I will leave the 0003 patch aside for now due to lack of
time, but here's a rebase of the first two patches. Since this is
really just more cleanup-obsolete-stuff background work, I'm going to
move it to the next CF.

0001 -- Teach mkdtemp() to care about permissions on Windows.
Reviewed by Juan José, who confirmed my blind-coded understanding and
showed an alternative version but didn't suggest doing it that way.
It's a little confusing that NULL "attributes" means something
different from NULL "descriptor", so I figured I should highlight that
difference more clearly with some new comments. I guess one question
is "why should we expect the calling process to have the default
access token?"

0002 -- Use AF_UNIX for pg_regress. This one removes a couple of
hundred Windows-only lines that set up SSPI stuff, and some comments
about a signal handling hazard that -- as far as I can tell by reading
manuals -- was never really true.

0003 -- TAP testing adjustments needs some more work based on feedback
already given, not included in this revision.

Attachments:

v3-0001-Make-mkdtemp-more-secure-on-Windows.patchtext/x-patch; charset=UTF-8; name=v3-0001-Make-mkdtemp-more-secure-on-Windows.patchDownload
From e7fac7a15ed0eda6516e7fa0917c06e005341b00 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 7 Sep 2022 07:35:11 +1200
Subject: [PATCH v3 1/2] Make mkdtemp() more secure on Windows.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Our POSIX mkdtemp() implementation in src/port/mkdtemp.c code would
create directories with default permissions on Windows.  Fix, using the
native Windows API instead of mkdir().

This function is currently used by pg_regress's make_temp_sockdir().

Reviewed-by: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGK30uLx9dpgkYwomgH0WVLUHytkChDgf3iUM2zp0pf_nA%40mail.gmail.com
---
 src/port/mkdtemp.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/src/port/mkdtemp.c b/src/port/mkdtemp.c
index 4578e8384c..9d3c4fce71 100644
--- a/src/port/mkdtemp.c
+++ b/src/port/mkdtemp.c
@@ -187,8 +187,35 @@ GETTEMP(char *path, int *doopen, int domkdir)
 		}
 		else if (domkdir)
 		{
+#ifdef WIN32
+			/*
+			 * Plain mkdir(path, 0700) would ignore the mode argument, so
+			 * we'll use the native Windows API to create the directory.  By
+			 * setting lpSecurityDescriptor to NULL, we get "the default
+			 * security descriptor associated with the access token of the
+			 * calling process.  [...]  By default, the default DACL in the
+			 * access token of a process allows access only to the user
+			 * represented by the access token."
+			 *
+			 * Note that a NULL lpSecurityDescriptor is not the same as a NULL
+			 * lpSecurityAttributes argument.  The latter would mean that the
+			 * ACL is inherited from the parent directory, which would
+			 * probably work out the same if it's the TMP directory, but by a
+			 * different route.
+			 */
+			SECURITY_ATTRIBUTES sa = {
+				.nLength = sizeof(SECURITY_ATTRIBUTES),
+				.lpSecurityDescriptor = NULL,
+				.bInheritHandle = false
+			};
+
+			if (CreateDirectory(path, &sa))
+				return 1;
+			_dosmaperr(GetLastError());
+#else
 			if (mkdir(path, 0700) >= 0)
 				return 1;
+#endif
 			if (errno != EEXIST)
 				return 0;
 		}
-- 
2.39.2

v3-0002-Always-use-AF_UNIX-sockets-in-pg_regress-on-Windo.patchtext/x-patch; charset=US-ASCII; name=v3-0002-Always-use-AF_UNIX-sockets-in-pg_regress-on-Windo.patchDownload
From 43ae66dc965d807dded8d434b7a1ea0b3f12e986 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Fri, 19 Aug 2022 11:28:38 +1200
Subject: [PATCH v3 2/2] Always use AF_UNIX sockets in pg_regress on Windows.

Since we can now rely on AF_UNIX sockets working on supported Windows
versions (10+), we can remove some Windows-only code for setting up
secure TCP in pg_regress.

Previously, we thought the socket cleanup code was unsafe, so we made
Unix-domain sockets an option with a "use-at-your-own-risk" note.  On
closer inspection, the concerns about signal handlers don't seem to
apply here.  (initdb.c has similar concerns but needs separate
investigation.)

Previously, commit f6dc6dd5 secured temporary installations using TCP/IP
on Windows, while commit be76a6d3 used file system permissions for Unix.
Now that our src/port/mkdtemp.c file creates non-world-accessible
directories on Windows, we can just do the same on Windows.

Note that this doesn't affect the TAP tests, which continue to use
TCP/IP on Windows by default.

Discussion: https://postgr.es/m/CA%2BhUKGK30uLx9dpgkYwomgH0WVLUHytkChDgf3iUM2zp0pf_nA%40mail.gmail.com
---
 src/test/regress/pg_regress.c | 274 ++++------------------------------
 1 file changed, 32 insertions(+), 242 deletions(-)

diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 7b23cc80dc..9c3251ca3b 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -55,6 +55,20 @@ char	   *host_platform = HOST_TUPLE;
 static char *shellprog = SHELLPROG;
 #endif
 
+/*
+ * The name of the environment variable that controls where we put temporary
+ * files, to override the defaut of "/tmp".
+ */
+#ifdef WIN32
+#define TMPDIR "TMP"
+#else
+#define TMPDIR "TMPDIR"
+#endif
+
+#if defined(WIN32) && defined(USE_ASSERT_CHECKING)
+static DWORD main_thread_id;
+#endif
+
 /*
  * On Windows we use -w in diff switches to avoid problems with inconsistent
  * newline representation.  The actual result files will generally have
@@ -285,9 +299,7 @@ stop_postmaster(void)
  * postmaster exit, so it is indeterminate whether the postmaster has yet to
  * unlink the socket and lock file.  Unlink them here so we can proceed to
  * remove the directory.  Ignore errors; leaking a temporary directory is
- * unimportant.  This can run from a signal handler.  The code is not
- * acceptable in a Windows signal handler (see initdb.c:trapsig()), but
- * on Windows, pg_regress does not use Unix sockets by default.
+ * unimportant.  This can run from a signal handler.
  */
 static void
 remove_temp(void)
@@ -304,6 +316,18 @@ remove_temp(void)
 static void
 signal_remove_temp(SIGNAL_ARGS)
 {
+#ifdef WIN32
+	/*
+	 * In general, it would not be acceptable to call remove_temp() in a
+	 * Windows signal handler.  It is safe in this program though, because
+	 * SIGHUP and SIGPIPE don't really exist and SIGTERM is never raised by the
+	 * system, leaving just SIGINT.  SIGINT doesn't interrupt the main
+	 * execution context on Windows, it runs the handler concurrently in
+	 * another thread.
+	 */
+	Assert(GetCurrentThreadId() != main_thread_id);
+#endif
+
 	remove_temp();
 
 	pqsignal(postgres_signal_arg, SIG_DFL);
@@ -326,7 +350,7 @@ static const char *
 make_temp_sockdir(void)
 {
 	char	   *template = psprintf("%s/pg_regress-XXXXXX",
-									getenv("TMPDIR") ? getenv("TMPDIR") : "/tmp");
+									getenv(TMPDIR) ? getenv(TMPDIR) : "/tmp");
 
 	temp_sockdir = mkdtemp(template);
 	if (temp_sockdir == NULL)
@@ -343,6 +367,10 @@ make_temp_sockdir(void)
 	/* Remove the directory during clean exit. */
 	atexit(remove_temp);
 
+#if defined(WIN32) && defined(USE_ASSERT_CHECKING)
+	main_thread_id = GetCurrentThreadId();
+#endif
+
 	/*
 	 * Remove the directory before dying to the usual signals.  Omit SIGQUIT,
 	 * preserving it as a quick, untidy exit.
@@ -753,211 +781,6 @@ initialize_environment(void)
 	load_resultmap();
 }
 
-#ifdef ENABLE_SSPI
-
-/* support for config_sspi_auth() */
-static const char *
-fmtHba(const char *raw)
-{
-	static char *ret;
-	const char *rp;
-	char	   *wp;
-
-	wp = ret = pg_realloc(ret, 3 + strlen(raw) * 2);
-
-	*wp++ = '"';
-	for (rp = raw; *rp; rp++)
-	{
-		if (*rp == '"')
-			*wp++ = '"';
-		*wp++ = *rp;
-	}
-	*wp++ = '"';
-	*wp++ = '\0';
-
-	return ret;
-}
-
-/*
- * Get account and domain/realm names for the current user.  This is based on
- * pg_SSPI_recvauth().  The returned strings use static storage.
- */
-static void
-current_windows_user(const char **acct, const char **dom)
-{
-	static char accountname[MAXPGPATH];
-	static char domainname[MAXPGPATH];
-	HANDLE		token;
-	TOKEN_USER *tokenuser;
-	DWORD		retlen;
-	DWORD		accountnamesize = sizeof(accountname);
-	DWORD		domainnamesize = sizeof(domainname);
-	SID_NAME_USE accountnameuse;
-
-	if (!OpenProcessToken(GetCurrentProcess(), TOKEN_READ, &token))
-	{
-		fprintf(stderr,
-				_("%s: could not open process token: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-
-	if (!GetTokenInformation(token, TokenUser, NULL, 0, &retlen) && GetLastError() != 122)
-	{
-		fprintf(stderr,
-				_("%s: could not get token information buffer size: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-	tokenuser = pg_malloc(retlen);
-	if (!GetTokenInformation(token, TokenUser, tokenuser, retlen, &retlen))
-	{
-		fprintf(stderr,
-				_("%s: could not get token information: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-
-	if (!LookupAccountSid(NULL, tokenuser->User.Sid, accountname, &accountnamesize,
-						  domainname, &domainnamesize, &accountnameuse))
-	{
-		fprintf(stderr,
-				_("%s: could not look up account SID: error code %lu\n"),
-				progname, GetLastError());
-		exit(2);
-	}
-
-	free(tokenuser);
-
-	*acct = accountname;
-	*dom = domainname;
-}
-
-/*
- * Rewrite pg_hba.conf and pg_ident.conf to use SSPI authentication.  Permit
- * the current OS user to authenticate as the bootstrap superuser and as any
- * user named in a --create-role option.
- *
- * In --config-auth mode, the --user switch can be used to specify the
- * bootstrap superuser's name, otherwise we assume it is the default.
- */
-static void
-config_sspi_auth(const char *pgdata, const char *superuser_name)
-{
-	const char *accountname,
-			   *domainname;
-	char	   *errstr;
-	bool		have_ipv6;
-	char		fname[MAXPGPATH];
-	int			res;
-	FILE	   *hba,
-			   *ident;
-	_stringlist *sl;
-
-	/* Find out the name of the current OS user */
-	current_windows_user(&accountname, &domainname);
-
-	/* Determine the bootstrap superuser's name */
-	if (superuser_name == NULL)
-	{
-		/*
-		 * Compute the default superuser name the same way initdb does.
-		 *
-		 * It's possible that this result always matches "accountname", the
-		 * value SSPI authentication discovers.  But the underlying system
-		 * functions do not clearly guarantee that.
-		 */
-		superuser_name = get_user_name(&errstr);
-		if (superuser_name == NULL)
-		{
-			fprintf(stderr, "%s: %s\n", progname, errstr);
-			exit(2);
-		}
-	}
-
-	/*
-	 * Like initdb.c:setup_config(), determine whether the platform recognizes
-	 * ::1 (IPv6 loopback) as a numeric host address string.
-	 */
-	{
-		struct addrinfo *gai_result;
-		struct addrinfo hints;
-		WSADATA		wsaData;
-
-		hints.ai_flags = AI_NUMERICHOST;
-		hints.ai_family = AF_UNSPEC;
-		hints.ai_socktype = 0;
-		hints.ai_protocol = 0;
-		hints.ai_addrlen = 0;
-		hints.ai_canonname = NULL;
-		hints.ai_addr = NULL;
-		hints.ai_next = NULL;
-
-		have_ipv6 = (WSAStartup(MAKEWORD(2, 2), &wsaData) == 0 &&
-					 getaddrinfo("::1", NULL, &hints, &gai_result) == 0);
-	}
-
-	/* Check a Write outcome and report any error. */
-#define CW(cond)	\
-	do { \
-		if (!(cond)) \
-		{ \
-			fprintf(stderr, _("%s: could not write to file \"%s\": %s\n"), \
-					progname, fname, strerror(errno)); \
-			exit(2); \
-		} \
-	} while (0)
-
-	res = snprintf(fname, sizeof(fname), "%s/pg_hba.conf", pgdata);
-	if (res < 0 || res >= sizeof(fname))
-	{
-		/*
-		 * Truncating this name is a fatal error, because we must not fail to
-		 * overwrite an original trust-authentication pg_hba.conf.
-		 */
-		fprintf(stderr, _("%s: directory name too long\n"), progname);
-		exit(2);
-	}
-	hba = fopen(fname, "w");
-	if (hba == NULL)
-	{
-		fprintf(stderr, _("%s: could not open file \"%s\" for writing: %s\n"),
-				progname, fname, strerror(errno));
-		exit(2);
-	}
-	CW(fputs("# Configuration written by config_sspi_auth()\n", hba) >= 0);
-	CW(fputs("host all all 127.0.0.1/32  sspi include_realm=1 map=regress\n",
-			 hba) >= 0);
-	if (have_ipv6)
-		CW(fputs("host all all ::1/128  sspi include_realm=1 map=regress\n",
-				 hba) >= 0);
-	CW(fclose(hba) == 0);
-
-	snprintf(fname, sizeof(fname), "%s/pg_ident.conf", pgdata);
-	ident = fopen(fname, "w");
-	if (ident == NULL)
-	{
-		fprintf(stderr, _("%s: could not open file \"%s\" for writing: %s\n"),
-				progname, fname, strerror(errno));
-		exit(2);
-	}
-	CW(fputs("# Configuration written by config_sspi_auth()\n", ident) >= 0);
-
-	/*
-	 * Double-quote for the benefit of account names containing whitespace or
-	 * '#'.  Windows forbids the double-quote character itself, so don't
-	 * bother escaping embedded double-quote characters.
-	 */
-	CW(fprintf(ident, "regress  \"%s@%s\"  %s\n",
-			   accountname, domainname, fmtHba(superuser_name)) >= 0);
-	for (sl = extraroles; sl; sl = sl->next)
-		CW(fprintf(ident, "regress  \"%s@%s\"  %s\n",
-				   accountname, domainname, fmtHba(sl->str)) >= 0);
-	CW(fclose(ident) == 0);
-}
-
-#endif							/* ENABLE_SSPI */
-
 /*
  * psql_start_command, psql_add_command, psql_end_command
  *
@@ -2015,7 +1838,6 @@ regression_main(int argc, char *argv[],
 		{NULL, 0, NULL, 0}
 	};
 
-	bool		use_unix_sockets;
 	_stringlist *sl;
 	int			c;
 	int			i;
@@ -2031,20 +1853,6 @@ regression_main(int argc, char *argv[],
 
 	atexit(stop_postmaster);
 
-#if defined(WIN32)
-
-	/*
-	 * We don't use Unix-domain sockets on Windows by default (see comment at
-	 * remove_temp() for a reason).  Override at your own risk.
-	 */
-	use_unix_sockets = getenv("PG_TEST_USE_UNIX_SOCKETS") ? true : false;
-#else
-	use_unix_sockets = true;
-#endif
-
-	if (!use_unix_sockets)
-		hostname = "localhost";
-
 	/*
 	 * We call the initialization function here because that way we can set
 	 * default parameters and let them be overwritten by the commandline.
@@ -2170,13 +1978,7 @@ regression_main(int argc, char *argv[],
 	}
 
 	if (config_auth_datadir)
-	{
-#ifdef ENABLE_SSPI
-		if (!use_unix_sockets)
-			config_sspi_auth(config_auth_datadir, user);
-#endif
 		exit(0);
-	}
 
 	if (temp_instance && !port_specified_by_user)
 
@@ -2295,18 +2097,6 @@ regression_main(int argc, char *argv[],
 
 		fclose(pg_conf);
 
-#ifdef ENABLE_SSPI
-		if (!use_unix_sockets)
-		{
-			/*
-			 * Since we successfully used the same buffer for the much-longer
-			 * "initdb" command, this can't truncate.
-			 */
-			snprintf(buf, sizeof(buf), "%s/data", temp_instance);
-			config_sspi_auth(buf, NULL);
-		}
-#endif
-
 		/*
 		 * Check if there is a postmaster running already.
 		 */
-- 
2.39.2

#11vignesh C
vignesh21@gmail.com
In reply to: Thomas Munro (#10)
Re: Using AF_UNIX sockets always for tests on Windows

On Wed, 22 Mar 2023 at 09:59, Thomas Munro <thomas.munro@gmail.com> wrote:

Thanks Tom, Andres, Juan José, Andrew for your feedback. I agree that
it's a better "OS harmonisation" outcome if we can choose both ways on
both OSes. I will leave the 0003 patch aside for now due to lack of
time, but here's a rebase of the first two patches. Since this is
really just more cleanup-obsolete-stuff background work, I'm going to
move it to the next CF.

0001 -- Teach mkdtemp() to care about permissions on Windows.
Reviewed by Juan José, who confirmed my blind-coded understanding and
showed an alternative version but didn't suggest doing it that way.
It's a little confusing that NULL "attributes" means something
different from NULL "descriptor", so I figured I should highlight that
difference more clearly with some new comments. I guess one question
is "why should we expect the calling process to have the default
access token?"

0002 -- Use AF_UNIX for pg_regress. This one removes a couple of
hundred Windows-only lines that set up SSPI stuff, and some comments
about a signal handling hazard that -- as far as I can tell by reading
manuals -- was never really true.

0003 -- TAP testing adjustments needs some more work based on feedback
already given, not included in this revision.

The patch does not apply anymore:
patch -p1 < v3-0002-Always-use-AF_UNIX-sockets-in-pg_regress-on-Windo.patch
patching file src/test/regress/pg_regress.c
...
Hunk #6 FAILED at 781.
...
1 out of 10 hunks FAILED -- saving rejects to file
src/test/regress/pg_regress.c.rej

Regards,
Vignesh

#12vignesh C
vignesh21@gmail.com
In reply to: vignesh C (#11)
Re: Using AF_UNIX sockets always for tests on Windows

On Sun, 21 Jan 2024 at 18:01, vignesh C <vignesh21@gmail.com> wrote:

On Wed, 22 Mar 2023 at 09:59, Thomas Munro <thomas.munro@gmail.com> wrote:

Thanks Tom, Andres, Juan José, Andrew for your feedback. I agree that
it's a better "OS harmonisation" outcome if we can choose both ways on
both OSes. I will leave the 0003 patch aside for now due to lack of
time, but here's a rebase of the first two patches. Since this is
really just more cleanup-obsolete-stuff background work, I'm going to
move it to the next CF.

0001 -- Teach mkdtemp() to care about permissions on Windows.
Reviewed by Juan José, who confirmed my blind-coded understanding and
showed an alternative version but didn't suggest doing it that way.
It's a little confusing that NULL "attributes" means something
different from NULL "descriptor", so I figured I should highlight that
difference more clearly with some new comments. I guess one question
is "why should we expect the calling process to have the default
access token?"

0002 -- Use AF_UNIX for pg_regress. This one removes a couple of
hundred Windows-only lines that set up SSPI stuff, and some comments
about a signal handling hazard that -- as far as I can tell by reading
manuals -- was never really true.

0003 -- TAP testing adjustments needs some more work based on feedback
already given, not included in this revision.

The patch does not apply anymore:
patch -p1 < v3-0002-Always-use-AF_UNIX-sockets-in-pg_regress-on-Windo.patch
patching file src/test/regress/pg_regress.c
...
Hunk #6 FAILED at 781.
...
1 out of 10 hunks FAILED -- saving rejects to file
src/test/regress/pg_regress.c.rej

With no update to the thread and the patch not applying I'm marking
this as returned with feedback. Please feel free to resubmit to the
next CF when there is a new version of the patch.

Regards,
Vignesh