Add support for EXTRA_REGRESS_OPTS for meson

Started by Andreas Karlsson11 months ago5 messages

andreas@proxel.se

11 months ago

1 attachment(s)

Hi,

We use EXTRA_REGRESS_OPTS to make sure the whole test suite passes with
our extension loaded and since I prefer develop in meson over using
autotools and make the lack of support for EXTRA_REGRESS_OPTS in meson
has bugged me for a while.

I have implemented support for it as an environment variable we read in
the testwrap script instead of adding it as a configuration option to
meson.build. The reason for this is that I do not like having to run
"meson reconfigure" all the time plus that for the PG_TEST_EXTRA we
ended up having to add an environment variable anyway.

To use this run e.g. the following:

EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test

Question: Would it make sense to rename it to PG_REGRESS_EXTRA_OPTS or
something similar while we already touch touch this code to make the
various options easier to remember?

Andreas

Attachments:

0001-meson-Add-support-for-EXTRA_REGRESS_OPTS.patchtext/x-patch; charset=UTF-8; name=0001-meson-Add-support-for-EXTRA_REGRESS_OPTS.patchDownload

From 87ce622a19315b679bbd5691e01c96261bc0c4c8 Mon Sep 17 00:00:00 2001
From: Andreas Karlsson <andreas@proxel.se>
Date: Thu, 27 Feb 2025 00:23:44 +0100
Subject: [PATCH] meson: Add support for EXTRA_REGRESS_OPTS

Add support for the EXTRA_REGRESS_OPTS environment variable in meson
which works just like with make and applies to all regress, ecpg and
isolation tests. TAP tests which support it under make will continue
to support the option.

Run it with e.g:

    EXTRA_REGRESS_OPTS="--temp-config=test.conf" meson test
---
 src/tools/testwrap | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/src/tools/testwrap b/src/tools/testwrap
index 8ae8fb79ba7..f52ca376da1 100755
--- a/src/tools/testwrap
+++ b/src/tools/testwrap
@@ -2,6 +2,7 @@
 
 import argparse
 import shutil
+import shlex
 import subprocess
 import os
 import sys
@@ -51,7 +52,12 @@ env_dict = {**os.environ,
 if "PG_TEST_EXTRA" not in env_dict and args.pg_test_extra:
     env_dict["PG_TEST_EXTRA"] = args.pg_test_extra
 
-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg'] and 'EXTRA_REGRESS_OPTS' in env_dict:
+    test_command = args.test_command + shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)
 # Meson categorizes a passing TODO test point as bad
 # (https://github.com/mesonbuild/meson/issues/13183).  Remove the TODO
 # directive, so Meson computes the file result like Perl does.  This could
-- 
2.47.2

Andres Freund

andres@anarazel.de

11 months ago

In reply to: Andreas Karlsson (#1)

Re: Add support for EXTRA_REGRESS_OPTS for meson

Hi,

On 2025-02-27 13:53:17 +0100, Andreas Karlsson wrote:

We use EXTRA_REGRESS_OPTS to make sure the whole test suite passes with our
extension loaded and since I prefer develop in meson over using autotools
and make the lack of support for EXTRA_REGRESS_OPTS in meson
has bugged me for a while.

Yep, we should add support for that. TEMP_CONFIG probably too.

Question: Would it make sense to rename it to PG_REGRESS_EXTRA_OPTS or
something similar while we already touch touch this code to make the various
options easier to remember?

I'd not tackle that at the same time personally.

@@ -51,7 +52,12 @@ env_dict = {**os.environ,
if "PG_TEST_EXTRA" not in env_dict and args.pg_test_extra:
env_dict["PG_TEST_EXTRA"] = args.pg_test_extra

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg'] and 'EXTRA_REGRESS_OPTS' in env_dict:
+    test_command = args.test_command + shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)
# Meson categorizes a passing TODO test point as bad
# (https://github.com/mesonbuild/meson/issues/13183).  Remove the TODO
# directive, so Meson computes the file result like Perl does.  This could

I hacked up something similar before, for TEMP_CONFIG, and found that I needed
to do something like this:

+if 'TEMP_CONFIG' in os.environ and \
+    args.testname in ['regress', 'isolation', 'ecpg']:
+        # be careful to insert before non-option args, otherwise it'll fail
+        # e.g. on windows
+        args.test_command.insert(1, '--temp-config='+os.environ['TEMP_CONFIG'])

Greetings,

Andres Freund

vignesh C

vignesh21@gmail.com

10 months ago

In reply to: Andres Freund (#2)

Re: Add support for EXTRA_REGRESS_OPTS for meson

On Thu, 27 Feb 2025 at 18:51, Andres Freund <andres@anarazel.de> wrote:

Hi,

On 2025-02-27 13:53:17 +0100, Andreas Karlsson wrote:

We use EXTRA_REGRESS_OPTS to make sure the whole test suite passes with our
extension loaded and since I prefer develop in meson over using autotools
and make the lack of support for EXTRA_REGRESS_OPTS in meson
has bugged me for a while.

Yep, we should add support for that. TEMP_CONFIG probably too.

Question: Would it make sense to rename it to PG_REGRESS_EXTRA_OPTS or
something similar while we already touch touch this code to make the various
options easier to remember?

I'd not tackle that at the same time personally.
@@ -51,7 +52,12 @@ env_dict = {**os.environ,
if "PG_TEST_EXTRA" not in env_dict and args.pg_test_extra:
env_dict["PG_TEST_EXTRA"] = args.pg_test_extra
-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg'] and 'EXTRA_REGRESS_OPTS' in env_dict:
+    test_command = args.test_command + shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)
# Meson categorizes a passing TODO test point as bad
# (https://github.com/mesonbuild/meson/issues/13183).  Remove the TODO
# directive, so Meson computes the file result like Perl does.  This could
I hacked up something similar before, for TEMP_CONFIG, and found that I needed
to do something like this:
+if 'TEMP_CONFIG' in os.environ and \
+    args.testname in ['regress', 'isolation', 'ecpg']:
+        # be careful to insert before non-option args, otherwise it'll fail
+        # e.g. on windows
+        args.test_command.insert(1, '--temp-config='+os.environ['TEMP_CONFIG'])

Could you please upload a v2 to address this? In the meantime, I’ve
updated the commitfest entry status to 'Waiting on Author.' Kindly
update the status once the new version is posted.

Regards,
Vignesh

Rustam ALLAKOV

rustamallakov@gmail.com

10 months ago

In reply to: vignesh C (#3)

Re: Add support for EXTRA_REGRESS_OPTS for meson

The following review has been posted through the commitfest application:
make installcheck-world: not tested
Implements feature: tested, failed
Spec compliant: tested, failed
Documentation: tested, failed

Hello everyone,
for v2 patch I suggest to refactor from

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg'] and 'EXTRA_REGRESS_OPTS' in env_dict:
+    test_command = args.test_command + shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg']:
+    test_command = args.test_command[:]
+    if 'TEMP_CONFIG' in env_dict:
+        test_command.insert(1, '--temp-config=' + env_dict['TEMP_CONFIG'])
+    if 'EXTRA_REGRESS_OPTS' in env_dict:
+        test_command += shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)

I double checked whether shlex module was built in Python, and yes it is,
so no need for additional requirement.txt input for pip to install.

in addition to the above, might be worth to add some documentation like

Environment Variables Supported:
EXTRA_REGRESS_OPTS: Additional options to pass to regression, isolation, or ecpg tests.
TEMP_CONFIG: Specify a temporary configuration file for testing purposes.
Example Usage:
# Use EXTRA_REGRESS_OPTS to load an extension
EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test
# Use TEMP_CONFIG to specify a temporary configuration file
TEMP_CONFIG="path/to/test.conf" meson test
# Use both EXTRA_REGRESS_OPTS and TEMP_CONFIG together
TEMP_CONFIG="path/to/test.conf" EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test

Should we cover these new lines with test? Asking, because I see EXTRA_REGRESS_OPTS
being tested for autotools in src/interfaces/ecpg/test/makefile

Regards.

Andreas Karlsson

andreas@proxel.se

12 days ago

In reply to: Rustam ALLAKOV (#4)

1 attachment(s)

Re: Add support for EXTRA_REGRESS_OPTS for meson

On 3/19/25 11:25 PM, Rustam ALLAKOV wrote:

Hello everyone,
for v2 patch I suggest to refactor from

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg'] and 'EXTRA_REGRESS_OPTS' in env_dict:
+    test_command = args.test_command + shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg']:
+    test_command = args.test_command[:]
+    if 'TEMP_CONFIG' in env_dict:
+        test_command.insert(1, '--temp-config=' + env_dict['TEMP_CONFIG'])
+    if 'EXTRA_REGRESS_OPTS' in env_dict:
+        test_command += shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)

Since commit 51da766494dcc84b6f8d793ecaa064363a9243c2 it is possible
that we no longer need to use .insert() to make sure the code works on
Windows but I need to think a bit more on it.

But added support for TEMP_CONFIG.

I double checked whether shlex module was built in Python, and yes it is,
so no need for additional requirement.txt input for pip to install.

Thanks!

in addition to the above, might be worth to add some documentation like

Environment Variables Supported:
EXTRA_REGRESS_OPTS: Additional options to pass to regression, isolation, or ecpg tests.
TEMP_CONFIG: Specify a temporary configuration file for testing purposes.
Example Usage:
# Use EXTRA_REGRESS_OPTS to load an extension
EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test
# Use TEMP_CONFIG to specify a temporary configuration file
TEMP_CONFIG="path/to/test.conf" meson test
# Use both EXTRA_REGRESS_OPTS and TEMP_CONFIG together
TEMP_CONFIG="path/to/test.conf" EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test

Yeah, we probably should. But not sure where, maybe at
https://www.postgresql.org/docs/current/install-meson.html or did you
imagine somewhere else?

Should we cover these new lines with test? Asking, because I see EXTRA_REGRESS_OPTS
being tested for autotools in src/interfaces/ecpg/test/makefile

As far as I can see they are not tested there, but maybe we should test
them.

Attached version 2 of the patch.

Andreas

Attachments:

v2-0002-meson-Add-support-for-EXTRA_REGRESS_OPTS-and-TEMP.patchtext/x-patch; charset=UTF-8; name=v2-0002-meson-Add-support-for-EXTRA_REGRESS_OPTS-and-TEMP.patchDownload

From 0e73ab29af1ccab9c4dd7453f07b41a4527b0340 Mon Sep 17 00:00:00 2001
From: Andreas Karlsson <andreas@proxel.se>
Date: Wed, 31 Dec 2025 01:48:56 +0100
Subject: [PATCH v2 2/2] meson: Add support for EXTRA_REGRESS_OPTS and
 TEMP_CONFIG

Add support for the EXTRA_REGRESS_OPTS and TEMP_CONFIG environment
variables in our Meson build which work just like with make and
apply to all regress, ecpg and isolation tests.
---
 src/tools/testwrap | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/src/tools/testwrap b/src/tools/testwrap
index e91296ecd15..165d35fba98 100755
--- a/src/tools/testwrap
+++ b/src/tools/testwrap
@@ -4,6 +4,7 @@ import argparse
 import shutil
 import subprocess
 import os
+import shlex
 import sys
 
 parser = argparse.ArgumentParser()
@@ -53,6 +54,14 @@ env_dict = {**os.environ,
 if "PG_TEST_EXTRA" not in env_dict and args.pg_test_extra:
     env_dict["PG_TEST_EXTRA"] = args.pg_test_extra
 
+# Add extra regress arguments before we add non-option arguments
+if args.testname in ['regress', 'isolation', 'ecpg']:
+    if 'TEMP_CONFIG' in env_dict:
+        args.test_command += ['--temp-config=' + env_dict['TEMP_CONFIG']]
+
+    if 'EXTRA_REGRESS_OPTS' in env_dict:
+        args.test_command += shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+
 if "TESTS" in env_dict:
     args.test_command += env_dict["TESTS"].split()
 else:
-- 
2.47.3