Add support for EXTRA_REGRESS_OPTS for meson

Hi,

On 2025-02-27 13:53:17 +0100, Andreas Karlsson wrote:

We use EXTRA_REGRESS_OPTS to make sure the whole test suite passes with our
extension loaded and since I prefer develop in meson over using autotools
and make the lack of support for EXTRA_REGRESS_OPTS in meson
has bugged me for a while.

Yep, we should add support for that. TEMP_CONFIG probably too.

Question: Would it make sense to rename it to PG_REGRESS_EXTRA_OPTS or
something similar while we already touch touch this code to make the various
options easier to remember?

I'd not tackle that at the same time personally.

@@ -51,7 +52,12 @@ env_dict = {**os.environ,
if "PG_TEST_EXTRA" not in env_dict and args.pg_test_extra:
env_dict["PG_TEST_EXTRA"] = args.pg_test_extra

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg'] and 'EXTRA_REGRESS_OPTS' in env_dict:
+    test_command = args.test_command + shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)
# Meson categorizes a passing TODO test point as bad
# (https://github.com/mesonbuild/meson/issues/13183).  Remove the TODO
# directive, so Meson computes the file result like Perl does.  This could

I hacked up something similar before, for TEMP_CONFIG, and found that I needed
to do something like this:

+if 'TEMP_CONFIG' in os.environ and \
+    args.testname in ['regress', 'isolation', 'ecpg']:
+        # be careful to insert before non-option args, otherwise it'll fail
+        # e.g. on windows
+        args.test_command.insert(1, '--temp-config='+os.environ['TEMP_CONFIG'])

Greetings,

Andres Freund

On Thu, 27 Feb 2025 at 18:51, Andres Freund <andres@anarazel.de> wrote:

Hi,

On 2025-02-27 13:53:17 +0100, Andreas Karlsson wrote:

We use EXTRA_REGRESS_OPTS to make sure the whole test suite passes with our
extension loaded and since I prefer develop in meson over using autotools
and make the lack of support for EXTRA_REGRESS_OPTS in meson
has bugged me for a while.

Yep, we should add support for that. TEMP_CONFIG probably too.

Question: Would it make sense to rename it to PG_REGRESS_EXTRA_OPTS or
something similar while we already touch touch this code to make the various
options easier to remember?

I'd not tackle that at the same time personally.

@@ -51,7 +52,12 @@ env_dict = {**os.environ,
if "PG_TEST_EXTRA" not in env_dict and args.pg_test_extra:
env_dict["PG_TEST_EXTRA"] = args.pg_test_extra

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg'] and 'EXTRA_REGRESS_OPTS' in env_dict:
+    test_command = args.test_command + shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)
# Meson categorizes a passing TODO test point as bad
# (https://github.com/mesonbuild/meson/issues/13183).  Remove the TODO
# directive, so Meson computes the file result like Perl does.  This could

I hacked up something similar before, for TEMP_CONFIG, and found that I needed
to do something like this:

+if 'TEMP_CONFIG' in os.environ and \
+    args.testname in ['regress', 'isolation', 'ecpg']:
+        # be careful to insert before non-option args, otherwise it'll fail
+        # e.g. on windows
+        args.test_command.insert(1, '--temp-config='+os.environ['TEMP_CONFIG'])

Could you please upload a v2 to address this? In the meantime, I’ve
updated the commitfest entry status to 'Waiting on Author.' Kindly
update the status once the new version is posted.

Regards,
Vignesh

The following review has been posted through the commitfest application:
make installcheck-world: not tested
Implements feature: tested, failed
Spec compliant: tested, failed
Documentation: tested, failed

Hello everyone,
for v2 patch I suggest to refactor from

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg'] and 'EXTRA_REGRESS_OPTS' in env_dict:
+    test_command = args.test_command + shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)

to

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg']:
+    test_command = args.test_command[:]
+    if 'TEMP_CONFIG' in env_dict:
+        test_command.insert(1, '--temp-config=' + env_dict['TEMP_CONFIG'])
+    if 'EXTRA_REGRESS_OPTS' in env_dict:
+        test_command += shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)

I double checked whether shlex module was built in Python, and yes it is,
so no need for additional requirement.txt input for pip to install.

in addition to the above, might be worth to add some documentation like

Environment Variables Supported:
EXTRA_REGRESS_OPTS: Additional options to pass to regression, isolation, or ecpg tests.
TEMP_CONFIG: Specify a temporary configuration file for testing purposes.
Example Usage:
# Use EXTRA_REGRESS_OPTS to load an extension
EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test
# Use TEMP_CONFIG to specify a temporary configuration file
TEMP_CONFIG="path/to/test.conf" meson test
# Use both EXTRA_REGRESS_OPTS and TEMP_CONFIG together
TEMP_CONFIG="path/to/test.conf" EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test

Should we cover these new lines with test? Asking, because I see EXTRA_REGRESS_OPTS
being tested for autotools in src/interfaces/ecpg/test/makefile

Regards.

On 3/19/25 11:25 PM, Rustam ALLAKOV wrote:

The following review has been posted through the commitfest application:
make installcheck-world: not tested
Implements feature: tested, failed
Spec compliant: tested, failed
Documentation: tested, failed

Hello everyone,
for v2 patch I suggest to refactor from

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg'] and 'EXTRA_REGRESS_OPTS' in env_dict:
+    test_command = args.test_command + shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)

to

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg']:
+    test_command = args.test_command[:]
+    if 'TEMP_CONFIG' in env_dict:
+        test_command.insert(1, '--temp-config=' + env_dict['TEMP_CONFIG'])
+    if 'EXTRA_REGRESS_OPTS' in env_dict:
+        test_command += shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)

Since commit 51da766494dcc84b6f8d793ecaa064363a9243c2 it is possible
that we no longer need to use .insert() to make sure the code works on
Windows but I need to think a bit more on it.

But added support for TEMP_CONFIG.

I double checked whether shlex module was built in Python, and yes it is,
so no need for additional requirement.txt input for pip to install.

Thanks!

in addition to the above, might be worth to add some documentation like

Environment Variables Supported:
EXTRA_REGRESS_OPTS: Additional options to pass to regression, isolation, or ecpg tests.
TEMP_CONFIG: Specify a temporary configuration file for testing purposes.
Example Usage:
# Use EXTRA_REGRESS_OPTS to load an extension
EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test
# Use TEMP_CONFIG to specify a temporary configuration file
TEMP_CONFIG="path/to/test.conf" meson test
# Use both EXTRA_REGRESS_OPTS and TEMP_CONFIG together
TEMP_CONFIG="path/to/test.conf" EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test

Yeah, we probably should. But not sure where, maybe at
https://www.postgresql.org/docs/current/install-meson.html or did you
imagine somewhere else?

Should we cover these new lines with test? Asking, because I see EXTRA_REGRESS_OPTS
being tested for autotools in src/interfaces/ecpg/test/makefile

As far as I can see they are not tested there, but maybe we should test
them.

Attached version 2 of the patch.

Andreas

in addition to the above, might be worth to add some documentation like

Environment Variables Supported:
EXTRA_REGRESS_OPTS: Additional options to pass to regression, isolation, or ecpg tests.
TEMP_CONFIG: Specify a temporary configuration file for testing purposes.
Example Usage:
# Use EXTRA_REGRESS_OPTS to load an extension
EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test
# Use TEMP_CONFIG to specify a temporary configuration file
TEMP_CONFIG="path/to/test.conf" meson test
# Use both EXTRA_REGRESS_OPTS and TEMP_CONFIG together
TEMP_CONFIG="path/to/test.conf" EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test

Yeah, we probably should. But not sure where, maybe at
https://www.postgresql.org/docs/current/install-meson.html or did you
imagine somewhere else?

Either at docs or/and directly in file /src/tools/testwrap as comments
Regards.

On 1/13/26 11:06 PM, Rustam ALLAKOV wrote:

in addition to the above, might be worth to add some documentation like

Environment Variables Supported:
EXTRA_REGRESS_OPTS: Additional options to pass to regression, isolation, or ecpg tests.
TEMP_CONFIG: Specify a temporary configuration file for testing purposes.
Example Usage:
# Use EXTRA_REGRESS_OPTS to load an extension
EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test
# Use TEMP_CONFIG to specify a temporary configuration file
TEMP_CONFIG="path/to/test.conf" meson test
# Use both EXTRA_REGRESS_OPTS and TEMP_CONFIG together
TEMP_CONFIG="path/to/test.conf" EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test

Yeah, we probably should. But not sure where, maybe at
https://www.postgresql.org/docs/current/install-meson.html or did you
imagine somewhere else?

Either at docs or/and directly in file /src/tools/testwrap as comments
Regards.

If we add it to the documentation the question is if we should also
document environment variables for builds with make. But maybe that is
for a separate patch.

As for documenting it in testwrap: that does not feel very useful to me
since people who know that this is where they should look likely already
knows about these. It is a pretty hidden place. Hmm, not sure where such
information belongs.

Andreas

On Wed, Jan 14, 2026 at 02:17:30AM +0100, Andreas Karlsson wrote:

If we add it to the documentation the question is if we should also document
environment variables for builds with make. But maybe that is for a separate
patch.

As for documenting it in testwrap: that does not feel very useful to me
since people who know that this is where they should look likely already
knows about these. It is a pretty hidden place. Hmm, not sure where such
information belongs.

On point is that this also make the gap in the documentation bigger,
and if we are just doing something, why not tackle it now. It is true
that TESTS is not documented as something that can work in meson, and
that it is not the fault of this patch, but I think that we should
spend a bit of time on the shape of an extra effort for:
1) closing the gap with the existing variables before adding more
stuff.
2) adding documentation for the new things added to testwrap, and
require that documentation is added for everything in the future.

If somebody sends a patch doing 1) for the existing things, and adds
the documentation of the new variables, satisfying 2), I would be OK
to merge the proposal (after checking it of course).

One issue for me with meson is that too many things feel implied, like
the regression test facilities that we make the effort to document for
configure/make.
--
Michael

On 3/10/26 7:18 AM, Michael Paquier wrote:

If somebody sends a patch doing 1) for the existing things, and adds
the documentation of the new variables, satisfying 2), I would be OK
to merge the proposal (after checking it of course).

One issue for me with meson is that too many things feel implied, like
the regression test facilities that we make the effort to document for
configure/make.

Agreed, the question is just where we should document them.

I would say one of these two pages:

https://www.postgresql.org/docs/current/install-meson.html
https://www.postgresql.org/docs/current/regress-run.html

Andreas

On Tue, Mar 31, 2026 at 10:45:32AM +0200, Andreas Karlsson wrote:

Agreed, the question is just where we should document them.

I would say one of these two pages:

https://www.postgresql.org/docs/current/install-meson.html
https://www.postgresql.org/docs/current/regress-run.html

regress-run would be a nicer fit in the long run, where it seems to me
that the existing documentation is wrong with its decision to document
the regression tests that can be run in meson in a section for the
build and install:
https://www.postgresql.org/docs/current/install-meson.html#INSTALL-PROCEDURE-MESON

The format that we use for the coverage code is something I find
pretty clear:
https://www.postgresql.org/docs/current/regress-coverage.html

We have two sub-sections, one for configure/make and one meson, which
is nice. Perhaps we don't need to have new sub-sections, limiting
ourselves to mention some of the equivalents that can be used with
meson?
--
Michael