Document how to run only specific regress tests

On 29 Oct 2024, at 05:40, Paul A Jungwirth <pj@illuminatedcomputing.com> wrote:

I never knew this was possible, but I ran across a note on the meson
wiki page[1]. I wish I knew about it years ago!

This patch adds a note to
https://www.postgresql.org/docs/current/regress-run.html so that other
hackers and potential hackers will know about it.

While not related to this patch per se, the lack of any meson related
information on that page isn't great.

+    Note that sometimes tests depend on each other, so you might get
+    extra failures.

Reading this pretending I don't know about regression test internals, I think
"extra" would be hard to interpret the meaning of. How about "unexpected"?

--
Daniel Gustafsson

On Tue, Oct 29, 2024 at 12:40 PM Paul A Jungwirth
<pj@illuminatedcomputing.com> wrote:

I never knew this was possible, but I ran across a note on the meson
wiki page[1]. I wish I knew about it years ago!

This patch adds a note to
https://www.postgresql.org/docs/current/regress-run.html so that other
hackers and potential hackers will know about it.

[1] https://wiki.postgresql.org/wiki/Meson

Thanks for your patch!
you can also check meson TESTS patch in
/messages/by-id/CAN55FZ3dSdvujT8yup1C=YOfTATrBrWcLtHmtXupcbog7Va-Xw@mail.gmail.com

Since this is mostly for postgres development, slightly expanding it
would be better.
I think we can also mention that:
1. all the available regress tests are on src/test/regress/sql.
2. regress test name is the same as these files omit file format.
For example "src/test/regress/sql/partition_join.sql" corresponding
regression test name is partition_join
3. individual tests will be executed in that specific order.
TESTS="test_setup copy'
will first execute 'test_setup' then 'copy'
4. Many tests depend on test_setup, some tests depend on create_index.
5. specify duplicate tests may make the tests fail.

Daniel Gustafsson <daniel@yesql.se> writes:

While not related to this patch per se, the lack of any meson related
information on that page isn't great.

Indeed. I think this is one of the main unfinished bits of business
for our meson conversion. It's not clear to me that our meson
infrastructure can do all the same testing tricks that the makefiles
can, and for sure it's not documented how.

regards, tom lane

hi.

maybe verbose, the following is the my intended changes based on your patch.
what do you think?

<sect2 id="regress-subset">
<title>Running Specific Tests</title>
<para>
You can run a subset of the regression tests through the <envar>TESTS</envar>
environment variable.
One way to do it is set <envar>TESTS</envar> environment variable beforehand:
<screen>
<userinput>export TESTS="test_setup boolean char"</userinput>
</screen>
You can also specify it while invoking the regress tests command,
one of example:
<screen>
make check-tests TESTS="test_setup boolean char"
</screen>
If the <envar>TESTS</envar> environment variable is set when the regression
tests are run, it overrides the original <envar>TESTS</envar> environment
variable.
</para>
<para>
All the regression tests are stored in <filename
class='directory'>src/test/regress/sql</filename>. Each
individual regression test
name is the corresponding file name omit the file name extension
(<literal>.sql</literal>). For example, the regression test name of
<literal>src/test/regress/sql/partition_join.sql</literal> is
<literal>partition_join</literal>. Tests specified by the
<envar>TESTS</envar>
environment variable will be run in a sequential order.
<literal>TESTS="test_setup copy' </literal> will first run
<literal>test_setup</literal> then <literal>copy</literal>. Note that
sometimes one test may depend on other tests, so you might get unexpected
failures if the dependent test is not specified. For example, to run
<literal>select_parallel</literal> test, to avoid failure, you can use
<screen>
make check-tests TESTS="test_setup create_misc create_index select_parallel"
</screen>
</para>
</sect2>