[PATCH] Allow TAP tests to be run individually
Per prior discussion, this small Makefile change allows anything that
uses $(prove_check) or $(prove_installcheck) to have the list of tests
overridden by passing a PROVE_TESTS as a list of space-separated test
file paths.
Particularly handy for src/test/recovery when you want to run
t/009_something_new.pl without running 001 through 008 each and every
time. The current workaround is deleting the tests you don't want,
which dirties up your git tree and is really annoying when checking
out and rebasing etc.
Please backpatch to at least 9.6 since it's trivial and we seem to be
doing that for TAP. 9.5 and 9.4 would be nice too :)
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
0001-Allow-individual-TAP-tests-to-be-run-via-PROVE_TESTS.patchtext/x-patch; charset=US-ASCII; name=0001-Allow-individual-TAP-tests-to-be-run-via-PROVE_TESTS.patchDownload
From 1180cfaca9b92c401095f9766cbb46e2ca4371f3 Mon Sep 17 00:00:00 2001
From: Craig Ringer <craig@2ndquadrant.com>
Date: Fri, 11 Nov 2016 16:21:41 +0800
Subject: [PATCH 1/2] Allow individual TAP tests to be run via PROVE_TESTS
Add a new optional Makefile variable PROVE_TESTS that, if passed
as a space-separated list of paths relative to the Makefile
invoking $(prove_check) or $(prove_installcheck), runs just
those tests instead of t/*.pl .
---
src/Makefile.global.in | 4 ++--
src/test/perl/README | 6 ++++++
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index ea61eb5..c5036d8 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -354,12 +354,12 @@ ifeq ($(enable_tap_tests),yes)
define prove_installcheck
rm -rf $(CURDIR)/tmp_check/log
-cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
+cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(or $(PROVE_TESTS),t/*.pl)
endef
define prove_check
rm -rf $(CURDIR)/tmp_check/log
-cd $(srcdir) && TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
+cd $(srcdir) && TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(or $(PROVE_TESTS),t/*.pl)
endef
else
diff --git a/src/test/perl/README b/src/test/perl/README
index 710a0d8..cfb45a1 100644
--- a/src/test/perl/README
+++ b/src/test/perl/README
@@ -6,6 +6,12 @@ across the source tree, particularly tests in src/bin and src/test. It's used
to drive tests for backup and restore, replication, etc - anything that can't
really be expressed using pg_regress or the isolation test framework.
+The tests are invoked via perl's 'prove' command, wrapped in PostgreSQL
+makefiles to handle instance setup etc. See the $(prove_check) and
+$(prove_installcheck) targets in Makefile.global. By default every test in the
+t/ subdirectory is run. Individual test(s) can be run instead by passing
+something like PROVE_TESTS="t/001_testname.pl t/002_othertestname.pl" to make.
+
You should prefer to write tests using pg_regress in src/test/regress, or
isolation tester specs in src/test/isolation, if possible. If not, check to
see if your new tests make sense under an existing tree in src/test, like
--
2.5.5
On Fri, Nov 11, 2016 at 6:10 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Please backpatch to at least 9.6 since it's trivial and we seem to be
doing that for TAP. 9.5 and 9.4 would be nice too :)
Yes please!
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 11 November 2016 at 18:13, Michael Paquier <michael.paquier@gmail.com> wrote:
On Fri, Nov 11, 2016 at 6:10 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Please backpatch to at least 9.6 since it's trivial and we seem to be
doing that for TAP. 9.5 and 9.4 would be nice too :)Yes please!
No immediate takers, so adding to CF.
I've taken the liberty of adding you as a reviewer based on your
response and the simplicity of the patch. if you get the chance to
test and verify please set ready for committer.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Nov 14, 2016 at 3:45 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 11 November 2016 at 18:13, Michael Paquier <michael.paquier@gmail.com> wrote:
On Fri, Nov 11, 2016 at 6:10 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Please backpatch to at least 9.6 since it's trivial and we seem to be
doing that for TAP. 9.5 and 9.4 would be nice too :)Yes please!
No immediate takers, so adding to CF.
I've taken the liberty of adding you as a reviewer based on your
response and the simplicity of the patch. if you get the chance to
test and verify please set ready for committer.
I don't mind. This patch uses the following pattern:
$(or $(PROVE_TESTS),t/*.pl)
While something more spread in Postgres source would be something like that:
$(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
It seems to me that we'd prefer that for consistency, but I see no
reason to not keep your patch as well. I am marking that as ready for
committer.
--
Michael
Attachments:
prove-tests-v2.patchtext/x-diff; charset=US-ASCII; name=prove-tests-v2.patchDownload
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index ea61eb5..aa1fa65 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -354,12 +354,12 @@ ifeq ($(enable_tap_tests),yes)
define prove_installcheck
rm -rf $(CURDIR)/tmp_check/log
-cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
+cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
define prove_check
rm -rf $(CURDIR)/tmp_check/log
-cd $(srcdir) && TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
+cd $(srcdir) && TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
else
diff --git a/src/test/perl/README b/src/test/perl/README
index 710a0d8..cfb45a1 100644
--- a/src/test/perl/README
+++ b/src/test/perl/README
@@ -6,6 +6,12 @@ across the source tree, particularly tests in src/bin and src/test. It's used
to drive tests for backup and restore, replication, etc - anything that can't
really be expressed using pg_regress or the isolation test framework.
+The tests are invoked via perl's 'prove' command, wrapped in PostgreSQL
+makefiles to handle instance setup etc. See the $(prove_check) and
+$(prove_installcheck) targets in Makefile.global. By default every test in the
+t/ subdirectory is run. Individual test(s) can be run instead by passing
+something like PROVE_TESTS="t/001_testname.pl t/002_othertestname.pl" to make.
+
You should prefer to write tests using pg_regress in src/test/regress, or
isolation tester specs in src/test/isolation, if possible. If not, check to
see if your new tests make sense under an existing tree in src/test, like
On 14 November 2016 at 16:52, Michael Paquier <michael.paquier@gmail.com> wrote:
On Mon, Nov 14, 2016 at 3:45 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 11 November 2016 at 18:13, Michael Paquier <michael.paquier@gmail.com> wrote:
On Fri, Nov 11, 2016 at 6:10 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
Please backpatch to at least 9.6 since it's trivial and we seem to be
doing that for TAP. 9.5 and 9.4 would be nice too :)Yes please!
No immediate takers, so adding to CF.
I've taken the liberty of adding you as a reviewer based on your
response and the simplicity of the patch. if you get the chance to
test and verify please set ready for committer.I don't mind. This patch uses the following pattern:
$(or $(PROVE_TESTS),t/*.pl)
While something more spread in Postgres source would be something like that:
$(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
It seems to me that we'd prefer that for consistency, but I see no
reason to not keep your patch as well. I am marking that as ready for
committer.
Thanks.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On 11/14/16 3:52 AM, Michael Paquier wrote:
I don't mind. This patch uses the following pattern:
$(or $(PROVE_TESTS),t/*.pl)
While something more spread in Postgres source would be something like that:
$(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
It seems to me that we'd prefer that for consistency, but I see no
reason to not keep your patch as well. I am marking that as ready for
committer.
($or ...) is a newer feature of GNU make, so we have avoided that so
far. I have committed your v2 with $(if ...).
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Tue, Nov 15, 2016 at 12:02 AM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
On 11/14/16 3:52 AM, Michael Paquier wrote:
I don't mind. This patch uses the following pattern:
$(or $(PROVE_TESTS),t/*.pl)
While something more spread in Postgres source would be something like that:
$(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
It seems to me that we'd prefer that for consistency, but I see no
reason to not keep your patch as well. I am marking that as ready for
committer.($or ...) is a newer feature of GNU make, so we have avoided that so
far. I have committed your v2 with $(if ...).
Thanks, I am just going to use it...
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers