running make check with only specified tests

Started by Andrew Dunstanalmost 12 years ago8 messages

andrew@dunslane.net

almost 12 years ago

1 attachment(s)

I've often wanted to be able to run "make check" and just have it run
the small number of tests I am interested in. Here's a tiny patch along
those lines. It creates a new targe which I have called "check-with" for
want of a better name. And with it I can do:

$ make check-with TESTS="json jsonb"

and have it do the temp install etc and then run just those two tests.

Thoughts?

cheers

andrew

Attachments:

check-with.patchtext/x-patch; name=check-with.patchDownload

diff --git a/GNUmakefile.in b/GNUmakefile.in
index 80116a1..32dd9bf 100644
--- a/GNUmakefile.in
+++ b/GNUmakefile.in
@@ -61,9 +61,9 @@ distclean maintainer-clean:
 # Garbage from autoconf:
 	@rm -rf autom4te.cache/
 
-check: all
+check check-with: all
 
-check installcheck installcheck-parallel:
+check check-with installcheck installcheck-parallel:
 	$(MAKE) -C src/test/regress $@
 
 $(call recurse,check-world,src/test src/pl src/interfaces/ecpg contrib,check)
diff --git a/src/test/regress/GNUmakefile b/src/test/regress/GNUmakefile
index 94762d5..4165a7d 100644
--- a/src/test/regress/GNUmakefile
+++ b/src/test/regress/GNUmakefile
@@ -142,6 +142,9 @@ REGRESS_OPTS = --dlpath=. $(EXTRA_REGRESS_OPTS)
 check: all tablespace-setup
 	$(pg_regress_check) $(REGRESS_OPTS) --schedule=$(srcdir)/parallel_schedule $(MAXCONNOPT) $(TEMP_CONF) $(EXTRA_TESTS)
 
+check-with: all tablespace-setup
+	$(pg_regress_check) $(REGRESS_OPTS) $(MAXCONNOPT) $(TEMP_CONF) $(TESTS) $(EXTRA_TESTS)
+
 installcheck: all tablespace-setup
 	$(pg_regress_installcheck) $(REGRESS_OPTS) --schedule=$(srcdir)/serial_schedule $(EXTRA_TESTS)

Pavel Stehule

pavel.stehule@gmail.com

almost 12 years ago

In reply to: Andrew Dunstan (#1)

Re: running make check with only specified tests

2014-01-26 Andrew Dunstan <andrew@dunslane.net>

I've often wanted to be able to run "make check" and just have it run the
small number of tests I am interested in. Here's a tiny patch along those
lines. It creates a new targe which I have called "check-with" for want of
a better name. And with it I can do:

$ make check-with TESTS="json jsonb"

Pavel

Show quoted text

and have it do the temp install etc and then run just those two tests.

Thoughts?

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Florian Pflug

fgp@phlo.org

almost 12 years ago

In reply to: Andrew Dunstan (#1)

Re: running make check with only specified tests

On Jan26, 2014, at 17:47 , Andrew Dunstan <andrew@dunslane.net> wrote:

I've often wanted to be able to run "make check" and just have it run the small number of tests I am interested in. Here's a tiny patch along those lines. It creates a new targe which I have called "check-with" for want of a better name. And with it I can do:

$ make check-with TESTS="json jsonb"

and have it do the temp install etc and then run just those two tests.

+1 for the feature (+Inf, actually), but will this work if the tests
depend on stuff created by other tests?

best regards,
Florian Pflug

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andrew Dunstan

andrew@dunslane.net

almost 12 years ago

In reply to: Florian Pflug (#3)

Re: running make check with only specified tests

On 01/26/2014 12:01 PM, Florian Pflug wrote:

On Jan26, 2014, at 17:47 , Andrew Dunstan <andrew@dunslane.net> wrote:

I've often wanted to be able to run "make check" and just have it run the small number of tests I am interested in. Here's a tiny patch along those lines. It creates a new targe which I have called "check-with" for want of a better name. And with it I can do:

$ make check-with TESTS="json jsonb"

and have it do the temp install etc and then run just those two tests.

+1 for the feature (+Inf, actually), but will this work if the tests
depend on stuff created by other tests?

No, if they do it will be up to you to include those in your test list
in the right order.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Tom Lane

tgl@sss.pgh.pa.us

almost 12 years ago

In reply to: Andrew Dunstan (#1)

Re: running make check with only specified tests

Andrew Dunstan <andrew@dunslane.net> writes:

I've often wanted to be able to run "make check" and just have it run
the small number of tests I am interested in. Here's a tiny patch along
those lines. It creates a new targe which I have called "check-with" for
want of a better name. And with it I can do:
$ make check-with TESTS="json jsonb"

The vast majority of the regression tests have interdependencies, which
would make any feature along these lines fairly useless IME. (And no,
I'm not interested in converting the tests to a no-dependencies style.)

Also, the tests themselves don't take that long, especially in parallel
mode. If you need to speed up repeated testing, it's more profitable to
avoid the install/initdb overhead of a "make check". I use a small
script that just reinstalls the postgres executable and does "make
installcheck-parallel" when I'm doing iterative development.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andrew Dunstan

andrew@dunslane.net

almost 12 years ago

In reply to: Tom Lane (#5)

Re: running make check with only specified tests

On 01/26/2014 12:08 PM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

I've often wanted to be able to run "make check" and just have it run
the small number of tests I am interested in. Here's a tiny patch along
those lines. It creates a new targe which I have called "check-with" for
want of a better name. And with it I can do:
$ make check-with TESTS="json jsonb"

The vast majority of the regression tests have interdependencies, which
would make any feature along these lines fairly useless IME. (And no,
I'm not interested in converting the tests to a no-dependencies style.)

Also, the tests themselves don't take that long, especially in parallel
mode. If you need to speed up repeated testing, it's more profitable to
avoid the install/initdb overhead of a "make check". I use a small
script that just reinstalls the postgres executable and does "make
installcheck-parallel" when I'm doing iterative development.

I have something similar, and prodded by your email I've just improved
it a bit ;-) But it doesn't work so well if you're changing the catalog,
as you need an initdb anyway. And there are are some cases (the one I
happen to be working on being one of them) where the tests have no prior
dependencies.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Tom Lane

tgl@sss.pgh.pa.us

almost 12 years ago

In reply to: Andrew Dunstan (#6)

Re: running make check with only specified tests

Andrew Dunstan <andrew@dunslane.net> writes:

On 01/26/2014 12:08 PM, Tom Lane wrote:

Also, the tests themselves don't take that long, especially in parallel
mode. If you need to speed up repeated testing, it's more profitable to
avoid the install/initdb overhead of a "make check". I use a small
script that just reinstalls the postgres executable and does "make
installcheck-parallel" when I'm doing iterative development.

I have something similar, and prodded by your email I've just improved
it a bit ;-) But it doesn't work so well if you're changing the catalog,
as you need an initdb anyway.

True. OTOH, when you're changing the catalogs it seems pretty foolish
to not run the whole test suite.

Anyway, I have no great objection to the proposed patch, I'm just dubious
that it's really worth the trouble. If you do go through with it, I'd
suggest adding an installcheck-with variant.

In the bikeshedding department, maybe "-tests" instead of "-with"?

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Alvaro Herrera

alvherre@2ndquadrant.com

almost 12 years ago

In reply to: Tom Lane (#7)

Re: running make check with only specified tests

Tom Lane wrote:

Anyway, I have no great objection to the proposed patch, I'm just dubious
that it's really worth the trouble. If you do go through with it, I'd
suggest adding an installcheck-with variant.

In the bikeshedding department, maybe "-tests" instead of "-with"?

No objection to the proposed idea either, but I wanted to point out that
I had the idea sometime ago that each test would declare which other
test it depended on; so if you specify one to run in isolation, the ones
it depended on got run beforehand. I never got around to implementing
it because running the whole bunch doesn't take that long anyway, but if
for some reason you want to run a specific test over and over, it is
useful.

--
ï¿½lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers