bin checks taking too long.

Started by Andrew Dunstanabout 11 years ago5 messages
#1Andrew Dunstan
andrew@dunslane.net

Currently the buildfarm animal crake (my development instance) is
running the bin check, but not any other animal. These tests still take
for too long, not least because each set of tests requires a separate
install.

I have complained about this before, but we don't seem to have made any
progress. I am very reluctant to put this check into a buildfarm client
release until that is addressed. Is there really nothing we can do about
it? This is getting more important since we now have another feature
that we want to get into a buildfarm client release.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#2Peter Eisentraut
peter_e@gmx.net
In reply to: Andrew Dunstan (#1)
Re: bin checks taking too long.

On 12/22/14 7:56 PM, Andrew Dunstan wrote:

Currently the buildfarm animal crake (my development instance) is
running the bin check, but not any other animal. These tests still take
for too long, not least because each set of tests requires a separate
install.

You can avoid that by using make installcheck.

Is there really nothing we can do about it?

There is, but it's not straightforward, as it turns out. Ideas are welcome.

Also, as you can imagine, any build system changes are bottlenecked on
Windows support.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#3Andrew Dunstan
andrew@dunslane.net
In reply to: Peter Eisentraut (#2)
Re: bin checks taking too long.

On 12/22/2014 10:41 PM, Peter Eisentraut wrote:

On 12/22/14 7:56 PM, Andrew Dunstan wrote:

Currently the buildfarm animal crake (my development instance) is
running the bin check, but not any other animal. These tests still take
for too long, not least because each set of tests requires a separate
install.

You can avoid that by using make installcheck.

Not working too well:

make -C pg_ctl installcheck
make[1]: Entering directory
`/home/bf/bfr/root/HEAD/pgsql.build/src/bin/pg_ctl'
cd /home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/bin/pg_ctl &&
TESTDIR='/home/bf/bfr/root/HEAD/pgsql.build/src/bin/pg_ctl'
PATH="/home/bf/bfr/root/HEAD/inst/bin:$PATH" PGPORT='65648' prove -I
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/test/perl/ --verbose
t/*.pl
Use of uninitialized value $ENV{"top_builddir"} in concatenation (.)
or string at t/001_start_stop.pl line 17.
file not found: /src/test/regress/pg_regress at
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/test/perl/TestLib.pm
line 142.

This was invoked like this:

cd $pgsql/src/bin && make NO_LOCALE=1 installcheck

Note that it's a vpath build.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

#4Andrew Dunstan
andrew@dunslane.net
In reply to: Andrew Dunstan (#3)
1 attachment(s)
Re: bin checks taking too long.

On 12/23/2014 11:11 AM, Andrew Dunstan wrote:

On 12/22/2014 10:41 PM, Peter Eisentraut wrote:

On 12/22/14 7:56 PM, Andrew Dunstan wrote:

Currently the buildfarm animal crake (my development instance) is
running the bin check, but not any other animal. These tests still take
for too long, not least because each set of tests requires a separate
install.

You can avoid that by using make installcheck.

Not working too well:

make -C pg_ctl installcheck
make[1]: Entering directory
`/home/bf/bfr/root/HEAD/pgsql.build/src/bin/pg_ctl'
cd /home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/bin/pg_ctl &&
TESTDIR='/home/bf/bfr/root/HEAD/pgsql.build/src/bin/pg_ctl'
PATH="/home/bf/bfr/root/HEAD/inst/bin:$PATH" PGPORT='65648' prove -I
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/test/perl/ --verbose
t/*.pl
Use of uninitialized value $ENV{"top_builddir"} in concatenation (.)
or string at t/001_start_stop.pl line 17.
file not found: /src/test/regress/pg_regress at
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/test/perl/TestLib.pm
line 142.

This was invoked like this:

cd $pgsql/src/bin && make NO_LOCALE=1 installcheck

Note that it's a vpath build.

The attached patch seems to fix this. However, running installcheck also
makes very little difference to the time. These checks still take around
100 seconds on my machine, including around 60 for the script tests.
That seems ridiculously long, when the whole core regression suite takes
23 seconds.

cheers

andrew

Attachments:

bincheckfix.patchtext/x-patch; name=bincheckfix.patchDownload
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index a77b5f1..7c39d82 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -317,7 +317,7 @@ endef
 ifeq ($(enable_tap_tests),yes)
 
 define prove_installcheck
-cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
+cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) t/*.pl
 endef
 
 define prove_check
#5Noah Misch
noah@leadboat.com
In reply to: Andrew Dunstan (#4)
Re: bin checks taking too long.

On Tue, Dec 23, 2014 at 01:55:46PM -0500, Andrew Dunstan wrote:

On 12/23/2014 11:11 AM, Andrew Dunstan wrote:

Use of uninitialized value $ENV{"top_builddir"} in concatenation (.)
or string at t/001_start_stop.pl line 17.
file not found: /src/test/regress/pg_regress at
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/test/perl/TestLib.pm
line 142.

This was invoked like this:

cd $pgsql/src/bin && make NO_LOCALE=1 installcheck

Note that it's a vpath build.

The attached patch seems to fix this.

Oops. Your fix looks good.

However, running installcheck also
makes very little difference to the time. These checks still take around 100
seconds on my machine, including around 60 for the script tests. That seems
ridiculously long, when the whole core regression suite takes 23 seconds.

I am satisfied with the performance of the TAP suites.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers