Extending USE_MODULE_DB to more test suite types

Started by Noah Mischalmost 7 years ago5 messages

noah@leadboat.com

almost 7 years ago

1 attachment(s)

Some buildfarm runs have failed like this:

============== dropping database "pl_regression" ==============
ERROR: database "pl_regression" is being accessed by other users
DETAIL: There is 1 other session using the database.

Affected runs:

axolotl │ PLCheck-C │ REL9_5_STABLE │ 2015-08-21 19:29:19 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=axolotl&dt=2015-08-21%2019:29:19
axolotl │ PLCheck-C │ REL9_6_STABLE │ 2017-03-16 17:43:16 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=axolotl&dt=2017-03-16%2017:43:16
mandrill │ PLCheck-C │ HEAD │ 2017-05-13 17:14:12 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mandrill&dt=2017-05-13%2017:14:12
tern │ PLCheck-C │ HEAD │ 2017-09-05 20:45:17 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tern&dt=2017-09-05%2020:45:17
mandrill │ PLCheck-C │ HEAD │ 2017-11-15 13:34:12 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mandrill&dt=2017-11-15%2013:34:12
mandrill │ PLCheck-en_US.ISO8859-1 │ REL_10_STABLE │ 2018-03-15 05:24:41 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mandrill&dt=2018-03-15%2005:24:41
frogfish │ TestModulesCheck-en_US.utf8 │ REL_11_STABLE │ 2019-01-29 01:32:51 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogfish&dt=2019-01-29%2001:32:51
hornet │ PLCheck-C │ REL_11_STABLE │ 2019-01-29 01:52:29 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hornet&dt=2019-01-29%2001:52:29

I can reproduce that reliably by combining "make -C src/pl installcheck" with
this hack:

proc_exit(int code)
{
+ pg_usleep(7000000);

This happens because dropdb()'s call to CountOtherDBBackends() waits up to 5s
for the database to become vacant. If the last plpython test backend takes
more than 5s to exit, the pltcl suite fails. Most test suites are unaffected,
thanks to USE_MODULE_DB=1 in the buildfarm script. However, PL suites ignore
USE_MODULE_DB. So do the three src/test/modules directories that contain no C
code and define $(REGRESS). Isolation suites, too, ignore USE_MODULE_DB.

I would like to fix this as follows. When MODULES and MODULE_big are both
unset, instead of using a constant string, derive the database name from the
first element of $(REGRESS) or $(ISOLATION). I considered $(EXTENSION), but
src/test/modules/commit_ts does not set it. $(REGRESS) and $(ISOLATION) are
robust; in their absence, a directory simply won't invoke pg_regress to drop
and/or create a database. I considered introducing a TESTDB_SUFFIX variable
that src/test/modules directories could define, but that felt like needless
flexibility. Treat src/pl in a similar fashion. With the attached patch,
installcheck-world and check-world no longer reuse any database name in a
given postmaster. Next, I'll mail this buildfarm client patch, after which
any non-MSVC, v9.5+ (due to ddc2504) buildfarm run would no longer reuse any
database name in a given postmaster:

--- a/run_build.pl
+++ b/run_build.pl
@@ -1677 +1677,2 @@ sub make_pl_install_check
-		@checklog = run_log("cd $pgsql/src/pl && $make installcheck");
+		my $cmd = "cd $pgsql/src/pl && $make USE_MODULE_DB=1 installcheck";
+		@checklog = run_log($cmd);

I plan to back-patch the PostgreSQL patch, to combat buildfarm noise. Perhaps
someone has test automation that sets USE_MODULE_DB and nonetheless probes the
exact database name "pl_regression", but I'm not too worried. The original
rationale for USE_MODULE_DB, in commit ad69bd0, was to facilitate pg_upgrade
testing. Folks using "make installcheck-world" to populate a cluster for
pg_upgrade testing will see additional test coverage, which may cause
additional failures. I'm fine with that, too.

Attachments:

USE_MODULE_DB-all-v1.patchtext/plain; charset=us-asciiDownload

diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index c118f64..6b98d53 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -597,16 +597,25 @@ submake-libpgfeutils: | submake-generated-headers
 #
 # Testing support
 
-PL_TESTDB = pl_regression
-CONTRIB_TESTDB = contrib_regression
-ifneq ($(MODULE_big),)
-  CONTRIB_TESTDB_MODULE = contrib_regression_$(MODULE_big)
-else
-  ifneq ($(MODULES),)
-    CONTRIB_TESTDB_MODULE = contrib_regression_$(MODULES)
+ifneq ($(USE_MODULE_DB),)
+  PL_TESTDB = pl_regression_$(NAME)
+  # Replace this with $(or ...) if we ever require GNU make 3.81.
+  ifneq ($(MODULE_big),)
+    CONTRIB_TESTDB=contrib_regression_$(MODULE_big)
+    ISOLATION_TESTDB=isolation_regression_$(MODULE_big)
   else
-    CONTRIB_TESTDB_MODULE = contrib_regression
+    ifneq ($(MODULES),)
+      CONTRIB_TESTDB=contrib_regression_$(word 1,$(MODULES))
+      ISOLATION_TESTDB=isolation_regression_$(word 1,$(MODULES))
+    else
+      CONTRIB_TESTDB=contrib_regression_$(word 1,$(REGRESS))
+      ISOLATION_TESTDB=isolation_regression_$(word 1,$(ISOLATION))
+    endif
   endif
+else
+  PL_TESTDB = pl_regression
+  CONTRIB_TESTDB = contrib_regression
+  ISOLATION_TESTDB = isolation_regression
 endif
 
 ifdef NO_LOCALE
diff --git a/src/makefiles/pgxs.mk b/src/makefiles/pgxs.mk
index 909a49f..271e7ea 100644
--- a/src/makefiles/pgxs.mk
+++ b/src/makefiles/pgxs.mk
@@ -380,12 +380,7 @@ distclean maintainer-clean: clean
 
 ifdef REGRESS
 
-# Select database to use for running the tests
-ifneq ($(USE_MODULE_DB),)
-  REGRESS_OPTS += --dbname=$(CONTRIB_TESTDB_MODULE)
-else
-  REGRESS_OPTS += --dbname=$(CONTRIB_TESTDB)
-endif
+REGRESS_OPTS += --dbname=$(CONTRIB_TESTDB)
 
 # When doing a VPATH build, must copy over the data files so that the
 # driver script can find them.  We have to use an absolute path for
@@ -413,6 +408,10 @@ ifndef PGXS
 	$(MAKE) -C $(top_builddir)/src/test/isolation all
 endif
 
+ifdef ISOLATION
+ISOLATION_OPTS += --dbname=$(ISOLATION_TESTDB)
+endif
+
 # Standard rules to run regression tests including multiple test suites.
 # Runs against an installed postmaster.
 ifndef NO_INSTALLCHECK

Andrew Dunstan

andrew.dunstan@2ndquadrant.com

almost 7 years ago

In reply to: Noah Misch (#1)

Re: Extending USE_MODULE_DB to more test suite types

On Mon, Apr 1, 2019 at 9:52 AM Noah Misch <noah@leadboat.com> wrote:

Some buildfarm runs have failed like this:

============== dropping database "pl_regression" ==============
ERROR: database "pl_regression" is being accessed by other users
DETAIL: There is 1 other session using the database.

Affected runs:

axolotl │ PLCheck-C │ REL9_5_STABLE │ 2015-08-21 19:29:19 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=axolotl&dt=2015-08-21%2019:29:19
axolotl │ PLCheck-C │ REL9_6_STABLE │ 2017-03-16 17:43:16 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=axolotl&dt=2017-03-16%2017:43:16
mandrill │ PLCheck-C │ HEAD │ 2017-05-13 17:14:12 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mandrill&dt=2017-05-13%2017:14:12
tern │ PLCheck-C │ HEAD │ 2017-09-05 20:45:17 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tern&dt=2017-09-05%2020:45:17
mandrill │ PLCheck-C │ HEAD │ 2017-11-15 13:34:12 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mandrill&dt=2017-11-15%2013:34:12
mandrill │ PLCheck-en_US.ISO8859-1 │ REL_10_STABLE │ 2018-03-15 05:24:41 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=mandrill&dt=2018-03-15%2005:24:41
frogfish │ TestModulesCheck-en_US.utf8 │ REL_11_STABLE │ 2019-01-29 01:32:51 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogfish&dt=2019-01-29%2001:32:51
hornet │ PLCheck-C │ REL_11_STABLE │ 2019-01-29 01:52:29 │ https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hornet&dt=2019-01-29%2001:52:29

I can reproduce that reliably by combining "make -C src/pl installcheck" with
this hack:

proc_exit(int code)
{
+ pg_usleep(7000000);

This happens because dropdb()'s call to CountOtherDBBackends() waits up to 5s
for the database to become vacant. If the last plpython test backend takes
more than 5s to exit, the pltcl suite fails. Most test suites are unaffected,
thanks to USE_MODULE_DB=1 in the buildfarm script. However, PL suites ignore
USE_MODULE_DB. So do the three src/test/modules directories that contain no C
code and define $(REGRESS). Isolation suites, too, ignore USE_MODULE_DB.

I would like to fix this as follows. When MODULES and MODULE_big are both
unset, instead of using a constant string, derive the database name from the
first element of $(REGRESS) or $(ISOLATION). I considered $(EXTENSION), but
src/test/modules/commit_ts does not set it. $(REGRESS) and $(ISOLATION) are
robust; in their absence, a directory simply won't invoke pg_regress to drop
and/or create a database. I considered introducing a TESTDB_SUFFIX variable
that src/test/modules directories could define, but that felt like needless
flexibility. Treat src/pl in a similar fashion. With the attached patch,
installcheck-world and check-world no longer reuse any database name in a
given postmaster. Next, I'll mail this buildfarm client patch, after which
any non-MSVC, v9.5+ (due to ddc2504) buildfarm run would no longer reuse any
database name in a given postmaster:
--- a/run_build.pl
+++ b/run_build.pl
@@ -1677 +1677,2 @@ sub make_pl_install_check
-               @checklog = run_log("cd $pgsql/src/pl && $make installcheck");
+               my $cmd = "cd $pgsql/src/pl && $make USE_MODULE_DB=1 installcheck";
+               @checklog = run_log($cmd);
I plan to back-patch the PostgreSQL patch, to combat buildfarm noise. Perhaps
someone has test automation that sets USE_MODULE_DB and nonetheless probes the
exact database name "pl_regression", but I'm not too worried. The original
rationale for USE_MODULE_DB, in commit ad69bd0, was to facilitate pg_upgrade
testing. Folks using "make installcheck-world" to populate a cluster for
pg_upgrade testing will see additional test coverage, which may cause
additional failures. I'm fine with that, too.

Excellent. Extending use of USE_MODULE_DB has been on my list of
things to do. I'll add the buildfarm patch right away. It should be
harmless before these changes are made.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Andres Freund

andres@anarazel.de

almost 7 years ago

In reply to: Noah Misch (#1)

Re: Extending USE_MODULE_DB to more test suite types

On 2019-04-01 06:52:13 -0700, Noah Misch wrote:

I plan to back-patch the PostgreSQL patch, to combat buildfarm noise. Perhaps
someone has test automation that sets USE_MODULE_DB and nonetheless probes the
exact database name "pl_regression", but I'm not too worried. The original
rationale for USE_MODULE_DB, in commit ad69bd0, was to facilitate pg_upgrade
testing. Folks using "make installcheck-world" to populate a cluster for
pg_upgrade testing will see additional test coverage, which may cause
additional failures. I'm fine with that, too.

+1 for all of that.

Greetings,

Andres Freund

Tom Lane

tgl@sss.pgh.pa.us

almost 7 years ago

In reply to: Andres Freund (#3)

Re: Extending USE_MODULE_DB to more test suite types

Andres Freund <andres@anarazel.de> writes:

On 2019-04-01 06:52:13 -0700, Noah Misch wrote:

I plan to back-patch the PostgreSQL patch, to combat buildfarm noise. Perhaps
someone has test automation that sets USE_MODULE_DB and nonetheless probes the
exact database name "pl_regression", but I'm not too worried. The original
rationale for USE_MODULE_DB, in commit ad69bd0, was to facilitate pg_upgrade
testing. Folks using "make installcheck-world" to populate a cluster for
pg_upgrade testing will see additional test coverage, which may cause
additional failures. I'm fine with that, too.

+1 for all of that.

I haven't tested the patch, but also +1 for the idea.

regards, tom lane

Noah Misch

noah@leadboat.com

almost 7 years ago

In reply to: Andrew Dunstan (#2)

Re: Extending USE_MODULE_DB to more test suite types

On Mon, Apr 01, 2019 at 01:01:11PM -0400, Andrew Dunstan wrote:

On Mon, Apr 1, 2019 at 9:52 AM Noah Misch <noah@leadboat.com> wrote:

I plan to back-patch the PostgreSQL patch, to combat buildfarm noise. Perhaps
someone has test automation that sets USE_MODULE_DB and nonetheless probes the
exact database name "pl_regression", but I'm not too worried. The original
rationale for USE_MODULE_DB, in commit ad69bd0, was to facilitate pg_upgrade
testing. Folks using "make installcheck-world" to populate a cluster for
pg_upgrade testing will see additional test coverage, which may cause
additional failures. I'm fine with that, too.

Pushed. It looks like XversionUpgradeSave relies on having a
"contrib_regression" database:
https://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=crake&br=REL9_5_STABLE

Excellent. Extending use of USE_MODULE_DB has been on my list of
things to do. I'll add the buildfarm patch right away. It should be
harmless before these changes are made.

Thanks.