pgsql: Add mode where contrib installcheck runs each module in a separa

Started by Andrew Dunstanabout 13 years ago9 messages

andrew@dunslane.net

about 13 years ago

Add mode where contrib installcheck runs each module in a separately named database.

Normally each module is tested in aq database named contrib_regression,
which is dropped and recreated at the beginhning of each pg_regress run.
This mode, enabled by adding USE_MODULE_DB=1 to the make command line,
runs most modules in a database with the module name embedded in it.

This will make testing pg_upgrade on clusters with the contrib modules
a lot easier.

Still to be done: adapt to the MSVC build system.

Backpatch to 9.0, which is the earliest version it is reasonably
possible to test upgrading from.

Branch
------
REL9_0_STABLE

Details
-------
http://git.postgresql.org/pg/commitdiff/c8f666abde2af3060af41afe4b03ced2f62d94a9

Modified Files
--------------
contrib/dblink/Makefile | 3 +++
src/Makefile.global.in | 9 +++++++++
src/makefiles/pgxs.mk | 6 +++++-
3 files changed, 17 insertions(+), 1 deletions(-)

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Alvaro Herrera

alvherre@2ndquadrant.com

about 13 years ago

In reply to: Andrew Dunstan (#1)

Re: pgsql: Add mode where contrib installcheck runs each module in a separa

Andrew Dunstan wrote:

Add mode where contrib installcheck runs each module in a separately named database.

Normally each module is tested in aq database named contrib_regression,
which is dropped and recreated at the beginhning of each pg_regress run.
This mode, enabled by adding USE_MODULE_DB=1 to the make command line,
runs most modules in a database with the module name embedded in it.

I'm seeing this when building contrib/dblink:

$ LC_ALL=C make
Makefile:15: invalid `override' directive

$ LC_ALL=C !!
LC_ALL=C make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for x86_64-pc-linux-gnu

It doesn't work (note the DB name in the command line):

$ LC_ALL=C make USE_MODULE_DB=q installcheck
Makefile:15: invalid `override' directive
make -C ../../src/test/regress pg_regress
make[1]: Entering directory `/home/alvherre/Code/pgsql/build/bgworker/src/test/regress'
make -C ../../../src/port all
make[2]: Entering directory `/home/alvherre/Code/pgsql/build/bgworker/src/port'
make -C ../backend submake-errcodes
make[3]: Entering directory `/home/alvherre/Code/pgsql/build/bgworker/src/backend'
make[3]: Nothing to be done for `submake-errcodes'.
make[3]: Leaving directory `/home/alvherre/Code/pgsql/build/bgworker/src/backend'
make[2]: Leaving directory `/home/alvherre/Code/pgsql/build/bgworker/src/port'
make[1]: Leaving directory `/home/alvherre/Code/pgsql/build/bgworker/src/test/regress'
../../src/test/regress/pg_regress --inputdir=/pgsql/source/bgworker/contrib/dblink --psqldir='/pgsql/install/bgworker/bin' --dbname=contrib_regression_dblink dblink
(using postmaster on Unix socket, default port)
============== dropping database "contrib_regression_dblink" ==============
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.55440"?
command failed: "/pgsql/install/bgworker/bin/psql" -X -c "DROP DATABASE IF EXISTS \"contrib_regression_dblink\"" "postgres"
make: *** [installcheck] Error 2

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Andrew Dunstan

andrew@dunslane.net

about 13 years ago

In reply to: Alvaro Herrera (#2)

Re: pgsql: Add mode where contrib installcheck runs each module in a separa

On 12/03/2012 01:45 PM, Alvaro Herrera wrote:

Andrew Dunstan wrote:

Add mode where contrib installcheck runs each module in a separately named database.

Normally each module is tested in aq database named contrib_regression,
which is dropped and recreated at the beginhning of each pg_regress run.
This mode, enabled by adding USE_MODULE_DB=1 to the make command line,
runs most modules in a database with the module name embedded in it.

I'm seeing this when building contrib/dblink:

$ LC_ALL=C make
Makefile:15: invalid `override' directive

$ LC_ALL=C !!
LC_ALL=C make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for x86_64-pc-linux-gnu

It doesn't work (note the DB name in the command line):

$ LC_ALL=C make USE_MODULE_DB=q installcheck
Makefile:15: invalid `override' directive
make -C ../../src/test/regress pg_regress
make[1]: Entering directory `/home/alvherre/Code/pgsql/build/bgworker/src/test/regress'
make -C ../../../src/port all
make[2]: Entering directory `/home/alvherre/Code/pgsql/build/bgworker/src/port'
make -C ../backend submake-errcodes
make[3]: Entering directory `/home/alvherre/Code/pgsql/build/bgworker/src/backend'
make[3]: Nothing to be done for `submake-errcodes'.
make[3]: Leaving directory `/home/alvherre/Code/pgsql/build/bgworker/src/backend'
make[2]: Leaving directory `/home/alvherre/Code/pgsql/build/bgworker/src/port'
make[1]: Leaving directory `/home/alvherre/Code/pgsql/build/bgworker/src/test/regress'
../../src/test/regress/pg_regress --inputdir=/pgsql/source/bgworker/contrib/dblink --psqldir='/pgsql/install/bgworker/bin' --dbname=contrib_regression_dblink dblink
(using postmaster on Unix socket, default port)
============== dropping database "contrib_regression_dblink" ==============
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.55440"?
command failed: "/pgsql/install/bgworker/bin/psql" -X -c "DROP DATABASE IF EXISTS \"contrib_regression_dblink\"" "postgres"
make: *** [installcheck] Error 2

Well, you seem to have more problems than just the database name.

This seems rather odd, though. What more can you tell us about the
platform etc.? I note you have "USE_MODULE_DB=q" where I'm using
"USE_MODULE_DB=1". Do you get the same failure when you don't define
USE_MODULE_DB?

cheers

andrew

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Alvaro Herrera

alvherre@2ndquadrant.com

about 13 years ago

In reply to: Andrew Dunstan (#3)

Re: pgsql: Add mode where contrib installcheck runs each module in a separa

Andrew Dunstan wrote:

On 12/03/2012 01:45 PM, Alvaro Herrera wrote:

../../src/test/regress/pg_regress --inputdir=/pgsql/source/bgworker/contrib/dblink --psqldir='/pgsql/install/bgworker/bin' --dbname=contrib_regression_dblink dblink
(using postmaster on Unix socket, default port)
============== dropping database "contrib_regression_dblink" ==============
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.55440"?
command failed: "/pgsql/install/bgworker/bin/psql" -X -c "DROP DATABASE IF EXISTS \"contrib_regression_dblink\"" "postgres"
make: *** [installcheck] Error 2

Well, you seem to have more problems than just the database name.

Obviously I didn't have the server running at that point, but surely
that's not relevant; what I'm pointing out is that make is not taking
notice of the variable override.

This seems rather odd, though. What more can you tell us about the
platform etc.?

This is Debian Squeeze running on amd64. There's nothing really odd
about it; it's my everyday platform.

I note you have "USE_MODULE_DB=q" where I'm using
"USE_MODULE_DB=1". Do you get the same failure when you don't define
USE_MODULE_DB?

Yes. I wasn't testing this patch, and I wasn't testing pg_upgrade
either; I was just running a full rebuild and noticed that message.
So initially I obviously hadn't set any variable at all.

I cleaned up everything up including autoconf cache, removed VPATH build
dir, git clean -dfx, and it's still there.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Tom Lane

tgl@sss.pgh.pa.us

about 13 years ago

In reply to: Andrew Dunstan (#3)

Re: [COMMITTERS] pgsql: Add mode where contrib installcheck runs each module in a separa

Andrew Dunstan <andrew@dunslane.net> writes:

On 12/03/2012 01:45 PM, Alvaro Herrera wrote:

$ LC_ALL=C make
Makefile:15: invalid `override' directive

Well, you seem to have more problems than just the database name.

I see this too. Given that I want to wrap the back branches in a few
minutes, please revert at least in the back branches. I can't wait
for you to sort this out.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andrew Dunstan

andrew@dunslane.net

about 13 years ago

In reply to: Tom Lane (#5)

Re: [COMMITTERS] pgsql: Add mode where contrib installcheck runs each module in a separa

On 12/03/2012 02:46 PM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

On 12/03/2012 01:45 PM, Alvaro Herrera wrote:

$ LC_ALL=C make
Makefile:15: invalid `override' directive

Well, you seem to have more problems than just the database name.

I see this too. Given that I want to wrap the back branches in a few
minutes, please revert at least in the back branches. I can't wait
for you to sort this out.

OK. I reverted the lot.

will do.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Tom Lane

tgl@sss.pgh.pa.us

about 13 years ago

In reply to: Andrew Dunstan (#6)

Re: pgsql: Add mode where contrib installcheck runs each module in a separa

Andrew Dunstan <andrew@dunslane.net> writes:

On 12/03/2012 02:46 PM, Tom Lane wrote:

I see this too. Given that I want to wrap the back branches in a few
minutes, please revert at least in the back branches. I can't wait
for you to sort this out.

OK. I reverted the lot.

Thanks, appreciate it. I'll help look into the problem late tonight,
if it's not sorted by then, but I have no time right now ...

regards, tom lane

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Andrew Dunstan

andrew@dunslane.net

about 13 years ago

In reply to: Tom Lane (#7)

Re: pgsql: Add mode where contrib installcheck runs each module in a separa

On 12/03/2012 03:11 PM, Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

On 12/03/2012 02:46 PM, Tom Lane wrote:

I see this too. Given that I want to wrap the back branches in a few
minutes, please revert at least in the back branches. I can't wait
for you to sort this out.

OK. I reverted the lot.

Thanks, appreciate it. I'll help look into the problem late tonight,
if it's not sorted by then, but I have no time right now ...

Looks like undefine is a new feature in gmake 3.82. I can reproduce this
on my SL6 box which has 3.81. I'll have to come up with something other
than ifdef, like

ifneq ($(USE_MODULE_DB),)

and use the override in dblink's Makefile to set it to the empty string.

cheers

andrew

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Andrew Dunstan

andrew@dunslane.net

about 13 years ago

In reply to: Andrew Dunstan (#8)

1 attachment(s)

Re: [COMMITTERS] pgsql: Add mode where contrib installcheck runs each module in a separa

On 12/03/2012 04:02 PM, Andrew Dunstan wrote:

Looks like undefine is a new feature in gmake 3.82. I can reproduce
this on my SL6 box which has 3.81. I'll have to come up with something
other than ifdef, like

ifneq ($(USE_MODULE_DB),)

and use the override in dblink's Makefile to set it to the empty string.

Here's a version that seems to work with pre 3.82 versions of gmake.

cheers

andrew

Attachments:

module_db.patch2text/plain; charset=UTF-8; name=module_db.patch2Download

diff --git a/contrib/dblink/Makefile b/contrib/dblink/Makefile
index a27db88..32314a0 100644
--- a/contrib/dblink/Makefile
+++ b/contrib/dblink/Makefile
@@ -11,6 +11,9 @@ DATA = dblink--1.1.sql dblink--1.0--1.1.sql dblink--unpackaged--1.0.sql
 
 REGRESS = dblink
 
+# the db name is hard-coded in the tests
+override USE_MODULE_DB =
+
 ifdef USE_PGXS
 PG_CONFIG = pg_config
 PGXS := $(shell $(PG_CONFIG) --pgxs)
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index e10eead..9cc14da 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -428,6 +428,15 @@ submake-libpgport:
 
 PL_TESTDB = pl_regression
 CONTRIB_TESTDB = contrib_regression
+ifneq ($(MODULE_big),)
+  CONTRIB_TESTDB_MODULE = contrib_regression_$(MODULE_big)
+else 
+  ifneq ($(MODULES),)
+    CONTRIB_TESTDB_MODULE = contrib_regression_$(MODULES)
+  else
+    CONTRIB_TESTDB_MODULE = contrib_regression
+  endif
+endif
 
 ifdef NO_LOCALE
 NOLOCALE += --no-locale
diff --git a/src/makefiles/pgxs.mk b/src/makefiles/pgxs.mk
index fd6473f..8acfe05 100644
--- a/src/makefiles/pgxs.mk
+++ b/src/makefiles/pgxs.mk
@@ -240,7 +240,11 @@ distclean maintainer-clean: clean
 ifdef REGRESS
 
 # Select database to use for running the tests
-REGRESS_OPTS += --dbname=$(CONTRIB_TESTDB)
+ifneq ($(USE_MODULE_DB),)
+  REGRESS_OPTS += --dbname=$(CONTRIB_TESTDB_MODULE)
+else
+  REGRESS_OPTS += --dbname=$(CONTRIB_TESTDB)
+endif
 
 # where to find psql for running the tests
 PSQLDIR = $(bindir)