Occasional tablespace.sql failures in check-world -jnn
Hi,
For fairly obvious reasons I like to run check-world in parallel [1]make -Otarget -j20 -s check-world && echo success || echo failed. In
the last few months I've occasionally seen failures during that that I
cannot recall seeing before.
--- /home/andres/build/postgres/13-assert/vpath/src/test/regress/expected/tablespace.out 2020-12-07 18:41:23.079235588 -0800
+++ /home/andres/build/postgres/13-assert/vpath/src/test/regress/results/tablespace.out 2020-12-07 18:42:01.892632468 -0800
@@ -209,496 +209,344 @@
ERROR: cannot specify default tablespace for partitioned relations
CREATE TABLE testschema.dflt (a int PRIMARY KEY USING INDEX TABLESPACE regress_tblspace) PARTITION BY LIST (a);
ERROR: cannot specify default tablespace for partitioned relations
-- but these work:
CREATE TABLE testschema.dflt (a int PRIMARY KEY USING INDEX TABLESPACE regress_tblspace) PARTITION BY LIST (a) TABLESPACE regress_tblspace;
SET default_tablespace TO '';
CREATE TABLE testschema.dflt2 (a int PRIMARY KEY) PARTITION BY LIST (a);
DROP TABLE testschema.dflt, testschema.dflt2;
-- check that default_tablespace doesn't affect ALTER TABLE index rebuilds
CREATE TABLE testschema.test_default_tab(id bigint) TABLESPACE regress_tblspace;
+ERROR: could not create directory "pg_tblspc/16387/PG_13_202007201/16384": No such file or directory
INSERT INTO testschema.test_default_tab VALUES (1);(many failures follow)
I suspect this is related to the pg_upgrade test and the main regression
test running at the same time. We have the following in src/test/regress/GNUMakefile
# Tablespace setup
.PHONY: tablespace-setup
tablespace-setup:
echo $(realpath ./testtablespace) >> /tmp/tablespace.log
rm -rf ./testtablespace
mkdir ./testtablespace
...
which pg_upgrade triggers. Even though it, as far as I can tell, never
actually ends up putting any data in it:
# Send installcheck outputs to a private directory. This avoids conflict when
# check-world runs pg_upgrade check concurrently with src/test/regress check.
# To retrieve interesting files after a run, use pattern tmp_check/*/*.diffs.
outputdir="$temp_root/regress"
EXTRA_REGRESS_OPTS="$EXTRA_REGRESS_OPTS --outputdir=$outputdir"
export EXTRA_REGRESS_OPTS
mkdir "$outputdir"
mkdir "$outputdir"/testtablespace
It's not clear to me why we have this logic in the makefile at all?
Somebody taught pg_regress to do so, but only on windows... See
convert_sourcefiles_in().
The other thing that confuses me is why I started getting that error in
*multiple* branches recently, even though I have used the parallel
check-world for ages.
Greetings,
Andres Freund
[1]: make -Otarget -j20 -s check-world && echo success || echo failed
On Tue, Dec 08, 2020 at 05:29:11PM -0800, Andres Freund wrote:
I suspect this is related to the pg_upgrade test and the main regression
test running at the same time. We have the following in
src/test/regress/GNUMakefile.
Yes, this one is not completely new to -hackers. See patch 0002 here
that slightly touched the topic by creating a specific makefile rule,
but I never got back to it as I never got annoyed by this problem:
/messages/by-id/20200511.171354.514381788845037011.horikyota.ntt@gmail.com
What we have here is not a solution though...
It's not clear to me why we have this logic in the makefile at all?
Somebody taught pg_regress to do so, but only on windows... See
convert_sourcefiles_in().
... Because we may still introduce this problem again if some new
stuff uses src/test/pg_regress in a way similar to pg_upgrade,
triggering again tablespace-setup. Something like the attached may be
enough, though I have not spent much time checking the surroundings,
Windows included.
The other thing that confuses me is why I started getting that error in
*multiple* branches recently, even though I have used the parallel
check-world for ages.
Perhaps you have just increased -j lately or moved to a faster machine
where there are higher changes of collision?
--
Michael
Attachments:
regress-tbspace-v1.patchtext/x-diff; charset=us-asciiDownload+13-24
On 2020-12-09 02:29, Andres Freund wrote:
I suspect this is related to the pg_upgrade test and the main regression
test running at the same time. We have the following in src/test/regress/GNUMakefile# Tablespace setup
.PHONY: tablespace-setup
tablespace-setup:
echo $(realpath ./testtablespace) >> /tmp/tablespace.log
rm -rf ./testtablespace
mkdir ./testtablespace
...which pg_upgrade triggers. Even though it, as far as I can tell, never
actually ends up putting any data in it:# Send installcheck outputs to a private directory. This avoids conflict when
# check-world runs pg_upgrade check concurrently with src/test/regress check.
# To retrieve interesting files after a run, use pattern tmp_check/*/*.diffs.
outputdir="$temp_root/regress"
EXTRA_REGRESS_OPTS="$EXTRA_REGRESS_OPTS --outputdir=$outputdir"
export EXTRA_REGRESS_OPTS
mkdir "$outputdir"
mkdir "$outputdir"/testtablespaceIt's not clear to me why we have this logic in the makefile at all?
Somebody taught pg_regress to do so, but only on windows... See
convert_sourcefiles_in().
I vaguely recall that this had something to do with SELinux (or
something similar?), where it matters in what context you create a file
or directory and then certain properties attach to it that are relevant
to subsequent programs that run on it. Again, vague.
On Fri, Jan 15, 2021 at 09:59:02AM +0100, Peter Eisentraut wrote:
I vaguely recall that this had something to do with SELinux (or something
similar?), where it matters in what context you create a file or directory
and then certain properties attach to it that are relevant to subsequent
programs that run on it. Again, vague.
Hmm. Does it? sepgsql has some tests for tablespaces involving only
pg_default, so it does not seem that this applies in the context of
the regression tests. The cleanup of testtablespace in GNUMakefile
comes from 2467394, as of June 2004, that introduced tablespaces.
--
Michael
On 09.12.20 08:55, Michael Paquier wrote:
It's not clear to me why we have this logic in the makefile at all?
Somebody taught pg_regress to do so, but only on windows... See
convert_sourcefiles_in().... Because we may still introduce this problem again if some new
stuff uses src/test/pg_regress in a way similar to pg_upgrade,
triggering again tablespace-setup. Something like the attached may be
enough, though I have not spent much time checking the surroundings,
Windows included.
This patch looks alright to me.
On Mon, Mar 08, 2021 at 11:53:57AM +0100, Peter Eisentraut wrote:
On 09.12.20 08:55, Michael Paquier wrote:
... Because we may still introduce this problem again if some new
stuff uses src/test/pg_regress in a way similar to pg_upgrade,
triggering again tablespace-setup. Something like the attached may be
enough, though I have not spent much time checking the surroundings,
Windows included.This patch looks alright to me.
So, I have spent more time checking the surroundings of this patch,
and finally applied it. Thanks for the review, Peter.
--
Michael
Hi,
On 2021-03-10 15:40:38 +0900, Michael Paquier wrote:
On Mon, Mar 08, 2021 at 11:53:57AM +0100, Peter Eisentraut wrote:
On 09.12.20 08:55, Michael Paquier wrote:
... Because we may still introduce this problem again if some new
stuff uses src/test/pg_regress in a way similar to pg_upgrade,
triggering again tablespace-setup. Something like the attached may be
enough, though I have not spent much time checking the surroundings,
Windows included.This patch looks alright to me.
So, I have spent more time checking the surroundings of this patch,
and finally applied it. Thanks for the review, Peter.
Thanks!
Greetings,
Andres Freund