[PATCH] Fix pg_upgrade test from v10

Started by Anton A. Melnikovover 3 years ago12 messages

aamelnikov@inbox.ru

over 3 years ago

2 attachment(s)

Hello!

Found out that test for pg_upgrade (test.sh for 11-14 and
002_pg_upgrade.pl for 15+) doesn't work from 10th versions to higher
ones due to incompatible options for initdb and default PGDATA permissions.

Here are the patches that may solve this problem.

Would be glad to your comments and concerns.

With best regards,
--
Anton A. Melnikov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachments:

v1-0001-Fix-upgrade-test-from-v10-to-v14.patchtext/x-patch; charset=UTF-8; name=v1-0001-Fix-upgrade-test-from-v10-to-v14.patchDownload

commit c62e484e73e0071bc00dffe3b2333fd702108ec6
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Thu Jun 2 03:40:09 2022 +0300

    Fix test for pg_upgrade from 10x versions.

diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
index f353e565b5..2f5ef1bb9f 100644
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -24,7 +24,13 @@ standard_initdb() {
 	# without increasing test runtime, run these tests with a custom setting.
 	# Also, specify "-A trust" explicitly to suppress initdb's warning.
 	# --allow-group-access and --wal-segsize have been added in v11.
-	"$1" -N --wal-segsize 1 --allow-group-access -A trust
+	initdbopt="-N -A trust"
+	if [ $OLD_PG_VERSION_NUM -ge 110000 ]; then
+		initdbopt="$initdbopt --wal-segsize 1 --allow-group-access"
+	fi
+
+	"$1" $initdbopt
+
 	if [ -n "$TEMP_CONFIG" -a -r "$TEMP_CONFIG" ]
 	then
 		cat "$TEMP_CONFIG" >> "$PGDATA/postgresql.conf"
@@ -136,6 +142,7 @@ PGHOSTADDR="";        unset PGHOSTADDR
 
 # Select a non-conflicting port number, similarly to pg_regress.c
 PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$newsrc"/src/include/pg_config.h | awk '{print $3}'`
+OLD_PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$oldsrc"/src/include/pg_config.h | awk '{print $3}'`
 PGPORT=`expr $PG_VERSION_NUM % 16384 + 49152`
 export PGPORT
 
@@ -240,18 +247,26 @@ pg_upgrade $PG_UPGRADE_OPTS -d "${PGDATA}.old" -D "$PGDATA" -b "$oldbindir" -p "
 
 # make sure all directories and files have group permissions, on Unix hosts
 # Windows hosts don't support Unix-y permissions.
+if [ $OLD_PG_VERSION_NUM -lt 110000 ]; then
+	NEW_DIR_PERM=700
+	NEW_FILE_PERM=600
+else
+	NEW_DIR_PERM=750
+	NEW_FILE_PERM=640
+fi
+
 case $testhost in
-	MINGW*|CYGWIN*) ;;
-	*)	if [ `find "$PGDATA" -type f ! -perm 640 | wc -l` -ne 0 ]; then
-			echo "files in PGDATA with permission != 640";
+	MINGW*) ;;
+	*)	if [ `find "$PGDATA" -type f ! -perm $NEW_FILE_PERM | wc -l` -ne 0 ]; then
+			echo "files in PGDATA with permission != $NEW_FILE_PERM";
 			exit 1;
 		fi ;;
 esac
 
 case $testhost in
-	MINGW*|CYGWIN*) ;;
-	*)	if [ `find "$PGDATA" -type d ! -perm 750 | wc -l` -ne 0 ]; then
-			echo "directories in PGDATA with permission != 750";
+	MINGW*) ;;
+	*)	if [ `find "$PGDATA" -type d ! -perm $NEW_DIR_PERM | wc -l` -ne 0 ]; then
+			echo "directories in PGDATA with permission != $NEW_DIR_PERM";
 			exit 1;
 		fi ;;
 esac

v1-0001-Fix-upgrade-test-from-v10-to-v15+.patchtext/x-patch; charset=UTF-8; name=v1-0001-Fix-upgrade-test-from-v10-to-v15+.patchDownload

commit aa96ef028482fc026850363c776145687cc60fc4
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Thu Jun 2 03:35:26 2022 +0300

    Fix test for pg_upgrade from 10x versions.

diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 75ac768a96..e9b4977fee 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -1,5 +1,6 @@
 # Set of tests for pg_upgrade, including cross-version checks.
 use strict;
+use version;
 use warnings;
 
 use Cwd qw(abs_path getcwd);
@@ -56,7 +57,13 @@ my $oldnode =
 # To increase coverage of non-standard segment size and group access without
 # increasing test runtime, run these tests with a custom setting.
 # --allow-group-access and --wal-segsize have been added in v11.
-$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
+my ($oldverstr) = `$ENV{oldinstall}/bin/pg_ctl --version` =~ /(\d+\.\d+)/;
+my ($oldver) =  (version->parse(${oldverstr}));
+$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ])
+		if $oldver >= version->parse('11.0');
+$oldnode->init()
+		if $oldver < version->parse('11.0');
+
 $oldnode->start;
 
 # The default location of the source code is the root of this directory.

Michael Paquier

michael@paquier.xyz

over 3 years ago

In reply to: Anton A. Melnikov (#1)

Re: [PATCH] Fix pg_upgrade test from v10

On Thu, Jun 02, 2022 at 04:22:52AM +0300, Anton A. Melnikov wrote:

Found out that test for pg_upgrade (test.sh for 11-14 and 002_pg_upgrade.pl
for 15+) doesn't work from 10th versions to higher ones due to incompatible
options for initdb and default PGDATA permissions.

Yeah, there are still TODOs in this stuff. Those tests can also break
easily depending on the dump you are pushing to the old node when
doing cross-version upgrades. Perl makes it a bit easier to reason
about improving this area in the future, though, and MSVC is able to
catch up on that.

# To increase coverage of non-standard segment size and group access without
# increasing test runtime, run these tests with a custom setting.
# --allow-group-access and --wal-segsize have been added in v11.
-$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
+my ($oldverstr) = `$ENV{oldinstall}/bin/pg_ctl --version` =~ /(\d+\.\d+)/;
+my ($oldver) =  (version->parse(${oldverstr}));
+$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ])
+		if $oldver >= version->parse('11.0');
+$oldnode->init()
+		if $oldver < version->parse('11.0');

A node's pg_version is assigned via _set_pg_version() when creating it
using PostgreSQL::Test::Cluster::new(). In order to make the
difference with the set of initdb options to use when setting up the
old node, it would be simpler to rely on that, no? Version.pm is able
to handle integer as well as string comparisons for the version
strings.
--
Michael

Tom Lane

tgl@sss.pgh.pa.us

over 3 years ago

In reply to: Anton A. Melnikov (#1)

Re: [PATCH] Fix pg_upgrade test from v10

"Anton A. Melnikov" <aamelnikov@inbox.ru> writes:

Found out that test for pg_upgrade (test.sh for 11-14 and
002_pg_upgrade.pl for 15+) doesn't work from 10th versions to higher
ones due to incompatible options for initdb and default PGDATA permissions.

The buildfarm animals that test cross-version upgrades are not
unhappy, so please be more specific about what problem you
are trying to solve.

regards, tom lane

Michael Paquier

michael@paquier.xyz

over 3 years ago

In reply to: Tom Lane (#3)

Re: [PATCH] Fix pg_upgrade test from v10

On Thu, Jun 02, 2022 at 12:36:30AM -0400, Tom Lane wrote:

The buildfarm animals that test cross-version upgrades are not
unhappy, so please be more specific about what problem you
are trying to solve.

Anton is complaining about the case where you try to use the in-core
upgrade tests with a set of binaries/dump/source tree older that the
current version tested passed down as environment variables. test.sh
and the new TAP tests authorize that but they have their limits in
portability, which is what Anton is proposing to improve here. The
client buildfarm does not make use of the in-core facility, as it has
its own module and logic to check after the case of cross-version
upgrades (see PGBuild/Modules/TestUpgradeXversion.pm).

My 2c.
--
Michael

Andrew Dunstan

andrew@dunslane.net

over 3 years ago

In reply to: Michael Paquier (#2)

Re: [PATCH] Fix pg_upgrade test from v10

On 2022-06-01 We 21:37, Michael Paquier wrote:

On Thu, Jun 02, 2022 at 04:22:52AM +0300, Anton A. Melnikov wrote:

Found out that test for pg_upgrade (test.sh for 11-14 and 002_pg_upgrade.pl
for 15+) doesn't work from 10th versions to higher ones due to incompatible
options for initdb and default PGDATA permissions.

Yeah, there are still TODOs in this stuff. Those tests can also break
easily depending on the dump you are pushing to the old node when
doing cross-version upgrades. Perl makes it a bit easier to reason
about improving this area in the future, though, and MSVC is able to
catch up on that.
# To increase coverage of non-standard segment size and group access without
# increasing test runtime, run these tests with a custom setting.
# --allow-group-access and --wal-segsize have been added in v11.
-$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
+my ($oldverstr) = `$ENV{oldinstall}/bin/pg_ctl --version` =~ /(\d+\.\d+)/;
+my ($oldver) =  (version->parse(${oldverstr}));
+$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ])
+		if $oldver >= version->parse('11.0');
+$oldnode->init()
+		if $oldver < version->parse('11.0');
A node's pg_version is assigned via _set_pg_version() when creating it
using PostgreSQL::Test::Cluster::new(). In order to make the
difference with the set of initdb options to use when setting up the
old node, it would be simpler to rely on that, no? Version.pm is able
to handle integer as well as string comparisons for the version
strings.

Both these patches look dubious.

1. There is no mention of why there's a change w.r.t. Cygwin and
permissions checks. Maybe it's ok, but it seems off topic and is
certainly not referred to in the patch submission.

2. As Michael says, we should not be using perl's version module, we
should be using the version object built into each
PostgreSQL::Test::Cluster instance.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Anton A. Melnikov

aamelnikov@inbox.ru

over 3 years ago

In reply to: Andrew Dunstan (#5)

4 attachment(s)

Re: [PATCH] Fix pg_upgrade test from v10

Hello!

On 02.06.2022 23:57, Andrew Dunstan wrote:

1. There is no mention of why there's a change w.r.t. Cygwin and
permissions checks. Maybe it's ok, but it seems off topic and is
certainly not referred to in the patch submission.

Thanks for the comments!
It was my error to change w.r.t. Cygwin. I've fixed it in the second
version of the patch. But change in permissons check is correct. If we
fix the error with initdb options, we've got the next one while testing
upgrade from v10:
"files in PGDATA with permission != 640"
and the test.sh will end immediately.
The thing is that the default permissions have changed in v11+ due to
this commit:
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=c37b3d08ca6873f9d4eaf24c72a90a550970cbb8.
Changes of permissions checks in test.sh fix this error.

On 2022-06-01 We 21:37, Michael Paquier wrote:

A node's pg_version is assigned via _set_pg_version() when creating it
using PostgreSQL::Test::Cluster::new(). In order to make the
difference with the set of initdb options to use when setting up the
old node, it would be simpler to rely on that, no? Version.pm is able
to handle integer as well as string comparisons for the version
strings.

2. As Michael says, we should not be using perl's version module, we
should be using the version object built into each
PostgreSQL::Test::Cluster instance.

Sure, very valuable note. Fixed it in the 2nd version of the patch attached.

Also find that i forgot to adjust initdb keys for new node in v15. So
there was an error due to wal-segsize mismatch. Fixed it in the 2nd
version too. And added patches for other versions.

The client buildfarm does not make use of the in-core facility, as it
has its own module and logic to check after the case of cross-version
upgrades (see PGBuild/Modules/TestUpgradeXversion.pm)..

Michael, thanks a lot for your 2c.

With best regards,

--
Anton A. Melnikov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachments:

v2-0001-For-v12-13-Fix-upgrade-test-from-v10.patchtext/x-patch; charset=UTF-8; name=v2-0001-For-v12-13-Fix-upgrade-test-from-v10.patchDownload

commit 5857e955aabe1a4eb8a8e826ddcba02a25fd2b00
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Sat Jun 4 12:24:16 2022 +0300

    Fix test for pg_upgrade from 10x versions.

diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
index 9f6fb3e018..4fc6e6657c 100644
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -24,7 +24,13 @@ standard_initdb() {
 	# without increasing test runtime, run these tests with a custom setting.
 	# Also, specify "-A trust" explicitly to suppress initdb's warning.
 	# --allow-group-access and --wal-segsize have been added in v11.
-	"$1" -N --wal-segsize 1 --allow-group-access -A trust
+	initdbopt="-N -A trust"
+	if [ $OLD_PG_VERSION_NUM -ge 110000 ]; then
+		initdbopt="$initdbopt --wal-segsize 1 --allow-group-access"
+	fi
+
+	"$1" $initdbopt
+
 	if [ -n "$TEMP_CONFIG" -a -r "$TEMP_CONFIG" ]
 	then
 		cat "$TEMP_CONFIG" >> "$PGDATA/postgresql.conf"
@@ -131,6 +137,7 @@ PGHOSTADDR="";        unset PGHOSTADDR
 
 # Select a non-conflicting port number, similarly to pg_regress.c
 PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$newsrc"/src/include/pg_config.h | awk '{print $3}'`
+OLD_PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$oldsrc"/src/include/pg_config.h | awk '{print $3}'`
 PGPORT=`expr $PG_VERSION_NUM % 16384 + 49152`
 export PGPORT
 
@@ -235,18 +242,26 @@ pg_upgrade $PG_UPGRADE_OPTS -d "${PGDATA}.old" -D "$PGDATA" -b "$oldbindir" -p "
 
 # make sure all directories and files have group permissions, on Unix hosts
 # Windows hosts don't support Unix-y permissions.
+if [ $OLD_PG_VERSION_NUM -lt 110000 ]; then
+	NEW_DIR_PERM=700
+	NEW_FILE_PERM=600
+else
+	NEW_DIR_PERM=750
+	NEW_FILE_PERM=640
+fi
+
 case $testhost in
 	MINGW*) ;;
-	*)	if [ `find "$PGDATA" -type f ! -perm 640 | wc -l` -ne 0 ]; then
-			echo "files in PGDATA with permission != 640";
+	*)	if [ `find "$PGDATA" -type f ! -perm $NEW_FILE_PERM | wc -l` -ne 0 ]; then
+			echo "files in PGDATA with permission != $NEW_FILE_PERM";
 			exit 1;
 		fi ;;
 esac
 
 case $testhost in
 	MINGW*) ;;
-	*)	if [ `find "$PGDATA" -type d ! -perm 750 | wc -l` -ne 0 ]; then
-			echo "directories in PGDATA with permission != 750";
+	*)	if [ `find "$PGDATA" -type d ! -perm $NEW_DIR_PERM | wc -l` -ne 0 ]; then
+			echo "directories in PGDATA with permission != $NEW_DIR_PERM";
 			exit 1;
 		fi ;;
 esac

v2-0001-For-v11-Fix-upgrade-test-from-v10.patchtext/x-patch; charset=UTF-8; name=v2-0001-For-v11-Fix-upgrade-test-from-v10.patchDownload

commit c3f47a7d857ee053b8b2c7c5de40b8ab9cb6bae7
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Sat Jun 4 12:49:55 2022 +0300

    Fix test for pg_upgrade from 10x versions.

diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
index d4c4320a04..cc710633fb 100644
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -23,7 +23,14 @@ standard_initdb() {
 	# To increase coverage of non-standard segment size and group access
 	# without increasing test runtime, run these tests with a custom setting.
 	# Also, specify "-A trust" explicitly to suppress initdb's warning.
-	"$1" -N --wal-segsize 1 -g -A trust
+	# --allow-group-access and --wal-segsize have been added in v11.
+	initdbopt="-N -A trust"
+	if [ $OLD_PG_VERSION_NUM -ge 110000 ]; then
+		initdbopt="$initdbopt --wal-segsize 1 --allow-group-access"
+	fi
+
+	"$1" $initdbopt
+
 	if [ -n "$TEMP_CONFIG" -a -r "$TEMP_CONFIG" ]
 	then
 		cat "$TEMP_CONFIG" >> "$PGDATA/postgresql.conf"
@@ -145,6 +152,7 @@ PGHOSTADDR="";        unset PGHOSTADDR
 
 # Select a non-conflicting port number, similarly to pg_regress.c
 PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$newsrc"/src/include/pg_config.h | awk '{print $3}'`
+OLD_PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$oldsrc"/src/include/pg_config.h | awk '{print $3}'`
 PGPORT=`expr $PG_VERSION_NUM % 16384 + 49152`
 export PGPORT
 
@@ -237,18 +245,26 @@ pg_upgrade $PG_UPGRADE_OPTS -d "${PGDATA}.old" -D "$PGDATA" -b "$oldbindir" -B "
 
 # make sure all directories and files have group permissions, on Unix hosts
 # Windows hosts don't support Unix-y permissions.
+if [ $OLD_PG_VERSION_NUM -lt 110000 ]; then
+	NEW_DIR_PERM=700
+	NEW_FILE_PERM=600
+else
+	NEW_DIR_PERM=750
+	NEW_FILE_PERM=640
+fi
+
 case $testhost in
 	MINGW*) ;;
-	*)	if [ `find "$PGDATA" -type f ! -perm 640 | wc -l` -ne 0 ]; then
-			echo "files in PGDATA with permission != 640";
+	*)	if [ `find "$PGDATA" -type f ! -perm $NEW_FILE_PERM | wc -l` -ne 0 ]; then
+			echo "files in PGDATA with permission != $NEW_FILE_PERM";
 			exit 1;
 		fi ;;
 esac
 
 case $testhost in
 	MINGW*) ;;
-	*)	if [ `find "$PGDATA" -type d ! -perm 750 | wc -l` -ne 0 ]; then
-			echo "directories in PGDATA with permission != 750";
+	*)	if [ `find "$PGDATA" -type d ! -perm $NEW_DIR_PERM | wc -l` -ne 0 ]; then
+			echo "directories in PGDATA with permission != $NEW_DIR_PERM";
 			exit 1;
 		fi ;;
 esac

v2-0001-For-v15-Fix-upgrade-test-from-v10.patchtext/x-patch; charset=UTF-8; name=v2-0001-For-v15-Fix-upgrade-test-from-v10.patchDownload

commit 2c8c78faba37f66c2ef88392f58ce8e241772300
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Fri Jun 3 23:50:14 2022 +0300

    Fix test for pg_upgrade from 10x versions.

diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 80437e93b7..d9d97b1b3d 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -58,7 +58,14 @@ my $oldnode =
 # To increase coverage of non-standard segment size and group access without
 # increasing test runtime, run these tests with a custom setting.
 # --allow-group-access and --wal-segsize have been added in v11.
-$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
+my $ver_with_newopts = 11;
+my $oldver = $oldnode->{_pg_version};
+
+$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ])
+		if $oldver >= $ver_with_newopts;
+$oldnode->init()
+		if $oldver < $ver_with_newopts;
+
 $oldnode->start;
 
 # The default location of the source code is the root of this directory.
@@ -145,7 +152,10 @@ if (defined($ENV{oldinstall}))
 
 # Initialize a new node for the upgrade.
 my $newnode = PostgreSQL::Test::Cluster->new('new_node');
-$newnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
+$newnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ])
+		if $oldver >= $ver_with_newopts;
+$newnode->init()
+		if $oldver < $ver_with_newopts;
 my $newbindir = $newnode->config_data('--bindir');
 my $oldbindir = $oldnode->config_data('--bindir');

v2-0001-For-v14-Fix-upgrade-test-from-v10.patchtext/x-patch; charset=UTF-8; name=v2-0001-For-v14-Fix-upgrade-test-from-v10.patchDownload

commit dd62bd663167918365ce92577a19d208961a2f2a
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Sat Jun 4 11:58:01 2022 +0300

    Fix test for pg_upgrade from 10x versions.

diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
index f353e565b5..ac9fc15646 100644
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -24,7 +24,13 @@ standard_initdb() {
 	# without increasing test runtime, run these tests with a custom setting.
 	# Also, specify "-A trust" explicitly to suppress initdb's warning.
 	# --allow-group-access and --wal-segsize have been added in v11.
-	"$1" -N --wal-segsize 1 --allow-group-access -A trust
+	initdbopt="-N -A trust"
+	if [ $OLD_PG_VERSION_NUM -ge 110000 ]; then
+		initdbopt="$initdbopt --wal-segsize 1 --allow-group-access"
+	fi
+
+	"$1" $initdbopt
+
 	if [ -n "$TEMP_CONFIG" -a -r "$TEMP_CONFIG" ]
 	then
 		cat "$TEMP_CONFIG" >> "$PGDATA/postgresql.conf"
@@ -136,6 +142,7 @@ PGHOSTADDR="";        unset PGHOSTADDR
 
 # Select a non-conflicting port number, similarly to pg_regress.c
 PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$newsrc"/src/include/pg_config.h | awk '{print $3}'`
+OLD_PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$oldsrc"/src/include/pg_config.h | awk '{print $3}'`
 PGPORT=`expr $PG_VERSION_NUM % 16384 + 49152`
 export PGPORT
 
@@ -240,18 +247,26 @@ pg_upgrade $PG_UPGRADE_OPTS -d "${PGDATA}.old" -D "$PGDATA" -b "$oldbindir" -p "
 
 # make sure all directories and files have group permissions, on Unix hosts
 # Windows hosts don't support Unix-y permissions.
+if [ $OLD_PG_VERSION_NUM -lt 110000 ]; then
+	NEW_DIR_PERM=700
+	NEW_FILE_PERM=600
+else
+	NEW_DIR_PERM=750
+	NEW_FILE_PERM=640
+fi
+
 case $testhost in
 	MINGW*|CYGWIN*) ;;
-	*)	if [ `find "$PGDATA" -type f ! -perm 640 | wc -l` -ne 0 ]; then
-			echo "files in PGDATA with permission != 640";
+	*)	if [ `find "$PGDATA" -type f ! -perm $NEW_FILE_PERM | wc -l` -ne 0 ]; then
+			echo "files in PGDATA with permission != $NEW_FILE_PERM";
 			exit 1;
 		fi ;;
 esac
 
 case $testhost in
 	MINGW*|CYGWIN*) ;;
-	*)	if [ `find "$PGDATA" -type d ! -perm 750 | wc -l` -ne 0 ]; then
-			echo "directories in PGDATA with permission != 750";
+	*)	if [ `find "$PGDATA" -type d ! -perm $NEW_DIR_PERM | wc -l` -ne 0 ]; then
+			echo "directories in PGDATA with permission != $NEW_DIR_PERM";
 			exit 1;
 		fi ;;
 esac

Justin Pryzby

pryzby@telsasoft.com

over 3 years ago

In reply to: Anton A. Melnikov (#6)

Re: [PATCH] Fix pg_upgrade test from v10

Would you add this to to the (next) CF ?

It's silly to say that v9.2 will be supported potentially for a handful more
years, but that the upgrade-testing script itself doesn't support that, so
developers each have to reinvent its fixups.

See also 20220122183749.GO23027@telsasoft.com, where I proposed some of the
same things.

--
Justin

Anton A. Melnikov

aamelnikov@inbox.ru

over 3 years ago

In reply to: Justin Pryzby (#7)

6 attachment(s)

Re: [PATCH] Fix pg_upgrade test from v10

Hello!

On 01.07.2022 20:07, Justin Pryzby wrote:

Would you add this to to the (next) CF ?

Yes, i've put it on september CF.

It's silly to say that v9.2 will be supported potentially for a handful more
years, but that the upgrade-testing script itself doesn't support that, so
developers each have to reinvent its fixups.

I've test the attached patch in all variants from v9.5..15 to supported
versions 10..master. The script test.sh for 9.5->10 and 9.6->10 upgrades
works fine without any patch.
In 9.4 there is a regress test largeobject to be patched to allow
upgrade test from this version.So i've stopped at 9.5.
This is clear that we limit the destination version for upgrade test to
the supported versions only. In our case destination versions
starting from the 10th inclusively.
But is there are a limit for the source version for upgrade test from?

See also 20220122183749.GO23027@telsasoft.com, where I proposed some of the
same things.

Thanks a lot, i've add some code for 14+ from
/messages/by-id/20220122183749.GO23027@telsasoft.com
to the attached patch.

With best regards,

--
Anton A. Melnikov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Attachments:

v3-0001-For-master-Fix-upgrade-test-from-v10-and-earl.patchtext/x-patch; charset=UTF-8; name=v3-0001-For-master-Fix-upgrade-test-from-v10-and-earl.patchDownload

commit 8cba3ca4a68a0d41ff8ac4cd7c92546f093f8c4d
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Fri Jun 3 23:50:14 2022 +0300

    Fix test for pg_upgrade from 10x and earlier versions.

diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 2f9b13bf0a..1fd1b6f028 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -60,7 +60,14 @@ my $oldnode =
 # To increase coverage of non-standard segment size and group access without
 # increasing test runtime, run these tests with a custom setting.
 # --allow-group-access and --wal-segsize have been added in v11.
-$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
+my $ver_with_newopts = 11;
+my $oldver = $oldnode->{_pg_version};
+
+$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ])
+		if $oldver >= $ver_with_newopts;
+$oldnode->init()
+		if $oldver < $ver_with_newopts;
+
 $oldnode->start;
 
 # The default location of the source code is the root of this directory.
@@ -147,7 +154,10 @@ if (defined($ENV{oldinstall}))
 
 # Initialize a new node for the upgrade.
 my $newnode = PostgreSQL::Test::Cluster->new('new_node');
-$newnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
+$newnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ])
+		if $oldver >= $ver_with_newopts;
+$newnode->init()
+		if $oldver < $ver_with_newopts;
 my $newbindir = $newnode->config_data('--bindir');
 my $oldbindir = $oldnode->config_data('--bindir');
 
diff --git a/src/bin/pg_upgrade/upgrade_adapt.sql b/src/bin/pg_upgrade/upgrade_adapt.sql
index 27c4c7fd01..d47d2075f5 100644
--- a/src/bin/pg_upgrade/upgrade_adapt.sql
+++ b/src/bin/pg_upgrade/upgrade_adapt.sql
@@ -84,8 +84,8 @@ DO $stmt$
 \if :oldpgversion_le13
 -- Until v10, operators could only be dropped one at a time, so be careful
 -- to stick with one command for each drop here.
-DROP OPERATOR public.#@# (pg_catalog.int8, NONE);
-DROP OPERATOR public.#%# (pg_catalog.int8, NONE);
-DROP OPERATOR public.!=- (pg_catalog.int8, NONE);
-DROP OPERATOR public.#@%# (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.#@# (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.#%# (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.!=- (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.#@%# (pg_catalog.int8, NONE);
 \endif

v3-0001-For-v11-Fix-upgrade-test-from-v10-and-earl.patchtext/x-patch; charset=UTF-8; name=v3-0001-For-v11-Fix-upgrade-test-from-v10-and-earl.patchDownload

commit bc69d06d6f5bdc31b60452d3af340e6af3faba31
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Sat Jun 4 12:49:55 2022 +0300

    Fix test for pg_upgrade from 10x versions.

diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
index d4c4320a04..cc710633fb 100644
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -23,7 +23,14 @@ standard_initdb() {
 	# To increase coverage of non-standard segment size and group access
 	# without increasing test runtime, run these tests with a custom setting.
 	# Also, specify "-A trust" explicitly to suppress initdb's warning.
-	"$1" -N --wal-segsize 1 -g -A trust
+	# --allow-group-access and --wal-segsize have been added in v11.
+	initdbopt="-N -A trust"
+	if [ $OLD_PG_VERSION_NUM -ge 110000 ]; then
+		initdbopt="$initdbopt --wal-segsize 1 --allow-group-access"
+	fi
+
+	"$1" $initdbopt
+
 	if [ -n "$TEMP_CONFIG" -a -r "$TEMP_CONFIG" ]
 	then
 		cat "$TEMP_CONFIG" >> "$PGDATA/postgresql.conf"
@@ -145,6 +152,7 @@ PGHOSTADDR="";        unset PGHOSTADDR
 
 # Select a non-conflicting port number, similarly to pg_regress.c
 PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$newsrc"/src/include/pg_config.h | awk '{print $3}'`
+OLD_PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$oldsrc"/src/include/pg_config.h | awk '{print $3}'`
 PGPORT=`expr $PG_VERSION_NUM % 16384 + 49152`
 export PGPORT
 
@@ -237,18 +245,26 @@ pg_upgrade $PG_UPGRADE_OPTS -d "${PGDATA}.old" -D "$PGDATA" -b "$oldbindir" -B "
 
 # make sure all directories and files have group permissions, on Unix hosts
 # Windows hosts don't support Unix-y permissions.
+if [ $OLD_PG_VERSION_NUM -lt 110000 ]; then
+	NEW_DIR_PERM=700
+	NEW_FILE_PERM=600
+else
+	NEW_DIR_PERM=750
+	NEW_FILE_PERM=640
+fi
+
 case $testhost in
 	MINGW*) ;;
-	*)	if [ `find "$PGDATA" -type f ! -perm 640 | wc -l` -ne 0 ]; then
-			echo "files in PGDATA with permission != 640";
+	*)	if [ `find "$PGDATA" -type f ! -perm $NEW_FILE_PERM | wc -l` -ne 0 ]; then
+			echo "files in PGDATA with permission != $NEW_FILE_PERM";
 			exit 1;
 		fi ;;
 esac
 
 case $testhost in
 	MINGW*) ;;
-	*)	if [ `find "$PGDATA" -type d ! -perm 750 | wc -l` -ne 0 ]; then
-			echo "directories in PGDATA with permission != 750";
+	*)	if [ `find "$PGDATA" -type d ! -perm $NEW_DIR_PERM | wc -l` -ne 0 ]; then
+			echo "directories in PGDATA with permission != $NEW_DIR_PERM";
 			exit 1;
 		fi ;;
 esac

v3-0001-For-v12-Fix-upgrade-test-from-v10-and-earl.patchtext/x-patch; charset=UTF-8; name=v3-0001-For-v12-Fix-upgrade-test-from-v10-and-earl.patchDownload

commit 9236b239921aa0946d1e87d908d41e57d442304f
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Tue Jul 5 07:14:34 2022 +0300

    Fix test for pg_upgrade from 10x and some earlier versions.

diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
index 7ca01ff38e..57787bba69 100644
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -24,7 +24,13 @@ standard_initdb() {
 	# without increasing test runtime, run these tests with a custom setting.
 	# Also, specify "-A trust" explicitly to suppress initdb's warning.
 	# --allow-group-access and --wal-segsize have been added in v11.
-	"$1" -N --wal-segsize 1 --allow-group-access -A trust
+	initdbopt="-N -A trust"
+	if [ $OLD_PG_VERSION_NUM -ge 110000 ]; then
+		initdbopt="$initdbopt --wal-segsize 1 --allow-group-access"
+	fi
+
+	"$1" $initdbopt
+
 	if [ -n "$TEMP_CONFIG" -a -r "$TEMP_CONFIG" ]
 	then
 		cat "$TEMP_CONFIG" >> "$PGDATA/postgresql.conf"
@@ -130,6 +136,7 @@ PGHOSTADDR="";        unset PGHOSTADDR
 
 # Select a non-conflicting port number, similarly to pg_regress.c
 PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$newsrc"/src/include/pg_config.h | awk '{print $3}'`
+OLD_PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$oldsrc"/src/include/pg_config.h | awk '{print $3}'`
 PGPORT=`expr $PG_VERSION_NUM % 16384 + 49152`
 export PGPORT
 
@@ -234,18 +241,26 @@ pg_upgrade $PG_UPGRADE_OPTS -d "${PGDATA}.old" -D "$PGDATA" -b "$oldbindir" -B "
 
 # make sure all directories and files have group permissions, on Unix hosts
 # Windows hosts don't support Unix-y permissions.
+if [ $OLD_PG_VERSION_NUM -lt 110000 ]; then
+	NEW_DIR_PERM=700
+	NEW_FILE_PERM=600
+else
+	NEW_DIR_PERM=750
+	NEW_FILE_PERM=640
+fi
+
 case $testhost in
 	MINGW*) ;;
-	*)	if [ `find "$PGDATA" -type f ! -perm 640 | wc -l` -ne 0 ]; then
-			echo "files in PGDATA with permission != 640";
+	*)	if [ `find "$PGDATA" -type f ! -perm $NEW_FILE_PERM | wc -l` -ne 0 ]; then
+			echo "files in PGDATA with permission != $NEW_FILE_PERM";
 			exit 1;
 		fi ;;
 esac
 
 case $testhost in
 	MINGW*) ;;
-	*)	if [ `find "$PGDATA" -type d ! -perm 750 | wc -l` -ne 0 ]; then
-			echo "directories in PGDATA with permission != 750";
+	*)	if [ `find "$PGDATA" -type d ! -perm $NEW_DIR_PERM | wc -l` -ne 0 ]; then
+			echo "directories in PGDATA with permission != $NEW_DIR_PERM";
 			exit 1;
 		fi ;;
 esac

v3-0001-For-v13-Fix-upgrade-test-from-v10-and-earl.patchtext/x-patch; charset=UTF-8; name=v3-0001-For-v13-Fix-upgrade-test-from-v10-and-earl.patchDownload

commit cc306db88c5edfe74e9290796c0bfa8749c86d29
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Sat Jun 4 12:24:16 2022 +0300

    Fix test for pg_upgrade from 10x and earlier versions.

diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
index 9f6fb3e018..4fc6e6657c 100644
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -24,7 +24,13 @@ standard_initdb() {
 	# without increasing test runtime, run these tests with a custom setting.
 	# Also, specify "-A trust" explicitly to suppress initdb's warning.
 	# --allow-group-access and --wal-segsize have been added in v11.
-	"$1" -N --wal-segsize 1 --allow-group-access -A trust
+	initdbopt="-N -A trust"
+	if [ $OLD_PG_VERSION_NUM -ge 110000 ]; then
+		initdbopt="$initdbopt --wal-segsize 1 --allow-group-access"
+	fi
+
+	"$1" $initdbopt
+
 	if [ -n "$TEMP_CONFIG" -a -r "$TEMP_CONFIG" ]
 	then
 		cat "$TEMP_CONFIG" >> "$PGDATA/postgresql.conf"
@@ -131,6 +137,7 @@ PGHOSTADDR="";        unset PGHOSTADDR
 
 # Select a non-conflicting port number, similarly to pg_regress.c
 PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$newsrc"/src/include/pg_config.h | awk '{print $3}'`
+OLD_PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$oldsrc"/src/include/pg_config.h | awk '{print $3}'`
 PGPORT=`expr $PG_VERSION_NUM % 16384 + 49152`
 export PGPORT
 
@@ -235,18 +242,26 @@ pg_upgrade $PG_UPGRADE_OPTS -d "${PGDATA}.old" -D "$PGDATA" -b "$oldbindir" -p "
 
 # make sure all directories and files have group permissions, on Unix hosts
 # Windows hosts don't support Unix-y permissions.
+if [ $OLD_PG_VERSION_NUM -lt 110000 ]; then
+	NEW_DIR_PERM=700
+	NEW_FILE_PERM=600
+else
+	NEW_DIR_PERM=750
+	NEW_FILE_PERM=640
+fi
+
 case $testhost in
 	MINGW*) ;;
-	*)	if [ `find "$PGDATA" -type f ! -perm 640 | wc -l` -ne 0 ]; then
-			echo "files in PGDATA with permission != 640";
+	*)	if [ `find "$PGDATA" -type f ! -perm $NEW_FILE_PERM | wc -l` -ne 0 ]; then
+			echo "files in PGDATA with permission != $NEW_FILE_PERM";
 			exit 1;
 		fi ;;
 esac
 
 case $testhost in
 	MINGW*) ;;
-	*)	if [ `find "$PGDATA" -type d ! -perm 750 | wc -l` -ne 0 ]; then
-			echo "directories in PGDATA with permission != 750";
+	*)	if [ `find "$PGDATA" -type d ! -perm $NEW_DIR_PERM | wc -l` -ne 0 ]; then
+			echo "directories in PGDATA with permission != $NEW_DIR_PERM";
 			exit 1;
 		fi ;;
 esac

v3-0001-For-v14-Fix-upgrade-test-from-v10-and-earl.patchtext/x-patch; charset=UTF-8; name=v3-0001-For-v14-Fix-upgrade-test-from-v10-and-earl.patchDownload

commit 136bd56f358638524bc11a459724c8c5c7327005
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Sat Jun 4 11:58:01 2022 +0300

    Fix test for pg_upgrade from 10x and earlier versions.

diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
index f353e565b5..ac9fc15646 100644
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -24,7 +24,13 @@ standard_initdb() {
 	# without increasing test runtime, run these tests with a custom setting.
 	# Also, specify "-A trust" explicitly to suppress initdb's warning.
 	# --allow-group-access and --wal-segsize have been added in v11.
-	"$1" -N --wal-segsize 1 --allow-group-access -A trust
+	initdbopt="-N -A trust"
+	if [ $OLD_PG_VERSION_NUM -ge 110000 ]; then
+		initdbopt="$initdbopt --wal-segsize 1 --allow-group-access"
+	fi
+
+	"$1" $initdbopt
+
 	if [ -n "$TEMP_CONFIG" -a -r "$TEMP_CONFIG" ]
 	then
 		cat "$TEMP_CONFIG" >> "$PGDATA/postgresql.conf"
@@ -136,6 +142,7 @@ PGHOSTADDR="";        unset PGHOSTADDR
 
 # Select a non-conflicting port number, similarly to pg_regress.c
 PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$newsrc"/src/include/pg_config.h | awk '{print $3}'`
+OLD_PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$oldsrc"/src/include/pg_config.h | awk '{print $3}'`
 PGPORT=`expr $PG_VERSION_NUM % 16384 + 49152`
 export PGPORT
 
@@ -240,18 +247,26 @@ pg_upgrade $PG_UPGRADE_OPTS -d "${PGDATA}.old" -D "$PGDATA" -b "$oldbindir" -p "
 
 # make sure all directories and files have group permissions, on Unix hosts
 # Windows hosts don't support Unix-y permissions.
+if [ $OLD_PG_VERSION_NUM -lt 110000 ]; then
+	NEW_DIR_PERM=700
+	NEW_FILE_PERM=600
+else
+	NEW_DIR_PERM=750
+	NEW_FILE_PERM=640
+fi
+
 case $testhost in
 	MINGW*|CYGWIN*) ;;
-	*)	if [ `find "$PGDATA" -type f ! -perm 640 | wc -l` -ne 0 ]; then
-			echo "files in PGDATA with permission != 640";
+	*)	if [ `find "$PGDATA" -type f ! -perm $NEW_FILE_PERM | wc -l` -ne 0 ]; then
+			echo "files in PGDATA with permission != $NEW_FILE_PERM";
 			exit 1;
 		fi ;;
 esac
 
 case $testhost in
 	MINGW*|CYGWIN*) ;;
-	*)	if [ `find "$PGDATA" -type d ! -perm 750 | wc -l` -ne 0 ]; then
-			echo "directories in PGDATA with permission != 750";
+	*)	if [ `find "$PGDATA" -type d ! -perm $NEW_DIR_PERM | wc -l` -ne 0 ]; then
+			echo "directories in PGDATA with permission != $NEW_DIR_PERM";
 			exit 1;
 		fi ;;
 esac
diff --git a/src/bin/pg_upgrade/upgrade_adapt.sql b/src/bin/pg_upgrade/upgrade_adapt.sql
index 27c4c7fd01..d47d2075f5 100644
--- a/src/bin/pg_upgrade/upgrade_adapt.sql
+++ b/src/bin/pg_upgrade/upgrade_adapt.sql
@@ -84,8 +84,8 @@ DO $stmt$
 \if :oldpgversion_le13
 -- Until v10, operators could only be dropped one at a time, so be careful
 -- to stick with one command for each drop here.
-DROP OPERATOR public.#@# (pg_catalog.int8, NONE);
-DROP OPERATOR public.#%# (pg_catalog.int8, NONE);
-DROP OPERATOR public.!=- (pg_catalog.int8, NONE);
-DROP OPERATOR public.#@%# (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.#@# (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.#%# (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.!=- (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.#@%# (pg_catalog.int8, NONE);
 \endif

v3-0001-For-v15-Fix-upgrade-test-from-v10-and-earl.patchtext/x-patch; charset=UTF-8; name=v3-0001-For-v15-Fix-upgrade-test-from-v10-and-earl.patchDownload

commit 1101b263ceb0009b495706e25b7aad569e2f78ce
Author: Anton A. Melnikov <a.melnikov@postgrespro.ru>
Date:   Fri Jun 3 23:50:14 2022 +0300

    Fix test for pg_upgrade from 10x adn earlier versions.

diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
index 2f9b13bf0a..1fd1b6f028 100644
--- a/src/bin/pg_upgrade/t/002_pg_upgrade.pl
+++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl
@@ -60,7 +60,14 @@ my $oldnode =
 # To increase coverage of non-standard segment size and group access without
 # increasing test runtime, run these tests with a custom setting.
 # --allow-group-access and --wal-segsize have been added in v11.
-$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
+my $ver_with_newopts = 11;
+my $oldver = $oldnode->{_pg_version};
+
+$oldnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ])
+		if $oldver >= $ver_with_newopts;
+$oldnode->init()
+		if $oldver < $ver_with_newopts;
+
 $oldnode->start;
 
 # The default location of the source code is the root of this directory.
@@ -147,7 +154,10 @@ if (defined($ENV{oldinstall}))
 
 # Initialize a new node for the upgrade.
 my $newnode = PostgreSQL::Test::Cluster->new('new_node');
-$newnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ]);
+$newnode->init(extra => [ '--wal-segsize', '1', '--allow-group-access' ])
+		if $oldver >= $ver_with_newopts;
+$newnode->init()
+		if $oldver < $ver_with_newopts;
 my $newbindir = $newnode->config_data('--bindir');
 my $oldbindir = $oldnode->config_data('--bindir');
 
diff --git a/src/bin/pg_upgrade/upgrade_adapt.sql b/src/bin/pg_upgrade/upgrade_adapt.sql
index 27c4c7fd01..d47d2075f5 100644
--- a/src/bin/pg_upgrade/upgrade_adapt.sql
+++ b/src/bin/pg_upgrade/upgrade_adapt.sql
@@ -84,8 +84,8 @@ DO $stmt$
 \if :oldpgversion_le13
 -- Until v10, operators could only be dropped one at a time, so be careful
 -- to stick with one command for each drop here.
-DROP OPERATOR public.#@# (pg_catalog.int8, NONE);
-DROP OPERATOR public.#%# (pg_catalog.int8, NONE);
-DROP OPERATOR public.!=- (pg_catalog.int8, NONE);
-DROP OPERATOR public.#@%# (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.#@# (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.#%# (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.!=- (pg_catalog.int8, NONE);
+DROP OPERATOR IF EXISTS public.#@%# (pg_catalog.int8, NONE);
 \endif

Justin Pryzby

pryzby@telsasoft.com

over 3 years ago

In reply to: Anton A. Melnikov (#8)

Re: [PATCH] Fix pg_upgrade test from v10

On Tue, Jul 05, 2022 at 09:01:49AM +0300, Anton A. Melnikov wrote:

On 01.07.2022 20:07, Justin Pryzby wrote:

It's silly to say that v9.2 will be supported potentially for a handful more
years, but that the upgrade-testing script itself doesn't support that, so
developers each have to reinvent its fixups.

I've test the attached patch in all variants from v9.5..15 to supported
versions 10..master. The script test.sh for 9.5->10 and 9.6->10 upgrades
works fine without any patch.
In 9.4 there is a regress test largeobject to be patched to allow upgrade
test from this version.So i've stopped at 9.5.
This is clear that we limit the destination version for upgrade test to the
supported versions only. In our case destination versions
starting from the 10th inclusively.
But is there are a limit for the source version for upgrade test from?

As of last year, there's a reasonably clear policy for support of old versions:

https://www.postgresql.org/docs/devel/pgupgrade.html
|pg_upgrade supports upgrades from 9.2.X and later to the current major release of PostgreSQL, including snapshot and beta releases.

See: e469f0aaf3c586c8390bd65923f97d4b1683cd9f

So it'd be ideal if this were to support versions down to 9.2.

This is failing in cfbot:
http://cfbot.cputube.org/anton-melnikov.html

..since it tries to apply all the *.patch files to the master branch, one after
another. For branches other than master, I suggest to name the patches *.txt
or similar. Or, just focus for now on allowing upgrades *to* master. I'm not
sure if anyone is interested in patching test.sh in backbranches. I'm not
sure, but there may be more interest to backpatch the conversion to TAP
(322becb60).

--
Justin

#10

Michael Paquier

michael@paquier.xyz

over 3 years ago

In reply to: Justin Pryzby (#9)

Re: [PATCH] Fix pg_upgrade test from v10

On Tue, Jul 05, 2022 at 02:08:24PM -0500, Justin Pryzby wrote:

..since it tries to apply all the *.patch files to the master branch, one after
another. For branches other than master, I suggest to name the patches *.txt
or similar. Or, just focus for now on allowing upgrades *to* master. I'm not
sure if anyone is interested in patching test.sh in backbranches. I'm not
sure, but there may be more interest to backpatch the conversion to TAP
(322becb60).

I am fine to do something for v15 and HEAD, as TAP makes that slightly
easier. Now, this patch is far from being complete, as it would still
generate a lot of diffs. Some of them cannot be easily avoided, but
there are areas where it is straightforward to do so:
- pg_dump needs to use --extra-float-digits=0 when dumping from a
version strictly older than v12.
- This does nothing for the headers ond footers of the logical dumps
that are version-dependent. One simple thing that can be done here is
to remove the comments from the logical dumps, something that the
buildfarm code already does.

That's the kind of things I already proposed on this thread, aimed at
improving the coverage, and this takes care of more issues than what's
proposed here:
/messages/by-id/Yox1ME99GhAemMq1@paquier.xyz

I'll rebase my patch to include fixes for --wal-segsize and
--allow-group-access when using versions older than v11.
--
Michael

#11

Anton A. Melnikov

aamelnikov@inbox.ru

over 3 years ago

In reply to: Michael Paquier (#10)

Re: [PATCH] Fix pg_upgrade test from v10

Hello!

On 06.07.2022 08:58, Michael Paquier wrote:

That's the kind of things I already proposed on this thread, aimed at
improving the coverage, and this takes care of more issues than what's
proposed here:
/messages/by-id/Yox1ME99GhAemMq1(at)paquier(dot)xyz
I'll rebase my patch to include fixes for --wal-segsize and
--allow-group-access when using versions older than v11.
--
Michael

Thanks!
I looked at this thread and tried to apply some changes from it in practice.
And found one strange error and describe it in a comment here:
/messages/by-id/cc7e961a-d5ad-8c6d-574b-478aacc11cf7@inbox.ru
It would be interesting to know if it occures on
my PC only or somewhere else.

On 05.07.2022 22:08, Justin Pryzby wrote:

..since it tries to apply all the *.patch files to the master branch, one after
another. For branches other than master, I suggest to name the patches *.txt
or similar. Or, just focus for now on allowing upgrades *to* master. I'm not
sure if anyone is interested in patching test.sh in backbranches. I'm not
sure, but there may be more interest to backpatch the conversion to TAP
(322becb60).

Yes, the backport idea seems to be interesting. I wrote more about this in a new thread:
/messages/by-id/e2b1f3a0-4fda-ba72-5535-2d0395b9e68f@inbox.ru
as the current topic has nothing to do with the backport of TAP tests.

With best regards,
--
Anton A. Melnikov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

#12

Michael Paquier

michael@paquier.xyz

over 3 years ago

In reply to: Anton A. Melnikov (#11)

Re: [PATCH] Fix pg_upgrade test from v10

On Mon, Aug 01, 2022 at 01:04:39AM +0300, Anton A. Melnikov wrote:

I looked at this thread and tried to apply some changes from it in practice.
And found one strange error and describe it in a comment here:
/messages/by-id/cc7e961a-d5ad-8c6d-574b-478aacc11cf7@inbox.ru
It would be interesting to know if it occures on
my PC only or somewhere else.

As mentioned upthread, please note that I'll send my arguments on the
other thread where I have sent my patch.
--
Michael